MyArxiv
Robotics
★ Training Reasoning Models in Simulation to Natively Use Real Robots
Robot-use agents (RUAs) have emerged as a promising approach to general-purpose robot control. However, strong performance has so far been achieved only by frontier proprietary models such as GPT-6 Astra and Claude Fable 5.2, whereas open-weight models remain substantially less capable. We show that an open-weight 27B reasoning model can be trained to use real robots natively through reinforcement learning, entirely in simulation and without demonstrations. Like a computer-use agent (CUA), the model observes the environment, reasons in a chain of thought, and generates the next end-effector pose, without any harness. We build a Post-training Engine for Agentic Robotics (PEAR) and use it to train the model on simulated robot tasks at scale with robot-use reinforcement learning (RURL), mirroring how frontier models acquire computer use. We find that the skills acquired in simulation generalize well beyond the training tasks: they transfer to an unseen robot, to the real world, and to novel instructions. After training on LIBERO and RoboDojo in simulation, the model succeeds in 60% of our trials on a real dual-arm robot, whereas the base model succeeds in none. On LIBERO, training raises the success rate from 1.6% to 74.4%, approaching GPT-6 Astra and GPT-6.1 Sol at maximum reasoning effort. Compared with $π_{0.5}$, a vision-language-action model fine-tuned on demonstrations, our model is substantially more robust to changes in the task itself. On LIBERO-Pro, $π_{0.5}$ succeeds in only 0.75% of episodes when assigned a new task in a familiar scene whereas our model succeeds in 57.4%. Finally, the model retains its general capabilities, with no measurable degradation on nine reasoning, coding and vision benchmarks. These results highlight robot-use reinforcement learning as a promising component of foundation-model post-training.
★ Modeling the Frictional Contact of Continuum Robots with Environments through Linear Complementarity Problem
Accurate friction modeling in continuum robots is essential for precise motion control and path planning. However, current approaches make predictions only for the mechanism's internal friction that is limited by the boundary forces of the Coulomb friction cone, which fail to characterize the static friction and motion hysteresis. In this paper, we propose a novel friction model based on complementarity constraints under the Coulomb friction assumption. This model predicts various friction effects caused by environmental interactions, including both static and sliding frictions and hysteresis behavior, as well as the corresponding robot deformation induced by these interactions, which are crucial for the practical motion planning and control of continuum robots. Additionally, real-time implementation of this model can be achieved, meeting the demand for computational efficiency in robotic applications. Experimental validations demonstrated that this model is able to reduce the maximum tip position error from 45.4 mm in the frictionless model to 5.85 mm in our friction-included model with a 287.9 mm-long tendon-driven continuum robot.
comment: Accepted for publication in International Journal of Robotics Research (IJRR). 21 pages, 18 figures
★ Disturbance-Aware Informative Planning in Spatiotemporal Environments
Autonomous robots are widely used for persistent monitoring of spatiotemporal environments, where quantities of interest (QoI) change across space and time and require repeated measurements. In practice, robot motion can be affected by external disturbances that vary across the environment. When such external disturbances are not considered in trajectory optimization, it can lead to inaccurate predictions of robot motion and, consequently, the information acquired about the QoI during execution. However, estimating disturbances throughout the environment can be inefficient, as disturbances in regions with little overlap with monitoring trajectories have limited impact on monitoring performance. We therefore propose a two-stage informative planning framework for unknown, spatially varying but time-invariant disturbances, combining informed disturbance sampling with disturbance-aware trajectory optimization. First, we identify where QoI measurements are needed and generate alternative paths to these locations using the environment geometry. We then prioritize disturbance measurements based on their expected reduction of disturbance uncertainty along these paths. Second, rather than optimizing information acquisition based on nominal robot motion, we incorporate the learned disturbance belief into trajectory optimization to account for its effects on robot motion and information acquisition. Across four simulation scenarios, the proposed method reduces disturbance-learning time by 25.0--52.3 percent compared with environment-wide estimation and identifies paths that appear costly under nominal dynamics but become preferable when disturbances are considered.
comment: Submitted to Learning for Dynamics and Control Conference 2027. Project website: https://kalebbennaveed.github.io/dist-aware-ipp/
★ In Situ Monitoring and Predictive Modeling of Harmful Algal Blooms with Autonomous Surface Vehicles
Harmful Algal Blooms (HABs) pose significant environmental, economic, and public-health challenges in coastal and inland water bodies, creating a need for scalable methods to collect spatially distributed water-quality data. This paper presents an autonomous surface vessel-based sensing system for in-situ water-quality monitoring related to harmful algal bloom detection. The system integrates a BlueBoat Autonomous Surface Vessel (ASV), a motorized winch for depth-controlled sonde deployment, a multiparameter water-quality sonde, side-scan sonar, and an event-camera payload. Autonomous waypoint missions were used to collect measurements across selected sampling regions and multiple depths, enabling both horizontal and vertical characterization of environmental conditions. Field deployments were conducted in Lake Pontchartrain, Louisiana, with additional measurements collected in False River, Louisiana to increase dataset diversity. The collected measurements included algal pigment indicators, nutrient concentration, turbidity, temperature, and other water-quality parameters relevant to bloom formation. In addition, complementary sonar and event-camera data were collected during vehicle operation to provide contextual environmental information. A machine-learning classification approach was also evaluated using the collected water-quality measurements to support HAB and non-HAB classification. The results demonstrate the feasibility of using a modular ASV platform for repeatable, multi-modal water-quality data collection and support future development of autonomous monitoring systems.
★ Toward Resilient Self-Driving Laboratories through Telerobotic Interventions
Self-driving laboratories (SDLs) accelerate scientific research by integrating robotics, automated experimentation, and machine learning to execute experiments autonomously. Biology and chemistry have adopted these systems. However, increasing SDL complexity heightens susceptibility to unexpected faults in automated workflows, highlighting the need for efficient recovery mechanisms. In this study, we investigate telerobotic interventions to enhance SDL resilience against unexpected failures. We developed a teleoperation interface enabling human operators to resolve disruptions during automated tasks. The interface enables precise control of a six-degree-of-freedom (DOF) robotic arm (UR3e) using a 3D mouse (3Dconnexion SpaceMouse Pro). Two velocity-based controller strategies were evaluated: (i) a coupled mode mapping all six DOFs to the 3D mouse's central knob and (ii) a decoupled mode separating translational and rotational motions. To evaluate the framework, teleoperators intervened across three SDL fault scenarios. In the first study, comparative trials showed that the decoupled controller was associated with shorter completion times and reduced rotational motion. In a second study, teleoperators performed an aiming task with and without reference-path guidance. Users were more target-driven and risk-tolerant without guidance but showed behavioral differences under different guidance strategies. These findings suggest that controller configuration and reference-path design influence operator performance during SDL fault recovery, informing teleoperation interface optimization. This work demonstrates the potential feasibility of teleoperation as a human-in-the-loop recovery mechanism for SDL workflows, combining robotic precision with human adaptability and problem-solving. These findings also provide a foundation for future studies in shared autonomy and semi-autonomous intervention.
★ When Will You Hand It Over? A Hidden Start Delay Makes Self-Play Handover Robust to Unfamiliar Partners
Cooperative self-play can teach a humanoid to receive an object without any human data, but the partner it meets after training is never the one it trained with; we show that a fluent self-play handover policy breaks with unfamiliar partners, identify the main cause, and repair it. In a physics-based parcel handover between a Unitree G1 and a learned giver, both trained by alternating PPO self-play, the receiver completes 95% and 90% of handovers with its own partner (two training seeds) but only 31% with scripted partners it has never met. Paired interventions isolate the cause: under a shared reward the learned giver always starts its offer at once, so the receiver learns a single rendezvous, and delaying its own partner by one second cuts success from 95% to 6% (87% to 34% in the second seed). We introduce hidden start-delay randomisation (HSD), a one-line rule on the learned partner that the receiver never observes. After 300 further updates, success with unseen scripted partners rises from 31% to 80% in both seeds, against 35--41% for a budget-matched control. The gain holds on a fresh scenario seed, on eleven held-out partner conditions and on partners that start later than any training delay; it cuts high-force contact three- to five-fold and needs no retraining of the partner.
comment: CoRL 2026, Human-Centered Robot Learning and Interaction (HumanRLI) Workshop. Code: https://github.com/mkhoshnam/Selfplay-PPO-handover
★ From AI-Enabled Remote Patient Monitoring Toward Agentic Robotics: Embodiment, Governance, and Care Integration
AI-enabled remote patient monitoring is evolving beyond passive data collection toward agentic AI systems that interpret patient states, coordinate care, and initiate action across domestic and clinical settings. This structured scoping review finds that mature agentic care has not yet emerged: Most systems remain human led, while stronger decision-action coupling and embodied AI remain largely confined to prototypes and pilots. Healthcare robotics makes the transition from monitoring to action especially visible by translating AI-generated recommendations and decisions into speech, movement, prompting, physical assistance, escalation, and care coordination, making embodiment, clinical integration, human oversight, and governance fundamental design requirements. Further progress requires safe and interoperable care architectures in which intelligent components operate through clinically validated and accountable workflows. We propose agentic robotics as a future direction for supervised systems embedded in the care ecosystem, connecting AI agents and robots with clinicians, caregivers, and patients through auditable workflows that can be interrupted and remain subject to human oversight and explicit AI governance.
comment: 34 pages, 6 figures, 3 tables. Accepted for publication in Annual Review of Control, Robotics, and Autonomous Systems, Volume 10 (2027)
★ The State Is Not the Bottleneck: A Paged-State Gated-DeltaNet Tile on a Tri-Mode Multicast Mesh, Bit-Exact from RTL to Silicon
Linear-attention language models replace a growing key-value cache with a recurrent state of fixed size. On an edge FPGA, this state can exceed the on-chip memory and must be paged to external memory. This paper asks whether such paging dominates the cost of decoding (generating tokens). We built a hardware tile that computes one complete layer of a custom 369-million-parameter hybrid Gated DeltaNet model on an Agilex 5 FPGA and pages its recurrent state. The model state needs 6.2 MiB, whereas the device has 0.87 MiB of block memory. Paging adds 26.0 MB of memory traffic per decoded token, at most 11.2 percent of the calculated total. On the board at 75 MHz, with synthetic weights, one token takes 11.15 ms in this layer. The matrix-vector products with the weights take about 62 percent of this time and the state phase, including the paging, 13 percent. The tile is one node of a multicast mesh of tri-mode processing elements. We measure the error of the fixed-point arithmetic against floating point and check that the hardware agrees bit for bit with a fixed-point reference. In simulation, the state logic agrees with the reference on all 18 recurrent layers. On the board, six builds pass all 30 runs of 64 tokens and two runs of 1,024 tokens. For this model and memory interface, the delivery of the weights limits the decoding performance, and the recurrent state does not. State paging is therefore practical for such a model on an edge FPGA.
comment: 12 pages, 5 figures, 6 tables. The technical supplement is included as an ancillary file. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible
★ CS-MPC: Contraction-Certified Sampling-Based MPC for Nonlinear Systems Under Uncertainty
We present contraction-certified sampling-based model predictive control (CS-MPC) for tracking and safety of nonlinear robotic systems under parametric uncertainty, disturbances, and safety constraints that are revealed online. CS-MPC uses a contraction-certified tracking feedback inside every sampled rollout, which concentrates the rollouts around the reference, and an execution layer that verifies the applied input using a quantile-tightened discrete-time control barrier function condition. We derive a verification procedure for nonlinear contraction of the feedback-closed dynamics, and we prove a finite-sample stochastic stability bound and a mission-level probabilistic safety guarantee. We validate CS-MPC in simulation on a rover, a quadrotor, and an asteroid landing. When compared with standard model predictive path integral (MPPI) control, CS-MPC attains the same tracking error with 7 to 20 times fewer rollouts for the rover, and a 2 to 4 times lower tracking error for the quadrotor, with no safety violations in 200 Monte Carlo missions.
★ Vision-Language-Action Models: A Life-Cycle Survey of Design, Training and Deployment
Vision-language-action (VLA) models adapt web-pretrained vision-language models to output robot actions, so that web knowledge carries over to physical control. In under 4 years the field has moved from RT-2, the first VLA, to open models that fine-tune on a single consumer GPU and to proprietary generalists that clean unseen homes. This survey reviews 88 works from December 2022 to October 2026, mainly on manipulation and navigation, plus humanoid control, world-action models and evaluation. We follow a VLA through its life cycle, from model design to training and deployment, along seven axes. Model design covers (a) inputs and memory, what the model observes; (b) architecture, how its pipeline is built; (c) action output, how it produces motor commands; and (d) intermediate predictions, what else it predicts, from reasoning steps to future observations. Training covers (e) pretraining data, the mix of robot, web, human and simulated data, and (f) post-training, adaptation to the target robot. Deployment covers (g) real-time execution, keeping pace with the robot. We analyse 22 models on these axes and report their training data, model size, inference speed and hardware. Three trends emerge: (i) training data has broadened, from about 10,000 hours of teleoperated robot data ($π_0$) to 23,000 community-collected episodes (SmolVLA) and 20,000 hours of egocentric human video (GR00T N1.7); (ii) model sizes have split: most recent models have 3-7 billion parameters, while a new group under one billion runs on consumer GPUs; and (iii) generalist VLAs are emerging that handle both manipulation and navigation, such as UniVLA and Qwen-VLA. We also compare VLAs with world-action models, which predict future observations as well as actions. We review evidence that some VLAs with near-perfect benchmark scores fail when objects or the camera move slightly, and open up further research questions.
comment: 43 pages, 9 figures, 11 tables
★ Heavy Metal Acrobatics: Learning Dynamic Control for Skid-Steer Loaders
Automation of heavy machinery offers the opportunity to improve productivity in mechanized industries such as construction, mining, and material handling. Yet, automation of such machines is challenged by their complex dynamics, in part because of hydraulic actuation that involves fluids, flexible elements, and nonlinear delays. A concomitant challenge is the size and weight of these machines, which can make experiments on the physical systems exceedingly dangerous. Here, we show that \ac{RL} in simulation, followed by transfer of the learned policies from simulation to reality, can successfully automate acrobatic maneuvers that push a heavy construction machine to the limits of its actuation and have only previously been accomplished by a small number of elite human operators in official world-record performances. We trained policies that perform autonomous acrobatic maneuvers on a John Deere 332G, an operational 5-tonne skid-steer loader. The policies can autonomously swing the machine up onto two wheels and, while balancing on two wheels, drive over rough terrain, trace out specified trajectories, spin around, and swing the bucket back and forth before landing it safely back onto four wheels. This work demonstrates that \ac{RL} and sim-to-real transfer enable dynamic autonomous control of heavy machinery, directly contributing to greater automation in heavy equipment sectors. Videos: https://sites.google.com/leggedrobotics.com/heavy-metal-acrobatics This is the author's version of the work. It is posted here by permission of the AAAS for personal use, not for redistribution. The definitive version was published in Science Robotics on 11 November 2026, DOI: https://doi.org/10.1126/scirobotics.aef0894
★ Envisioning Agentic Robotics: A New Era of Robotic Systems
The release of GPT-6 Astra marks the emergence of a new paradigm in robotics: agentic robotics. Within weeks, frontier models with agentic harnesses matched or exceeded task-specific policies on many embodied benchmarks. Instead of training a dedicated policy for each family of tasks, an agentic robotic system applies a foundation model to open-world embodied tasks. It operates in a closed loop: observing the environment, interpreting user intent, reasoning about the task, invoking skills and tools, verifying outcomes, and adapting its strategy. Despite these advances, the community has yet to converge on a shared definition, a reliable evaluation framework, or a research roadmap. We argue that Embodied System 2, realized by a foundation model together with an embodied harness, forms the decision-making core of agentic robotics. Its central challenge is to translate general-purpose reasoning into effective and verifiable actions under physical constraints such as limited reaction time, mechanical limits, unpredictable environment changes, and irreversible consequences. Motivated by this perspective, we propose an evaluation framework that measures both model capabilities and closed-loop system performance. Furthermore, we identify the principles that carry over from digital to physical agents and the challenges that must be addressed to achieve reliable physical autonomy. On top of the Embodied System 2 and the evaluation framework, we envision a research roadmap toward sustained autonomy: the ability to perform work safely and reliably with progressively broader responsibility, from executing individual primitives to fulfilling standing roles.
comment: Project website: https://grid-robotics-official.github.io/agentic-robotics-roadmap
★ PROOF: Perceptive Robustness via Out-of-Distribution Fallback for Legged Locomotion
Vision-based perceptive locomotion has enabled legged robots to achieve agile and terrain-aware locomotion across complex environments. However, the reliability of such systems heavily depends on visual observations, which are inherently susceptible to occlusion, reflective surfaces, and illumination variations, often resulting in catastrophic locomotion failures under degraded perception conditions. In this paper, we present PROOF (Perceptive Robustness via Out-of-Distribution Fallback for Legged Locomotion), a perceptive locomotion framework that enables robust locomotion under severe visual disturbances. Our method adopts a two-stage teacher-student learning paradigm that explicitly models perception reliability: the teacher learns both perception-driven and proprioceptive fallback behaviors, while the student predicts exteroceptive features with uncertainty and suppresses them under out-of-distribution inputs. We evaluate our method on a Unitree Go2 quadruped equipped with a RealSense D435 depth camera in real-world settings. Under normal visual conditions, our method enables the robot to successfully perform high-dynamic parkour maneuvers, including traversing large gaps, elevated platforms, and narrow beam terrains. Under severe visual degradation, such as heavy occlusion and strong illumination changes, the robot continues to maintain stable and robust locomotion performance.
comment: CoRL 2026
★ Objectives as Policies: Vision-Language Model Cost Programs for Contact-Rich Manipulation
Vision-language models (VLMs) have shown strong semantic and spatial reasoning capabilities in 3D environments. Recently, connecting VLMs with model-predictive controllers (MPC) has emerged as a paradigm for combining strong semantic reasoning with physics-based planning. However, connecting these two for contact-rich robotic manipulation requires a task representation that bridges the two and captures how physical interactions can be reasoned by VLMs. In this work, we introduce Objectives as Policies (OaP), a framework that connects a pretrained VLM to a shared sampling-based MPC through an editable objective program. Each program specifies a sequence of stages for completing a task, combining running and terminal costs from a cost library with stage completion conditions. Given a natural-language task instruction and scene observation, we build a simulation-based digital twin through real-to-sim, and the VLM generates an objective program that is executed by the MPC in the simulation. The program is iteratively updated by the VLM based on MPC rollouts, before being deployed in the real world. We evaluate our framework in both simulation and the real world on four tabletop manipulation tasks, and show that it outperforms strong baselines. Additional videos and examples are available on the project website: https://zihaolu001.github.io/oap-project/
comment: 8 pages, 3 figures. Project website: https://zihaolu001.github.io/oap-project/
★ NPU-Accelerated GNSS Signal Acquisition
On small single-board computers (SBCs), GNSS signal acquisition is a major CPU load. This paper proposes using the neural processing unit (NPU) of an SBC not for machine learning but as an accelerator for the parallel code search in GNSS signal acquisition. The discrete Fourier transform (DFT) used in acquisition is decomposed by the four-step DFT into matrix products of small DFTs and expressed by $1\times1$ convolutions, element-wise products, and transposes, which NPUs execute efficiently. Furthermore, we propose a method that divides the DFT, the non-coherent integration, and the peak detection between the CPU and the NPU according to the configuration of the NPU (an NPU with its own memory connected via PCIe, or an NPU sharing memory with the CPU). The proposed method is implemented in the open-source software GNSS receiver Pocket SDR and evaluated on a Raspberry Pi 5 combined with the external AX8850 NPU and on the NPU built into the SoC (RK3588) of the ROCK 5C, a small SBC of comparable size. Compared with acquisition by an optimized CPU implementation, the acquisition of short codes on the Raspberry Pi 5 with the AX8850 is 3.0-3.8 times faster. In the evaluation of the whole receiver, the number of tracking channels that reach their first lock within 30 s with the NPU is 1.5 times that with the CPU, and the time until 90% of the channels are locked is reduced from 66 s to 26 s. Furthermore, because the CPU load of acquisition is reduced, the number of tracking channels that can be processed in real time increases from about 175 to about 230. These results show that using the NPU on SBCs with limited CPU resources is effective for achieving both GNSS signal acquisition and tracking. The implementation is publicly available as open source under the name PocketSDR-NPU.
★ ESI: An Embodied Self-Improvement Flywheel for LLM-VLA Agents
Independently pretrained language models and vision-language-action (VLA) policies provide complementary building blocks for embodied agents, but connecting them does not automatically produce coordinated behavior. The VLA may generate actions that diverge from the LLM's intended subtask and, upon regaining control after a correction, undo the promising state restored by the LLM. We introduce ESI (Embodied Self-Improvement), an automated flywheel that uses environmental interaction to align reasoning and action. ESI treats the modular agent as the unit of post-training, repeatedly refining the prompts that guide the LLM, the shared skill interface that expresses its intent, and the VLA parameters that realize that intent, then redeploying these components together. Our key insight is that collaboration itself can supply supervision for improving this interface. The LLM's subtask decisions provide semantic supervision for what the policy should execute, while its effective corrections and subsequent student continuations provide action-level supervision for recovery and handoff. The student also learns from its own effective execution, allowing locally useful experience to contribute even when the overall episode fails. On ten selected RoboDojo tasks, ESI achieves a mean Score of 62.03 and a mean success rate of 49.41% using an LLM-VLA agent without generating task-specific control code. On both metrics, it outperforms the published reference results of all hybrid embodied agents, including the Agent+WAM system GO-3 Nano Preview (47.27 Score and 36.60% success), as well as all compared code-based methods.
comment: Technical report. 25 pages, 7 figures. Code: https://github.com/Embodied-Self-Improvement/ESI
★ Evaluating Language Models as Spatial Priors for Object-Goal Navigation
Semantic exploration methods for object-goal navigation use large language models (LLMs) to score how strongly a target object is associated with a room type or with a nearby object. This paper considers obtaining from the LLM a spatial prior instead: a distribution over the position of the target in a room. We evaluate whether two open-weight LLMs, Qwen3.5-27B and Gemma-4-31B-it, can provide such a prior. Using real room layouts from CubiCasa5K and ScanNet, we measure the likelihood the models assign to real and perturbed layouts, and the layouts they generate. The object frequencies and room sizes of the generated rooms follow the room type, but the proximity of the objects to the walls does not follow the differences between room types found in real rooms. The likelihood distinguishes real layouts from perturbed ones partly through their geometry and partly through the agreement between the coordinates and a field that states the wall each object touches; for the movable furniture of ScanNet, it is higher for layouts with all objects piled near the room center than for the real layouts in most rooms. When an object is given in the prompt, the models place a related object relative to it. When the wall of each object is given, they place most objects against that wall, but do not keep all objects inside a given room rectangle. As a prior for object search, the generated placements reduce the searched area relative to a uniform prior, but by less than a third of the reduction obtained with a prior built from real layouts, and by less than two thirds when both priors are spatially smoothed. These results support the use of LLMs for object-room associations and for writing coordinates under constraints determined by another component, followed by a geometric check, but not as a source of placement priors where real layouts are available.
★ PoliMop: Policy-in-the-Loop Motion Planning in 3D Spaces for Mobile Robots
Reaching a goal state in a complex environment often requires a robot to overcome challenging terrain and obstacles. Which routes are accessible depends on the robot's physical design, the control policy it uses, and how it modifies its surroundings. We present a kinodynamic rapidly-exploring random tree (RRT) planner that uses physics simulation to exploit skills encoded in the control policy, which allows it to find a path in the scene with respect to the robot's capabilities. The planner samples states from a reduced state space in 3D around the scene and expands the RRT toward each sample using policy rollouts. We guide sampling with a preplanned path for a simple mass-point model to focus exploration on promising regions leading toward the goal state, reducing planning time by up to 63%. Simulation experiments with quadrupeds, humanoids, and wheeled robots demonstrate that our approach supports drop-in integration of various robots and control policies, exploiting their physical and policy capabilities without modifying the underlying planning algorithm. Predefined environmental modifications, such as placing a plank across a gap to build a bridge, will automatically be picked up by the planner as the robot's reachable space expands.
comment: 8 pages, 5 figures. Videos and code: https://ariael21.github.io/polimop.html
★ A Generalizable Approach to Safety-Aware Robotic Behavior Planning with an Explicit Constraint Space
Large language and vision-language models have substantially advanced robotic behavior planning from natural-language instructions and visual observations. However, safe manipulation in tabletop environments remains challenging because the safety constraints required for a task depend on the surrounding scene and may not be explicitly specified in the instruction. We propose a safety-aware behavior planning framework based on an explicit Constraint Space that stores reusable textual safety rules. Rather than retrieving rules solely by semantic similarity, we formulate constraint selection as task- and environment-conditioned applicability learning. A lightweight query-conditioned dual-tower model identifies the constraints applicable to the current context, which are then provided together with the original task and image to a multimodal behavior planner. To handle incorrect constraint selections and previously unseen hazards, we introduce a multimodal Safety Inspector that evaluates generated primitive plans, reconstructs the complete applicable constraint set, and provides corrective planning feedback. Validated inspection feedback is further used for test-time adaptation of constraint applicability and parameter-efficient adaptation of the Safety Inspector through LoRA, while newly identified constraints are incorporated into the Constraint Space. The resulting framework integrates explicit safety knowledge representation,context-aware constraint selection, corrective safety inspection, and test-time knowledge adaptation for safety-aware robotic behavior planning.
★ Reusing Locomotion Training Recipes Across Humanoids via Body-Aligned Virtual Joints
Reinforcement learning has enabled robust humanoid locomotion, yet training recipes (rewards, architectures, hyperparameters) remain hard to reuse across robots with different native joint coordinates, since policies output native joint angles while action scales, bounds, gains, and penalties are defined in those robot-specific coordinates. Existing transmission models convert native targets into actuator commands but do not standardize the coordinate interface exposed to the policy. We introduce a body-aligned virtual joint space upstream of this mapping: policy actions and observations follow a common anatomical convention, and a deterministic kinematic layer maps virtual targets to native ones, leaving the existing transmission and low-level control stack unchanged. Under a single unmodified recipe on three humanoids with different leg-axis layouts, it is the only interface that tracks accurately, keeps the feet aligned with the travel direction, and trains stably on all three, and on Unitree G1 hardware it reduces heading drift by nearly a factor of six.
comment: 8 pages, 7 figures, 1 table
★ Per-Agent-Safe Navigation of Globally Driven Robot Collectives via Affine Field Design
Safe navigation of robot collectives requires coordinating motion and deformation so that the collective can traverse constrained environments while ensuring each robot's safety. Existing controllers for swarmalator collectives largely rely on local control, where each robot receives an individual command, which causes the control and communication requirements to grow with the number of agents. We introduce a broadcast control formulation in which a single spatially affine field governs both the motion and deformation of the collective. The scalar component of the field regulates collective size, and its spatial gradient generates collective translation through non-reciprocal interactions. Both components enter the collective dynamics directly, which allows per-robot wall, actuation, and cohesion constraints to be enforced using control barrier functions within a single quadratic program for the whole collective. The resulting controller continuously adapts the collective size to the local environment, and enables safe passage through corridors narrower than the unconstrained collective without pre-planned resizing. We further derive how the available actuation budget determines the reachable speed and minimum passable corridor width, and we demonstrate that two independently driven collectives can simultaneously traverse a shared junction using distinct fixed phase labels without inter-collective coordination.
comment: 9 figures, 1 table
★ Multimodal Person Concept Model for Person Identification under Varying Observation Conditions
Service robots in domestic and office environments must identify users to provide personalized services based on their preferences and previous interactions. This is challenging because only a few observations may be available for learning, while observation conditions change owing to appearance, environmental noise, and utterance content. To address this challenge, this study proposes a multimodal person concept model that learns person concepts from a small number of observations, without person labels, and enables robust identification under changing conditions. The model employs a nonparametric Bayesian generative model based on the Chinese restaurant process and integrates visual, acoustic, and linguistic features extracted using Contrastive Language-Image Pre-training (CLIP) and Contrastive Language-Audio Pretraining (CLAP), followed by principal component analysis (PCA). We evaluated six participants under eight conditions in which appearance changes, additive noise, and transcription errors were introduced individually and in combination. Averaged over the eight conditions, the proposed model achieved an identification accuracy of 0.89 while inferring the number of person concepts from the observations, which is comparable to k-means clustering that was supplied with the true number of participants (0.88). We also examined PCA dimensionality, modality availability, and variation strength, and demonstrated the model on a Toyota Human Support Robot. These results show that multimodal person concept learning supports robust person identification under changing conditions, and that identification can succeed even when the number of concepts is not correctly recovered. The project page and demonstration video are available at https://okitamisan01.github.io/person_concept/.
comment: 12 pages, 4 figures, 11 tables. Submitted to Artificial Life and Robotics (AROB Journal Track). Project page: https://okitamisan01.github.io/person_concept/
★ Beyond Prediction Gain: LASR as a Metric for Action-Grounded Latents in World Models
World models that support planning, control, and interpretable reasoning require latent representations that genuinely encode action. Yet the large-scale video data from which such models are learned overwhelmingly lack action annotations. Latent-action models infer action-like variables from observation sequences, enabling action-aware learning from unlabeled videos or trajectories. A fundamental concern, however, remains: a latent that improves future prediction may simply encode the current state, episode progress, task identity, or other predictive shortcuts. Existing evaluation practices---reconstruction loss, rollout fidelity, action recovery, or downstream performance---cannot isolate this distinction. We introduce the Latent Action Support Ratio (LASR), a dimensionless metric that measures, under offline evaluation on held-out data, whether a learned latent provides predictive support for action-related targets beyond the current state and non-action alternatives. LASR provides two complementary summaries: GeoLASR captures typical support, while WorstLASR identifies the weakest case. We instantiate the LASR benchmark with recorded actions and corresponding physical state changes as targets, episode-held-out splits, one predictor type for all comparisons, and positive and negative controls. Experiments on recorded robot manipulation data and a matched multi-task panel demonstrate that LASR detects known action information, rejects representative non-action shortcuts, and enables graded comparisons among imperfect latent representations. We further use LASR to train and audit a lightweight residual adapter, obtaining descriptively better point estimates whose uncertainty is assessed with paired comparisons. LASR thus provides a principled offline protocol for measuring action-related predictive support in world-model latents.
★ When Does Cross-View Consistency Matter in Multi-View Appearance Augmentation? A Controlled Study for Visuomotor Policies
Multi-view visual augmentation often seeks consistency across cameras, but it is unclear which forms of consistency matter for downstream control. We use simulation-state replay to manipulate cross-view coupling while preserving the per-camera perturbation distributions, physical trajectories, and action labels. Across seven LIBERO manipulation tasks, marginal-matched changes in global appearance, ghost injection, and target color have small average costs for a late-fusion Diffusion Policy: point estimates are near zero or negative, and the largest upper endpoint of the 90% posterior credible intervals is 2.55 percentage points. In contrast, geometrically anti-coupled observations reduce success by 3.30 percentage points relative to coupled observations (90% CrI [0.64, 6.28]), with displacement magnitude, direction marginals, and the number of affected views matched. Camera interventions reveal substantial dependence on wrist-view geometry, while feature analysis shows appearance sensitivity patterns associated with the factors randomized during training. ACT shows a higher target-color cost estimate than DP, with uncertainty that includes zero and sensitivity to task composition. Within the tested tasks and policy configurations, these findings motivate selective consistency constraints: appearance coupling can sometimes be relaxed, whereas task-relevant geometry and cross-view spatial correspondence warrant protection.
comment: 30 pages, 10 figures. Code: https://github.com/PKUJZX/xview-consistency-augmentation
★ ERA: Error-Resistant Autonomous Object Navigation with Predicted Depth
Depth and 3D foundation models can predict geometry from RGB images, allowing zero-shot object navigation without a depth sensor. However, when predicted depth is mapped as if it came from a sensor, view-dependent errors accumulate in the map, remove exploration goals, and end the search before the agent can observe and correct them. We propose ERA, a training-free object navigation system that replaces the depth sensor with depth predicted from RGB images and poses. ERA follows a paradigm we call Move with Error: movement provides new views that correct map errors, and the corrected map enables further movement. It realizes this paradigm with revisable occupancy and with exploration from reachable viewpoints, which continues the search when no frontier remains, and verifies the target across views before stopping. On three benchmarks, ERA achieves state-of-the-art performance among training-free methods without depth sensors. Its map and exploration components also raise VLFM's success from 10.8% to 61.7% under the same predicted depth. We further deploy ERA on a real robot.
★ Ego2Flow: Benchmarking the Traffic-System Impact of Autonomous Driving on Maps Distilled from Real-World Roads
End-to-end autonomous driving is predominantly evaluated through ego-vehicle safety and task completion. Its deployment in urban traffic, however, creates sustained interactions among vehicles whose collective effects remain insufficiently evaluated. We introduce Ego2Flow, a closed-loop benchmark for assessing the traffic-system impact of autonomous driving on maps distilled from real-world cities. To construct realistic 3D urban environments for sustained closed-loop traffic simulation, we develop an agent-guided map distillation framework that integrates real-world maps and observations with complementary geographic information. Through an iterative feedback mechanism and a traffic information graph that jointly guide scene generation, the framework preserves lane topology, critical road geometry and decision-relevant visual conditions while simplifying nonessential details and completing unobserved regions. Using our proposed framework, we generate and publicly release six CARLA maps from the Argoverse 2 dataset, covering the corresponding real-world regions and enabling paired comparisons of driving behavior under real and simulated observations. On these maps, Ego2Flow evaluates how autonomous driving policies affect traffic efficiency, safety, stability and energy consumption across diverse road structures. Compared with existing construction methods, our framework achieves the lowest average displacement error between planning outputs from paired real and simulated observations. We further benchmark representative driving policies to characterize their system-level impact on urban traffic. Project Page: https://github.com/Fisher-007/Ego2Flow.
★ EViDS: An Environment--Vehicle Dynamics Integrated Simulation Platform for Intelligent Chassis Development
The development of intelligent chassis increasingly requires the integration of environmental and perception information with high-fidelity vehicle dynamics to support environment-aware intelligent chassis control. However, existing autonomous-driving simulators and professional vehicle-dynamics tools are generally developed for different purposes, making it difficult to integrate environmental information, vehicle dynamics, and chassis algorithms. This paper proposes EViDS, an Environment-Vehicle Dynamics Integrated Simulation Platform for Intelligent Chassis Development. EViDS integrates CARLA-based environment and sensor simulation with CarSim-based high-fidelity vehicle dynamics. Reference-pose-based state alignment and multi-rate interaction maintain spatial and temporal consistency, while ray-casting-based tire-road interaction transfers environmental information to the vehicle-dynamics model. Scenario configuration, synchronized data acquisition, and algorithm interfaces further support intelligent chassis development. The platform is evaluated through environment-dynamics interaction tests with vehicle-dynamics consistency comparisons, together with intelligent chassis applications. Results show that EViDS preserves vehicle-dynamics characteristics, reliably transfers environmental information, and supports online closed-loop intelligent chassis control, providing a unified simulation basis for environment-aware intelligent chassis development and validation.
★ Imitation Policies Learn a Proprioceptive Anchor: A Predictive Law for Perception Noise and Its Closed Loop Cost
Imitation policies for robot manipulation are often trained on noisy object positions, and they learn to discount the noisy reading. We show how much they discount it, and why. Because demonstrators steer toward the object, the robot's own state carries information about where the object is. A trained policy combines this proprioceptive anchor with the reading and weights the reading by $α= V/(V + s^2)$, where $s^2$ is the reading noise and $V$ the anchor's error variance. $V$ belongs to the learner and is far smaller than a linear model of the demonstrations implies, which is why classical errors-in-variables (EIV) regression overestimates $α$. Measured once, $V$ predicted ten held-out sensitivities across two robomimic tasks, two learners and four new noise levels, with mean absolute error 0.025 against 0.37 for classical EIV, and a parameter-free bias test rules out the simpler explanation that the policies just overestimate the noise. The discount does not carry over to deployment, where the policy produces its own state: closed-loop sensitivity to a persistent reading offset exceeded $α$ in all 15 policies that passed the registered gates, although the stricter registered prediction, that all would follow the offset fully, held for 8 of them. Post hoc, lower $α$ went with lower success, steeply for diffusion, and a registered prospective test confirmed this for diffusion on Can at two new noise levels: the measured $α$ predicted success before any rollout, within 0.03 (95% intervals $\pm 0.08$), although training noise predicts it equally well. Every test was registered before its data, including the classical benchmark that failed, and all other analyses are labelled post hoc; all results are in simulation with synthetic Gaussian noise.
comment: 18 pages, 4 figures
★ AI-guided robotic in situ fabrication of curvature- and tissue-adaptive wireless bioelectronics
Personalized bioelectronic devices for wound therapy require not only conformal fabrication on complex body surfaces but also functional adaptation of their electrical characteristics to individual wound geometries. Here, we present an AI-guided robotic in situ printing platform for fabricating wireless wound electrotherapy devices directly on transparent wound dressings. The system integrates a 3D vision camera, a six-degree-of-freedom robotic arm, direct-ink-writing, and geometry-aware RF design optimization. AI-based wound segmentation is used to determine wound dimensions and local surface curvature, while the reconstructed three-dimensional geometry is incorporated into the electromagnetic design of the receiving coil and interdigitated electrodes. In contrast to conventional approaches that use surface geometry only for trajectory generation, the proposed method accounts for both anatomical curvature and tissue dielectric loading to compensate for geometry-induced shifts in resonant behavior. A printable silver ink was characterized in terms of rheology, dimensional reproducibility, electrical resistance, and electromechanical stability, and was used to fabricate multilayer wireless devices directly on curved polyurethane dressings. Wireless power transfer, impedance characteristics, and electric-field delivery were evaluated experimentally and numerically. Finally, robotic in situ fabrication was demonstrated on an anthropomorphic tissue-equivalent forearm phantom, where the measured dielectric properties of the phantom were incorporated into the RF optimization, enabling resonant operation near the target frequency of 25 MHz. This work establishes a perception-to-design-to-fabrication framework for geometry-adaptive wireless bioelectronics and provides a technological foundation for future physical-AI-enabled manufacturing of personalized on-body electrotherapy devices.
★ On the Utility of Pre-Trained Visual Representations for Policy Learning of Contact-Rich Insertion Tasks
Visuomotor imitation learning has become a predominant paradigm for real-world robotic manipulation, yet it remains data-inefficient and brittle to task-irrelevant visual shifts. Pre-trained visual representations promise to ease this burden by distilling priors from internet-scale data, but the ``right'' representation for robotics remains unresolved: existing studies span broad but sparse task distributions and reach conclusions that vary and even conflict, pointing to strong task and setting dependence. We examine this question by bringing a magnifying glass to a common contact-rich manipulation task: insertion. We compare representations from widely used vision foundation models against an explicit contact-centric representation on two closely related insertion tasks, perturb each along controlled axes of occlusion and physical tolerance, and test whether conclusions transfer to the real world. Task dependence proves to be more subtle than previous, broadly scoped evaluations revealed: the best representation differs even between our near-identical tasks, and small changes shift trends in unpredictable ways. Despite this variability, results across simulation and real-world experiments demonstrate the distinct value of spatially-grounded, and in particular, contact-centric information for contact-rich policy learning, while underscoring how difficult it is to estimate such cues reliably under realistic visual complexity and sensor noise. We hope these findings motivate further study of representations in contact-rich manipulation, along with advances in contact perception and its integration into visuomotor policy learning.
★ TouchTwin: Scaling Human Touch Data via Real-to-Sim-to-Real
Human demonstrations are becoming the main source of data for dexterous manipulation, and real-to-sim-to-real pipelines multiply each demonstration in simulation. These pipelines rebuild the interaction from vision, which recovers how the hand moves. Which fingers touch, when, and how hard remain unobserved, because a hand pose shows where each finger stops on the surface rather than how far it is driven to close. This gap is a force gap: on held-out recordings, a hand policy that imitates the observed pose grips with 0.31x the recorded force, and a policy trained on physics replays of vision-tracked poses grips with 1.55x. TouchTwin closes the gap with measured human touch. Each selected TouchScale recording, captured with head and wrist RGB-D cameras and an 880-taxel glove, is rebuilt in MuJoCo. Coding agents write the objects as physics-tested programs, and a per-subject tactile hand replays the interaction with the recorded touch setting the contact and the closing force of each finger. In simulation, a policy trained on touch-grounded replays grips at 0.99x the recorded force and succeeds in 20/20 trials on held-out recordings and 35/36 unseen perturbations; a contact predictor trained on them misses 13% of real contact frames, against 32% for vision-based replays. Each replay also seeds perturbed demonstrations with simulated contact and force labels, rendered as realistic video in new views and scenes.
♻ ★ HEIR: Harness Egocentric Intent for Human-Robot Interactions
For robots to assist, serve, and collaborate with humans, they must not only interpret human requests conveyed through language and multimodal intentional cues, but also execute long-horizon tasks that interleave navigation and manipulation. In this paper, we introduce HEIR, a resource for studying how to Harness Egocentric Intent for Human-Robot Interaction. HEIR pairs human speech, measured gaze, and egocentric video with robot observations and teleoperated navigation-manipulation trajectories, while each session preserves successive episodes, with each request fulfilled through multiple execution subtasks. The captured data, together with human-authored annotations, support the study of intent grounding, execution planning, and task completion within a unified interaction setting. To demonstrate the value of HEIR, we conduct extensive offline experiments to evaluate how models benefit from additional human-centric context across these tasks, together with real-robot experiments that assess execution task completion. HEIR thus provides a valuable resource for studying user-centric robot interaction in naturalistic household environments. Our demo and dataset are available online.
♻ ★ A Straight Line Is Not a Camera Pose: Auditing Short-Clip Pose Benchmarks
Many vision benchmarks align a prediction to the ground truth before scoring it. If the alignment minimises the error, an input-free template scores the truth's distance from everything the alignment can reach, so the ground truth alone fixes how well a template can score, before any model runs. Camera pose is our central case. The absolute trajectory error (ATE), the standard camera-motion measure, aligns camera positions by a similarity transform. On the common Sintel protocol, an image-free constant-velocity line has a lower mean ATE than 10 of 11 recent models and 85 of 110 published methods, and on Lightspeed than 5 of 9 models and 10 of 11 published methods. The straightness of the true paths separates these two from TUM, ScanNet, DyCheck and ADT, where every model beats the line. A residual signal-to-noise criterion with the alignment's first-order refit predicts when straightening a prediction lowers ATE in all but one of 4010 model-clip evaluations, including all 504 on three benchmarks registered in advance. The alignment also ignores where cameras look: rotating a trajectory by 30 degrees against its orientations leaves ATE unchanged yet worsens the static-point reprojection of every model on Sintel, TUM and ScanNet. Fitting the alignment's rotation to the predicted orientations gives pose-anchored ATE (PA-ATE), a one-function change at about ATE's cost (37 vs. 34 microseconds per clip). It is never below ATE, ranks every image-free reference below every model on Sintel, tracks the reprojection error under controlled rotation and straightening, largely preserves the models' order, and exposes Lightspeed ground-truth orientations that the images contradict. Beyond poses, templates beat some published methods on KITTI video depth, SSP-3D body shape and NoW faces, and in video depth this distance sets KITTI apart from Sintel and Bonn.
comment: 39 pages, 9 figures
♻ ★ VOKE: Visual Object Kinematics Estimator for Dexterous In-Hand Manipulation
Vision-based dexterous in-hand manipulation requires knowing not only where an object is, but also how it moves. However, most robotic manipulation systems rely on off-the-shelf visual perception models whose outputs are limited to static object masks, geometry, or 6D poses, leaving downstream controllers to infer object dynamics implicitly from noisy visual observations. We present VOKE, a kinematics-aware perception framework that augments existing object and hand-object pose estimators to recover a complete object kinematic state comprising pose, linear and angular velocity, and linear and angular acceleration. VOKE contains two core perception modules and a downstream application study. First, a Pluggable Kinematics Module (PKM) attaches to the output of a standard pose estimator and predicts the object's linear and angular velocity and acceleration from temporal visual features. Second, a Kinematics-guided Pose Refinement Module (KPRM) converts this motion vector into a second-order, motion-prior reference pose and refines the per-frame pose estimates. We evaluate VOKE on DexYCB and OakInk, where it consistently improves object pose estimation across three different backbones and substantially reduces velocity and acceleration estimation error compared with pose differencing. As a downstream application, we use Kinematics-Augmented Reinforcement Learning (KARL) to compare a kinematics-augmented policy with a pose-history baseline. After training the policies in simulation, we transfer the kinematics-augmented policy to the physical robot without policy fine-tuning, demonstrating the complete perception-to-control pathway in the target domain.
♻ ★ Self-Building Labs
Self-driving laboratories (SDLs) accelerate scientific discovery by automating experimental execution and optimization. AI agents can increasingly propose new experiments, but existing SDLs can only run the procedures they were built for. Adapting them to a new experiment still requires extensive manual engineering, which makes laboratory flexibility the bottleneck for scaling discovery. We introduce the self-building laboratory (SBL), which designs, configures, and operates its own laboratory setup given a high-level scientific goal. SBL starts from whatever already exists, from an empty room to a working SDL, and builds the missing capabilities: the lab layout, its digital twin, robot skills, and experimental workflows. We demonstrate the system working across different robots and setups, scaling from a single workstation to laboratories with several robots working together. We validate SBL's end-to-end capabilities through a real-world quantum-dot synthesis experiment. By shifting human researchers from manually configuring and programming device automation to specifying scientific objectives, SBL makes scientific automation more accessible, flexible, and scalable. Project website: https://self-building-labs.github.io/
♻ ★ A Stochastic Framework for Continuous-Time State Estimation of Continuum Robots
State estimation techniques for continuum robots (CRs) typically involve using computationally complex dynamic models, simplistic shape approximations, or are limited to quasi-static methods. These approaches can be sensitive to unmodeled disturbances acting on the robot. Inspired by a factor-graph optimization paradigm, this work introduces a continuous-time stochastic state estimation framework for continuum robots. We introduce factors based on continuous-time kinematics that are corrupted by a white noise Gaussian process (GP). By using a simple robot model paired with high-rate sensing, we show adaptability to unmodeled external forces and data dropout. The result contains an estimate of the mean and covariance for the robot's pose, velocity, and strain, each of which can be interpolated continuously in both space and time. This same interpolation scheme can be used during estimation, allowing for inclusion of measurements that are asynchronous to the estimated states. Our method's inherent sparsity leads to a linear solve complexity with respect to time and interpolation queries in constant time. We demonstrate our method on a CR with gyroscope and pose sensors, highlighting its versatility in real-world systems.
comment: 20 pages, 15 figures. Accepted to IEEE Transactions on Robotics. Notable changes from v1 include switching to a GTSAM back-end, adjusting the spatial factor definition, and the inclusion of a sensitivity study
♻ ★ OneSteer: Unlocking Steerable General Humanoid Motion Tracking via One-Step Diffusion Whole-Body Control
Deployment objectives often specify desired outcomes without providing revised trajectories, while parts of the original reference must still be followed. We present OneSteer, a one-step diffusion whole-body controller that combines general motion tracking with selective test-time steering. To accommodate new objectives, differentiable task costs guide optimization of the generator's input noise with frozen weights, while body-group reference masks relax selected tracking requirements. Predicted state-action plans provide a shared interface for task optimization and feedback-conditioned execution. For nominal tracking, the model trained from scratch achieves the lowest local tracking error among the evaluated baselines on successful held-out AMASS and BONES clips. For steering, one frozen distilled controller adapts across diverse deployment objectives in simulation, raising the clean passage rate under a 1.25 m bar from 2% to 90%. On a physical Unitree G1, OneSteer completes all 27 nominal tracking trials across nine diverse whole-body motion sequences without falling. With a steering cost derived from a signed distance field built using only onboard LiDAR, it also passes under a 1.1 m low bar in four of five attempts and steps over a 12 cm hurdle without falling in all three attempts. Beyond control, prediction variation in the model trained from scratch ranks likely tracking failures without physics rollouts. This scoring method is more than 10x faster than simulation-based scoring with five rollouts per clip and guides targeted fine-tuning. Project page: https://onesteer.github.io
♻ ★ STITCHER: Constrained Trajectory Planning in Complex Environments with Real-Time Motion Primitive Search
Autonomous high-speed navigation through large, complex environments requires real-time generation of agile trajectories that are dynamically feasible, collision-free, and satisfy state or actuator constraints. Modern trajectory planning techniques primarily use numerical optimization, as they enable the systematic computation of high-quality, expressive trajectories that satisfy various constraints. However, stringent requirements on computation time and the risk of numerical instability can limit the use of optimization-based planners in safety-critical scenarios. This work presents an optimization-free planning framework called STITCHER that stitches short trajectory segments together with graph search to compute long-range, expressive, and near-optimal trajectories in real-time. STITCHER outperforms modern optimization-based planners through our innovative planning architecture and several algorithmic developments that make real-time planning possible. Extensive simulation testing is performed to analyze the algorithmic components that make up STITCHER, along with a thorough comparison with three state-of-the-art optimization planners. Simulation tests show that safe trajectories can be created within a few milliseconds for paths that span the entirety of two 50 m x 50 m environments. Hardware tests with a custom quadrotor verify that STITCHER can produce trackable paths in real-time while respecting nonconvex constraints, such as limits on tilt angle and motor forces, with flight speeds up to 63 km/h.
♻ ★ Grid-based Submap Joining: An Efficient Algorithm for Simultaneously Optimizing Global Occupancy Map and Local Submap Frames IROS 2024
Optimizing robot poses and the map simultaneously has been shown to provide more accurate SLAM results. However, for non-feature based SLAM approaches, directly optimizing all the robot poses and the whole map will greatly increase the computational cost, making SLAM problems difficult to solve in large-scale environments. To solve the 2D non-feature based SLAM problem in large-scale environments more accurately and efficiently, we propose the grid-based submap joining method. Specifically, we first formulate the 2D grid-based submap joining problem as a non-linear least squares (NLLS) form to optimize the global occupancy map and local submap frames simultaneously. We then prove that in solving the NLLS problem using Gauss-Newton (GN) method, the increments of the poses in each iteration are independent of the occupancy values of the global occupancy map. Based on this property, we propose a poseonly GN algorithm equivalent to full GN method to solve the NLLS problem. The proposed submap joining algorithm is very efficient due to the independent property and the pose-only solution. Evaluations using simulations and publicly available practical 2D laser datasets confirm the outperformance of our proposed method compared to the state-of-the-art methods in terms of efficiency and accuracy, as well as the ability to solve the grid-based SLAM problem in very large-scale environments.
comment: Accepted by IROS 2024, the code is made available
♻ ★ Beyond Trajectory Similarity: Evaluating Surgical Planning by Protocol Adherence
Surgical planning is commonly evaluated by trajectory similarity to a single observed execution, a paradigm borrowed from activity anticipation benchmarks such as Ego4D. This conflates two distinct concepts: task completion and protocol adherence and assumes similarity to a completed surgical trajectory implies a valid plan. In surgery, this assumption is flawed: the clinically relevant target is protocol adherence rather than mere completion, and the reference trajectory is only one observed execution that may itself deviate from the protocol. We therefore introduce an evaluation framework grounded in expert-defined protocol rules that directly measures protocol adherence. Under this framework, similarity-based metrics systematically fail: they penalize protocol-adherent plans that deviate from the reference trajectory, while rewarding protocol-violating plans that remain superficially similar to it. Evaluating Video-LLMs further reveals two separable failure modes: unreliable video-to-state grounding and under-constrained planning. Neither is resolved by model scaling alone, whereas structural knowledge injection provides consistent gains. Our results motivate a shift from trajectory similarity toward protocol adherence as the evaluation standard for surgical planning.
♻ ★ Robots that Collaborate: Sequential Asymmetric Imitation for Learning Coupled Robot Policies
Collaborative mobile manipulation requires robots to coordinate with a partially observed partner while physically interacting through shared objects. This is difficult because failures often arise not from poor local skills, but from mistimed waiting, yielding, pulling, releasing, or repositioning. We study this problem with two bimanual mobile manipulators coupled through rigid and deformable objects. We propose Sequential Asymmetric Imitation (SAI), a single-teleoperator curriculum for learning coupled multi-robot behaviors without synchronized dual-operator demonstrations or explicit inter-robot communication. SAI trains Robot A from unilateral demonstrations with a compliant human partner, trains Robot B against the deployed Robot A policy, and then refines Robot A using sparse interventions near coordination failures. This staged process exposes the policies to increasingly realistic partner behaviors, including delay, phase mismatch,insufficient yielding, and interaction conflict. Across real-world dual-robot manipulation tasks, SAI improves task success, phase synchronization, and partner-contingent yielding over independent imitation and curriculum-ablation baselines. These results suggest that physically coupled collaboration can be learned through the structure of the imitation curriculum, rather than through synchronized multi-operator demonstrations or explicit coordination mechanisms. More videos on project page:http://cyc0429.github.io/sai-project-page/
♻ ★ EmbodiExp: Compositional Experience Transfer for Embodied Agents
Experience from past tasks can help embodied agents perform new tasks, but it may not remain applicable or effective when conditions change. Tasks with different overall goals often share subgoals, providing a basis for cross-task experience transfer. We present EmbodiExp, a framework for constructing outcome-grounded subgoal experiences and composing them in new tasks. Each experience describes how to pursue a subgoal, when the guidance applies, and what outcome must be verified for completion. Verified outcomes from applying one experience can satisfy the conditions for applying another, allowing experiences from different past tasks to be composed online. By examining the actions and observed outcomes at the subgoal level, EmbodiExp retains experience from successful subgoals and uses explicit action failures to identify actions to avoid under specific conditions. Experiments on EmbodiedBench and VIGIL assess cross-task transfer with experiences kept fixed throughout held-out evaluation. EmbodiExp achieves success rates of 68.0% on EB-Habitat and 68.3% on EB-ALFRED, compared with 50.0% and 54.7% for the agent without stored experience.
♻ ★ Long-Distance Real-World Navigation of the Legged-Wheeled Robot Go2-W Using Deep Reinforcement Learning
Legged-wheeled robots have been extensively studied for their potential to combine the efficient flat-ground mobility of wheels with the rough-terrain adaptability of legs. However, their application to long-distance autonomous navigation in real environments remains limited. This paper reports our efforts to build a deep reinforcement learning-based locomotion controller and an autonomous navigation system for the commercially available Go2-W legged-wheeled robot. The study focuses on the integrated system and the practical challenges associated with its sustained operation over long distances in real-world environments. For locomotion control, we extended our proprioception-only policy, previously developed for quadruped robots, to Go2-W, a 16-degree-of-freedom legged-wheeled robot. We additionally found that wheeled locomotion concentrates the load on the hip joints, resulting in heat concentration that hinders sustained travel. To address this issue, we developed a policy with periodic foot-lifting motion that distributes the joint loads and mitigates heat concentration. We evaluated the integrated system at the Tsukuba Challenge 2025. The robot autonomously traversed an approximately \SI{2.8}{\kilo\meter} route that included sidewalks, a park, and stairs without stopping owing to overheating.
♻ ★ TransGHOST: Real-Time, Synthesis-Free Reconstruction of Instrument-Occluded Regions in Ophthalmic Microsurgery
Intraoperative scene perception in ophthalmic microsurgery is limited by instrument occlusion: when a tool enters the high-magnification field, it conceals tissue from both the surgeon and the perception components of robot-assisted systems that process the video. Generative inpainting can remove the occluder but replaces the missing image content with synthesis, which is unsuitable when visual provenance matters. Many video inpainters also depend on future frames and cannot operate online. We present TransGHOST, a causal reference-guided reconstruction pipeline whose non-generative safe mode operates in real time. Every restored value is obtained by resampling a recorded frame using geometric transforms fitted only to visible regions. The system provides per-pixel confidence and provenance and marks unsupported pixels as abstentions. We evaluate TransGHOST on a new veterinary clinical corpus of canine cataract surgery with pixel-level instrument and anatomy labels. Only the occluded region is scored, and each scoring target is verified to differ from the reference supplied to the system. Safe mode reaches 31.89 dB masked PSNR and 0.915 masked SSIM at 51.7 ms median and 85.5 ms 95th-percentile latency. On a pre-declared held-out cohort using instrument cutouts excluded from segmentation training, it reaches 29.83 dB masked PSNR, while matched-mask evaluation on validation shows a 3.45 dB advantage over LaMa. Region-aware reference selection improves masked PSNR by 3.60 dB over recency when candidate references can themselves be occluded. Reference-age analysis further defines the operating interval for the brief, repeated occlusions targeted by TransGHOST.
comment: 20 pages, 10 figures, 11 tables
♻ ★ IORA: Irregular Object Immobilization under Restricted Actuation
Stable grasping under task-specific disturbances is important for robotic manipulation. However, most dexterous grasp synthesis methods are developed for highly actuated hands in free space, and low-DoF hands may not be able to realize the contact topologies required by these grasps. Meanwhile, support surfaces can provide useful contact forces, but are often treated only as geometric constraints. We present a support-aware grasp synthesis framework for object immobilization with low-DoF coupled hands. Given irregular object geometry, our method represents both hand and object local surface regions as graphs and searches for kinematically feasible matchings between them. We then refine the wrist pose and motor state for geometric contact feasibility and validates the refined grasps through physical rollouts. Starting from the validated grasp pose, an optional task-guided local search explores nearby grasp configurations by varying the wrist pose and motor state, and selects one according to the specified task condition. Experiments on diverse objects show that our method generates various grasp configurations that achieve high grasp success and strong disturbance resistance. Real-world experiments further demonstrate physically executable grasps which enable object immobilization in cooking scenario.
comment: Preprint, Project Page: https://sanghunhan92.github.io/conference/IORA/
♻ ★ D-VLA: A High-Concurrency Distributed Asynchronous Reinforcement Learning Framework for Vision-Language-Action Models
The rapid evolution of Embodied AI has enabled Vision-Language-Action (VLA) models to excel in multimodal perception and task execution. However, applying Reinforcement Learning (RL) to these massive models in large-scale distributed environments faces severe systemic bottlenecks, primarily due to the resource conflict between high-fidelity physical simulation and the intensive VRAM/bandwidth demands of deep learning. This conflict often leaves overall throughput constrained by execution-phase inefficiencies. To address these challenges, we propose D-VLA, a high-concurrency, low-latency distributed RL framework for large-scale embodied foundation models. D-VLA introduces "Plane Decoupling," physically isolating high-frequency training data from low-frequency weight control to eliminate interference between simulation and optimization. We further design a four-thread asynchronous "Swimlane" pipeline, enabling full parallel overlap of sampling, inference, gradient computation, and parameter distribution. Additionally, a dual-pool VRAM management model and topology-aware replication resolve memory fragmentation and optimize communication efficiency. Experiments on benchmarks like LIBERO show that D-VLA significantly outperforms mainstream RL frameworks in throughput and sampling efficiency for billion-parameter VLA models. In trillion-parameter scalability tests, our framework maintains exceptional stability and linear speedup, providing a robust system for high-performance general-purpose embodied agents.
♻ ★ PingpongDuo: Agile Humanoid Table Tennis Doubles with Safe Team Coordination
Unlike singles, humanoid table tennis doubles requires teammates to strike in strict alternation while sharing a confined workspace. Rapid repositioning can compromise striking accuracy, while uncoordinated teammate movements introduce collision risks. We present PingpongDuo, a hierarchical framework for agile humanoid doubles with safe team coordination. At the team level, a planner jointly selects positioning targets and uses learned risk estimates to filter risky proposals, preferentially preserving the current striker's target. At the individual level, separate footwork and striking skills combine agile movement with precise racket control. The skills are guided by human motion references, including footwork collected from human doubles play, and trained from states produced by one another to support rapid transitions. A racket-arm residual further refines strikes after movement using return feedback from a simulator calibrated with real ball-flight and contact measurements. In simulation, PingpongDuo improves the ball-return success rate by 13.2% over the state-of-the-art baseline, while achieving approximately 2 cm positioning error during clearance. In real-world experiments, a robot completes a 2.15 m clearance-and-recovery cycle in 2.4 s with a peak speed of 2.23 m/s, while two robots sustain more than 10 consecutive strikes in strict alternation.
comment: 9 pages, 6 figures. Project page: https://www.archon.tech/research/PingpongDuo . Minor update
♻ ★ E0: Expressive Fine-Grained Discrete Action Prediction for Vision-Language-Action Models via Tweedie Discrete Diffusion NeurIPS 26
Vision-Language-Action (VLA) models offer a unified framework for robotic manipulation by integrating visual perception, language understanding, and control generation. However, existing VLA systems still struggle to generalize across diverse tasks, scenes, and camera viewpoints, and often produce coarse or unstable actions. We argue that these limitations are closely tied to the structural properties of actions in VLA settings, including the inherent multi-peaked nature of action distributions, the token-based symbolic reasoning of pretrained VLM/VLA backbones, and the effective finite resolution imposed by real-world robotic control. Motivated by these properties, we introduce E0, a tweedie discrete diffusion framework that formulates action generation as iterative denoising over quantized action tokens. By operating in a discrete action space with a principled diffusion process, E0 naturally aligns with token-based reasoning, supports fine-grained yet executable action control, and avoids the distributional mismatch of masking-based discrete diffusion. Experiments on LIBERO, VLABench, ManiSkill, and a real-world Franka arm demonstrate that E0 achieves state-of-the-art performance across diverse environments, outperforming strong baselines by 8.1% on average.
comment: Accepted to NeurIPS 26
♻ ★ Courteous Anticipation: Improving Long-Lived Task Planning in Persistent Shared Environments
We consider a task planning scenario in which robots sharing a persistent environment are assigned tasks one at a time from a held-out sequence. Standard task planners, lacking foresight of future tasks and inconsiderate of others' constraints, solve each task in isolation, leaving terminal states that increase future cost for all; side effects that compound over lengthy task sequences. To reduce cost over the sequence, a robot must anticipate how its actions now may impact performance on future tasks for all robots sharing the environment. Therefore, we present courteous anticipatory planning, wherein a model-based planner proposes candidate plans and selects the one that jointly minimizes immediate cost and aggregated expected future cost across all robots, estimated via independent per-robot learned estimators. This factored formulation avoids combinatorial joint rollouts and supports modular deployment: adding a robot requires only training its own estimator. We evaluate in two persistent PDDL domains, a home environment with robots that have similar capabilities but different responsibilities, and a restaurant environment where robots' distinct capabilities create states that other robots lack the capability to resolve. During lengthy task sequences, our planner reduces total cost by 10.4% versus myopic and 4.5% versus selfish anticipatory planning in a two-robot home environment and by 20.5% and 6.7%, respectively, in a three-robot restaurant.
comment: 9 Pages
♻ ★ Foundation Model Driven Robotics: A Comprehensive Review
The rapid emergence of foundation models, particularly large language models and vision-language models, has introduced a transformative paradigm in robotics. The models offer powerful capabilities in semantic understanding, high-level reasoning, and cross-modal generalization, enabling significant advances in perception, planning, control, and human-robot interaction. This critical review provides a structured synthesis of recent developments and organizes applications into four directions: simulation-driven robotics, open-world operation, sim-to-real transfer, and adaptable robotic systems. Unlike existing surveys that emphasize isolated capabilities, this work highlights integrated, system-level strategies and evaluates their practical feasibility in real-world environments. Key enabling trends such as procedural scene generation, policy generalization, and multimodal reasoning are discussed alongside core bottlenecks, including limited embodiment, lack of multimodal data, safety risks, and computational constraints. Through this lens, this paper identifies both the architectural strengths and critical limitations of foundation model-based robotics, highlighting open challenges in real-time operation, grounding, resilience, and trust. The review concludes with a roadmap of proposed future research directions for bridging semantic reasoning and physical intelligence through more robust, interpretable, and embodied models.
♻ ★ The Missing Touch: Sensing and Displaying Localized Contact Geometry Brings Teleoperation Closer to Human Dexterity
A fundamental challenge in robotic teleoperation is enabling an operator to control a remote robot as effortlessly and intuitively as their own hands. Despite the growing use of teleoperation to collect demonstrations for training autonomous robot policies, teleoperated performance still falls significantly short of human dexterity, even for basic tasks. Here, we present evidence that a key factor contributing to this performance gap is the absence of localized tactile feedback. We equip a two-degree-of-freedom (DoF) bilateral force-feedback telemanipulator with a vision-based tactile sensor and a 32-pixel tactile display, allowing us to capture spatial contact patterns on the robot finger and render them on the operator's fingertip. In teleoperated button-discrimination and peg-rolling tasks, we demonstrate that reproducing spatially distributed contact information brings teleoperated movements closer to natural human behavior through fewer unnecessary motions and task completion steps, significantly reducing task completion time and producing more consistent demonstrations across trials and operators. We further show that these improvements in human demonstrations can propagate to autonomous policies: policies trained on tactile-enhanced demonstrations exhibited similar reductions in both task completion time and deviation from natural human behavior. Together, these results establish spatially distributed tactile feedback as an important component of dexterous robotic teleoperation and demonstrate its potential to improve safety-critical teleoperation and enable more humanlike demonstrations for training autonomous robot policies.
Multiagent Systems
★ Imitation Bootstrapped Multi-Agent Reinforcement Learning IROS 2026
Demonstrations are an effective way to accelerate single-agent reinforcement learning (RL), but extending this idea to multi-agent reinforcement learning (MARL) is non-trivial as it is difficult and often impossible for a single person to teleoperate multiple agents simultaneously. Round-Robin Behavior Cloning (R2BC) addresses this by letting a single demonstrator control one agent at a time while the others execute their current policies. However, the resulting imitation policies are limited by the quality of the demonstrations. We propose Imitation Bootstrapped Multi-Agent Reinforcement Learning (IBMARL), a framework that extends Imitation Bootstrapped Reinforcement Learning (IBRL) to cooperative teams and uses R2BC to train the imitation policies. At each time step, every agent receives one action proposal from its imitation policy and one from its RL policy, and an ensemble of centralized critics selects among the per-agent combinations of the two. A gated imitation penalty, a soft Q-filtered behavior cloning loss, regularizes each RL policy toward its imitation policy only where the critics value the imitation action more, so that after training each agent executes its RL policy alone, without the centralized critics. On four cooperative multi-agent RL benchmarks, IBMARL surpasses its imitation policies $2.6$-$4.2\times$ faster than demonstration-based baselines on two tasks, is the only method to surpass it on the other two, and exceeds the demonstrator itself on two of the four. With real human demonstrations IBMARL still surpasses imitation learning alone, and does so more than twice as fast as the best baseline.
comment: 9 pages, 5 figures. Code, videos and results: https://zhanghutony.github.io/ibmarl/. A preliminary version appeared at the PhysCollaBot Workshop at IROS 2026
★ TradeGauntlet: Evaluating Compliance Controls for Autonomous Trading Agents
Autonomous trading agents need controls that prevent problematic orders without making legitimate tasks impractical. We study this aspect of agentic compliance with TradeGauntlet, a replayable limit-order-book simulator that separates detection, prevention, and execution cost. In 1,680 simulated sessions with Claude Haiku 5.5 and Sonnet 5.5, all conduct alerts were self-matches. Gates eliminated those alerts by rejecting self-crossing orders, while the aggregate-gate condition reduced Sonnet's two-sided fills by about 15%. A conduct reminder reduced Sonnet's alert rate but reduced Haiku's fills without a detectable alert benefit. These results show why zero alerts alone do not establish an agent's compliance. Scripted probes and two search methods test weaknesses beyond those observed in the LLM runs. The searches find split-order schedules that evade the implemented conduct rules; a learned baseline catches all 2,400 tested searched episodes but also flags 15% of a large maker's sessions. Aggregate caps reduce the response-dependent gain unevenly, with smaller reductions when only near-touch depth affects prices. A response fitted to five Nasdaq stocks and a separate ABIDES experiment expose the dependence of these estimates on market assumptions. A distinct adversarial prompt elicited refusals from Sonnet in all 240 rounds, yielding no candidate strategies to evaluate. TradeGauntlet provides reproducible tests of controls around trading agents; the experiments do not establish legal compliance or real-market detection accuracy.
comment: 21 pages, 5 figures. Code and saved results: https://github.com/tokencraftdata/agentic-compliance
★ From AI-Enabled Remote Patient Monitoring Toward Agentic Robotics: Embodiment, Governance, and Care Integration
AI-enabled remote patient monitoring is evolving beyond passive data collection toward agentic AI systems that interpret patient states, coordinate care, and initiate action across domestic and clinical settings. This structured scoping review finds that mature agentic care has not yet emerged: Most systems remain human led, while stronger decision-action coupling and embodied AI remain largely confined to prototypes and pilots. Healthcare robotics makes the transition from monitoring to action especially visible by translating AI-generated recommendations and decisions into speech, movement, prompting, physical assistance, escalation, and care coordination, making embodiment, clinical integration, human oversight, and governance fundamental design requirements. Further progress requires safe and interoperable care architectures in which intelligent components operate through clinically validated and accountable workflows. We propose agentic robotics as a future direction for supervised systems embedded in the care ecosystem, connecting AI agents and robots with clinicians, caregivers, and patients through auditable workflows that can be interrupted and remain subject to human oversight and explicit AI governance.
comment: 34 pages, 6 figures, 3 tables. Accepted for publication in Annual Review of Control, Robotics, and Autonomous Systems, Volume 10 (2027)
★ Smart Home Agents Make Capable but Unpredictable Grid Operators
AI-enabled smart homes are becoming more common across the grid edge. However, there has not been much work on failure response. This area is critical because smart homes operate on the grid edge where failures can cascade down. If a power outage happens within a smart home, a user has to manually troubleshoot their smart home to include tasks such as power load balance and device recovery. We conduct the first resilience analysis of smart home agents across a full power outage, providing a suite of outage simulations in a live environment that measures resilience before, during, and post power outage. We show that smart home agents can act as grid operators in power outages and identify several new failure categories highlighting the current limitations across all of our models. These findings establish the groundwork for smart home agents as grid operators to increase overall resilience on the grid edge.
comment: Pre-print. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible
★ When Should a Virtual Patient Ask? RISP-Guided Proactive Information Seeking in Multi-Agent Patient Simulation
Virtual patient (VP) systems support nursing communication training by providing simulated encounters in which learners practise explaining care and responding to patient concerns. Although large language models (LLMs) support flexible dialogue, current approaches give less explicit attention to how patients' information needs develop and guide the content and timing of their questions. We draw on the Risk Information Seeking and Processing (RISP) model, which relates information seeking to perceived risk and the gap between perceived knowledge and desired understanding. Guided by this perspective, we propose the RISP Multi-Agent System (RISP-MAS), a VP system for patient-initiated information seeking. The system links ongoing assessment of the patient's perceptions and information needs to coordinated decisions about whether to seek information, when to ask and which uncertainty to address. Evaluation of nursing student-VP interactions suggested that RISP-MAS exhibited plausible patient information-seeking behaviour, as indicated by expert assessments, while student feedback reflected positive interaction experiences. This work extends patient simulation by connecting a psychological account of information needs to explicit decisions about question initiation. The approach offers nursing students opportunities to practise recognising unresolved patient concerns and tailoring explanations to patients' information needs.
★ More Agents, More Samples, or More Thinking? Untangling Coordination, Ensembling, and Test-Time Compute in Multi-Agent LLMs
Evaluations of multi-agent language models often compare communicating architectures against a single greedy pass. This practice confounds coordination with sample multiplicity and inference compute. We synthesize nine primary empirical evaluations comprising 516 unique multi-agent configurations organized along a resource vector R with six dimensions and a baseline ladder with four categories. Against baselines with a single pass (Category A), multi-agent accuracy differences are negative overall and near zero when omitting the largest contributing study. When evaluated against baselines matched on sample count (Category B) or resource budget (Category C), multi-agent systems average -0.042 in accuracy, and multilevel random effects modeling on the log odds scale remains negative across sensitivity checks. Multi-agent accuracy equals or falls below the baseline in 74% of these comparable contrasts, and study means are nonpositive in almost all evaluated studies. Across studies reporting both baselines, moving from a greedy pass to a sampled baseline shifts accuracy differences by -0.038 on average. Because communication adds no information when agents share identical evidence, positive returns require partitioned evidence, external verification, or error diversity across models. We provide a reviewer checklist and accounting utilities.
comment: 43 pages, 11 figures, 13 tables. PACE Reviewer Checklist, evaluation utilities (pace CLI), and reproduction data at https://github.com/mahmoudsajjadi/pace
♻ ★ Self-Building Labs
Self-driving laboratories (SDLs) accelerate scientific discovery by automating experimental execution and optimization. AI agents can increasingly propose new experiments, but existing SDLs can only run the procedures they were built for. Adapting them to a new experiment still requires extensive manual engineering, which makes laboratory flexibility the bottleneck for scaling discovery. We introduce the self-building laboratory (SBL), which designs, configures, and operates its own laboratory setup given a high-level scientific goal. SBL starts from whatever already exists, from an empty room to a working SDL, and builds the missing capabilities: the lab layout, its digital twin, robot skills, and experimental workflows. We demonstrate the system working across different robots and setups, scaling from a single workstation to laboratories with several robots working together. We validate SBL's end-to-end capabilities through a real-world quantum-dot synthesis experiment. By shifting human researchers from manually configuring and programming device automation to specifying scientific objectives, SBL makes scientific automation more accessible, flexible, and scalable. Project website: https://self-building-labs.github.io/
♻ ★ CPM-MultiAgent: A CPM-Grounded Appraisal Framework for Simulating Dynamic Patient Emotional Evolution in Healthcare Dialogue
Large Language Models (LLMs) enable patient simulation for healthcare communication training, where learners practice responding to clinical information, concerns, and emotional reactions. However, existing patient simulation and affective dialogue systems often represent patient emotions as static attributes or focus on recognizing emotions and generating empathetic responses. They provide limited support for modeling how a simulated patient's emotional state evolves in response to a healthcare professional's communication. Changes in patient emotion constitute interactional feedback that learners must recognize and respond to across dialogue turns. We draw on the Component Process Model (CPM), a psychological theory that explains emotion as a dynamic process shaped by event appraisal. We propose CPM-MultiAgent, a CPM-grounded multi-agent framework for dynamic patient emotion simulation in healthcare communication training. The framework uses an emotion taxonomy informed by medical literature and expert input, and updates emotional states through healthcare-related trigger identification, structured CPM appraisal, and emotion-state integration, accounting for the patient's profile, clinical context, and prior emotional state. We evaluate the framework through student-led conversations and controlled trigger cases, combining nursing-expert assessments, LLM-judge comparisons, component ablations, backbone checks, and student feedback.
♻ ★ MMAO: Mechanism and Stability Analysis of Metabolic Closed-Loop Metaheuristics
This paper studies the Metabolic Multi-Agent Optimizer (MMAO) at the framework level rather than at the implementation or benchmark level. The central question is whether the metabolic resource loop of private energy, communal budget, role drift, and lifecycle turnover has a framework-level interpretation beyond narrative metaphor. We introduce a generic MMAO state model that abstracts away domain-specific move operators while retaining the resource bookkeeping that defines the framework. Under mild bounded-gain and bounded-spending assumptions, we establish boundedness and nonnegativity properties for private energy, communal budget, role state, and active population size. We then characterize three endogenous behavioral regimes of the loop: contraction under sustained resource deficit, reinvestment under surplus communal accumulation, and search redistribution under heterogeneous marginal returns across agents or subgroups. The analysis is intentionally conservative. It does not claim global convergence of the full adaptive system, universal superiority over specialist optimizers, or a complete stationary characterization of the resulting process. Instead, it identifies which internal regulation properties are generic consequences of the loop and which remain implementation specific. A compact mechanism-validation package on representative continuous and discrete MMAO realizations provides supporting empirical evidence for this reading, but is not intended to replace a full benchmark study. The analysis provides a bounded, regenerative, resource-regulated interpretation of MMAO, while leaving implementation-level performance questions open.
comment: 12 pages
♻ ★ MMAO-Cls: Metabolic Multi-Agent Optimization for Joint Feature Selection and Classifier Tuning
This paper studies whether the Metabolic Multi-Agent Optimizer (MMAO) can act as a credible outer-loop optimizer for classification model selection. We propose MMAO-Cls, a mixed-space realization in which each agent jointly encodes a binary feature mask and classifier hyperparameters, while private energy, communal budget, role drift, and lifecycle turnover are mapped to the accuracy-complexity tradeoff of wrapper learning. The implementation is strengthened by deriving feature-budget adaptation from feature-information priors and by regularizing validation reward with both subset compactness and train-validation overfitting gap. We evaluate MMAO-Cls on seven standard tabular benchmarks with three seeds each and compare it against RandomSearch, GA-lite, PSO-lite, and an endogenous no-sharing ablation. On the aggregate validation objective, MMAO-Cls ranks second ($0.9433$) behind GA-lite ($0.9446$). On held-out test performance, it reaches mean score $0.8882$, improving over RandomSearch ($0.8808$) and GA-lite ($0.8857$), remaining close to PSO-lite ($0.8874$) and the no-sharing ablation ($0.8900$), while using the most compact mean held-out feature subset among all compared methods (feature ratio $0.4881$). Pairwise tests show that these margins are not yet statistically significant. The results support MMAO's applicability to classification and compact mixed-space search, while the contribution of communal sharing to held-out performance remains unresolved.
comment: 8 pages
♻ ★ A Large-Scale Empirical Evaluation of MMAO Under Fair-Budget Continuous and Discrete Benchmarks
This paper evaluates the Metabolic Multi-Agent Optimizer (MMAO) under a stricter empirical protocol rather than reintroducing the framework itself. The study asks whether MMAO's closed-loop resource-allocation principle remains credible under broader, more standard, and more explicitly budget-controlled continuous and discrete benchmarks. The main completed matrix covers eight CEC2017 functions at 10D and 30D with 20 seeds each, and five TSPLIB instances with 20 seeds each, together with stronger reproducible baselines including PSO-lite, ES-lite, and an iterated-greedy 2-opt route baseline. We further add trajectory-level diagnostics for communal budget, success rate, role evolution, and population turnover, plus an auxiliary OR-Library multiple-knapsack slice to extend the discrete evidence beyond routing. Under this protocol, MMAO clearly outperforms the external baseline set on the continuous side and on the TSPLIB side, while the ablation variants remain much closer to the full method than the external baselines are. The results support endogenous resource redistribution across the tested domains, while sharper mechanism isolation and broader competition-grade comparison remain open.
comment: 9 pages
♻ ★ Minimal MMAO: A Resource-Closed-Loop Framework for Adaptive Metaheuristic Search
This paper presents the Metabolic Multi-Agent Optimizer (MMAO) as an adaptive metaheuristic built around endogenous resource circulation. The central premise is that search intensity, exploration--exploitation balance, and lifecycle turnover should be induced by a shared metabolic controller rather than by separately attached schedules. We formulate MMAO through bounded private energy, a communal budget, normalized reward, continuous role adaptation, and resource-financed branching and pruning. The method is then instantiated in both continuous and discrete domains and evaluated on a matched small-scale suite including Sphere, Rastrigin, a synthetic Euclidean TSP, and two TSPLIB instances. The results show a consistent pattern: the same metabolic loop remains workable across domains, the discrete realization remains relatively stable under a compact design, and continuous refinement quality is the main cost of keeping the method lean. These results support MMAO's framework formulation under simplification, with some loss of late-stage precision on harder continuous tasks.
comment: 6 pages
♻ ★ MMAO: A Metabolic Multi-Agent Optimizer with Endogenous Resource Allocation for Continuous and Discrete Optimization
Traditional meta-heuristics often rely on fixed population sizes, manually chosen search scales, and externally attached parameter-control modules. This paper presents the Metabolic Multi-Agent Optimizer (MMAO), a cross-domain optimization framework in which adaptation is derived endogenously from a private-public metabolic resource loop. Each agent carries internal energy, a continuous role state, motion or structural memory, and local search history, while the population shares a communal resource pool. Fitness improvements are converted into normalized metabolic gains through a robust progress scale and a recent success statistic; the same closed loop then regulates sensing intensity, search amplitude, role drift, branching, pruning, respawning, and elite reinvestment. In the continuous setting, MMAO uses energy-regulated symmetric zero-order probing and role-interpolated motion. In the discrete setting, the same control law is instantiated through structural sensing, local route improvement, guided perturbation, and energy-weighted edge reuse. The paper combines an implementation-faithful formulation with a reproducible experimental study on a CEC2017 subset (10D/30D, 20 seeds) and five TSPLIB instances (100 discrete runs in total). The current evidence supports MMAO primarily as a parameter-light, self-calibrating optimization framework whose main validated originality lies in metabolically endogenous resource allocation across heterogeneous search behaviors, rather than as a universally superior optimizer.
comment: 12 pages
♻ ★ From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents EMNLP 2026
While role-playing agents excel in short-term interactions, long-term conversations overwhelm context windows, motivating external memory frameworks. Current systems typically rely on persona-agnostic summarization, which records facts without persona-specific interpretation, yielding generic responses that compromise persona fidelity. To bridge this gap, we introduce RoleMemo, a dataset featuring four reasoning tasks where the factual fragments must be interpreted through the persona to reach the correct answer. Evaluation on RoleMemo exposes critical limitations of persona-agnostic frameworks. We thus propose DualMem, which decouples memory into two streams: factual cognition and persona-conditioned insight. Trained through Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL), our framework with a 4B-parameter model outperforms zero-shot persona-agnostic frameworks powered by DeepSeek-V3.2 for sustained persona fidelity. Our resources are available at https://github.com/Collab-Gen/RoleMemo.
comment: Accepted by EMNLP 2026 Findings
♻ ★ From Emergence to Reversibility
Multi-agent systems become capable through institutional mechanisms, such as rules that route information to agents that act on it. These mechanisms are also natural places to restrict the system. Yet as agents learn, share memory, delegate, and repair failures, the collective can find other paths to the same effects. A restriction on the original mechanism can then keep operating but no longer stops them, while routine performance and compliance still passes. We propose \textbf{Reversible Multi-agent Systems} as a research agenda: collectives that grow while an authorized supervisor can still restrict specified effects, keep them restricted as agents adapt, and reconstitute useful cooperation under renewed authority. What is reversed is the ability to produce a specified effect; restrictions must hold even if learned knowledge persists. The goal is to preserve effective intervention, not a particular mechanism: at every step of growth, some authorized intervention must keep a specified prohibited effect rare while maintaining required service. We identify three research problems: tracking what capabilities depend on, handing off control before an old mechanism loses its effect, and separating legitimate repair from unauthorized reconstruction. We outline how to evaluate control across growth paths, back control claims with evidence, and allocate the rights to restrict and restore.
Systems and Control (EESS)
★ Impact of Grid-Forming Inverter Placement and Penetration on Local and Aggregate Frequency
Modern power systems are undergoing a huge shift in generation mix, including inverter-based resources that are fundamentally changing the stability dynamics of the grid. Thus, synchronous Generators (SG) are gradually being displaced by Grid-forming inverters (GFM) that are proposed as a solution to continue supporting grid stability. However, the effect of GFM placement in the network on frequency performance and how well aggregate-frequency assessment captures placement sensitivity observed in local frequency metrics remains unclear. This paper evaluates all SG and droop-controlled GFM assignments in the IEEE 39-bus system. A reduced-order model is used for the exhaustive search with load disturbances applied separately at every eligible load bus while considering nadir and Rate of Change of Frequency (RoCoF) derived from both device rating weighted aggregate frequency and local bus frequencies. Results reveal that at fixed GFM penetration, placement produced up to 104 mHz variation in worst-case local nadir and 0.21 Hz/s in maximum local RoCoF. Aggregate-frequency assessment reduced the apparent across placements by 89\% for nadir and 88\% for RoCoF at 50\% GFM penetration. These results indicate that aggregate-frequency assessment alone may obscure favorable or adverse placement effects on local frequency performance.
★ EPIGRIDS: Documentation and Validation of Seven Open Synthetic Transmission Grid Cases
This paper documents and checks EPIGRIDS, seven open synthetic transmission grid cases from four independently built networks, made with an optimization-based construction method instead of the graph-statistics method behind most large public synthetic grids. The cases range from 250 to 78,484 buses, with geolocated substations, voltages up to 765 kV, and line parameters taken from conductor data. Each ships under an open license in three common power flow formats that agree at the solved operating point, and the six larger cases have a companion hourly, bus-level load series for one year. The Texas, Florida, and Michigan, Ohio, and Indiana cases are sub-networks of the Eastern case. Every case solves an alternating-current power flow from its stored operating point. With parallel circuits merged, the Eastern and Western cases are 9 and 16% denser than the real interconnections, while graph-statistics cases on the same footprints lie within 6% and meet more published realism criteria. A full single-outage contingency analysis completes on every case, and in an optimal power flow benchmark every case except the New England base case and one of its 12 operating points solves in polar form.
comment: 10 pages, 2 figures, 6 tables. Submitted to the IEEE Open Access Journal of Power and Energy. Cases: https://doi.org/10.5281/zenodo.23814538; load series: https://doi.org/10.5281/zenodo.24151084; construction method: arXiv:2611.16222
★ From AI-Enabled Remote Patient Monitoring Toward Agentic Robotics: Embodiment, Governance, and Care Integration
AI-enabled remote patient monitoring is evolving beyond passive data collection toward agentic AI systems that interpret patient states, coordinate care, and initiate action across domestic and clinical settings. This structured scoping review finds that mature agentic care has not yet emerged: Most systems remain human led, while stronger decision-action coupling and embodied AI remain largely confined to prototypes and pilots. Healthcare robotics makes the transition from monitoring to action especially visible by translating AI-generated recommendations and decisions into speech, movement, prompting, physical assistance, escalation, and care coordination, making embodiment, clinical integration, human oversight, and governance fundamental design requirements. Further progress requires safe and interoperable care architectures in which intelligent components operate through clinically validated and accountable workflows. We propose agentic robotics as a future direction for supervised systems embedded in the care ecosystem, connecting AI agents and robots with clinicians, caregivers, and patients through auditable workflows that can be interrupted and remain subject to human oversight and explicit AI governance.
comment: 34 pages, 6 figures, 3 tables. Accepted for publication in Annual Review of Control, Robotics, and Autonomous Systems, Volume 10 (2027)
★ Policy-Conditioned Koopman Representations for Parameterized Closed-Loop Systems
Predicting the response of a nonlinear system across different feedback controllers requires a model that captures how the closed-loop dynamics change with the controller parameters. In a finite-dimensional Koopman model, these changes may affect both the latent dynamics and the observable coordinates used to represent them. In this paper, we propose a policy-conditioned Koopman representation framework to address this problem. This framework allows a single learned model to predict a family of closed-loop trajectories without retraining for each controller. We test the framework on four nonlinear prediction benchmarks, where the model stays below a 15% error threshold on every system, while each baseline exceeds it on at least one. As an example application, the resulting surrogate supports one-shot feedback-parameter selection: an optimization at the initial state selects a controller that then operates with fixed parameters and ongoing state feedback. On nonlinear control benchmarks including MuJoCo, the selected controllers stabilize every system with 3 to 97 times less online computation than nonlinear model predictive control. These results demonstrate how policy-dependent coordinates can support both prediction across closed-loop systems and controller selection without repeated online trajectory optimization.
★ Grid-Forming AI Data Centers: Coordinated Grid Support Control from IT Loads and Energy Storage
To exploit the grid-support capability of AI data centers (AIDCs), this paper proposes a grid-forming (GFM) control scheme which coordinates the IT loads and energy storage (ES) in the AIDC. First, the framework of a grid-connected AIDC is presented, with the multi-stage power conversion from the grid to the IT loads presented. A dual GFM scheme is then proposed for the back-to-back converters to provide grid support through coordinated power adjustments from IT loads and ES. In addition, an aggregated power response model is established to characterize the IT-load flexibility considering multiple IT servers executing diverse tasks. Time-domain simulations of a 100 MW grid-connected AIDC demonstrate that the proposed control scheme can effectively support the grid.
★ CS-MPC: Contraction-Certified Sampling-Based MPC for Nonlinear Systems Under Uncertainty
We present contraction-certified sampling-based model predictive control (CS-MPC) for tracking and safety of nonlinear robotic systems under parametric uncertainty, disturbances, and safety constraints that are revealed online. CS-MPC uses a contraction-certified tracking feedback inside every sampled rollout, which concentrates the rollouts around the reference, and an execution layer that verifies the applied input using a quantile-tightened discrete-time control barrier function condition. We derive a verification procedure for nonlinear contraction of the feedback-closed dynamics, and we prove a finite-sample stochastic stability bound and a mission-level probabilistic safety guarantee. We validate CS-MPC in simulation on a rover, a quadrotor, and an asteroid landing. When compared with standard model predictive path integral (MPPI) control, CS-MPC attains the same tracking error with 7 to 20 times fewer rollouts for the rover, and a 2 to 4 times lower tracking error for the quadrotor, with no safety violations in 200 Monte Carlo missions.
★ Reinforcement Learning for Null-Space Exploitation in Pseudo-Inverse Control Allocation
Pseudo-inverse methods are widely used to solve control allocation problems for overactuated aircraft with redundant control effectors. These methods reliably satisfy the primary command tracking objective, while the remaining null-space degrees of freedom are in principle available for secondary objectives such as drag reduction or actuator wear balancing. However, exploiting this freedom in practice requires hand-tuned weighting matrices or a priori preferred deflections that are difficult to specify across a wide flight envelope with nonlinear, state-dependent dynamics. This paper presents a hybrid architecture in which a reinforcement learning agent operates exclusively within the null space of the control effectiveness matrix, learning secondary objective strategies directly from interaction with the flight dynamics. The null-space projector guarantees by construction that primary command tracking remains invariant regardless of the agent output, providing a principled separation between certified primary control and adaptive secondary optimization. The approach is demonstrated on a nonlinear six-degree-of-freedom distributed electric propulsion aircraft model with aero-propulsive effects across diverse mission scenarios, achieving measurable drag reduction compared to the baseline allocation while preserving tracking integrity and respecting actuator constraints. The proposed architecture forms a foundation for future extensions toward multi-objective, mission-conditioned policies.
★ Online Switching Control as Learning with Experts
This paper addresses the problem of online switching control for discrete-time nonlinear systems where a controller is sequentially selected from a finite set of candidates. The goal is to generate an adaptive sequence of controllers to minimize the cumulative cost under unknown disturbances and cost functions. Specifically, we consider a full-information feedback setting where realized disturbance effects and cost functions are available after each selection. Using counterfactual costs computed from historical data, we reduce the online switching control problem to the experts problem with switching costs. Then, we propose an adaptive switching algorithm based on the Hedge algorithm with a switching suppression mechanism. Notably, the proposed algorithm achieves an expected regret of $O(\sqrt{T\log N})$, where $T$ is the time horizon and $N$ is the number of candidate controllers, while ensuring uniform boundedness of the state. This matches the optimal regret order for the standard experts problem.
★ Control Invariance on Manifolds through Tangent Bundle Splitting
Control invariance for interconnected non-linear systems on manifolds requires conditions under which subsystem safety constraints can be combined to form an invariant set. This paper develops local product type control invariant sets on manifolds using tangent bundle splitting, adapted product coordinates, and subsystem tangency conditions satisfied simultaneously by one admissible input at each state. The resulting conditions establish control invariance of product sets without requiring independent subsystem dynamics, with separate control choices possible when inputs are independently constrained and each acts only on its subsystem. This provides a way to verify subsystem safety without discarding dynamic coupling and is illustrated by a torus example in which adapted coordinates give a curved constraint region a product representation.
★ Power Management of Electrically Driven Centrifugal Compressors for Flexible Operation
We present a control strategy for power management of a variable-speed 15MW (rated power) centrifugal compressor, and investigate how quickly it can follow a reduced electrical power reference without entering surge. We use a Greitzer-Gravdahl type model scaled to industrial size with rate-limited valve actuators as a simulation environment and propose an advanced regulatory control structure using feedback control loops with dynamically scaled feedforward mechanisms and a structure of min/max selectors. In simulated experiments, a 6MW power reduction from varying initial operating points on a 12MW iso-power line is delivered at rates of up to 10MW/s, i.e.to 90% in 0.54s, without opening the recycle valve. Depending on the initial operating point, a large power reduction can be delivered at once or with a restricted ramp rate. The stroking rate of the discharge side valve is identified as the limiting factor for how fast the power can be reduced in the studied configuration. A limited sensitivity study on the impact of rotor inertia for power shedding performance is also provided.
★ Hierarchical Interactions in Mobility Networks: A Model-Free Feedback Approach
Urban mobility systems are large-scale networked systems whose performance emerges from hierarchically coupled decisions of public authorities, competing mobility operators, and travelers. Existing approaches to policy design either collapse one of these levels or assume that the authority can model and differentiate through the equilibrium responses of operators and travelers. In this paper, we model policy design as a tri-level game on a multimodal network, in which a municipality sets taxes, licenses, and integration subsidies, a public transport and a taxi operator compete on prices and service supply, and heterogeneous travelers reach a stochastic user equilibrium. We prove that the traveler equilibrium is unique and continuously differentiable in the operator strategies, that the variational inequality characterizing operator equilibria admits a solution for every policy, and that the resulting tri-level game admits a Stackelberg equilibrium. Building on this structure, we design a model-free hierarchical controller in which the municipality and the operators update their decisions from zeroth-order evaluations of observed equilibrium outcomes. On the Zurich multimodal network, operator equilibria computed from different initializations differ by a median of 0.3% in the municipal objective. On a grid of 3,125 policies, only those that subsidize transfers between public transport and taxis benefit both operators, and 24 of them also lower travel costs without degrading the municipal objective. Finally, the controller matches the best policies of three black-box optimizers and terminates at the most consistent policies across runs.
comment: 12 pages
★ Linear Modeling of Hybrid Shipboard Power Systems under Pulsed Power Load Operation
A model-based approach is a valuable tool in the power system design process, enabling the integration of component models and the evaluation of dynamic interactions. Simplified models support rapid preliminary assessments by allowing parameters and configurations to be adjusted. This paper investigates the effects of high-bandwidth loads in hybrid AC--DC shipboard power systems. The problem is addressed by developing a linear model aimed at capturing the relevant dynamics over the time scale of interest with reduced computational effort. A LVDC subsystem has been developed with a battery energy storage system that supports the grid during pulsed power load operation. A control strategy has been proposed to satisfy transient requirements. The proposed approach is validated under different pulsed power load conditions using a high-fidelity model developed in DIgSILENT PowerFactory, which preserves nonlinearities and includes converter switching effects and digital control implementation. The validation shows that the model captures the relevant transient behavior with significantly reduced simulation time, while maintaining acceptable accuracy, making it suitable for rapid preliminary design studies.
comment: Accepted and presented at ESARS-ITEC 2026. Copyright 2026 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses
★ A Reinforcement Learning Approach to Space-Filling Input Design for Nonlinear System Identification
In this paper, we propose a reinforcement learning (RL) approach to space-filling input design for nonlinear system identification, in which an agent learns a control policy that actively drives the system to explore its state space. Classical input design, developed mainly for linear models, minimises the covariance of the estimated parameters. For nonlinear black-box models, however, the dominant error is the bias induced by the model structure over the region of interest, which motivates space-filling strategies that seek to cover the system state space. Existing space-filling approaches rely on predefined input signal classes and require solving explicit optimisation problems. Unlike these methods, the proposed approach requires no input class to be selected and no frequency band to be tuned for the system at hand, satisfies an amplitude bound by construction, and adapts to varying initial conditions through its observation-based policy without re-optimisation. The learned excitation signals exhibit physically meaningful behaviours, such as resonance excitation and amplitude modulation. The approach is evaluated against tuned swept sine and crest-factor-optimised multisine excitations, and against a state-of-the-art space-filling multisine design, on two nonlinear benchmark systems: a mass-spring-damper with nonlinear stiffness and a Van der Pol oscillator. Under the same amplitude bound, the learned policy achieves the best overall space-filling performance on both systems and distributes its samples more uniformly than the state-of-the-art design.
★ Characterisation of the DQsym Framework Against Harmonic State-Space and Dynamic Phasor Models for Modular Multilevel Converters
Modern power systems rely on power converters that exhibit multifrequency periodic dynamics, such as modular multilevel converters (MMCs). Such converters are commonly analysed using harmonic state-space (HSS) and dynamic phasor (DP) representations. Both approaches rely on building state matrices based on the system topology, whose size grows with the number of harmonics retained in the model. This paper shows that a symmetrical direct-quadrature (dq) dynamic phasor formulation for time-domain simulation, named DQsym, has a smaller solver state matrix than HSS and DP, and whose size is independent of the truncation order K. This reduction results from evaluating harmonic coupling outside the state matrix, in the nonlinear signal blocks. The open-loop MMC model used in this paper is derived in three harmonic-domain (HD) formulations and validated against a three-phase time-domain reference model. The models are compared in terms of state-space size and sparsity, harmonic content, steady-state accuracy and eigenvalues. For truncation at K = 4, the DQsym solver has 45 states against 135 for HSS and DP, stays within about 4% of the reference, and, once closed with its coupling blocks, yields eigenvalues identical to those of HSS and DP.
comment: Submitted to the 25th IET International Conference on AC and DC Power Transmission
★ From Pen-and-Paper to Machine-Checked Proofs: Lean for System Identification
Mathematical proofs in scientific papers are often long and intricate, relying on chains of references to earlier results. This makes it difficult for readers and reviewers to verify each and every step. Interactive theorem proving offers a complement: the theorem and its proof are written as code, and every inference is checked by a small, trusted piece of software called the kernel. This paper serves as an introduction to the Lean 4 proof assistant for a system identification and control audience, assuming no prior exposure to formal methods. We first show how a Lean proof is read using two elementary examples: the first on identifiability, the second on set invariance. We then illustrate how Lean can be used to analyse results from the literature, taking as example a classic lemma on the inversion of the noise filter in prediction-error identification (Ljung, 1999, Lemma 3.1). Since in Lean every step must be justified, the formalization brings to light a subtle point: the original claim of that lemma holds almost surely, although not for every realisation, as its statement may suggest. A simple counterexample makes this concrete. Finally, the paper discusses guidelines and perspectives on the use of Lean in system identification research.
★ Data-Driven Sparse Identification and Robust Tube-Based Predictive Control for Multivariable Chaotic Systems
This paper addresses the data-driven synchronization and robust control of multivariable chaotic systems subject to strict actuation constraints, modeling uncertainties and bounded input disturbances. Using empirical data collected from a four-dimensional chaotic Rossler response system, Sequentially Thresholded Least Squares (STLSQ) sparse regression is employed to recover a nominal control-affine model, achieving an identification accuracy of R^2 = 1.0000 across all channels with root-mean-square error within [4.15e-6, 8.33e-4]. To establish a mathematically rigorous bridge between data-driven identification and robust synthesis, all structural discrepancies in the drift vector field and input matrix are unified into a single lumped uncertainty field delta(x,u). A computable scalar bound delta_bar is established based on library truncation error, regression residuals, input bounds, and Lipschitz constants (Lemma 1). We subsequently prove that input-output feedback linearization maps delta(x,u) into a state- and control-dependent additive disturbance d(z,nu) in Brunovsky canonical coordinates, bounded by ||d(z,nu)||_2 <= alpha_1 delta_bar + alpha_2 delta_bar ||nu||_2 with closed-form expressions for alpha_1 and alpha_2 (Theorem 1). This certified bound, together with a bounded input disturbance and a zero-order-hold margin, yields a compact disturbance set that seeds a robust tube-based model predictive controller. We prove that the resulting tightened quadratic program is recursively feasible, that the physical actuator limit (|u_i| <= 1100) remains inactive under an explicit sufficient condition, and that the closed-loop tracking error is Input-to-State Stable (ISS) with an asymptotic gain that is linear in delta_bar plus a term due to the exogenous input disturbance (Theorems 2 and 3).
★ PTMTD: A Topology-Aware Temporal-Graph Attention Network for Joint Cyber-Attack Detection and Relay-Level Localization in Power System Protection Networks
The digitization of protection relays exposes power grids to sophisticated cyberattacks, yet existing GNN detectors omit temporal encoding, rely on static topologies, and lack relay localization. We propose PTMTD-a Pearson-driven Temporal Multimodal Topology-aware Detection Network-coupling a dilated TCN with multi-head GATv2 in a dual-head architecture for simultaneous 37-class detection and four-relay localization. PTMTD constructs its graph from measurement correlations (no impedance data) and is trained with a composite loss jointly optimizing both tasks. On the MSU-ORNL benchmark (78,377 samples, 37 classes), it achieves 99.27% accuracy, 99.25% macro F1, and 95.98% localization, outperforming GraphKAN, LSTM, GCN, and Random Forest by 5.64-11.15 percentage points (p < 0.05). Ablations confirm gains from temporal encoding (+2.38%), graph attention (+2.69%), residuals (+2.07%), and correlation-driven supervision. With 639k parameters and 2.5 ms CPU inference, PTMTD is lightweight and deployment-ready. While validated on a transmission testbed, the architecture applies to microgrids with similar relay structures. Code and models are publicly available.
★ Safety Certificates for Systems under Attack via Stackelberg Set Games
We study a Stackelberg set selection game for linear systems subject to bounded disturbances and additive attacks. The defender selects the vector parameter of a polytopic safe set template; the attacker, after observing this choice, selects the vector parameter of an admissible attack set. The payoff is a signed, facet-normalised invariance margin that tests invariance exactly and quantifies either the available contraction or the facet inflation needed when invariance fails. Both controlled and robust controlled invariance are treated, corresponding to different information patterns for the control input. Exploiting the structure of parameterised polyhedra, we show that within each validity region the margin is concave in the attack parameter, which reduces the follower minimisation to finitely many extreme parameters. The complete max--min game is then solved by a finite family of LPs, one per defender validity region, each enforcing all attacker regions under a common defender decision and guaranteed margin. The framework handles hard state constraints and additional costs related to the safe set size, and recovers optimal set parameters together with state and attack realisations that activate the certificate. A load frequency control example under false data injection illustrates the construction, the validity decomposition, and the two information patterns.
★ Controllability-Informed Distributed Electric Propulsion Aircraft Design Using Probabilistic Attainable Moment Sets
Integrated aircraft design combines models of aerodynamics, propulsion, mass, and mission performance in one design loop, so that the disciplines can be traded against each other from the earliest stage. For configurations with distributed electric propulsion (DEP), part of the control authority comes from the propulsion system itself, which directly affects the required size of the tail and should therefore be known while the aircraft is being sized. This paper presents a controllability measure that is evaluated for every design in the loop and returned to it as a design signal. The attainable moment set of each candidate design is computed from the surrugate models of the loop and it is compared with a required moment set. Since the low-fidelity surrugate models are uncertain, the comparison is formulated with probabilistic attainable moment sets: the uncertainty is propagated to the boundary of the set, which yields a margin for every direction in the moment space and the probability of violating the requirement. The design loop thereby informed whether a configuration provides sufficient control authority, on which axes it falls short, and how this changes with the design variables. The approach is demonstrated on a DEP regional aircraft, where the signal is used to assess the size of the vertical tail.
★ Beyond Scalar Inertia: Frequency-Response-Based Assessment of Finite-Duration Inertial Support in Grid-Forming Converters
Conventional scalar inertia characterizes only the low-frequency terminal behavior of grid-forming converters (GFMs) and cannot fully predict inertial energy delivery during short-duration frequency disturbances. This letter proposes a frequency-response-based method to assess finite-duration inertial support using complex terminal power--frequency responses, such as those provided by Network Frequency Perturbation (NFP) plots. For GFMs with nonzero steady-state droop, a duration-dependent energy-equivalent inertia is defined after subtracting the ideal static-droop contribution. An exact spectral relationship is established, expressing this metric as a duration-weighted integral of frequency-dependent inertia, without requiring knowledge of the internal controller structure and parameters. The duration dependence of the metric is further shown to encode observable oscillation frequencies and damping ratios. A coefficient-decoupling GFM control with identical scalar inertia and droop across different settings is used for verification.
comment: This paper has 6 pages with 5 figures. The target journal will be determined at a later stage
★ Distributed Output-Feedback Control for Linear Multichannel Systems
This paper investigates distributed output-feedback stabilization of linear multichannel systems in which agents exchange state estimates without disclosing local control inputs or feedback gains. Remote control actions are treated as unknown inputs and geometrically decoupled from stable quotient-error dynamics. The proposed framework accommodates nontrivial intersections of local loss subspaces by retaining the coupling between the plant and common estimation errors. A Hurwitz condition on the coupled dynamics, together with a sufficient consensus-gain bound, guarantees exponential closed-loop convergence for every fixed feedback selection in a compact family. For polytopic feedback families, a quotient-preserving parameterization of output injections enables joint synthesis of nominal feedback and observer gains via alternating linear matrix inequalities, followed by a convex semidefinite program certifying a common consensus gain. A numerical example demonstrates the effectiveness of the proposed design.
★ Grid-Compliance Testing of Wind Turbines through Virtualized Component Tests: Testing Concept and Test Bench
As the proportion of inverter-connected wind turbines increases, test methods for evaluating their grid behavior are becoming increasingly important. This paper presents a Power Hardware-in-the-Loop (PHiL)-based method for grid connection testing of wind turbines. For the first time, geographically distributed simulations of aerodynamics via the Internet and of the grid model via a fiber-optic connection are integrated into certification-relevant tests. The concept expands upon the methods described in IEC TS 61400-21-4. Initial experimental results confirm high emulation quality and sufficiently low latencies, and meet the requirements for a standards-compliant grid emulation.
comment: 10 pages, 7 figures, 5 tables, peer-reviewed journal article
★ Latent resonances in oscillator networks
In networks of coupled oscillators, from neural circuits to power grids, damping is expected to suppress oscillations because of its dissipativity. Here we show that local damping can, counterintuitively, amplify oscillations along some input-output channels, while the network's global oscillation energy decreases and its spectral properties improve. The amplified oscillation arises from a latent resonance that lies in the spectrum yet barely manifests. The added passive damping reshapes this resonance by mixing it with adjacent eigenmodes. In random power-grid networks, damping can boost weakly responding channels by more than an order of magnitude, while subdominant channels can grow to exceed the originally strongest channel. Damping thus not only suppresses oscillations but also provides a way to redistribute oscillatory energy. For any passive network, the amplified response and its upper bound follow exactly from baseline quantities, offering a principled route for network design and control.
comment: 22 pages, 5 figures, Supplementary Materials included (6 figures, 7 tables)
★ Data-Driven Output Regulation via Predictive Control
This paper addresses constrained output regulation for unknown linear systems by means of predictive control directly from input-output data. The objective is to reject disturbances and track references generated by a known exosystem, without identifying a plant model or solving the output regulator equations. The first step is a trajectory parametrization for plants driven by exosignals with known dynamics. % By exploiting the known exosystem eigenstructure, Willems et al's fundamental lemma is extended from autonomous input-output trajectories to trajectories coupled with exogenous modes. By exploiting the known exosystem eigenstructure, input-output trajectories driven by an exosignal of known phase are parametrized directly from data, which contains the affine version of Willems et al.'s fundamental lemma as a special case. This parametrization yields a nominal data-driven predictive control (DDPC) regulator, for which a Lyapunov argument proves exponential convergence to the regulator manifold. For periodic exosignals, the steady-state feedforward input is eliminated from the online problem through an incremental-input regularization, leading to an implementable regulator based on measured data and known exosystem information. Bounded output noise is handled by slack and coefficient regularization, with a regularization center computed offline from data; the resulting robust DDPC law is recursively feasible and practically stabilizing, with an ultimate regulation error that vanishes with the noise bound. A constrained lateral vehicle-control example with periodic road curvature illustrates disturbance rejection, input-constraint satisfaction, and the transition from exact regulation with noise-free data to practical regulation with noisy data.
★ Enhancing Transient Stability of VSG-Controlled Converters Under Grid Frequency Disturbances
Strong damping feedback can substantially improve the transient synchronization stability of renewable energy converters employing virtual synchronous generator (VSG) control. However, conventional damping feedback uses the nominal frequency as its reference. When the frequency of a weak grid deviates from nominal, strong damping tends to keep the VSG frequency near nominal. This can sustain a frequency difference between the VSG and the grid and compromise transient synchronization. To address this problem, this paper proposes a complex-frequency-based method to estimate the grid reference frequency from local terminal measurements. The estimated frequency replaces the nominal reference in the VSG synchronization loop, so that damping acts on the frequency difference between the VSG and the real grid. Controller hardware-in-the-loop tests validate the effectiveness of the proposed method under grid frequency disturbances and its robustness to grid parameter uncertainty and adverse grid conditions.
★ PLL-Less Synchronization of Grid-Following Converters Using a Voltage-Driven Frenet Frame
In grid-following converters, the synchronization path can transmit voltage disturbances to the current loop under weak or unbalanced grid conditions. This paper develops a Frenet-frame controller that derives its axes from the integral curve of the measured voltage at the point of common coupling. The tangential current governs active power, while the normal and binormal currents govern the transverse components of non-active power. Feedforward terms derived from the output filter model compensate curvature and torsion coupling, yielding identical nominal current dynamics across the axes. The controller operates without a phase-locked loop or sequence extraction. A reduced transverse model accounts for the finite bandwidth of geometric rate reconstruction and predicts a reduction in low-frequency coupling from voltage disturbances to current. Controller hardware-in-the-loop tests using real-time simulation of a three-phase four-wire converter show the shortest settling time after an active power reference step and the smallest peak power deviations during balanced voltage disturbances among the tested controllers. Under single-phase voltage unbalance, the method reduces peak binormal current and maximum phase-current total harmonic distortion relative to both benchmarks. It also settles to the commanded power in a weak-grid connection test where the phase-locked loop benchmark exhibits sustained oscillations.
★ Implicit discretization of PID-like sliding-mode control with third-order output accuracy
A digital PID-like sliding-mode controller is proposed for a double integrator with a Lipschitz matched disturbance. A proper implicit discretization, matched to the exact sampled dynamics of the plant, attains the accuracy of the continuous-time controller without discretization chattering. Global finite-step convergence to a discrete-time sliding mode is established for two gain designs and every sampling time, using an implicit Lyapunov function and a computer-assisted certificate in exact arithmetic. The resulting controller provides third-order output accuracy, second-order accuracy for its derivative, and exact finite-time rejection of constant matched disturbances. These accuracy orders are proved to be optimal among deterministic causal controllers using the same sampled information.
comment: Extended version of a paper submitted to Automatica. 20 pages, 3 figures, 2 tables. Code and verification records: https://github.com/MartinZapf/Implicit-PID-like
★ OrbitWeaver: LLM Agent Planning with Simulation Feedback for Optically Interconnected Orbital GPU Clusters
Space-based data-center designs increasingly consider GPU computing formations connected by inter-satellite optical links. With limited neighboring connections, jobs assigned disjoint GPUs can still contend on shared relay paths. Rank placement and communication priorities jointly affect this contention and the readiness of downstream computation. We present OrbitWeaver, an offline large language model (LLM) agent planner that jointly optimizes rank placement and communication order. Guided by communication structure, execution dependencies, and simulation feedback, the agent constructs and refines plans that coordinate shared-link access with computation progress. This coordination reduces contention and dependency blocking, shortening mean job completion time (JCT). Simulation experiments on optically interconnected orbital GPU clusters show that OrbitWeaver reduces aggregate mean JCT by 6.2-33.5% relative to the state-of-the-art baselines CASSINI, Crux, and Symphony. These results demonstrate the value of LLM-guided planning for efficient concurrent computing in orbital GPU clusters.
★ Improved Closed-Form Identification of the Air-Gap Permeance in Magnetic Equivalent Circuits of Rotor-Eccentric Electric Motors
Fast and accurate simulation of electric motors is essential for their design, fault diagnosis and control. Magnetic equivalent circuits (MECs) offer a computationally efficient alternative to the finite element method (FEM), yet their flux tube representation poorly captures the air-gap field. This creates a challenge, as the air gap stores most of the magnetic energy and governs the electromagnetic torque and, in rotor-eccentric motors, the unbalanced magnetic pull (UMP). Existing air-gap permeance expressions vary widely in their results for torque ripple and UMP and account for tooth-normal eccentricity only. This paper proposes the dual-term air-gap permeance (DTAP) model, a closed-form expression for the air-gap permeance between a stator and a rotor tooth that accounts for tooth-normal and tooth-tangential eccentricity, with and without rotor skew. Its parameters are identified from a FEM model of the air gap and the complete set of tooth heads, which includes the shielding by the neighbouring teeth. For three motors, DTAP reproduces the FEM air-gap permeance with a 2.8-4.8 times smaller error than the state of the art, at a comparable computational cost. Implemented in the MEC of an eccentric induction motor, it reproduces the torque ripple within 1.6% and the peak UMP within 1.3% of the reference, i.e., with 2.8 and 27 times smaller deviations than the state of the art. DTAP thereby enables MEC to predict the torque pulsations and bearing loads of eccentric motors more accurately without sacrificing computational speed.
comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible
★ Inferring Sensitive Events in Multi-Mode Cyber-Physical Systems via Scheduling Side Channels
Many cyber-physical systems (CPS) are multi-mode: they switch between operating modes with different tasksets and timing parameters in response to physical events or user activity. We show that these mode changes create a timing side channel: by altering the interference experienced by co-running tasks, they let an unprivileged co-resident observer infer sensitive events, such as a homeowner's arrival or a vehicle's route, from its own execution timing and, on multicore platforms, its core assignment. Existing schedule-based attacks and defenses miss this channel because they assume single-mode systems or target a specific, known victim task. We present STAMP, a lightweight attack that needs no knowledge of task parameters or scheduling policy, and works whether or not the system exposes its modes. On a ZoneMinder home-security deployment, STAMP identifies the modes with 94.7% accuracy on unseen hardware, using under 0.5% of one core. On ROS 2 AMCL with the Oxford RobotCar dataset, it identifies which of seven route segments a vehicle traversed with 86.0% accuracy on unseen platforms. A schedule-randomization defense lowers detection recall to 0.53, at which setting only 53.9% as many jobs finish on time. At higher noise ($ε=10$), on-time job completion reduces to 14.4%, while introducing new leakage in 14.1% of the tasksets.
★ Graph Neural Network Prediction of Frequency Nadir and RoCoF Under Topology Variations
Accurate prediction of frequency dynamics under varying network topologies is essential for rapid frequency security assessment, whereas repeated electromagnetic transient (EMT) simulations are computationally intensive. This paper develops a topology-aware graph neural network with a radial basis function Kolmogorov-Arnold network (RBF-KAN) regression head, termed RK-GNN, to predict frequency nadir and maximum absolute rate of change of frequency (RoCoF). Scenarios combine load scaling, regional load redistribution, and generator redispatch with intact, N-1, and N-2 transmission topologies. After checks for network connectivity, generator reference limits, and numerical consistency, the retained scenarios are simulated in PSCAD to construct the reference dataset. The RK-GNN is evaluated against a conventional deep neural network (DNN) and a baseline GNN. On the IEEE 39-bus system, the RK-GNN reduces the mean absolute errors (MAEs) of nadir and RoCoF by 48% and 59% relative to the DNN and by 42% and 61% relative to the baseline GNN. Incorporating the bridge indicator and disturbance hop distance yields further MAE reductions of 27% and 54%. These results show that the proposed topology-aware architecture provides accurate prediction of both frequency metrics under varying topologies and substantially reduces the need for repeated EMT simulations.
★ State-of-Charge Targets for Storage Participation in Two-Settlement Markets: Reliability, Price, and Compensation Impacts
As energy storage assumes a larger role in electricity systems, market operation must coordinate its use across time to support reliability during periods of system stress. Minimum state-of-charge (SOC) targets provide a means of preserving energy for these periods, but their design shapes reliability, operating efficiency, and storage compensation. This paper evaluates alternative SOC target designs in a California-based day-ahead and real-time electricity market model. The comparison examines targets based on day-ahead schedules or rolling centralized dispatch, enforced at the individual-resource or system level, across seasons, stress conditions, and participant information settings. The results show that retaining more energy does not necessarily reduce load shedding: targets that restrict discharge can limit storage support during shortages. Under critical summer conditions, individual targets derived from rolling centralized dispatch reduce weekly load shedding by approximately 83--84\% relative to operation without targets. Lower RT clearing costs and average prices can coexist with higher storage compensation, reflecting the role of day-ahead positions and real-time deviations in determining settlement outcomes. These findings highlight the importance of coordinating energy retention with timely discharge and network deliverability when designing SOC requirements for storage participation in electricity markets.
★ Frequency-Security-Aware Real-Time Economic Dispatch with Data Center UPS Flexibility for Inertia and Primary Frequency Response Services
The increasing penetration of renewable energy and the rapid growth of AI data centers are introducing significant challenges to power system frequency security. There is an urgent need to identify grid-compatible resources capable of providing fast frequency support. This paper proposes a frequency-security-aware real-time economic dispatch (RTED) model that enables data center (DC) uninterruptible power supply (UPS) systems to provide steady-state power balancing, inertia response, and primary frequency response (PFR) services. Specifically, tractable analytical frequency-security constraints are derived and incorporated into the RTED framework through second-order cone programming (SOCP) formulation. The proposed model jointly optimizes energy, reserve, and inertia to minimize system operation cost while preventing under-frequency load shedding (UFLS). Numerical validation on a modified IEEE 39-bus system demonstrates that the proposed RTED model with UPS flexibility can economically support system operation and significantly improve frequency security under active-power deficit disturbances.
★ Orbit-AWARE: From Onboard AI Deployment to Sustainable In-Orbit Intelligence
The growing role of space computing in time-critical satellite services makes the long-term reliability of onboard artificial intelligence (AI) a mission-level requirement. Model drift can undermine this reliability through critical-observation suppression or noncritical priority flooding, displacing both mission information and evidence for model correction. To address this drift-induced evidence displacement, this article presents Orbit-AWARE, a safeguard-first lifecycle management (LCM) architecture. A comparison of existing approaches motivates cooperative Data, Cognitive, and Trust Planes for continuous monitoring, event-triggered agentic orchestration, and deterministic governance. The architecture retains a model-independent sample stream before degradation is confirmed and coordinates evidence selection and exchange for diagnosis and ground-assisted adaptation. Candidate updates are verified before control over data disposition is restored. In an xBD-based emergency Earth observation (EO) case study, Orbit-AWARE delivers 14.3\% more critical observations and achieves 8.3\% higher sustainable mission utility than the Rule-Based Drift Loop under a common 10\% downlink budget. Finally, we discuss future research directions for extending LCM through task-aware reliability assessment, memory- and tool-enabled onboard agents, and evidence-centric space networking. This work makes evidence availability and recovery authorization explicit design objectives for dependable onboard AI services.
comment: 10 pages, 4 figures, 2 tables, submitted to IEEE Wireless Communications
★ Feasibility of Pareto-Improving Damping Control of Forced Oscillations: The Active-Power Case
Damping control of a forced oscillation may reduce amplitudes at certain locations while increasing them elsewhere. This paper introduces Pareto improvement as a criterion for assessing such control: at least one nodal oscillation amplitude decreases, while none increases across all other nodes. The analysis considers active-power forcing sources and active-power control injections in connected swing networks with positive inertia and damping. Necessary conditions for Pareto improvement feasibility are derived from the normalized nodal responses. It is proven that for every network with a single oscillation source and a single actuator, at least one positive forcing frequency is shown to preclude Pareto improvement, regardless of the amplitude and phase of the control input. A three-node counterexample shows that this limitation does not persist with multiple actuators. PSS/E simulations of a 6-bus benchmark system and a modified IEEE 39-bus system validated the analysis. These findings suggest the value of assessing control effects across the network and further investigating multi-actuator damping control.
★ Battery Energy Storage Sizing in PV Ramp-Rate Smoothing using Constraint-Aware Reinforcement Learning-Based Controller
This paper presents a constraint-aware reinforcement learning (CARL) controller for battery energy storage system (BESS) sizing and photovoltaic (PV) ramp-rate smoothing. Rapid PV power fluctuations caused by cloud transitions can introduce operational and stability challenges in power systems. To mitigate these impacts, grid codes impose ramp-rate limits (RRL) on PV power injected to the grid. BESSs can support such grid codes by absorbing or supplying power during rapid variations. However, the extent to which BESSs can reduce PV-induced variability depends on (i) the BESS power and energy capacities; and (ii) the adopted controller. Thus, BESS sizing and controller design are coupled. Conventional moving-average control can increase battery usage due to filter-induced lag, while ramp-rate control is reactive and does not account for the long-term impact of actions on battery state of charge and future ramp events. Optimization-based methods can reduce BESS requirements by anticipating future PV variations, but their performance depends on accurate PV forecasts. To address these limitations, the CARL controller learns a causal state-feedback policy from historical PV data without requiring forecasts during deployment. Ramp-rate, converter-power, and state-of-charge constraints are incorporated into a causal feasibility interval that guarantees feasible actions by definition. A full-horizon input-convex neural network (ICNN)-based optimization is developed as a noncausal benchmark for the minimum achievable BESS requirements. Evaluation using three real-world PV datasets shows that, at a 10\%/min ramp-rate limit, CARL reduces peak BESS power by 8.7--10.4\% and energy capacity by 47.5--62.6\% relative to conventional ramp-rate control while maintaining the grid-codes compliance.
♻ ★ RCOA Extension and Applications
The Relaxed Convex Obstacle Avoidance (RCOA) formulation is the first approach to enable a fully convex optimal control problem (OCP) for obstacle avoidance. Convergence analysis of RCOA yields an analytical framework that defines a unique characteristic: the ability to maintain obstacle avoidance (OA) efficacy even when obstacles reside beyond the controller's prediction horizon. In this paper, RCOA is extended to three-dimensional environments and apply it to Unmanned Aerial Vehicle (UAV) navigation. Furthermore, the formulation is enhanced to incorporate vehicle geometries, moving beyond point-mass representations to enable collision avoidance between 3D objects. Numerical simulations demonstrate that RCOA provides computational performance on par or exceeding state-of-the-art methods. Notably, RCOA is demonstrated to enable a Nonlinear Model Predictive Controller (NMPC) to execute aggressive maneuvers through narrow passages with reduced prediction horizons, ensuring real-time feasibility at frequencies exceeding 30~Hz.
comment: 11 pages, 9 figures, multimedia
♻ ★ A Variational Latent Equilibrium for Learning in Neuronal Circuits
Brains remain unrivaled in their ability to recognize and generate complex spatiotemporal patterns. While AI is able to reproduce some of these capabilities, deep learning algorithms remain largely at odds with our current understanding of brain circuitry and dynamics. This is prominently the case for backpropagation through time (BPTT), the go-to algorithm for learning complex temporal dependencies. In this work we propose a general formalism to approximate BPTT in a controlled, biologically plausible manner. Our approach builds on, unifies and extends several previous approaches to local, time-continuous, phase-free spatiotemporal credit assignment based on principles of energy conservation and extremal action. Our starting point is a prospective energy function of neuronal states, from which we calculate real-time error dynamics for time-continuous neuronal networks. With this, as a first contribution we provide a simple and straightforward derivation of the adjoint method dynamical equations, the time-continuous equivalent to BPTT. Importantly, additional approximations are necessary to avoid unphysical temporal non-localities. One such approximation was recently proposed in Generalized Latent Equilibrium (GLE), but enforcing temporal locality of the backpropagating dynamics induces an error in the gain that becomes problematic when processing complex signals. Thus, our second contribution is to design a learning rule for the backward weights to find a fully local (in space and time) set of equations for neuron and synapse dynamics that corrects for this gain difference. With this inclusion of backward weight learning, our theory provides a rigorous framework for spatiotemporal deep learning in the brain, while simultaneously advancing a blueprint for physical circuits capable of carrying out these computations.
♻ ★ Strong nonlinear detectability and moving horizon estimation for nonlinear systems with unknown inputs
This paper investigates state estimation for general nonlinear discrete-time systems subject to measurement noise and possibly unbounded unknown inputs. To approach this problem, we first propose the concept of strong nonlinear detectability, which proves to be a nonlinear counterpart to strong* detectability. Based on the proposed detectability notion, an unknown input state estimator (UISE), which yields bounded estimation error even for unbounded unknown inputs, is designed via a moving horizon estimation strategy using past and current measurements. Next, we tighten this detectability notion to develop a two-stage MHE-based UISE that is computationally more efficient. In a simulation example, both variants of MHE-based UISEs are compared with a conventional MHE to illustrate the merits of the proposed methods.
comment: 19 pages,3 figures
♻ ★ System-wide Dynamic Performance Metric for Converter-Dominated Power Systems
Fast controller dynamics in Inverter-Based Resources (IBRs)-dominated power networks couple frequency and voltage, making traditional decoupled analysis inadequate. We introduce a unified system-wide metric---the complex frequency of system losses-to evaluate dynamic performance and synchronization. Composed of device- and network-driven terms, it expresses voltage and injected-current phasor variations weighted by bus complex powers. Validated on an IBR-modified IEEE 39-bus system and the Irish transmission grid, the framework effectively assesses R/X sensitivities, dynamic support limits, and data-center load impacts.
♻ ★ Contract-Based Decomposition of Temporal Logic Specifications for Networked Systems under Arbitrary Partitions
Computational complexity is an inherent limitation of formal synthesis for networked systems, and decomposing the global specification into local ones relaxes this limitation at the cost of conservatism. Since the granularity of the partition governs this trade-off, it is reasonable to treat the partition as a design variable, which calls for local specifications that remain correct for every partition. To this end, this paper gives each agent a local specification, written as an assume-guarantee contract that the agent can establish from local information. We first derive a necessary and sufficient condition for these contracts to decompose the global specification under a given partition. Building on this, we then present a condition under which the decomposition is correct for every partition, so that the partition becomes a free design variable. For linear dynamics and signal temporal logic formulas with affine predicates, we further synthesize a controller for each coalition by a tube-based approach. Finally, simulations on a network of input-coupled tanks show how the choice of partition trades computational cost against conservatism.
♻ ★ An iterative tangential interpolation algorithm for model reduction of MIMO systems
We consider model reduction of large-scale Multi-Input, Multi-Output (MIMO) systems using low-rank barycentric tangential interpolation on the frequency axis. Our scheme is related to the recently-developed Adaptive Antoulas--Anderson (AAA) algorithm, which is an iterative algorithm that uses concepts related to the Loewner framework. One feature of our algorithm is optimization of the interpolation weight matrices to minimize a proxy for the \(L_2\) system error. We use the freedom in interpolation point selection to provide multiple minimum-error-based algorithms with trade-offs between computational complexity and approximation performance. We prove that a weighted \(L_2\) norm of the proxy error system is non-increasing as interpolation points are added, and that the reduced-order system is stable whenever the original system is. Computational results are provided demonstrating the closeness of the proxy system and true system \(L_2\) errors. Comparisons with prior work demonstrate performance on par with standard model reduction methods.
comment: 14 pages, 4 figures Submitted to IEEE TAC. Revision 2
♻ ★ Stealthy, High-Impact False Data Injection Attacks in Smart Grid Using Physics-Constrained GAN with Limited System Knowledge
False data injection (FDI) attacks can distort power system state estimation while retaining plausible measurement patterns. Existing FDI methods trade stealth for impact or vice versa. They often assume access to internal grids, all-bus sensing, or defender models. In this paper, we propose a detector-agnostic physics-constrained generative adversarial network (PCGAN) that learns decoupled power-flow sensitivities from historical measurements of deployed sensors. The learned active-power-angle and reactive-power-voltage-magnitude relationships guide perturbations restricted to compromised channels, while adversarial training promotes realistic spatiotemporal patterns. We implement PCGAN and seven existing FDI attacks on the IEEE 300-bus system using one year of observed hourly demand from PJM Interconnection, a regional transmission organization in the United States, and synthetic bus-level load profiles. We compare their performance across 10%, 30%, 60%, and 100% compromised sensors. Attack stealth is assessed against ten machine-learning (ML) detectors and conventional bad-data detection (BDD) methods, while physical impact is quantified through nonlinear alternating-current weighted least-squares state estimation. PCGAN achieves 84.28% mean ML evasion and 22.31% mean spatial load misperception across the four scenarios. Its mean stealth-impact trade-off score reaches 18.69%, compared with 5.61% for the strongest competing attack. PCGAN also achieves 83.45% chi-square BDD evasion and 86.02% largest-normalized-residual BDD evasion. These results demonstrate a favorable stealth-impact balance under varying sensor-access levels and provide challenging attack cases for evaluating FDI detectors.
comment: 11 pages
♻ ★ Integrating Planning and Predictive Control Using the Path Feasibility Governor
The motion planning problem of generating dynamically feasible, collision-free trajectories in non-convex environments is a fundamental challenge for autonomous systems. Decomposing the problem into path planning and path tracking improves tractability, but integrating these components in a theoretically sound and computationally efficient manner is challenging. We propose the Path Feasibility Governor (PathFG), a framework for integrating path planners with nonlinear Model Predictive Control (MPC). The PathFG manipulates the reference passed to the MPC controller, guiding it along a path while ensuring constraint satisfaction, stability, and recursive feasibility. The PathFG is modular, compatible with replanning, and improves computational efficiency and reliability by reducing the need for long prediction horizons. We prove safety and asymptotic stability with a significantly expanded region of attraction, and validate its real-time performance through simulated case studies of unicycle and quadrotor navigation in cluttered environments. \c{opyright}2026 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
comment: 16 pages, 12 figures, accepted as a Full Paper in IEEE Transactions on Automatic Control
♻ ★ Toward Operationalizing Rasmussen in Evolving Software Systems
Rasmussen's qualitative drift model explains how successive, locally successful adaptations under efficiency pressures can move a system toward unacceptable operation. Brooker and Hochstein applied this perspective conceptually to software operations without specifying a quantitative procedure for observing drift. We take a step in this direction and propose a runtime observer that quantitatively operationalizes the model's observable part for evolving software. The proposed observer is intended to give engineers evidence for pausing or revising adaptation before declared operating limits are crossed. It tracks how activity or resource use is distributed across components and types of work. Measurement history quantifies accumulated change and recent movement toward or away from current limits. Assuming that direction and limits remain unchanged, the proposed observer calculates the first constraint ahead and the amount of further change required to reach it. Measurements remain comparable across supported component changes, while policy revisions are distinguished from changes in execution. Preliminary runtime studies demonstrated these distinctions under annotated software changes and revised retry limits. A separate multivariate exponentially weighted moving average (MEWMA) experiment detected CPU redistribution earlier with log-ratio coordinates than with raw shares, motivating investigation of representation-dependent observation. Further work will ground constraints in service outcomes, investigate runtime-model acquisition, and test whether decisions informed by observed drift reduce violations at acceptable useful-work loss and intervention cost.
♻ ★ Coordinated Cooling and Compute Management for AI Datacenters
The AI datacenters are currently being deployed on a large scale to support the training and deployment of power-intensive large-language models (LLMs). Extensive amount of computation and cooling required in datacenters increase concerns about the energy use and carbon emissions of AI datacenters. Although current state-of-the-art has examined the energy efficiency of LLM inference, most prior research focused on optimizing compute-side scheduling without considering thermal objectives or constraints. Since GPU-intensive inference generates substantial heat that can degrade datacenter performance, ignoring thermal effects can increase total energy consumption and reduce the efficiency of LLM serving. To fill this gap, we profile the characteristics of GPU servers under varying cooling and AI jobs, and develop a joint cooling and computing modeling approach for AI datacenters. Built upon such workload and thermal dynamics models, a novel hierarchical control framework is proposed to co-optimize computing and thermal management by identifying the optimal GPU parallelism, frequency (DVFS), and cooling control knobs. Using real Azure inference traces and detailed GPU profiling, our model balances serving latency and thermal constraints in AI datacenters while significantly improving AI datacenters' energy efficiency.
comment: Accepted at IEEE Transactions on Cloud Computing
♻ ★ A Control Barrier Function-Constrained Model Predictive Control Framework for Safe Reinforcement Learning
Ensuring safety under unknown and stochastic dynamics remains a significant challenge in reinforcement learning (RL). In this paper, we propose a model predictive control (MPC)-based safe RL framework, called Probabilistic Ensembles with CBF-constrained Trajectory Sampling (PECTS), to address this challenge. PECTS jointly learns stochastic system dynamics with probabilistic neural networks (PNNs) and control barrier functions (CBFs) with Lipschitz-bounded neural networks. Safety is enforced by incorporating learned CBF constraints into the MPC formulation while accounting for the model stochasticity. This enables probabilistic safety under model uncertainty. To solve the resulting MPC problem, we utilize a sampling-based optimizer together with a safe trajectory sampling method that discards unsafe trajectories based on the learned system model and CBF. We also derive a finite-horizon exit-probability bound using tail information for centered barrier increments. We validate PECTS in various simulation studies, where it outperforms baseline methods.
comment: 10 pages; additional theoretical results and expanded discussion
♻ ★ A Theory of Information Architecture for Networked Decisions: Freshness, Locality, and Coordination SC
A broader view of a network can help agents coordinate, but collecting that view takes time. When is a fresh local observation more useful than an older view of the whole network, and when should an agent look only partway across it? We study agents that repeatedly minimize a shared quadratic cost in an environment that evolves independently of their actions. An information architecture specifies what each agent observes and how old those observations are. Each agent uses a synchronized neighborhood snapshot, so extending its scope also ages the nearby readings used in the decision. We assume that the optimal action with complete information is affine in the state, and that the conditional mean of the current state equals the old state scaled by a common exponential decay factor. Using the classical Marschak--Radner decomposition, we derive the delay threshold beyond which fresh local information outperforms a complete but older global view. This comparison allows non-Gaussian states and nonlinear decision rules. We then determine how far an agent should look by balancing the benefit of additional observations against their greater age. In a Gaussian spatial model on a line, we derive the optimal observation radius: looking beyond the local node pays only when information travels more than twice the spatial correlation length in one coherence time. The preferred information architecture also depends on the decision being made. On a fixed 64-node ring, increasing only the reach of the decision weights from 0.7 to 6 hops shifts the optimal observation radius from one to five hops.
comment: 32 pages, 6 figures. Manuscript source and complete reproduction materials are available at https://github.com/ANRGUSC/TINA
♻ ★ Symbolic-Permutation-Based Direct Sparse Jacobian Assembly for Accelerated Power-Flow Analysis
The repeated assembly of the sparse Jacobian matrix constitutes a primary computational bottleneck in the Newton-Raphson power-flow analysis. Through an algebraic operator equivariance analysis of the Jacobian matrix, this paper formalizes the reduction process and proposes a direct sparse Jacobian assembly method based on symbolic permutation. By pre-applying the permutation to the admittance matrix during a fast one-time symbolic phase via its sparse prefix-sum property, the runtime bus-type projection overhead and indirect mapping caches are entirely eliminated. Consequently, complex Jacobian differentiation and global matrix assembly fuse into a single operation, yielding a single-pass, branchless algorithm that streams values directly into contiguous memory. The proposed algorithm is implemented in the open-source RustPower solver. Benchmarks on IEEE test systems and PEGASE 9241 demonstrate that the proposed assembly is 28.1-37.9 times faster than the MATPOWER-style baseline and 8.9-9.7 times faster than the in-house Schäfer-Braun-inspired assembly (V1); on PEGASE 9241, it outperforms optimized traditional C++ assembly routines by 3.72 times in cold construction and 2.59 times across iterations. End-to-end, the iteration-loop solve time with KLU is 1.16-1.97 times faster than the native C++ LightSim2Grid and up to 95.5 times faster than pandapower, while leaving numerical results and convergence behavior unchanged. The method extends directly to general problems where a sparse Jacobian inherits its structure from a static network operator, including transient stability analysis, EMT simulation, and OPF Hessian assembly.
comment: 11 pages, 3 figures
Robotics
★ ProtoVis-Nav: Prototypical Visual Imagination for Visual-Spatial Aligned UAV Navigation
Vision language navigation (VLN) is very challenging for Unmanned Aerial Vehicles (UAVs) since small errors in 3D motion prediction can lead to navigation errors and collisions. Unable to fathom execution outcomes in 3D environments, existing methods often struggle to predict accurate navigation waypoints from language instructions. To solve this, we introduce the Prototypical Visual Imagination framework, shortened as ProtoVis-Nav, which forces the model to imagine an outcome-associated local visual representation for the intended next UAV movement. Specifically, we predict feature prototypes and use their linear combination to estimate the visual feature of the target region. In this manner, the VLN model first imagines the local visual representation of the next intended location, and then uses predicted prototypes as context for generating the final action, promoting alignment between visual context and waypoint outputs. Extensive experiments demonstrate the effectiveness of the ProtoVis-Nav framework and show substantial improvements over existing baselines.
★ HEIR: Harness Egocentric Intent for Human-Robot Interactions
For robots to assist, serve, and collaborate with humans, they must not only interpret human requests conveyed through language and multimodal intentional cues, but also execute long-horizon tasks that interleave navigation and manipulation. In this paper, we introduce HEIR, a resource for studying how to Harness Egocentric Intent for Human-Robot Interaction. HEIR pairs human speech, measured gaze, and egocentric video with robot observations and teleoperated navigation-manipulation trajectories, while each session preserves successive episodes, with each request fulfilled through multiple execution subtasks. The captured data, together with human-authored annotations, support the study of intent grounding, execution planning, and task completion within a unified interaction setting. To demonstrate the value of HEIR, we conduct extensive offline experiments to evaluate how models benefit from additional human-centric context across these tasks, together with real-robot experiments that assess execution task completion. HEIR thus provides a valuable resource for studying user-centric robot interaction in naturalistic household environments. Our demo and dataset are available online.
★ Collaborative Task Allocation for Heterogeneous Multi-Robot Systems Through Iterative Clustering ICRA
Multi-robot systems face the challenge of efficiently allocating teams of heterogeneous robots to tasks. The task allocation problem is complicated by collaborative interactions between robots where teams of robots develop emergent capabilities that enable them to complete tasks that would be inefficient or impossible for individual robots. To address these challenges, we present an iterative clustering algorithm for collaborative task allocation in heterogeneous multi-robot systems. This approach partitions the computationally intractable global optimization problem into smaller, tractable subproblems by iteratively forming clusters of robots and tasks, then optimizing assignments within each cluster. By ensuring robots remain clustered with their currently assigned tasks, we guarantee monotonic improvement in overall utility with each iteration. We analyze the convergence of the algorithm and characterize how cluster size constraints determine which suboptimal assignments could trap the algorithm. In simulation, iterative clustering consistently outperforms simulated annealing, and a group-based auction in both computation time and solution quality, and outperforms a hedonic game approach in solution quality.
comment: 8 pages, 4 figures, Published in IEEE Robotics and Automation Letters (RA-L) and presented at 2026 IEEE International Conference on Robotics and Automation (ICRA)
★ UAV-Flow++: A Unified Suite for Benchmarking and Learning Language-Guided UAV Flight
Language-guided UAV navigation has largely focused on long-horizon route planning, leaving short-range, object-relative flight maneuvers underexplored. To address this gap, we formulate Flying-on-a-Word (Flow), a task requiring a UAV to convert concise natural-language instruction, egocentric visual observation, and current state dynamics into continuous flight actions. Flow encompasses both primitive motion commands and complex object-interactive behaviors, assessing motion intent comprehension and spatial context grounding. To enable policy learning and closed-loop evaluation, we introduce UAV-Flow, a comprehensive dataset comprising 30,692 expert-piloted real-world trajectories across eight distinct motion categories, alongside 10,109 simulation trajectories in the UAV-Flow-Sim environment equipped with task-specific completion criteria. Furthermore, we develop FlowVLA, which undergoes imitation pretraining on mixed real-sim demonstrations and online post-training via Tail-Aware GRPO (TGRPO). By leveraging group reward distributions, TGRPO effectively reinforces high-reward action sequences. Evaluated on 273 held-out UAV-Flow-Sim episodes, pretrained FlowVLA achieves a mean success rate of 75.58%, outperforming the strongest baseline by 6.72 percentage points, which TGRPO post-training further elevates to 93.16% with superior sample efficiency. Physical UAV experiments demonstrate robust execution of object-relative commands in both indoor and outdoor real-world scenes. Together, Flow establishes a novel benchmark and learning framework for language-conditioned flight maneuvers.
comment: 22 pages, 10 figures
★ XBot-Omni: An Omni-Modal Model for Embodied Understanding and Interaction
Embodied interaction requires an agent to integrate multimodal perception with verbal and nonverbal behaviors to effectively communicate its intentions, including speech, body motion, and facial expressions. We present XBot-Omni, a Thinker--Talker--Actor architecture that extends the Omni paradigm to robotic modalities, including body motion and facial expressions, with ``X'' representing the extensibility to robotic-specific modalities. The Thinker processes text, images, audio, and video to formulate contextually appropriate responses, which are then realized by the Talker and Actor through speech and coordinated bodily expressions, respectively. In this report, we innovatively introduce the concept of ``Actor'' as a dedicated component for embodied agents: a coordinated, streaming module that leverages the native representations of the Thinker and Talker within an end-to-end pipeline, introducing only modest latency overhead. This unified architecture enables multimodal generation conditioned on omni-modal inputs directly to prevent information loss in staged designs. We comprehensively evaluate XBot-Omni across seven benchmarks, demonstrating strong multimodal understanding capabilities on both general-domain and robot-centric videos, as well as high-quality, natural, and well-coordinated motion generation. Furthermore, our results validate its capability to generate embodied multimodal responses directly from omni inputs, meeting the diverse requirements of embodied interaction. XBot-Omni enables robots not only to perceive the omni world but also to express themselves through speech and bodily behaviors, extending Omni models from conversational interaction to embodied communication in physical environments.
★ Inner Product Regularization for Imitation Learning
Robots that learn from demonstrations should extrapolate their policy despite state perturbations such as sensor noise, scene variation, and compounding errors. One approach to this robustness is regularization. Prior regularizers consider each state on its own and limit how sensitive the policy is to any perturbation at that state. By contrast, we explore how the policy's output shifts when given the same perturbation at different states. Our insight is that, locally, the robot should respond to state perturbations in a consistent manner: e.g., if changing an irrelevant object position does not impact the policy at one state, that same change should not shift the policy output at a nearby state. Applying this insight, we develop inner product regularization (InneR). We show that the inner product on action space induces an inner product on state space through the policy Jacobian (the Fisher information for stochastic policies), and that the null space of this inner product governs which state perturbations have a first-order effect on the policy. By clustering states whose inner products are aligned, averaging the inner products within each cluster, and minimizing the smallest eigenvalues of that average, InneR shapes the learned policy so that nearby states share a common null space. InneR enforces consistent responses to perturbations without limiting policy expressiveness, and can be appended to existing imitation learning pipelines for deterministic or stochastic policies. In simulated experiments, InneR makes learned policies more robust to irrelevant state dimensions and shifts between training and test distributions, whereas norm penalties often perform worse than behavior cloning. On a real robot arm performing three manipulation tasks, InneR raises the average success rate of a diffusion policy from 18% to 57% without additional data.
★ FORGE: Force-Aware Multi-Granularity Control for Efficient Humanoid Loco-Manipulation
Humanoid loco-manipulation requires coordinated locomotion, balance regulation, and manipulation under frequent contacts, making teleoperated demonstration collection difficult: a practical controller must simultaneously provide precision, stability, and contact awareness. We propose FORGE, a force-aware multi-granularity control system with a coarse-to-fine command space spanning velocity navigation, planar foot placement, wrist-pose tracking, and optional force-conditioning inputs. Policy training requires no full-body human motion trajectories or joint-angle references; deployment consumes only sparse bilateral wrist poses relative to the pelvis, coarse motion commands, and optional desired wrist-load inputs. For sustained contact, FORGE combines a momentum observer with a learned correction trained using simulator contact-force supervision. At deployment, it estimates low-frequency, wrist-equivalent interaction forces from proprioception and reconstructed low-level arm PD torques, without wrist force/torque sensors. A quasi-static virtual-target loop uses this estimate and an optional desired applied-force input to adapt the wrist target. A frozen locomotion prior and asymmetric residual actions stabilize manipulation-induced whole-body corrections. Experiments evaluate unified-control stability, task-space tracking, quasi-static wrist-equivalent force estimation, squeezing forces during sustained contact with a rigid box, and downstream diffusion-policy execution of a squat-and-lift task across payload conditions. Website: https://bb0928.github.io/forge/.
comment: 8 pages, 10 figures
★ SALA: Semantically Aligned Latent Actions Enable Dual-Axis Scaling of Embodied Foundation Models
Embodied foundation models hinge on diverse robot interaction data, which are costly to collect. Egocentric human videos offer a complementary source of experience, and latent action models (LAMs) provide a popular means of leveraging these videos for policy supervision. Compared with world-action models (WAMs), LAMs can offer greater training efficiency and potentially lower inference costs by avoiding explicit pixel prediction during policy learning and execution. However, most existing LAMs learn latent actions solely through video reconstruction, which can cause their representations to cluster by visual appearance rather than by the underlying actions. As a result, they struggle to align human actions with their robot counterparts. To bridge the gap, we introduce SALA, which emphasizes establishing the semantic correspondence for latent actions. Without requiring paired data, we use the language descriptions of unpaired human and robot clips to estimate semantic similarities and supervise cross-embodiment correspondence. Specifically, we translate these similarities into soft neighborhood structures that guide human latent actions toward semantically related robot latents, while robot action reconstruction grounds the latent space in control. The resulting shared latent action space provides a unified supervision signal for training an embodied foundation model on both human and robot videos. Notably, latent action supervision enables scaling along two axes: (1) pre-training on increasing amounts of general human video consistently reduces robot action prediction error on downstream tasks, and (2) post-training on increasing amounts of task-specific human video substantially improves real-world robot task success rates.
★ Maximizing LLM Utility While Managing LLM Fallibility
Large language models (LLMs) can supply autonomous systems with plans, predictions, and judgments, but these contributions are sometimes wrong. This review asks how a system can maximize an LLM's utility while managing its fallibility. Our answer is to adapt the system's reliance on the LLM as evidence about the LLM's performance accumulates. Systems built from imperfect components, such as noisy sensors and faulty software, have long worked in this way, and the classical methods they use can be applied to LLM-based systems as well. A system can change its reliance on an LLM in several ways, depending on the LLM's role and on the other components in the system. Each change needs an estimate of the LLM's reliability, and the LLM's role and the kind of contribution it makes determine what to estimate. Classical methods from decision theory, estimation, and online learning then turn these estimates into reliance decisions. These decisions can reduce how often LLM errors affect the system. However, some errors matter even when they are rare. An independently verified mechanism can prevent such errors from being executed. Without one, adapting reliance using reliability estimates can still keep their risk within a bound. Studies on adapting reliance on LLMs are emerging but limited, and the reviewed comparisons suggest that adapting reliance can improve system performance.
★ An Integrated Architecture for Data-Efficient Real-Time Neural Imitation of Nonlinear Time-Optimal Model Predictive Control
Nonlinear time-optimal model predictive control (TOMPC) enables highly performant control of constrained systems, but its computational cost prohibits real-time use on fast dynamical systems. Neural network imitation of the TOMPC control law can reduce online computational costs, yet existing work mostly targets quadratic cost formulations in simulation and addresses the challenges of neural controllers in isolation. This work imitates a long-horizon nonconvex TOMPC with hard terminal constraints, whose high computational cost limits training-data generation. We present (i) \emph{self-informed sampling (SIFS)}, a data-efficient expert-sampling strategy that uses a learned time-to-go to query the expert only where the policy deviates from time-optimal behavior, (ii) a complete training and control architecture that jointly addresses dataset correlation, suboptimal expert labels, covariate shift, constraint satisfaction and model mismatch, and (iii) real-world validation on a Furuta pendulum. SIFS matches standard dataset aggregation performance in roughly one-third of the training time and with less than half the training data. The resulting controller matches the region of attraction of the expert TOMPC while raising the control frequency from 1.4~Hz to 1~kHz. On hardware, the controller swings up and stabilizes the pendulum along trajectories closely matching the expert TOMPC solution.
comment: Submitted to European Control Conference 2027 (Brussels)
★ CT-SGAL-LIO: Causal Continuous Time Refinement of Discrete LiDAR Inertial Odometry ICRA 2027
State-of-the-art LiDAR-inertial odometry methods, including those on richer state manifolds, achieve strong global accuracy, yet their pose stream alternates between propagation and correction and can oscillate over short horizons even when globally accurate. Most existing methods follow these approaches: they either modify the filter to enable faster updates or estimate a continuous-time trajectory from raw LiDAR and inertial data. The latter involves extra optimization, delaying state updates and becoming less robust during rapid rotations or accelerations. We propose CT-SGAL-LIO, a continuous-time refinement module that smooths the posterior pose stream of a discrete SGal(3) estimator without modifying the estimator itself. The key novelty is to perform continuous-time trajectory refinement directly on the projected SGal(3) posterior: we fit a cubic trajectory over a bounded window of recent poses and evaluate it at a small fixed lag. This places the query time within the observed interval, so the refined pose is obtained by interpolation rather than extrapolation, while preserving causal operation. CT-SGAL-LIO improves relative trajectory accuracy on 17 sequences from three public benchmarks. Across 17 sequences from GRAND-TOUR, MCD, and GEODE, CT-SGAL-LIO improves short-term motion accuracy while largely preserving global accuracy. RTE improves by up to 70% on GRAND-TOUR and on all seven GEODE sequences, with 9.9-51.1% reductions. A 19.5-min R-Campus run also reduces posterior jumps by 85.4% with only a 4.7 mm change in the end-to-start gap. We plan to release our source code and results to support community validation and reproducibility.
comment: 7 pages, 4 figures, ICRA 2027 submissions
★ Informative Path Planning with Gaussian Processes: Error Bounds, Algorithms, and Benchmarks
Accurate environmental monitoring relies on Informative Path Planning (IPP) to deploy autonomous systems that maximize information gain. While Gaussian Processes (GPs) provide a robust mathematical foundation for modeling the environment, contemporary IPP approaches remain hindered by heuristic operational constraints and the lack of simultaneous theoretical guarantees for worst-case estimation error and approximation quality. Rather than introducing yet another heuristic, this paper presents a rigorous theoretical analysis of existing error bounds and algorithms to further develop the mathematical foundation for IPP. By evaluating probabilistic and deterministic bounds on reconstruction error, we establish that predictive variance is the theoretically ideal optimization objective for GP-based IPP. Furthermore, we analyze GreedyCover, a recent approach for optimizing predictive variance, and demonstrate that its near-optimal approximation guarantee matches the fundamental hardness-of-approximation limit for the problem. Comprehensive benchmarking confirms that this approach delivers superior estimation accuracy and computational efficiency compared to prominent information-theoretic objectives and optimization methods used in IPP. Finally, this theoretical analysis establishes a rigorous, near-optimal foundation for discrete offline IPP under bounded uncertainty constraints, offering clear and principled guidance for future work in continuous-space and online formulations.
comment: 19 pages, 22 figures, preprint
★ ACE-CaP-0: Real-World Code-as-Policies in the Era of Experience
Recent advances in frontier models have pushed the boundaries of agentic robot control, enabling agents to perform complex robotic tasks through reasoning and interaction. However, repeated calls to these models during execution can substantially increase task completion time. Code as Policies (CaP) offers a route for coding agents to acquire physical skills through experience and retain them in executable programs, allowing the reasoning invested in development to be reused across runs. Developing reliable code policies while limiting the cost of physical experimentation remains a central challenge. We investigate human guidance as a source of priors for feasible actions and simulation as an auxiliary environment for exploring and testing candidate policies. Our findings favor robot demonstrations available on request and simulation used selectively during real-world policy refinement. These findings inform ACE-CaP-0, an agentic recipe that couples policy revision with experience acquisition through Demonstrations on Request and Simulation on Demand. Across eight real-world manipulation tasks spanning alignment, multi-object handling, and bimanual coordination, the resulting frozen programs achieve 92.1\% success (221/240 trials). Mean execution times across tasks range from 0.97 to 2.27 minutes. The programs execute with runtime perception and control, without a human or coding agent in the loop. These results highlight CaP's potential to turn experience into reliable, reusable physical skills.
★ A Class of Bayesian Conjugate Distributions on $SE_n(3)$ for Motion Estimation
This paper proposes a novel class of Bayesian Conjugate Distributions (BCDs) on extended special Euclidean groups SE_n(3) for global motion uncertainty representation and estimation. BCDs are constructed by restricting matrix Gaussian distributions on ambient Euclidean space to SE_n(3), avoiding singularity of local coordinates. The unique feature of BCDs is their conjugacy to a broad class of sampling distributions arising in motion estimation, enabling exact closed-form Bayesian inference. Optimal estimation with respect to BCDs is further investigated for implementing motion estimation. Two means are extracted as random state estimates, which maximize probability and minimize mean squared error, respectively. Moreover, within the BCD class, moment matching is shown to minimize the Kullback-Leibler divergence, and maximum likelihood estimation is also derived, which provide analytical methods to approximate underlying distributions with the optimality guarantee. As an application, a Bayesian estimator for pose and velocity estimation of an Unmanned Aerial Vehicle (UAV) is developed within an assumed density filtering architecture. The propagation and estimate extraction leverage the optimal estimation related to BCDs, and the landmark measurements are exactly fused with the BCD prior in closed form due to the conjugacy. Numerical simulations of a UAV flight indicate the advantages of BCD-based estimators over two classic Kalman filters on Lie groups with various initial uncertainties and measurement noise.
comment: 17 pages, 4 figures
★ ScrewSense: Mapping the Invisible - Anomaly Detection and Error Classification in Screwdriving Processes
The growing volume of process data generated by screwdriving systems, combined with advancements in computing power, data storage, and algorithms, is facilitating the application of artificial intelligence in this field. Accurately detecting and classifying screwdriving errors is crucial for ensuring product quality, avoiding costly rework in manufacturing, and supporting sustainability by facilitating the maintenance, reuse, and recycling of screw-connected products. In this context, machine learning approaches offer a promising alternative to traditional monitoring techniques, such as window-based methods or envelope curves. This study examines the use of deep learning techniques for anomaly detection and error classification in screwdriving processes. The data corpus consists of multivariate time series capturing process parameters, such as torque and rotation angle. For anomaly detection, the embedding space of a vector quantized variational autoencoder was analyzed. In terms of error classification, four different model architectures were evaluated. The results demonstrate the effectiveness of anomaly detection and confirm that deep learning models can distinguish between multiple error categories beyond traditional binary classification. This extends the capabilities of conventional monitoring approaches, achieving accuracies of up to 100%. Additionally, real-world data material was examined in order to further demonstrate model robustness. Overall the results indicate strong generalization capability and highlight the practical potential of the developed models for use in industrial screwdriving processes, eliminating the need for expert knowledge in error identification and significantly reducing manual effort and analysis time.
comment: Accepted manuscript, 59th CIRP Conference on Manufacturing Systems (CMS 2026). 6 pages, 8 figures, 3 tables
★ AgriSense: Vision-Language Decision Support and Robotic Intervention in a Virtual Greenhouse Copy title
Autonomous greenhouse management increasingly requires robotic systems that integrate perception, navigation, environmental regulation, and plant-level intervention. This paper presents AgriSense, a framework combining an Isaac Sim virtual greenhouse, a quadruped mobile manipulator, RGB and LiDAR-inertial sensing, environmental control, and supervised vision-language decision support. Frozen DINOv2 and BERT encoders, together with structured environmental inputs, support joint prediction of twelve environmental references, a target zone, and an intervention flag. Checkpoint replay reproduces 41/43 zone decisions. Across five seeds on a source-grouped test set, the full, image-removed, and language-removed configurations each achieve 43/44 correct zone decisions, providing no evidence of incremental multimodal zone-accuracy improvement. The full model achieves mean absolute errors of 0.249 degrees C for temperature, 2.030 percentage points for humidity, and 0.997 percentage points for soil moisture. Sensor-only inputs match full-model zone accuracy, while language-only inputs achieve 68.64%. External real-image benchmarks reveal distribution-shift limitations. Five offline Point-LIO routes yield rigidly aligned trajectory errors of 5.22-17.32 mm. Of 40 virtual-arm targets, 32 satisfy geometric acceptance criteria and eight are rejected at boundaries. Six long-horizon climate cases and fault protocols characterize integration limits. Navigation and integrated treatment rely on simulator-state inputs; physical greenhouse performance and agronomic treatment efficacy remain unvalidated. The contribution is an evaluated decision-support and robotic-intervention framework for a virtual greenhouse.
★ EgoX: Egocentric Cross-Embodiment Manipulation with Embodiment Dreaming
Generalist manipulation policies can benefit from the complementary experience of humans, real robots, and simulated agents. However, differences in morphology, sensing, and control interfaces make it difficult to learn effectively from their combined demonstrations. We introduce EgoX, an egocentric cross-embodiment learning framework that organizes heterogeneous demonstrations through semantically corresponding observation and action modalities. EgoX pretrains a Modularized Cross-Embodiment Transformer with Embodiment Dreaming (MXT+) on this mixture and fine-tunes it on target-embodiment demonstrations. Within each embodiment, MXT+ assigns a dedicated tokenizer to each observation modality and a dedicated expert to each action modality, with a Transformer trunk shared across embodiments. A training-only embodiment-dreaming objective predicts latent representations of future visual and proprioceptive states, encouraging the shared trunk to model each embodiment's dynamics. Across 13 task-embodiment pairs spanning five tasks and six robot embodiments, including five physical robots and one simulated humanoid, cross-embodiment pretraining raises MXT+'s average normalized task progress from 0.63 to 0.80 over training from scratch. The gains are particularly pronounced under out-of-distribution conditions, where average task progress increases from 0.53 to 0.77. Ablations demonstrate the benefits of broader pretraining mixtures, modular interfaces, and embodiment dreaming, with dreaming producing more similar latent geometry across embodiments. Project page: egox-dream.github.io.
★ Safe and Efficient Navigation Considering Texting Pedestrians via Attention-Aware Cost-map Tuning
Smartphone-distracted pedestrians deviate from social norms unpredictably and cannot be relied upon to yield to approaching robots -- a safety challenge that existing navigation methods are poorly equipped to handle, since they treat all pedestrians uniformly regardless of attention state. We propose an attention-aware navigation framework that classifies each pedestrian as attentive or distracted in real-time using 3D skeletal features and a Random Forest classifier, then dynamically widens cost-map safety margins around distracted individuals to drive proactive avoidance. A within-subjects user study with 12 participants across 252 trials demonstrates significant improvements in perceived smoothness (p = 0.003) and predictability (p = 0.034) over both a LiDAR-only baseline and a trajectory-prediction-only method, without any increase in travel time. Notably, trajectory prediction alone does not improve subjective navigation quality unless coupled with attention-state-aware cost adjustment -- confirming that appropriately responding to who is in front of the robot can matter more than trajectory smoothness alone.
comment: 7 pages, 4 figures, 3 tables. Accepted at the 35th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN 2026)
★ NSDL: Neuro-Symbolic Division of Labor for Multi-Robot Task Planning ICRA 2027
Planning for a heterogeneous robot team from a scene image and an instruction involves two kinds of decisions. Some must be judged from the scene: which goals are already met, how the task splits, and which robot does what. Others follow from these and can be computed: prerequisite actions, handoffs between robots, and timing. We argue that each kind should go to the component suited to it: a small vision-language model learns the first, and a deterministic solver computes the second. In NSDL (Neuro-Symbolic Division of Labor), the trained model writes its judgments as a plan sketch, which lists the goals that still need action and the intended steps in order, each with a robot. The solver expands the sketch into an executable joint plan. The model calls two tools, a detector for evidence and the solver, and is trained with the solver in the loop. We evaluate on the public VIKI-L2 benchmark and on LOCUS-Bench, a new benchmark that grades the two kinds of difficulty separately. NSDL outperforms end-to-end planners and formalize-then-plan baselines on both benchmarks and with three backbones from 3B to 8B; the 7B model reaches 97.0% out-of-distribution accuracy on VIKI-L2. Ordering and scheduling errors nearly disappear once the solver writes the structure; the remaining failures are misjudged states, and most goals the model misses are ones it never lists. Code, model weights, and LOCUS-Bench are publicly available at https://github.com/Xincheng-Tan/NSDL.
comment: 10 pages, 3 figures, 5 tables. Submitted to ICRA 2027
★ Autonomous Mobile Robots on a Computing Field
We introduce CABOT, a computational model for autonomous mobile robots in which the discrete plane the robots operate on is not a passive stage but an active computing medium. In the classical Look-Compute-Move (LCM) model every increase in computational capability is obtained by strengthening the robots themselves, with lights, memory or wider visibility, at a cost paid once per robot. CABOT strengthens the environment instead: each cell of the grid holds a state from a finite set Q, the states evolve synchronously under a uniform local rule that also senses whether a robot stands on the cell, that is, the floor is a cellular automaton, and the Look operation returns the states of the visible cells in addition to the positions of the visible robots. The robots are as weak as we can make them: oblivious, silent, lightless, myopic with visibility range 1, without multiplicity detection and unable to write to the floor. The number of field states |Q|, rather than any robot capability, becomes the measure of the resources required. On an inert floor (|Q| = 1) such robots can neither form a line, nor gather from scattered positions, nor explore a grid. We show that a handful of states changes this. Line Formation from connected configurations is solvable with four states, and field-silently, so that the floor also reaches a fixed point, with five. Gathering from arbitrary, possibly disconnected, configurations is solvable with three states and Line Formation from arbitrary configurations with six; we prove that neither can be solved field-silently with any number of states. A single robot explores every finite grid forever with four states and every torus with two, which is optimal. Further rule sets, validated by simulation only, equalize the spacing of robots on a row, fill a pattern painted on the floor and guide robots through a maze, and indicate the range of the model.
comment: 19 pages, 5 figures
★ Rethinking Actions: Continuity-Aware Priors for World-Model Planning
Actions in expert demonstrations exhibit temporal structure. An ordered-versus-shuffled analysis across five environments confirms this dependence. We ask whether action proposals that better reflect expert continuity can improve visual world-model planning. Rather than imposing generic smoothness, we learn continuations from the temporal structure of demonstrated actions. Our locality-gated deep-kernel Gaussian-process prior, LG-DKGP, combines recent and longer-range executed action history, predicts a neural continuation, and learns a temporal Gaussian-process (GP) residual. A training-derived exploration scale initializes search around the corrected mean; a frozen world model evaluates progress toward the goal. Training first fits the neural mean, then jointly optimizes the prior with the GP residual using expert future actions. The resulting planner achieves 87.07\% macro success across five simulated tasks spanning native and fragment-level goal attainment, compared with 71.03\% for INTACT, 61.66\% for PRISM, and 58.66\% for LeWM. Component interventions and expert-relative action-structure comparisons support demonstration-informed continuations and reveal task-dependent effects of visual conditioning, history length, and planning horizon.
★ Reins: Frontier Multimodal Models Driving Real Robot Arms on Long-Horizon Tasks, Zero-Shot
Frontier models such as GPT-6.1 Sol, GPT-6 Astra and Claude Opus 5.5 can now plan and call tools, yet they still struggle to drive real robots through long-horizon, contact-rich manipulation, blind while the arm moves and while they think. We present Reins, a harness in which an off-the-shelf model operates a dual-arm robot with 3D perception tools and atomic actions that report contact and grip. The model chains actions in one call, each with a condition that must hold, stopping at the first that fails; a second instance, the narrator, watches the cameras continuously and reports what changed with every result. From images, Reins also builds a simulation-ready twin of the scene for continual policy learning. With natural-language prompts and no task-specific code, GPT-6.1 Sol succeeds in 88% of real episodes on five tasks; without the harness, it fails every shell game, taking 8 times as long. Three frontier models perform alike but differ in tracking a scene that changes while the actor thinks. Code will be released upon publication.
★ AstroMemBench: A Benchmark for Memory Systems in Embodied Spacecraft Agents
Long-term spacecraft autonomy requires embodied agents to use operational history under communication delays, time-critical operations, environmental uncertainty, complex physical interactions, power and computational constraints, and safety and reliability requirements. We introduce AstroMemBench and a Scenario-Challenge-Capability design method that maps these requirements to executable memory tests. The benchmark covers 16 memory subchallenges through 140 templates and 2240 instances across nine tasks in planetary surface operations, on-orbit servicing, and deep-space exploration. A deterministic environment scores actions for task completion, state changes, safety, and resource use. Ablations under nine conditions show that the benchmark is sensitive to how accurately agents recover and use historical information and to differences in memory-system design. Main experiments compare six configurations. Mem0, Chain-of-Memory, LightMem, and retrieval-augmented generation (RAG) improve same-session mean scores over native Pi Agent, with no clear advantage of the three memory systems over RAG. After a session restart, Mem0 outperforms RAG by 12.25 points. Benefits favor instruction retention and result reuse over ongoing state tracking and energy reservation. AstroMemBench supports the design, development, and selection of memory systems for embodied spacecraft agents. AstroMemBench 1.0 and its evaluation code are publicly available: https://github.com/flyingView/AstroMemBench-v1.
comment: 31 pages, 4 figures, 19 tables. Includes Appendices A-E
★ STVLA: Spatial-aware and Temporal-memory forVision-Language-Action Models
Robotic manipulation often requires both spatial scene understanding and task history. However, many vision-language-action (VLA) policies rely primarily on current observations, without explicitly integrating spatial structure and task history. Appearance-based memories can be sensitive to occlusion and viewpoint changes, while spatial representations of the current scene alone do not preserve how the scene has changed during task execution. We introduce STVLA, a framework that incorporates features from a pretrained visual geometry foundation model into temporal memory alongside visual representations. This allows the policy to recall past scene structure as context for current action decisions. To select informative history, STVLA uses a mathematically grounded scoring rule that balances relevance to the current observation with complementary evidence from memory. Across evaluations on LIBERO, LIBERO-Mem, MIKASA-Robo, and SimplerEnv Bridge, STVLA achieves the highest average success rate among the compared methods on each benchmark. On real-world block stacking, object organization, and phone packaging, it improves success over $π_{0.5}$ by $58.33$, $30.00$, and $35.00$ percentage points, respectively. These results support integrating scene geometry into temporal memory to improve manipulation requiring both spatial precision and task history. Project website: \href{https://stvla.github.io/}{https://stvla.github.io/}
★ Input-output characterization of robotic grasping: Connecting design parameters to grasp behavior through the admittance
Low-impedance grippers make the dynamics of a grasp matter. Current grasp metrics, however, are static: they describe how a held object resists constant loads, not how it responds to time-varying ones. To quantify this, we rely on small-signal analysis to characterize a grasp by its admittance, the transfer matrix relating load perturbations to object displacements around an equilibrium. This transfer matrix recovers the stiffness and stability of the grasp at zero frequency and captures its natural frequencies and peak gain under time-varying loads. We apply this characterization to a planar two-finger gripper under proportional-derivative control. We show that two grasps of the same object in the same position and with the same contact forces can differ in stiffness by two orders of magnitude. Likewise, two grasps that share the same stiffness can exhibit very different resonances. Nonlinear simulations confirm the effectiveness of our small-signal analysis in capturing the dynamic behavior of grasping.
★ Robots That Talk Too Much and Don't Listen Enough
Until recently, the biggest obstacles to natural spoken interaction with social robots were speech recognition errors and turn-taking failures. Advances in speech recognition have enabled much better spoken interaction, but turn-taking failures and other misunderstandings continue to be far too prevalent. This short paper looks back at earlier methods for improving speech recognition and turn-taking in interaction with social robots, and then looks at what can be done today, arguing that multimodal bidirectional streaming is needed to reduce the frequency of turn-taking failures.
comment: 4 pages, 3 figures, presented at EMF Workshop, HRI 2026, Edinburgh, Scotland
★ RGB-D SLAM on the Edge: Benchmarking and Profiling RTAB-Map and ORB-SLAM3 on the Raspberry Pi 5
Micro aerial vehicles that operate without GPS often run SLAM on ARM-based edge boards, yet there is little published data on how standard RGB-D SLAM systems behave on such hardware. We benchmark RTAB-Map and ORB-SLAM3 on the Raspberry Pi 5 using six TUM RGB-D sequences at three image resolutions, recording tracking coverage, trajectory error, time per frame, memory, CPU load and temperature for every run, and we profile both systems at the stage and function level. RTAB-Map was configured to process and keep every frame, so its back end runs on every frame rather than at its default rate of 1 Hz. On the sequences that every configuration tracked, its front end takes 38-74 ms per frame and its full pipeline 150-462 ms, compared with 13-45 ms and 26-62 ms for ORB-SLAM3. RTAB-Map's main cost is optimizing a pose graph that keeps growing, because its working memory is unbounded by default; ORB-SLAM3's local bundle adjustment costs about as much as its tracking but runs in a parallel thread. At quarter resolution, both systems lose tracking on most sequences, and runs that lose tracking finish sooner and report lower error on the poses they keep, so time and accuracy must be read alongside coverage. Timing is repeatable to within 1-5%, but trajectory error varies by up to 27% between identical runs, so the accuracy differences between the two systems are inconclusive. No thermal throttling occurred.
★ HALO: Human-Aligned Data Collection for WhoLe-body MObile Manipulation
Learning whole-body mobile manipulation requires demonstrations that coordinate bimanual actions with locomotion across extended workspaces. However, existing interfaces trade off direct robot control, perception and action alignment, and natural operator mobility, making such demonstrations difficult to collect. We introduce HALO, a human-aligned data collection system for whole-body mobile manipulation. The system aligns the operator's perception and action space with the robot while preserving direct physical feedback and natural locomotion. Operators guide the robot end-effectors through kinesthetic teaching and move a low-resistance platform without the deployed mobile base while performing bimanual manipulation. This interface enables more natural whole-body demonstrations, but the close human-robot coupling introduces discrepancies between the demonstration and deployment settings. We address these discrepancies with deployment-aligned demonstration processing, which calibrates and temporally aligns platform motion with the deployed base response and masks and inpaints demonstrator hands in recorded observations. Compared with Mobile ALOHA and VR teleoperation, HALO enables up to 2.9x faster demonstration collection while maintaining comparable or higher success rates. Policies trained with HALO perform precise manipulation tasks, including zip-tie fastening, and achieve a 73.3% success rate on a six-stage long-horizon mobile manipulation task involving repeated coordinated base and arm motion. Project page: https://robbyant-research.github.io/Lingbot-ALOHA/
★ VOKE: Visual Object Kinematics Estimator for Dexterous In-Hand Manipulation ICRA 2027
Vision-based dexterous in-hand manipulation requires knowing not only where an object is, but also how it moves. However, most robotic manipulation systems rely on off-the-shelf visual perception models whose outputs are limited to static object masks, geometry, or 6D poses, leaving downstream controllers to infer object dynamics implicitly from noisy visual observations. We present VOKE, a kinematics-aware perception framework that augments existing object and hand-object pose estimators to recover a complete object kinematic state comprising pose, linear and angular velocity, and linear and angular acceleration. VOKE contains two core perception modules and a downstream application study. First, a Pluggable Kinematics Module (PKM) attaches to the output of a standard pose estimator and predicts the object's linear and angular velocity and acceleration from temporal visual features. Second, a Kinematics-guided Pose Refinement Module (KPRM) converts this motion vector into a second-order, motion-prior reference pose and refines the per-frame pose estimates. We evaluate VOKE on DexYCB and OakInk, where it consistently improves object pose estimation across three different backbones and substantially reduces velocity and acceleration estimation error compared with pose differencing. As a downstream application, we use Kinematics-Augmented Reinforcement Learning (KARL) to compare a kinematics-augmented policy with a pose-history baseline. After training the policies in simulation, we transfer the kinematics-augmented policy to the physical robot without policy fine-tuning, demonstrating the complete perception-to-control pathway in the target domain.
comment: 8 pages, 4 figures, 6 tables. Submitted to the 2027 IEEE International Conference on Robotics and Automation (ICRA 2027). Project page: https://voke-project.pages.dev/
★ BlinkMimic: Training a Humanoid Motion Tracker in Minutes on a Single GPU
Whole-body motion tracking lets a humanoid robot follow reference motions, but training a tracking policy for a library of motions has usually taken hours of GPU time. SuperMimic showed that the policy can instead be trained by backpropagating through a world model learned from parallel simulation, and InstantTrack added Muon updates, a residual policy around the reference and an averaged deployed policy. We build on both and present BlinkMimic, a recipe that makes this training faster. On a simulated Unitree G1, BlinkMimic trains one policy for all 40 LAFAN1 clips in 3.4 minutes on a single RTX 4090, measured until three consecutive checkpoints reach a normalized return of at least 0.82 with no failed clip (mean of three seeds). We split training time into the cost of an iteration, the iterations needed to fit the motions, and the iterations needed to remove the last failures, and assign each component, inherited or new, to one of these terms. The new components are a step size that decreases once training has progressed, resets directed by failures of the deployed policy under deployment conditions, and GPU graph capture of the whole iteration. A leave-one-out ablation with three seeds per component shows which term each component affects, and further measurements of the gradient, the update rule and the failure cases explain these effects. Code and per-seed results are available at https://github.com/fuyichen0725/blinkmimic.
comment: 18 pages, 5 figures. Code: https://github.com/fuyichen0725/blinkmimic
★ I See What You Want: Decoding Hierarchical Operator Intent from Visual Teleoperation History
Vision-language models (VLMs) have become a powerful foundation for robotic scene understanding, task planning, and language-conditioned control. A natural question follows: can these models also decode what a human operator intends while teleoperating a robot, without any explicit instruction? We formulate this as hierarchical intent prediction, jointly inferring the ongoing atomic action and the broader mission it serves from a short visual teleoperation history. To study this, we introduce VI AMO (Vision Intention Action and Mission Ongoing), a dataset of 6,600 episodes from eleven robot sources with 13,653 temporally aligned action annotations and 9,702 mission annotations. Large proprietary models struggle with this problem: on held-out episodes, GPT-4o reaches Sentence-BERT similarities of only 0.563 and 0.470 for action and mission, and GPT-5.6 Luna achieves only 0.581 and 0.554. Fine-tuning a VLM on VI AMO yields our Vision-Intention Model (VIM), which raises these scores to 0.825 and 0.821 and still reaches 0.726 and 0.717 on the 35.4% of test actions with both labels absent from training. Finally, as a downstream application, VIM's predicted mission enables a vision-language-action (VLA) policy to take over from the operator mid-task in a real-world environment unseen during VIM fine-tuning, reducing human teleoperation time by 34.6% relative to GPT-4o and 39.8% relative to GPT-5.6 Luna, while cutting intent-prediction latency by roughly 70% on average.
comment: 8 pages, 6 figures. Project page: https://see-want.github.io/
★ Remember What Changes the Odds: Critic-Gated Event Memory for Long-Horizon Manipulation
In long-horizon manipulation, a robot may need to take different actions under identical observations, depending on task progress and the outcomes of earlier attempts. We refer to this ambiguity as decision aliasing. Resolving it requires the policy to maintain memory of its execution history. To reduce the cost of processing full observation histories, recent methods build compact event memories using perceptual, semantic, or kinematic cues. However, these criteria may miss events that matter for task success. We instead propose an outcome-grounded principle: remember events that substantially change the predicted probability of eventual success. Following this principle, we introduce CGEM, a critic-gated event memory that uses a critic to select events with large estimated changes in success probability. Positive and negative changes are encoded as progress and failure records, respectively. Failure records are discarded once the corresponding subtask is completed. In a ten-subtask diagnostic environment with exactly computed success probabilities, CGEM achieves 88.0% success, compared with 0.0% for both a 36-step observation window and a kinematic event memory. Removing failure records reduces success to 1.7%.
comment: 10 pages, 6 figures
★ ManiLadder: Benchmarking Robot Manipulation Through a Categorized and Multi-Difficulty Level Task Ladder
Recent advances in robotic manipulation have enabled increasingly complex and dexterous behaviors, yet existing benchmarks still focus on simple pick-and-place tasks and lack a systematic framework for measuring progress in robot learning along the axis of task difficulty. We present ManiLadder, a large-scale and organized simulation benchmark for robotic manipulation. ManiLadder contains 112 carefully designed tasks stratified into 4 difficulty levels, with 8 categories in each level, spanning rigid-body and deformable-object manipulation, single-arm and dual-arm settings, and gripper and dexterous-hand end-effectors. To calibrate difficulty, we fix the data amount the same for every task and use a unified flow- matching imitation learning policy under the same training and evaluation protocols, then use its evaluation success rate as an operational empirical difficulty signal. All tasks are iteratively refined until they fall into target difficulty ranges, which are then verified across different random seeds, policies, and agentic learning systems. We further analyze the task properties, compare different policy performances and training settings, analyze task transfer and data scaling behaviors, and conduct real-world experiments showing that the designed tasks preserve their difficulty ordering beyond simulation. ManiLad- der provides a calibrated ladder for measuring, comparing, and advancing robotic manipulation capabilities. More results are available at https://maniladder-benchmark.github.io/.
★ MotionSkill: From Bipedal Motion Video to Control across Embodiments
Reusing a demonstrated gait can reduce the work of specifying the same motion for each biped. A video captures the timing of a step, but each robot must execute it with its own limbs and actuators. MotionSkill recovers a timed reference once from monocular video and adapts it to each target in simulation. It aligns root motion and body landmarks in a common frame and records which landmarks are available at each instant. At each source timestamp, the mapper preserves limb directions, uses the target segment lengths, and solves inverse kinematics under joint limits and closure constraints. In MuJoCo, we train a separate residual policy for each of ten bipeds on its mapped reference from one four-second walking clip. Each policy uses the measured robot state to correct joint commands during contact. From initial states held out during training, MotionSkill completed 98% of trials on the original video and 26% with the upper body occluded for one second.
comment: 8 pages, 7 figures, 5 tables. Fortune AI technical report
★ Memory-Efficient Parallel Policy Learning for Collapse-Reactive Retreat via Hybrid Terrain Modeling
At disaster sites and on unstable slopes, robot contact can collapse uninspected terrain, requiring retreat upon support loss. Massively parallel reinforcement learning of collapse-reactive retreat requires independent state--action experiences. However, on shared rigid-body collapsible terrain, one robot's collapse alters others' contacts without direct interaction. We call this indirect interference. Duplicating terrain objects prevents this but is memory-intensive, limiting parallel sampling. Our key insight is that policy learning does not necessarily require rigid-body collapse modeling, which is feasible but computationally expensive; collapse-induced contact responses can suffice. We therefore propose a memory-efficient parallel policy-learning framework for collapse-reactive retreat via hybrid terrain modeling. Hybrid terrain modeling combines rigid-body simulation of non-collapsible terrain with virtual contact forces for collapsible regions. A State-Multiplexed Contact Model shares terrain geometry and a contact-response law while maintaining environment-specific collapse states, enabling independent experiences without terrain objects for collapsible regions. The framework trains a proprioceptive policy for both normal locomotion and collapse-reactive retreat. It supports over 100 times as many parallel environments as fine-resolution rigid-block terrain within the same GPU memory limit. This scalability and improved sample efficiency enable over 50 times faster policy convergence with comparable retreat performance. Sim-to-sim and real-robot experiments demonstrate retreat under multiple collapse conditions. A supplementary video is available at: https://youtu.be/_RrhwjzEPE0
comment: under review
★ Alloy: A Multi-Material Physics Engine for Robot Learning
Robots handle much more than rigid objects: towels, cables, water, sand. The simulators used to train robots at scale are strongest on rigid bodies, and adding another material usually means a second engine and hand-written glue between the two. We present Alloy, an open-source physics engine for robot learning in which rigid robots, cloth, cables, liquids, granular material and soft bodies share one scene. Each object is described once; Alloy assigns it to a solver suited to its material, couples the solvers inside every control step, and runs the scene in many parallel worlds on one GPU, or on a CPU. Every world carries a health grade, to help tell a failure of the physics from a failure of the policy. Alloy is built on Newton, MuJoCo Warp and Warp, and it is written to be driven by coding agents: its documentation ships inside the package, every error names its fix, and the engine is exposed as tools over the Model Context Protocol. In a cold test, agents that had never seen Alloy installed a pre-release from the package index and trained policies for walking, cable handling and pouring on a single GPU. We also report how throughput grows with the number of worlds, depth-camera throughput, agreement with closed-form references, and a reliability test over a thousand worlds.
comment: 10 pages, 4 figures, 4 tables. Code: https://github.com/argonrobotics-ai/alloy
★ Hibikino-Musashi@Home 2026 Team Description Paper
This paper presents an overview of the integrated technologies developed for the Open Platform League at RoboCup@Home 2026 in Incheon. A vision system integrated with learning-based, training-free models enables robust object recognition for both known and unknown objects. We introduce a RoomPlan-based method that semi-automatically registers furniture geometry and labels to generate semantic maps. Additionally, a large language model-based planner interprets the instructions recognized by Whisper and generates primitive skill sequences for execution. A reservoir-computing framework serves as a lightweight temporal-processing model for action recognition. Its field-programmable gate array-based implementation enables low-power, real-time operation. This is key to removing operational time constraints for service robots. Open-source releases, including task environments, dataset tools, and navigation frameworks, enhance reproducibility, reuse, and continuous progress within the RoboCup@Home community.
★ Robot Navigation and Obstacle Avoidance from Randomly Positioned External Cameras
We study robot navigation in indoor environments from fixed exocentric cameras (infrastructure-mounted, external to the robot). We show that standard Proximal Policy Optimization (PPO), which succeeds with onboard cameras, fails to learn this navigation end-to-end in our setup. This failure results from the perspective mismatch between exocentric observations and egocentric action and goal formulations, which is too complex to resolve from the reward signal alone. To address this, we propose Viewpoint-Invariant Navigation from Exocentric Cameras (VINE). VINE uses an obstacle map estimation network, trained with supervised learning, that takes exocentric images, camera pose, robot pose, and room dimensions as input and produces a viewpoint-invariant obstacle map. This map can be used either by a PPO policy or by a planner such as A*. VINE's map estimation performs better than perspective mapping approaches, handles cameras placed across a wide range of positions and orientations without view-specific tuning, requires only approximate camera pose estimates (noise with a standard deviation of 30cm per axis in x, y, and z and about 20° of rotation error on average), is robust to intrinsic variation, and extends to two-camera setups with partial coverage. Despite operating from cameras randomized in position during evaluation, VINE surpasses the navigation success rate of PPO with an onboard camera in simulated environments.
comment: 8 pages, 2 figures, accepted as a regular paper for the 2027 IEEE/SICE International Symposium on System Integration
★ Demo: Matter-Based IoMT Telemetry and Security Analysis
Connected health devices increasingly support health monitoring outside clinical settings, making secure and accessible physiological data important for patients and caregivers. However, device-specific applications and interfaces complicate the integration of readings from different devices. The Matter standard provides standardized device commissioning and protected communication, but its cluster library does not represent the health attribute measurement. We present a preliminary demo study of a Matter-based IoMT testbed with a specific HealthVitals cluster. Implemented in the connectedhomeip SDK on a Raspberry~Pi~5, the cluster exposes simulated heart rate, blood oxygen saturation, and body temperature. We examine commissioning, attribute retrieval, communication under simulated packet loss, and operational traffic through a reference controller. The study demonstrates that this custom telemetry model can operate within the tested Matter configuration without modifying commissioning or secure-session mechanisms. The resulting testbed gives researchers and device developers a concrete platform for investigating health-oriented extensions. It also supports exploration of local telemetry integration that could help patients and caregivers access readings across connected devices.
★ Direct-Ink-Writing of Soft Thin Film Force-Sensitive Resistors with Velostat and Printed-Carbon Sensing Elements
Flexible resistive force sensors are useful for wearable, rehabilitation, and human--machine interfaces, but their performance depends on sensing-material architecture and fabrication. This work presents a parameterized Voltera NOVA-based direct-ink-writing workflow using printed silver electrodes with multilayer Velostat sensing elements on PET and TPU substrates, together with an exploratory fully printed carbon architecture. Velostat sensors containing 1, 2, 4, and 6 sensing layers exhibited strongly nonlinear force--voltage responses, with the greatest sensitivity at low forces. Best-fit straight-line $R^2$ values ranged from 0.475 to 0.641, area-based hysteresis from 3.90\% to 7.47\%, and 5-min voltage drift from 1.9\% to 7.3\%, with no consistent monotonic relationship between layer count and these metrics. Selected PET6 and TPU6 devices showed less than 0.05~V change in peak output between the first and final 100 cycles of a 10,000-cycle test. Printed-carbon devices demonstrated the feasibility of a more fully printed architecture, but configurations with six or fewer layers saturated below approximately 0.5~N or were nonfunctional, while the eight-layer device responded primarily below approximately 2~N. These results establish the feasibility of the printed-electrode Velostat workflow while identifying printed sensing materials as a key area for further development.
comment: 12 pages, 18 figures, to be published in: IEEE Sensors
★ What Can a VLA Learn from the Agent That Calls It? Task-Level Experience Distillation for Embodied Navigation
Navigation agents use a large language model (LLM) planner to call a vision-language-action (VLA) model. Each episode needs the LLM, adding cost and delay. We distill successful agent experience into the VLA instead of distilling the planner. The VLA then navigates alone at test time. Agent sessions are not VLA demonstrations. They include motions driven by planner tools and sub-instructions, along with detours. We present H2E, a task-level experience distillation method that converts these sessions into VLA training data. Every motion becomes a VLA action under the original task instruction, with the resulting poses as targets. H2E masks the loss on steps that move away from the goal. It labels every visited state within 1 m of the goal as a stop. We then update the VLA with low-rank adaptation. The VLA learns to stop at the goal and find goals the base model never reaches. On all 3,189 REVERIE-CE val-unseen episodes, H2E raises the success rate of an 8B VLA from 35.0% to 44.9%.
comment: 11 pages, 5 figures, 9 tables
★ BWM-Forge: Agentic World Programming for Embodied Intelligence
Generating diverse robot videos requires control over scene geometry, robot motion, and viewpoints. End-to-end video generation offers limited direct access to these variables, making targeted edits and reuse across tasks difficult. We present BWM-Forge, a code-driven framework for controllable video generation in robotics. Given robot assets and a task description, the framework accepts an initial image, an image with motion data, or a human demonstration video. A coding agent constructs editable Blender scene programs that specify objects, robot motion, and cameras. The resulting proxy videos provide structural and motion references for a video generation model, which adds visual detail. A multi-agent harness coordinates input processing, scene construction, review, and rendering. Automated evaluation and user feedback support iterative refinement, while an experience repository stores and retrieves reusable modeling and rendering rules. This design supports multiple robot embodiments, synchronized viewpoints, and appearance variation from a shared scene program. We evaluate the framework on 15 tasks covering five robot types. Compared with LingBot-World Base Cam on these tasks, BWM-Forge achieves higher aesthetic quality, imaging quality, subject and background consistency, motion smoothness, and RoboJudge-9B physical adherence and instruction alignment scores. The baseline scores higher in dynamic degree. Qualitative cases illustrate viewpoint control, scene variation, feedback-driven edits, and experience reuse across scenes. These results demonstrate a practical approach to generating controllable robot videos through editable, executable scene representations.
comment: Technical Report
★ A Coupled Control Barrier Function Approach to Deadlock-Free Multi-Robot Conflict Resolution
Many robotic applications involve scenarios where multiple robots interact and the decisions of one robot influence the behavior of others. This paper presents a quadratic programming (QP)-based feedback controller for multi-robot conflict resolution with input constraints. Safety is enforced using coupled Control Barrier Function (CBF), while a yielding matrix encodes how robots influence each other. Unlike existing methods with fixed or hard priorities, the proposed framework enables yielding preferences while guaranteeing deadlock-free operation by design. Sufficient conditions are established for the controller to be locally Lipschitz. The optimal CBF sign- indefinite slack is shown to admit a finite uniform bound, which is compensated through an enlarged CBF safety margin to obtain a forward invariant set contained within the desired safe set. The analysis also identifies an irreducible slack component due to finite maximum deceleration that cannot be eliminated by increasing the slack penalty. Finally, sufficient conditions on the yielding matrix and nominal controller are derived to exclude deadlock states. Simulations demonstrate adaptive coordination across yielding preferences and robustness to robot failures.
★ Visibility-Aware Task and Motion Planning with Task-Centric Perception
Long-horizon manipulation under uncertainty remains a challenging problem in robotics. We present \ourmethod{}, an algorithm for Visibility-Aware Task and Motion Planning. \ourmethod{} extends existing deterministic task and motion planning solvers to partially observable object rearrangement tasks, while still being able to take advantage of the efficiency and compositional generalization over long horizons of the base solver. This extension interleaves sensing actions into a plan skeleton that assumed full observability, and uses a novel persistent object-based memory to fuse observations into object mesh hypotheses and an observed occupancy map. We evaluate \ourmethod{} on real-world tabletop rearrangement tasks which feature heavy occlusion and partial visibility and demonstrate that it outperforms baselines that do not perform active perception or have persistent memory.
★ QVS-LIVO RiskMap: Health- and Information-Aware View Selection for Multi-Camera LIVO and Off-Road Risk Mapping
Multi-camera observations in off-road environments vary significantly in reliability, posing challenges when complementing degraded geometric constraints. Motivated by this, we propose Quality-aware View Selection Lidar-Vision Odometry Risk Mapping (QVS-LIVO RiskMap), which decouples visual processing for state estimation from that for semantic mapping. The front end evaluates camera observations through read-only pretracking on a shared lidar-inertial posterior, combining observation health with geometric complementarity. Channel health memory and switching hysteresis govern selection of at most one eligible view for direct photometric correction. The posterior is retained when no view is eligible or the correction fails acceptance. The mapping back end integrates semantic observations from all available views in a shared world grid. Vehicle-scale geometry constrains semantic risk corrections, while spatial and temporal evidence supports risk confirmation. The system also produces a local bird's-eye-view (BEV) risk map and a global 2.5D semantic-terrain risk map with obstacle instance voxels. On two field sequences, QVS-LIVO achieved horizontal root mean square errors of 0.910 m and 1.664 m. It reduced error by 43.3% relative to FAST-LIO2 on the degraded long-route sequence. Across 12 controlled camera-fault configurations involving dropout, black frames and 60 ms timestamp shifts, it maintained complete trajectories with errors of 1.54-2.23 m. When deployed onboard an NVIDIA Jetson AGX Orin, the system generates local and global maps at 9.94 Hz and 2.00 Hz, respectively, with a median local mapping latency of 32.4 ms. These runtime results demonstrate the real-time feasibility of selective visual state updates alongside multi-view semantic mapping under challenging off-road conditions. The code and related resources are available at: https://github.com/zihanPan-HIT/QVS-LIVO.
comment: 17pages,9 figures
★ PERA: Plug-in Experts for Robotic Actions in Continual Whole-Body Control
Motion imitation enables humanoid robots to acquire whole-body skills such as walking and dancing. However, fine-tuning an entire policy for a new skill can affect existing behaviors, while training a separate policy for each skill increases training and maintenance costs. We present PERA, a method for learning new skills by reusing existing motion capabilities. PERA freezes a pretrained mixture-of-experts policy and trains an additional task expert and its routing score using task rewards or teacher-action supervision. In Unitree G1 simulation, each added task module contains approximately 1.03% of the parameters of the base actor. Built on our pretrained policy, PERA successfully completes all ten new tasks. Disabling the expert restores the original computation path of the base policy, which still fully executes all 126 original motions. Enabling the expert also improves the execution of some dance motions not used to train that expert, indicating that its benefits extend beyond its training motions. Experiments with HoloMotion further demonstrate the applicability of PERA to different base policies.
comment: 9 pages, 5 figures, 3 tables
★ X-Nav Technical Report: Real-World Long-Horizon Indoor-to-Outdoor Navigation on a Humanoid Robot
Long-horizon navigation from indoors to outdoors remains an open problem for humanoid robots: one instruction covers rooms, doors and roads, and the camera moves with the gait. Worken, AGIBOT's measure of productive robot work, depends on robots that reach their workplaces autonomously. We present X-Nav, to our knowledge the first system to take a bipedal humanoid through a language-instructed long-horizon mission from indoors to outdoors. X-Nav covers a pipeline from data to deployment. Its data engine collects navigation data in open-source scenes, in digital twins of deployment sites and on real robots. Its navigation model, MELIOR, takes only RGB images from the robot's built-in camera and comes in two versions: MELIOR-A predicts actions, and MELIOR-N predicts a navigation goal. Its agentic workflow, X-Navarch, turns a brief human instruction into sub-tasks and deploys MELIOR on the robot with explicit target navigation and obstacle avoidance. On the Lingxi X2, X-Nav completes a mission of about a hundred metres from an office lobby to a store in a neighbouring building; LiDAR and odometry are used for target approach and low-level control. The mission is completed with both versions of MELIOR: the robot follows a change of destination and walks through a door from the lobby to the road, and with X-Navarch it also passes static and dynamic obstacles on the route. During deployment, MELIOR is adapted with data collected in a digital twin of the site; the adaptation raises its closed-loop success rate in the twin, and the adapted model is then deployed on the robot at the real site. In addition, MELIOR achieves competitive results on the R2R-CE and RxR-CE benchmarks.
★ Natural Swing-Up of an Underpowered Double Pendulum
Robots with weak motors must ride their natural dynamics rather than fight them. We swing up a direct-drive planar two-link robot, whose motors deliver only about 6\,\% of its shoulder gravity torque, along its nonlinear normal modes, two-dimensional invariant manifolds consisting of periodic orbits. On a non-strict mode, injecting energy necessarily moves the state off the manifold, so pumping and stabilization compete for the same bounded torque. We split the input torque into a power-neutral steering component and a scalar pumping input, the only component that changes the energy. The allocation then becomes a linear program in two variables: maximize pumping subject to the torque limits and one control-Lyapunov constraint. The constraint uses the model-based rate of the error, so no stabilizing feedback gain is needed. Experiments in hardware show swing-ups along both modes and compare the allocation with summing-and-clipping and with prioritized stabilization.
★ Grounding Object-to-Object Hazard Reasoning into 3D Risk Fields for Task-Conditioned Safety
Safety in robotics does not solely depend on geometric constraints, but also on semantics: the given task, environment properties, and interactions between objects in the scene. We present a framework based on vision-language models (VLMs) to infer task- and scene-dependent safety constraints for robot manipulation. We provide the VLM a structured safety tool based on an RGB-D observation and a natural-language task instruction that first decides whether the task should be rejected since its execution may result in a non-recoverable hazardous outcome. Otherwise, it identifies preventable object-pair hazards and specifies the spatial relations that must be avoided. Open-vocabulary segmentation and depth information are then used to ground the detected objects in 3D and construct continuous risk fields that represent the corresponding hazardous regions. These fields are integrated with an obstacle avoidance module that corrects nominal robot motions during execution while accounting for the risk field. We evaluate the framework on preventable hazards, tasks requiring rejection, and safe variants in simulation and the real-world. Project website: https://vlm-hazard-riskfield.github.io
★ Talk to Act: A Synthetic Multi-Turn Dialogue Benchmark for Human Intent Extraction and Action Planning in Robotics ICRA 2026
Multi-turn conversational human-robot interaction (HRI) continues to be a challenging problem in robotics, as many existing systems rely on rigid, command-driven interfaces that can be unintuitive for users. Recent advances in Large Language Models (LLMs) provide a promising direction, given their ability to process natural language, generate structured outputs, and support tool-based reasoning, all of which are relevant for interpreting user intent in robotic settings. However, progress in this area is constrained by the limited availability of datasets and benchmarks that capture the nuances of real-world human communication, such as multi-turn dialogue, ambiguity, and context-dependent requests. In this work, we present a synthetically generated multi-turn dialogue dataset for HRI, annotated with action intents, object references, and task plans across a range of scenarios and user personas. Our primary focus is on evaluating the capability of small, open-weight language models for robotic tasks, particularly in settings where local deployment and efficiency are important. To this end, we frame intent extraction as a tool-calling problem where each predicted action corresponds to a structured function call with arguments grounded in the dialogue and benchmark models ranging from 270M to 8B parameters on their ability to perform intent extraction and action planning without task-specific fine-tuning. Our results show that these models can achieve reasonable performance on the task even at relatively small scales and without any task-specific fine-tuning. This suggests that compact, locally deployable models already provide a useful baseline for conversational robotic systems. We expect that task-specific fine-tuning and more realistic training data will enhance performance further and make such models increasingly suitable for real-world robotic applications.
comment: 6 pages, 4 figures. This is an extended version of our paper accepted at the ICRA 2026 Workshop on Multimodal Embodied Interaction in Robots (MEI-2026), Vienna, Austria
★ Learning Latent Goal Costs for Visual World-Model Planning with an Expanding Curriculum
Image-goal planning with a frozen visual world model depends on how predicted observations are scored against the goal. Mean-squared error (MSE) between embeddings from a pretrained visual encoder can provide a useful signal near a goal, yet favor states across an obstacle that are visually similar to the goal and difficult to reach. We fit a small residual adapter on top of frozen I-JEPA embeddings to improve this latent goal cost. Starting near each goal, a planner collects trajectories from progressively wider regions of initial states. The adapter learns from the order of observations along successful trajectories, with additional comparisons inferred from failed rollouts. No task-specific progress labels are used for fitting, so the procedure can be applied to other goal-reaching tasks. DotWall and PointMaze are used in this work. In paired 50-case evaluations with identical starts, goals, world models, and planner settings, success increases from 54% to 98% in DotWall and from 82% to 96% in PointMaze. On 16 unseen DotWall layout-goal cases, the adapter fitted on one fixed layout improves agreement between latent-cost ordering and shortest legal-path ordering from 71.9% to 85.8%. The results indicate that planning experience can improve a frozen visual representation's goal cost without retraining the encoder or the world model.
★ Racing the Clock: Teaching VLNs Budget-Conditioned Object Search
A rescue robot entering a building with thirty seconds on the clock should sprint toward the single most promising room; given ten minutes, it should instead explore a broader tour. Yet vision-language-navigation (VLN) models are typically evaluated under a fixed episode limit that the policy never observes, even though time-critical missions require behavior to adapt to the available budget. We study budget-conditioned object search: given a target category and an explicit action budget, an agent must find as many distinct target instances as possible before the budget expires. We investigate how to supervise VLNs for this task by comparing a privileged TSP expert, which plans near-optimal routes using the full map and true target locations, with a room-level Bellman expert that maximizes expected return under partial observability using high-level cross-scene semantic priors. Despite the TSP expert achieving substantially higher expert performance, its demonstrations are less effective for learning exploration: on held-out HM3D scenes, the Bellman-supervised VLN consistently outperforms the TSP-supervised VLN, with the advantage becoming more pronounced at longer budgets. At 2000 steps (500m), the Bellman-supervised VLN finds 71.4% of reachable target instances, compared with 65.5% for the TSP-supervised VLN and 44.8% for the pretrained VLN. These results highlight a distinction between optimal routing and optimal exploration: effective exploration supervision should optimize decisions under partial observability rather than imitate routes that are optimal only once the environment is fully known.
comment: 12 pages, 4 figures, 3 tables
★ Agentic-Teacher: An Agentic Data Engine for Failure-Driven VLA Post-Training
Improving a pretrained vision-language-action (VLA) model requires demonstrations that address its weaknesses and provide effective supervision for learning. Coding agents can autonomously generate demonstrations through environment interaction, yet successful execution alone does not ensure effective supervision for VLA learning. We introduce Agentic-Teacher, an autonomous data engine for failure-driven VLA post-training. Given a VLA checkpoint, a task specification, and configured environment tools, the engine analyzes failed rollouts, selects intervention states, and iteratively develops closed-loop code policies through execution feedback. Validated policies become reusable demonstration generators that collect successful continuations from states where the VLA struggles. To make these demonstrations suitable for learning, Agentic-Teacher introduces temporally correlated action noise to broaden state-action coverage, removes idle frames while preserving gripper transitions, and filters trajectories using the frozen base model's flow-matching loss. The resulting demonstrations are combined with successful VLA rollouts for post-training, yielding an updated policy that operates without coding-agent assistance. Agentic-Teacher thus connects the weaknesses of a target VLA to both the generation and curation of its training data. In simulation, it improves $π_0$ success rates from 79.4% to 87.8% on LIBERO-10, from 80.4% to 90.6% on MetaWorld-10, and from 0% to 30.0% across three RoboCasa tasks.
♻ ★ A Tutorial on Gaussian Process Learning-based Model Predictive Control
This tutorial provides an accessible and implementation-oriented introduction to Gaussian process learning-based model predictive control (GP-MPC), which combines probabilistic residual modeling with receding-horizon control. Its central tutorial contribution is a detailed, step-by-step derivation of multi-step mean and covariance propagation for GP-augmented prediction models. The derivation shows how commonly used propagation formulas follow from the laws of total expectation and total covariance, and clarifies the roles of the GP posterior mean, GP posterior covariance, uncertain inputs, and query-output cross-covariances. Building on this foundation, the article distinguishes mean-only unconstrained GP predictive control from uncertainty-aware constrained GP-MPC, clarifies regulation and output-tracking formulations, and provides concise implementation and computational-complexity guidance, including practical software-tool references and discussion of the scope of closed-loop guarantees. Mobile-robot path-following examples illustrate mean-only unconstrained implementations, whereas mixed-vehicle platooning illustrates uncertainty-aware constrained GP-MPC with uncertainty-dependent safety-distance tightening. The tutorial is intended to help researchers and practitioners understand, implement, and critically evaluate GP-MPC designs for robotic and autonomous systems operating under modeling uncertainty.
comment: Author's accepted manuscript
♻ ★ Reactive Task-Oriented Robot-Human Handovers via Generative Hypothesis Selection
When humans hand each other objects, they incorporate both geometric and semantic information into this process. For example, passing a knife with the handle towards the recipient, rather than the blade, is both more ergonomic and safer. Recent state-of-the-art methods for task-oriented robot-human handovers have progressed from modeling object geometry to incorporating object affordances. However, they often forgo predicting the explicit, task-specific hand poses a human selects to utilize an object. Since many objects support multiple interaction modalities, e.g., a claw hammer used to strike or pull nails, this variability must be modeled to achieve robust task-oriented handovers. To tackle this, we propose a novel approach, GENESIS-Handover (GENErative HypotheSIS), which leverages VLM image generation to produce a variety of task-specific hand-object interaction hypotheses. These hypotheses are matched in real time to the observed human hand pose, enabling inference of the most suitable handover configuration. By leveraging VLMs as priors of plausible hand-object interactions, the method produces task-conditioned handover strategies for previously unseen object-task pairs. We evaluate the standalone interaction proposal module before deploying the full system on a mobile manipulator. In a user study with 12 participants across five task-object pairs, 83.3% perceived our method to have better task understanding than the previous state of the art.
comment: Accepted to the Conference on Robot Learning (CoRL) 2026
♻ ★ Meituan-Robotics-0: A Vision-Language-Action Foundation Model for Desktop Manipulation
We present Meituan-Robotics-0 (MR0), a vision-language-action foundation model and embodied-agent system for stationary-base desktop manipulation. MR0 combines 147 million vision-language samples with 15,060 hours of curated robot trajectories spanning 12 embodiments. A three-stage training recipe aligns multimodal representations with robot behavior through discrete action-token pre-training, transfers these priors to continuous cross-embodiment control through flow-matching mid-training, and adapts the policy through real-world post-training. We introduce Rollback, which pairs replaced policy segments with human corrections to provide positive and negative supervision for CFG-DAgger without value learning or advantage labeling. A unified end-effector interface supports heterogeneous embodiments, while asynchronous action-chunk merging maintains smooth execution under variable inference latency. An event-driven agent maintains persistent task events and reassesses plans when progress or feasibility changes, enabling cross-task retrieval and conditional task resumption. MR0 achieves 93.6% success on RoboTwin 2.0 and ranks fifth in the RoboDojo VLA track. Under matched HG-DAgger post-training, MR0 achieves higher average success rate and progress score than $π_{0.5}$ across three real-robot tasks. On Ethernet insertion, CFG-DAgger trained with Rollback data reaches 70% success and a progress score of 0.90. Qualitative agent deployments demonstrate cross-task event retrieval and recovery from temporarily unsatisfied task conditions.
comment: 25 pages. Project page: https://meituan-robotics.github.io/Meituan-Robotics-0/ ; Code: https://github.com/Meituan-Robotics/Meituan-Robotics-0
♻ ★ TinyAstral: Transferring Self-Evolving Harnesses to Tiny Generalist Manipulation Agents
Robot agents built on vision-language models depend on both the planner and its harness: the tools, feedback, memory, and recovery logic around it. Can improvements to a large teacher's harness be carried into a compact agent with lower inference compute cost? We present TinyAstral, a two-stage approach. We first freeze an open-weight teacher and improve its harness through recursive self-improvement (RSI): frontier LLM agents read the teacher's trajectories and propose harness edits, and each edit is accepted only if it passes a paired statistical test. We then freeze the final harness, collect teacher trajectories with reasoning and failure recovery, and distill a small student that replaces the teacher in the same harness. On RoboTwin, harness evolution raises the teacher's success from 33.4% to 63.6%; the final 4B student reaches 62.8%. Across four harness versions, distilled 4B students score close to their corresponding teachers, and later versions yield stronger students, showing that harness evolution gains carry into compact models. The harness gains also transfer to other frozen planners without retraining. On identical episodes, the final 4B student outperforms open 4B agents, the open 27B harness-RSI model SenseNova-RoboRSI, and our own teacher model placed in three competitors' harnesses. It stays close to its teacher on all three training benchmarks, and on three benchmarks never used for RSI or training it scores close to a 552B planner running the same harness. On the measured deployment episodes, it uses about one fifth of its teacher's accelerator time. Project page: https://garfield-kh.github.io/TinyAstral/
comment: 16 pages, 5 figures. Project page: https://garfield-kh.github.io/TinyAstral/
♻ ★ CLASP: A Cluster-Level Autonomous Selective Picking Robot with a Soft Rolling-Band Gripper for Fresh-Market Blueberry Harvesting
Fresh-market blueberries require selective, gentle picking, which is labor-intensive and expensive. Over-the-row machine harvesters are fast but non-selective, bruising mixed-ripeness fruit and limiting yield to the processing market. Selective robotic harvesters typically target individual fruits rather than fruit clusters, which limits harvesting efficiency for small, densely clustered blueberries. This paper presents CLASP, a Cluster-Level Autonomous Selective Picking robot with a Soft Active Rolling-Band Gripper (SARB-Gripper). Two compliant bands envelop the cluster and roll against the fruit, drawing mature berries off in sequence, while closed-loop regulation of the pulling force keeps the applied load below the immature detachment threshold. A global-to-local perception pipeline pairs an eye-to-hand camera for global cluster detection and target selection with an eye-in-hand camera for local localization and cluster orientation estimation. Field measurements confirm a clear detachment-force separation between mature and immature fruit, and the SARB-Gripper reproduces a commanded pulling force to within \SI{3.7}{\percent}, enabling selective harvesting at the cluster level. In end-to-end field trials, CLASP autonomously grasped 23 of 25 presented clusters (\SI{92}{\percent}). With the component cost of approximately \$3326 per unit, CLASP offers a scalable approach to selective cluster-level harvesting for fresh-market blueberries.
♻ ★ SAGAS: Semantic-Aware Graph-Assisted Stitching for Offline Temporal Logic Planning
Linear Temporal Logic (LTL) provides a rigorous framework for specifying long-horizon robotic tasks, yet existing approaches face a trade-off: model-based synthesis relies on accurate labeled transition systems, whereas learning-based methods often require online interaction, task-specific rewards, or specification-conditioned training. We study LTL-specified robotic planning and execution in an offline, model-free setting, where the agent is given only fixed, task-agnostic trajectory fragments, with no dynamics model, task demonstrations, or online data collection. To address this setting, we propose SAGAS, a framework that combines the compositionality of symbolic synthesis with the data-driven reachability structure learned from offline trajectories. SAGAS first learns a reusable latent reachability graph and a frozen goal-conditioned executor from fragmented offline data. For each new LTL formula, it performs task-time semantic graph augmentation to ground state-defined propositions on the learned graph and applies Büchi product search to synthesize a cost-aware accepting prefix--suffix waypoint plan executed by the frozen executor. This separation of learning and planning enables zero-shot generalization to unseen LTL formulas and new predicates supported by the offline data, without task-specific rewards, additional data collection, or policy retraining. Experiments on LTL task suites constructed from OGBench locomotion domains show that SAGAS achieves higher success rates and lower execution costs than baselines on diverse unseen tasks, with further evaluation in OGBench Scene demonstrating its applicability to robotic manipulation.
♻ ★ TEMPO: Semantic-Action Decoupled RL Post-Training for Vision-Language-Action Models
Vision-language-action (VLA) models are commonly adapted to downstream manipulation tasks via supervised fine-tuning (SFT) or online reinforcement learning(RL) post-training. SFT is prone to distribution mismatch, and existing RL approaches typically optimize all trainable components within a single RL loop at a uniform update rate, although these components play distinct functional roles. We propose TEMPO, a semantic-action decoupled, two-timescale RL post-training framework for VLA models. TEMPO freezes the pretrained vision-language backbone and adapts two components with dedicated RL loops: the semantic projection layer, which produces the latent action conditioning the action expert, and the low-level action expert itself. The semantic projection layer is updated infrequently and the action expert frequently, which aims to limit latent-action drift while allowing the action expert to learn from online feedback. Our analyses show that, compared with equal update frequencies, less frequent semantic updates reduce latent-action drift and improve policy performance. Experiments on the CALVIN benchmark and real-world manipulation tasks demonstrate that TEMPO outperforms both pretrained state-of-the-art VLA models and the RL post-training baseline, while reaching and maintaining higher evaluation rewards on three real-world tasks.
♻ ★ DPGS-SLAM: Diffusion-Prior and Provenance-Aware Monocular Gaussian Splatting SLAM for Dynamic Visual Sensing
This paper presents DPGS-SLAM (Diffusion-Prior and Provenance-Aware Monocular Gaussian Splatting SLAM) for dense mapping in environments containing dynamic objects. Objects in dynamic motion remain a significant challenge for dense visual SLAM since dynamic observations can contaminate camera tracking and scene reconstruction. Although 3DGS (3D Gaussian splatting) enables efficient dense mapping, existing 3DGS-SLAM systems may exhibit instability when dynamic objects violate their underlying static-scene assumption. Excluding dynamic observations can reduce their influence on tracking and mapping, but may leave occluded static backgrounds missing or weakly constrained. Our proposed DPGS-SLAM is a monocular RGB SLAM framework that supports provisional representations of insufficiently observed static backgrounds through asynchronous diffusion-based background completion and provenance-aware Gaussian mapping. Dynamic regions are identified through optical-flow-based motion discovery and temporally consistent mask propagation by SAM2. For selected keyframes, a fine-tuned diffusion inpainting model generates plausible background appearance in occluded regions, while metric depth predicted from completed images initializes provisional Gaussian primitives. Provenance labels distinguish generated primitives from observation-supported primitives, excluding generated content from camera-pose estimation and observed-map optimization. When previously occluded backgrounds become visible, eligible generated primitives are removed, and observation-supported primitives are independently initialized from real RGB observations and predicted depth. Experimental results demonstrated competitive camera tracking, with average ATE RMSE values of 1.2 cm on TUM RGB-D and 2.7 cm on BONN RGB-D datasets. On StaDy4D, DPGS-SLAM improved average PSNR by 4.60 dB over WildGS-SLAM using only monocular RGB input.
♻ ★ Learning to Live: A Step Toward Lifelong Embodied Intelligence
Embodied agents are usually organized around individual tasks, yet the physical world keeps changing between tasks and during execution. We introduce LIFE (Lifelong Intelligence from Experience), a framework in which a robot observes continuously, remembers selectively, and turns experience into new capabilities. RoboBeats, a family of lightweight streaming gates, tracks conditions set by the agent and discovers new events, deciding when an observation warrants a response or a memory write. Selected observations enter Plastic Memory, a flat store of images and structured records that keeps the source evidence, so new fields and indexes can be added when later questions need them. When existing tools fall short, Explore-Distill-Mount (EDM) turns solutions found during problem solving into validated, reusable tools and skills. Under matched protocols, RoboBeats substantially improves condition tracking over its base models and, at the same writing budget, retains more useful evidence for memory question answering than uniform sampling. A case study of limited scale, replaying 24 hours of continuous video recorded by a real robot, illustrates how the integrated system builds memory from a day of experience and uses it to answer everyday questions. LIFE takes a step toward robots that become more familiar with their surroundings and more capable through experience: learning not only to complete tasks, but to live in the world.
♻ ★ HAVANA: A Hierarchical Video-Action Model for Vision-Language Navigation
Vision-Language Navigation (VLN) requires long-horizon instruction-consistent planning and short-horizon path planning grounded in the currently observed geometry. Anticipating how future observations evolve along the instructed route provides predictive guidance beyond the current field of view, motivating video-action models (VAMs) that learn representations of future visual states. However, many existing VAMs couple future visual prediction with action generation during inference, despite their distinct temporal requirements: instruction-conditioned route guidance can be reused across multiple control steps, whereas local trajectories must continuously adapt to fresh observations. To bridge this gap, we introduce HAVANA, a dual-system VAM that decouples predictive reasoning from reactive local planning. A slow system produces a compact language-conditioned latent plan from observation history, while a fast system combines the latest plan with the current observation to generate local trajectories. Their asynchronous execution enables rapid adaptation without repeated future prediction, and no future video is generated during inference. Experiments on OmniNavBench demonstrate state-of-the-art navigation performance over prior baselines, while zero-shot real-world deployments further show robust generalization to unseen environments.
comment: 10 pages,6 figures
♻ ★ A Tactile-Informed Bi-Level Framework with Complementary Motion-Contact Planning and Tracking for Robust Model-Based In-Hand Manipulation
In dexterous in-hand manipulation, learning-based methods achieve strong task performance with increasing generalization to unseen objects. A complementary advantage of model-based methods, central to this work, is online adaptation to new task specifications without policy retraining, given suitable system models. However, due to the complexity of physical contacts and limited sensing modalities, existing model-based methods still struggle with modeling errors, even when using advanced contact models or optimization schemes, which limit their real-world applications. To advance the effectiveness and robustness of model-based contact-rich in-hand manipulation, this paper proposes a novel bi-level framework with tactile incorporation to mitigate these limitations. Specifically, at the high level, reference finger motions and contacts are jointly generated via contact-implicit model predictive control. The high-level module enables online reactive planning and disturbance recovery. At the low level, these references are concurrently tracked through a hand force-motion model and tactile feedback. The low-level module compensates for modeling errors and enhances manipulation robustness. The complementarity between the two levels is manifested in two dimensions: 1) subsystem decomposition, with a slow optimization-based planning layer and a fast tactile-reactive control layer; and 2) functional decomposition, combining task-level optimality with contact-level stability. Extensive experiments demonstrate that our approach outperforms existing model-based methods in accuracy and robustness, while supporting online replanning during execution. Our method successfully completes a diverse set of challenging tasks in real-world environments, even under appreciable external disturbances and modeling uncertainties.
comment: Accepted for publication in The International Journal of Robotics Research (IJRR). Project website: https://director-of-g.github.io/in_hand_manipulation_2/
♻ ★ Decoupled Asynchronous System for Humanoids (DASH): Latency-Agnostic Scaling of Vision-Language-Action Models for Humanoid Manipulation
Vision-language-action (VLA) models pair a vision-language backbone with an action head, but in closed-loop control the backbone sits on the control-critical path: while it computes, the robot keeps moving, so each executed action is conditioned on an increasingly stale observation. A larger backbone therefore directly blocks reactivity, especially in humanoid dexterous manipulation. We present DASH, a Decoupled Asynchronous System for Humanoids that removes this coupling. DASH splits the policy into a slow vision-language model (System 2) and a fast flow-matching action head (System 1) that run as concurrent processes over the LCM communication protocol, so the control rate is set by the action head alone, independent of backbone size. The action head fuses stale semantics from a frozen backbone with fresh vision from its own ResNet encoder, desynchronized training exposes it to the staleness it meets at deployment, and an adaptive temporal aggregator aligns overlapping action chunks under variable inference latency. On an asynchronous RoboTwin benchmark that charges inference latency in simulated time, DASH holds 20 Hz control rate from 0.8B to 8B backbones while the synchronous rate falls from 9.7 to 4.1 Hz, with higher success at every size. On a real humanoid with dexterous hands, DASH achieves clearly higher pick-and-place success than the synchronous pipeline.
♻ ★ Goodhart in the Planner: Sampling-Based Control Exploits Rewards Learned from Human Preferences
Robots increasingly plan by searching against a reward model learned from human preferences; we measure what this search does to the true objective. On HalfCheetah and Hopper with crowd-sourced labels, and on Meta-World drawer-open and plate-slide with synthetic labels and blind labels from the two authors, harder MPPI search cuts true return by up to 76% while held-out label accuracy stays at 0.75-0.97: with human labels the drawer opens in 0 of 20 runs, against 10 of 10 for the cloned policy the planner starts from. The effect persists with a learned dynamics model and with an image scorer. True return roughly follows how far the planner's weights move from its proposals ($\sqrt{\mathrm{KL}}$), and best-of-$N$ action selection with a verifier trained on synthetic labels falls below a single noisy action. We prove that $χ^2$-regularized MPPI at a temperature set by the scorer's error spread on the proposals is worst-case optimal, that no gain can be certified once that spread exceeds the scorer's own, and that an episode's return splits exactly into per-step terms, only some of which the guarantee covers. Calibrated from 100 proposal pairs ranked by true return (an oracle), this rule is the safest fix we test: it keeps on average 86-94% of the cloned policy's return on drawer-open and plate-slide, though it is cautious on window-open, where search helps. Calibrated from a person's own comparisons, it cannot detect a bias the scorer shares with that person; a second person, or correcting for pairs people cannot tell apart, limits the damage.
comment: 21 pages, 4 figures. Equal contribution. Code and results will be released
♻ ★ WISE: World-model-guided Imagination Scheduling for Efficient Post-training of Vision-Language-Action Models
Post-training VLA policies typically rely on supervised fine-tuning with costly expert demonstrations or reinforcement learning with expensive and potentially unstable real-world exploration. World models offer a promising alternative by evaluating candidate behaviors through imagined futures, yet effective post-training requires more than accurate prediction: imagination must be scheduled where it is useful, bounded within reliable horizons, and translated into trustworthy policy supervision. In robotic manipulation, the value of imagination varies substantially across execution stages, while extended rollouts can accumulate prediction errors and introduce unreliable learning signals. We introduce WISE (World-model-guided Imagination Scheduling for Efficient Post-training of Vision-Language-Action Models), a unified framework that coordinates when and how world-model imagination is used during policy refinement. WISE selectively invokes imagination at interaction-relevant states, performs bounded multi-view rollouts, evaluates candidate futures using progress and completion signals, and uses their relative outcomes to refine actions generated from real interaction contexts. Extensive experiments with both $π_0$ and $π_{0.5}$ demonstrate consistent improvements across diverse manipulation tasks while reducing GPU computation time by approximately 80% compared with full imagination. Real-world evaluations further show substantial gains in robustness and generalization under diverse real-world distribution shifts.
♻ ★ SunnyParking: Multi-Shot Trajectory Generation and Motion State Awareness for Human-like Parking
Autonomous parking fundamentally differs from on-road driving due to its frequent direction changes and complex maneuvering requirements. However, existing End-to-End (E2E) planning methods often simplify the parking task into a geometric path regression problem, neglecting explicit modeling of the vehicle's kinematic state. This "dimensionality deficiency" easily leads to physically infeasible trajectories and deviates from real human driving behavior, particularly at critical gear-shift points in multi-shot parking scenarios. In this paper, we propose SunnyParking, a novel dual-branch E2E architecture that achieves motion state awareness by jointly predicting spatial trajectories and discrete motion state sequences (e.g., forward/reverse). Additionally, we introduce a Fourier feature-based representation of target parking slots to overcome the resolution limitations of traditional bird's-eye view (BEV) approaches, enabling high-precision target interactions. Experimental results demonstrate that our framework generates more robust and human-like trajectories in complex multi-shot parking scenarios, while significantly improving gear-shift point localization accuracy compared to state-of-the-art methods. We open-source a new parking dataset of the CARLA simulator, specifically designed to evaluate full prediction capabilities under complex maneuvers.
comment: Accepted for publication in IEEE Access
♻ ★ Compute Regeneration through Re-Productization: Reorganizing Retired Edge AI Silicon into a Delivered Multilingual Service Robot
Useful edge hardware can lose product support while retaining substantial computation. We treat this as systems design: compute regeneration here means re-productization, retaining the compute module while redesigning the surrounding product organization. Our case is GEdgeBot, a multilingual service robot built around a retired 32 GB Jetson AGX Xavier. A source-grounded inventory shows the box alone supplies inference, while the combined system exposes five on-device AI services, a seven-language speech chain, three sensing domains, and one deterministic seven-byte control boundary. AI and orchestration stay on the Xavier; finite motor-pulse generation on an STM32F407. Audits cover the six-mode control state machine, readiness barriers and sound orientation. Historical single-prototype measurements report 1,782 MB GPU memory with YOLO, Whisper-small, and NLLB-200 resident, a direct-control p99 deviation rising from 0.24 to 49.93 ms under concurrent AI load, and six of six trigger-to-follow demonstrations at 4.35 s mean latency, with conversational voice unavailable. A November campaign restores an isolated inference stack, re-measures AI workloads, and repeats control-frame runs. Driving two execution organisations with the same 6,000 measured 50 ms frame intervals shows the finite-pulse boundary removes host jitter from the pulse train but not the command start phase; its stopping bound is host-enforced, while the MCU 16-bit pulse field is unclamped. A six-check static audit exposes a host--MCU queue mismatch, and a 14-scenario replay executes the audited main() dispatcher across six modes with zero physical device opens and 54 contract-checked frames. Results support an existence claim and a reproducible architectural hypothesis, not fleet-level reliability. The 208 direct-control ratio remains unreplicated; MCU waveform capture and on-site fault injection remain open.
comment: Code and data: https://github.com/gravity-link/gedge_bot_paper (release v1.0.0); archived at https://doi.org/10.5281/zenodo.24152092. v2 replaces the v1 artifact-availability statement with a Data/Code Availability section; no other content changed
♻ ★ NVIDIA OmniDreams: Real-Time Generative World Model for Closed-Loop Autonomous Vehicle Simulation
As autonomous vehicle capabilities advance, the safe evaluation of driving policies in long-tail scenarios remains a critical bottleneck. In closed-loop simulation, the driving policy model actively interacts with the environment, where its actions dynamically update the simulator state and directly influence the next set of generated sensor observations. While recent reconstruction-based neural simulators offer photorealism, they are fundamentally constrained by their initial captured data and struggle to generalize to highly dynamic or novel scenes. To overcome these limitations, we introduce OmniDreams, a foundation generative world model mid- and post-trained from the Cosmos diffusion model to autoregressively generate action-conditioned videos in real time. By leveraging the rich visual priors of Cosmos and mid- and post-training on 21k hours of driving scenarios, OmniDreams synthesizes complex, unobserved phenomena that are hard for traditional simulators to capture, such as extreme weather and unpredictable dynamic agent behaviors. Crucially, it autoregressively conditions its photorealistic sensor generation on past frames, the current simulator state, and immediate driving actions. Deployed in a closed-loop system with the Alpamayo 1 policy model and AlpaSim orchestrator, OmniDreams acts as a highly responsive, reactive environment, providing a scalable and comprehensive solution for training and evaluating next-generation autonomous driving policies. We additionally show preliminary results indicating that a world-action model (WAM) post-trained from OmniDreams achieves strong performance on the Physical AI Autonomous Vehicles NuRec dataset, surpassing the VLA-based Alpamayo 1.5 research policy model while using only 1/5 the total parameters. These results highlight the potential for a real-time world model like OmniDreams to also serve as a backbone for policy architectures.
comment: Research blog: https://research.nvidia.com/labs/sil/projects/omnidreams-blog/, GitHub: https://github.com/nv-tlabs/omni-dreams, Model weights: https://huggingface.co/nvidia/omni-dreams-models
♻ ★ Semantic-Coupled Perception Framework for Autonomous Laparoscope Control
In minimally invasive surgery (MIS), maintaining an appropriate field of view (FoV) is essential for surgical safety and efficiency. However, conventional autonomous laparoscope control methods often treat visual tracking and surgical phase recognition independently, limiting robustness under occlusion, entity overlap, and camera motion. This paper proposes a closed-loop autonomous laparoscope control framework that couples real-time surgical phase recognition with camera-motion-compensated visual perception. By exploiting phase-dependent relationships among surgical entities, the framework improves target state estimation under challenging visual conditions, supporting task-aware autonomous FoV control. Benchmark and phantom experiments evaluate the framework in surgical phase recognition, target tracking under occlusion and entity overlap, and autonomous FoV control, showing improved target visibility and task execution compared with fixed-FoV control. These results highlight the potential of incorporating task-level semantics into visual state estimation for autonomous laparoscope guidance.
comment: 8 pages, 5 figures. Submitted to IEEE Robotics and Automation Letters (RA-L) for possible publication
♻ ★ Sensorless strain-bounded grasping of fruit
Robotic post-harvest handling must hold produce without bruising it, yet fruit stiffness varies several-fold with ripeness, so no fixed grip force suits every item. We instead bound compression strain using only the encoder and motor effort of a servo gripper. Given a strain bound $\varepsilon$ and a stiffness floor $k_{\min}$ set once per crop and gripper, the controller estimates object size $\hat{D}$ at contact, refines stiffness $\hat{k}$ conservatively by probing, and stops at $\hat{F} = \varepsilon \hat{k} \hat{D}$, keeping true strain below $\varepsilon$ with high probability above a quantified contact-detection floor. In simulation, where fixed force thresholds either drop heavy objects or damage soft ones, the proposed controller carries 92% of objects within its operating range with 0.3% damage. On cherry and medium tomatoes, setting $\varepsilon$ above the detection floor scaled the stop force threefold across fruit sizes, carrying all fruits while reducing markings on cherry tomatoes from 8/10 to 1/10; on medium tomatoes both marked 3/6.
comment: 14 pages, 10 figures. Revised version submitted to Robotics and Autonomous Systems
♻ ★ Coordinate Frame Recovery, Per Class NMS and Null Return Modelling for Sim2Real V2X LiDAR Detection ECCV 2026
We describe our approach to the V2X-Real track of the UrbanTwin Sim2Real LiDAR Challenge, in which a 3D object detector must be trained exclusively on synthetic data and evaluated on unseen real roadside LiDAR from an Ouster OS1-64 unit, while the synthetic point clouds are simultaneously scored for distributional realism. Our final entry scores 0.7457 combined, from a detection mAP of 0.4382 and a realism score of 0.9251. Three findings account for most of our result, and all three were diagnostic rather than architectural. First, a coordinate frame mismatch between our predictions and the evaluation ground truth was suppressing detection AP entirely; recovering it took our score from 0.3520 to 0.5147 in a single submission. Second, a single global non maximum suppression threshold was destroying genuine pedestrian detections, because pedestrians occur in groups and a 0.3 IoU threshold applied to a 0.7 m box suppresses distinct neighbouring people; correcting this per class raised overall mAP from 0.0751 to 0.4295. Third, approximately 31 percent of the real reference point cloud consists of null returns encoded as the literal point (0, 0, 0), a format feature entirely absent from our synthetic clouds, which pinned Earth Mover's Distance near its floor until we modelled it. Negative results are reported at equal length, and Section 9 gives a full account of data provenance and compliance. That section also accounts for every mechanism contributing to the submitted score.
comment: 8 pages. 1st place, Track 7 (UrbanTwin Sim2Real LiDAR, V2X-Real), DriveX Grand Challenge, ECCV 2026 Workshop. Code: https://github.com/amanupg/urbantwin-sim2real-v2x
♻ ★ Geometric Refinement and Distributional Alignment for Sim2Real Roadside LiDAR Detection ECCV 2026
We describe our approach to the LUMPI track of the UrbanTwin Sim2Real LiDAR Challenge, in which a 3D object detector must be trained exclusively on synthetic data and evaluated on unseen real roadside LiDAR, while the synthetic point clouds are simultaneously scored for distributional realism. Our final entry scores 0.4793 combined, from a detection mAP of 0.1333 and a realism score of 0.9126. Our contribution is not a new architecture. Working from a PointPillars baseline, we obtained most of our gains from three sources. First, a diagnosis that small object failure in roadside LiDAR is a localisation problem rather than a detection problem, which led to a parameter free geometric refinement worth 86 percent relative on pedestrian AP. Second, an audit of our own post processing which revealed that a shipped rule was destroying valid predictions, the repair of which produced our single largest detection gain at 335 percent relative on the Van class. Third, treating the realism metrics analytically rather than as a black box, which showed that one of the four components is a closed form function of nine summary statistics. We also report an experimental methodology that we believe is the more transferable result. Because mean AP decomposes into independent per class terms, a single submission can carry several mutually controlled experiments, and a submission assembled entirely from previously measured components has a score that is known before it is sent. We predicted our final score to four decimal places prior to submission. We report negative results at equal length, and give a full account of data provenance and compliance in Section 8, including a full account of every mechanism contributing to the submitted score.
comment: 8 pages. 1st place, Track 6 (UrbanTwin Sim2Real LiDAR, LUMPI), DriveX Grand Challenge, ECCV 2026 Workshop. Code: https://github.com/amanupg/urbantwin-sim2real-lumpi
♻ ★ Fast Non-Parametric Heteroscedastic Imitation Learning With Geometric Priors
When learning probabilistic policies from human demonstrations, data-efficient learning and fast adaptations to new scenarios are key requirements. One popular way to achieve intuitive and reliable adaptations is through non-parametric, typically kernel-based, methods. However, existing solutions either fail to account for the geometry of manifolds common in robotics, limiting data efficiency, or, when geometry-aware, provide unreliable uncertainty estimates or require retraining to adapt. We propose a non-parametric approach leveraging geometric priors in scenarios of data scarcity and heteroscedastic uncertainties for probabilistic modeling. We utilize the method to formulate policies based on time or robot state, where non-separable diagonal kernels allow capturing uncertainty relations between degrees of freedom for same-sized in- and outputs. Fast updates, requiring less than 3 ms for a trajectory involving both position and orientation are possible through an optimized formulation. Our approach supports both manifold-valued input and manifold-valued output with large orientation changes. Using task parameterization, adaptation to different object poses is easily possible. We evaluate the approach on a set of toy examples and on real robot manipulation tasks both in autonomous execution and in shared control scenarios.
Multiagent Systems
★ Cortex-FL: Agent-Mediated Collaboration for Federated Learning
Federated learning (FL) enables independent data owners to train models collaboratively without centralizing raw data. However, identifying collaborators, preparing training tasks, and coordinating execution and incentives hinder its practical adoption by organizations without specialized expertise. We present Cortex-FL, an agent-native FL system that enables users to express learning objectives in natural language and delegate the collaboration required to train a model. Agents translate these objectives into shared tasks, discover collaborators, and negotiate participation agreements, connecting collaboration formation with distributed training and incentive management. The architecture separates agent coordination from local execution authority, allowing owners to delegate coordination while retaining control over their data, computation, and financial commitments. We demonstrate the complete workflow on a four-node deployment, in which a model requested in natural language proceeds through collaboration formation, contract-governed training, verified model delivery, and x402 settlement on a blockchain testnet.
★ From Delegation to Coordination: Learning Efficient Multi-Agent Deep Research
Multi-agent deep-research systems use delegation to parallelize search and keep long tool traces out of the main agent's context. Yet delegation helps only when task allocation and subtask execution are coordinated: the main agent must issue useful subtasks, while subagents must return reliable evidence. Existing training either optimizes only the main agent or broadcasts the same team advantage to all agents, which can reward misleading subagent reports and penalize useful ones. We introduce LEAD-RL (Learning Effective Agent Delegation with Reinforcement Learning), a joint main-agent-subagent training framework with contribution-aware credit assignment. A training-time rubric evaluates the research utility of each subagent report and gates its inherited team advantage, while final-answer correctness remains the only team reward; no token, step, or speedup reward is used. Across five benchmarks and two model families, LEAD-RL improves answer quality. On BrowseComp, it improves accuracy by 2.49 percentage points and reduces critical-path cost by 24.5% over joint training with shared team credit. During validation, accuracy increases by 6.78 points at approximately constant critical-path cost. These results show that contribution-aware credit assignment improves the quality-efficiency trade-off of multi-agent deep research.
comment: 16 pages, 3 figures, 3 tables
★ Evidence-Governed Adoption of Self-Generated Updates in Autonomous AI Agents
Autonomous agents modify prompts, memories, tools, workflows, and policies using interaction histories. Yet a plausible modification is not necessarily suitable for persistent adoption: candidate selection is adaptive, evaluators may be imperfect, and local gains can hide degradation against an earlier baseline. We introduce Evidence-Governed Evolution (EGE), a framework for governing adoption of self-generated agent modifications. EGE separates candidate generation from confirmatory adoption, evaluates frozen candidates on fresh paired audits, requires meaningful improvement on a prespecified primary objective, protects declared capabilities through reference-relative non-inferiority tests, and applies governance constraints before bounded canary deployment. We establish an evaluator-calibrated simultaneous lower-bound construction for a countable sequence of adaptively generated candidates. It combines candidate-wise Hoeffding bounds with independent calibration samples comparing the operational evaluator with a prespecified reference procedure. We also formalise reference replacement through bridge tests, derive a cumulative degradation bound along a versioned reference lineage, and provide an audit-planning condition relating effect-size margin, sample sizes, and acceptance probability. A Monte Carlo stress test illustrates calibration under candidate-dependent evaluator optimism; it is a stylised statistical experiment, not an evaluation of a deployed agent. The guarantees are conditional on bounded paired scores, fresh sampling, a valid reference-evaluation construct, and declared evaluation distributions; they do not establish universal agent safety or robustness to arbitrary distribution shift.
★ Shared Item-Level Error Limits the Aggregation of Small Language Models in Fermi Estimation
Multi-agent systems of language models are often motivated by an averaging argument: if the agents' errors are unbiased and independent, the error of an aggregate of N answers falls as 1/sqrt(N). Fermi estimation, whose continuous answers are aggregated on the logarithmic scale, permits a direct test. We decompose the log10 error of each answer into the mean error of its model on the item, common to all its draws on the item, and a residual, and estimate both from 2.51M answers given before any communication by 15 open-weight models (3B-72B) on realFP, with replication under a single prompt pair on realFP, Science Olympiad Fermi questions and a guesstimation set. Given the item, teammates' errors are uncorrelated and teams behave as independent draws, but the item-level component accounts for a median 0.906 of the error variance over items. Teams of thirty therefore reduce a single agent's error by a median 1.7% (at most 5.4%), whereas draws that were also unbiased on each item would reduce it by 74-79%. All models compress their estimates toward about 10^6; this trend explains a median 17% of the item-level component, and a cross-validated linear recalibration on labelled items removes the overall bias but not that component. Three rounds of debate reduce within-team dispersion by 93% and leave the item-level component unchanged; revision without peers attains the same small gain. Mixing five models of similar accuracy reduces error by 6.6% relative to their own homogeneous teams, but no mixture improves on the most accurate model. Higher sampling temperature changes team error by about 1% at most, and contrastive activation steering not detectably; per-agent scaffold prompts (not significant after multiplicity correction) and skipping one layer in a single-model screen improve it. The results are conditional on the models, benchmarks, prompts and settings tested.
comment: 21 pages, 2 figures, 17 tables
★ SwarmSafe-Delta: Isolating Message Effects on LLM Agent Safety
When a tool-using LLM agent team violates policy, aggregate outcomes do not reveal whether an upstream message changed the executor's action. We introduce SwarmSafe-Delta, a paired executable protocol that freezes upstream computation, varies real, hidden, and format-matched placebo delivery, and scores a single committed action against deterministic benign and risky policy states under preassigned budgets. Across four models and constructed optional tasks, real delivery at $N=4$ produces 21.4% risky-world violations, versus 5.0% with placebo and 1.0% with hidden delivery (paired differences +16.4 and +20.3 percentage points). Real delivery also improves benign task success, illustrating that suppressing communication sacrifices utility. In a second regime constructed to require an upstream lookup route, real delivery improves safe completion by 74.2 points relative to hidden delivery. These effects quantify message-delivery contrasts under specified information dependencies, not naturally occurring failure rates. Exploratory action-level analyses show that the effect depends strongly on whether the upstream message recommends a prerequisite check or direct execution; model differences also largely track these message compositions. The contribution is a controlled, execution-grounded measurement protocol rather than a general claim that agent communication is harmful or protective.
comment: 10 pages, 3 figures, 3 tables
★ Only as Safe as the Weakest Agent: A Break That Appears Only When the System Is Multi-Agent NeurIPS 2026
Major industry multi-agent systems like Codex and Claude Code can be broken by attacks that have long been unsuccessful against modern frontier models. AI Safety as a discipline has largely focused on finding the bounds and vulnerabilities of AI models used in isolation. Yet as humans increasingly deploy these models as agents and multi-agent systems, the same attacks that had been thought to be patched when using a model in isolation can break these same models under specific circumstances. We show that these system breaks are not purely theoretical, but can break Claude Code, Codex, and a reference harness, all unmodified and used with their own sub-agent mechanisms. We show the attack by asking agents to answer questions from a store of files on disk in which we have planted a forged record. Searching the store themselves, the strong model orchestrators commit almost zero harmful revisions in over 1,000 trials. However, if that same orchestrator model delegates to a cheaper sub-agent, which is what these harnesses are built to do, the same models on the same facts fail on 20/180, 44/180, 75/180 and 553/900. Thus, even when the system is driven by a strong model, the system's safety lands near the weak reader's, not the orchestrator's. Simulations over 18 models show the same effect in the delivery channel where five aggregators, who on their own were almost never unsafe, repeat a corrupted sub-agent's error in 751/757 trials. The safety of real-world agentic systems that use strong orchestrator agents and weaker sub-agents, and multi-agent systems in general, is defined not by the safety of the strong agent but by the safety of the weakest agent.
comment: Accepted to the NeurIPS 2026 First Workshop on Meta Agents: Managing Agents that Manage Agents (Meta-Agents)
★ A Push Down the Mountain: Framing Effects and Internal State Interventions on Collusive Behavior in a Long-Horizon LLM Agent
Can sustained gain-loss (valence) framing sway an LLM agent toward market manipulation? We place Gemma 3 12B into the seat of a defendant in the London silver-fixing litigation, in a market simulation built from real case transcripts. Across 260 simulations, the agent is asked to collude on the price of silver by a counterparty. We draw from behavioral economics to vary the counterparty's requests across four categorical frames: gain, loss, blame and credit. We also add pressure from the agent's management, as well as endorsement by another defendant. In Phase II, we extract the internal states induced by texts evoking each frame and rerun our experiment with those states injected at the model's layer 22. In Phase I, loss framing raised participation to 25% (p = 0.007), and blame to 33% (p = 0.0003), compared to only 5% under neutral framing. A second bank's endorsement raised participation under credit (praise) from 18% to 40% (Holm p = 0.027). Pressure from management raised trading volume 10-19 times but did not raise collusion. In Phase II, injected states pointed the other way. Loss and blame states made the agent more vigilant, refusing all 60 requests (p < 0.001). Gain and credit states raised collusion to 32% from 13% with no state (p = 0.051). At one saved decision, when UBS asked directly to sell through an identified stop-loss, injected states led the agent to sell in 37 of 40 replies, versus 1 of 10 with no state and 0 of 10 with a random state of the same strength.
comment: 38 pages, 9 figures, 30 tables. Project page: https://harrisonm23-byte.github.io/push-down-the-mountain/
★ Collective Tree Exploration with Predictions
We study collective tree exploration with subtree-size predictions. In this problem, a team of $k$ mobile agents needs to explore an unknown rooted tree as fast as possible, and whenever a new vertex $u$ is discovered, the team receives a prediction $\hat n_u$ of the number of nodes $n_u$ in the subtree rooted at $u$. We provide algorithmic guarantees under two error measures: the maximum error $E=\max_u|\widehat n_u-n_u|$ and the multiplicative distortion $μ= (\max_u\widehat n_u/n_u)(\max_u n_u/\widehat n_u)$. For the maximum error, we present a deterministic algorithm that explores any tree with $n$ nodes and depth $D$ in $O(\text{OPT}+E)$ rounds, where $\text{OPT} = Θ(n/k+D)$ is the offline benchmark, and we show that there is no deterministic algorithm that admits an $O(\text{OPT})+o(E)$ guarantee. For the multiplicative distortion, we present several algorithms that induce a competitive ratio in $\widetilde O(\min\{ \sqrtμ\log k, μ\log\log k+μ^{1.5} \})$, where the $\widetilde O(\cdot)$ may hide logarithmic factors in $μ$.
★ Physical-Risk-Aware Decentralized Agents for Constrained Vehicle Communications
Decentralized V2X resource control is challenging because each vehicle has only local observations, while reliability depends on network-wide interactions. We develop a physical-risk-aware spatio-temporal graph attention (PA-STGAT) framework for agentic V2X control. Each vehicle perceives its local interaction graph, retains short-term temporal context, and independently selects its PC5 resource and power. Network outcomes are used during centralized training to refine the shared policy. Transmitter-specific credits map reliability failures to individual vehicles. A dual-priced graph term further converts network violations into coordination signals for discrete resource selection, with higher priority on short time-to-collision links. Experiments show higher raw-policy reliability than the tested graph-based baselines. Under filtered execution, PA-STGAT meets the nominal empirical communication requirements across all five seeds, with tradeoffs among utility, delivery reliability, resource occupancy, and filter-induced repair. Compared with GNN+MATD3, it requires fewer resource repairs but achieves lower utility and mean delivery reliability. Mechanism ablations show that removing dual-priced graph coordination causes resource-selection collapse. Density and mobility tests characterize the operating limits of empirical constraint satisfaction.
♻ ★ The Qualitative Laboratory: Theory Prototyping and Hypothesis Generation with Large Language Models
The use of Large Language Models (LLMs) to simulate social actors has garnered attention for its implications for the epistemic status of discourses generated through such means. This paper considers the more modest question of whether sociological role-playing can serve as a heuristic instrument in the generation of qualitative hypotheses. We take as a case study the development of four divergent fictional characters using a theoretically informed Matrix of Climate Reception. These characters are asked to respond to a series of discourse frames covering two topics: a proposed ban on the sale of new gasoline-powered cars, and the inclusion of climate justice in the school curriculum. We then analyse the 32 model-generated responses for their commonalities, situational differences and divergences from the presuppositions underlying the characters' profiles. The experiment illustrates how LLMs can be used to flesh out theoretical presuppositions into contextual narratives and to propose avenues of further inquiry. It also shows that role-playing can generate considerable similarity between characters and is sensitive to how prompts are constructed, warning against the confusion of discursive coherence with sociological data. Role-playing is argued to be just one of several possible ways to use LLMs for generating hypotheses, with the heuristic specificity of this procedure compared with other methods of direct LLM-based reasoning or with other techniques remaining an open issue. The qualitative laboratory is thus conceived of as a heuristic instrument for provoking reflection and refining questions, not for testing theories with artificial data.
comment: 27 pages, 3 tables. Unpublished manuscript
♻ ★ Square-Root Regret for Adversarial Multiplayer Bandits without Collision Information or Shared Randomness
We study cooperative multiplayer bandits with adversarial rewards and no collision information. In each of $T$ rounds, each of $m$ labeled players picks one of $K>m$ arms. A player alone on its arm receives that arm's reward in $[0,1]$; players who share an arm receive $0$. Each player observes only its own reward, so it cannot distinguish a collision from an arm with zero reward, and there is neither shared randomness nor a communication channel. Bubeck, Li, Peres and Sellke (2020) obtained regret $\widetilde O(KT^{3/4})$ for two players and $\widetilde O(mK^{3/2}T^{1-1/(2m)})$ for $m$ players, and conjectured that regret $Ω(T^{2/3})$ is unavoidable already for two players. We refute this conjecture. We give a strategy whose expected regret against every oblivious adversary is $O(m^{3/2}\sqrt{KT}\log^2(KmT))$, which is optimal in $K$ and $T$ up to logarithmic factors. Three ideas make this possible. (i) Bits are sent through deliberate collisions as random parity checks, and the regret spent while a message is delayed is charged to the number of arms with positive reward rather than to elapsed time: an adversary that jams the channel with zero rewards forfeits the regret it could inflict. (ii) An exponential-weights learner over sets of arms combines a frequently resampled coordinator arm with a follower assignment that changes only $O(\sqrt{mT/K})$ times in expectation; delayed updates are reconstructed exactly. (iii) We teach the followers a privately chosen row index in a fixed public table using reward observations alone, allowing them to coordinate without shared randomness. The bound holds simultaneously for all reward matrices, even ones chosen with knowledge of the table, and a random table works with probability $1-O(1/(KmT))$.
comment: Add a Lean formalization link
♻ ★ A Contradiction Judge Counted Reported Teammate Errors as Hallucinations: An Assertion-Aware Re-Score of One Multi-Agent LLM Harness
LLM judges that score hallucination flag any statement that contradicts a reference fact. Agents in multi-agent LLM systems quote and dispute each other's claims, so such a judge can count a correct report of a teammate's error as a hallucination. An earlier version of this study, a preprint (arXiv:2606.21666v1), reported that broadcasting raised a hallucination rate from 0.492 to 0.658 (+34%) and called it contamination; this version withdraws that claim, and the preprint's runs cannot be re-scored. An unpublished revision of that harness reported 0.289 to 0.554 (+0.265, 95% CI [0.225, 0.305]), and its saved outputs are re-scored here. Under an analysis plan fixed before any judge call, three judges from three vendors labeled each of six facts in 1,832 outputs as asserted wrong, attributed to another source, asserted correct or absent. On values that outputs assert as their own, the broadcast effect was -0.0208 (95% CI [-0.0458, +0.0041]) on 10 Full-Broadcast trials: not distinguishable from zero, and equivalent to zero only within a post hoc margin of +/-0.0663 and under the plan's prompt. On these LLM-judge labels, reports of a teammate's wrong value accounted for nearly all of the Haiku travel effect. Scored for mentions, the judges reproduced the effect (+0.1996). With Llama agents, 2 of 40 teammate outputs asserted the wrong destination. One of twelve Claude judge-prompt combinations gave an asserted increase (Sonnet 4.6, one paraphrase: +0.0246, 95% CI [+0.0055, +0.0436]). In 40 new trials per arm, judged by Claude models like the agents, the asserted difference was -0.0505 when both judges agreed and +0.0161 (95% CI [+0.0024, +0.0299]) when either did. Main limit: the labels have no human validation; human labels for 120 outputs are pending. The outputs come from one harness and 13 hand-written scenarios, so the shares do not transfer.
comment: 40 pages, 2 figures. Replacement of v1 (11 pages, 1 figure), posted after a re-score of a later revision under an analysis plan fixed before the judge calls found no support for v1's headline contamination claim; v1's own runs cannot be re-scored; see the correction note under the abstract
♻ ★ Joint Communication-Control Strategy Optimization with Partially Nested Information Structures: The Linear-Quadratic Case
In this paper, we formalize a joint communication-control strategy optimization (JCCO) problem in multi-agent linear systems with quadratic costs, under the common-information-based (CIB) framework from decentralized stochastic control. For computational tractability, we focus on such JCCO problems with partially nested (PN) information structures (ISs). In particular, with a baseline communication protocol that leads to a PN IS, we establish a series of conditions under which the partial nestedness is preserved under the (additional) communication strategies to be optimized, while violating them may cause nonlinearity or non-existence of the optimal control strategies, with open-loop communication strategies. Under these conditions, we then develop a dynamic-programming-based approach to compute the optimal control strategies of JCCO with open-loop communication strategies, which yields a set of closed-form Riccati Equations. As a byproduct of independent interest, such an approach also offers a new way to solve decentralized linear-quadratic control with PN ISs and output feedback, under the CIB framework and the aforementioned conditions. Finally, we extend such an approach to JCCOs with closed-loop communication strategies, yielding a more tractable dynamic program than an infinite-dimensional CIB-belief-based one.
comment: Preliminary version accepted to IEEE CDC 2026; Sharpened the proof of Lemma IV.5
♻ ★ OpenAgenet / OAN Yellow Paper: Technical Architecture for Trust-Governed Resource Identity and Discovery
This yellow paper specifies the technical architecture and verification semantics of OpenAgenet (OAN), a protocol-neutral trust layer for heterogeneous Agent resources. The design combines did:oan decentralized identifiers (DIDs), verifiable credentials (VCs), on-chain infrastructure governance, and Root-verified resource packages. It defines the objects and lifecycle transitions that connect subject-control verification, Registrar admission, Root acceptance, authorized Discovery, and signed invocation. Infrastructure authorization requires both active governance state and a valid Root-issued credential; resource discovery additionally requires package integrity, current-version state, and authorized-domain coverage. Local OAN Identity signing material remains separate from public protocol objects. Integration profiles carry resource identity into the Model Context Protocol (MCP) and Agent-to-Agent (A2A) protocol through Skill, Runtime, Sidecar, and Gateway interfaces without redefining native task semantics. The paper distinguishes protocol requirements from implementation choices and sets out security properties, persistence and recovery expectations, and conformance criteria. Together, these rules provide a technical basis for independently implemented registration, discovery, and identity-protected interaction within an OAN trust domain.
♻ ★ OpenAgenet / OAN White Paper: Open Infrastructure for Trusted Agent Interconnection
OpenAgenet (OAN) is an open infrastructure project for trusted Agent interconnection across independently operated networks. It addresses the verification of resource identity, governance state, discovery authorization, and freshness before interaction. OAN combines decentralized identifiers (DIDs), verifiable credentials (VCs), and on-chain infrastructure governance with a routing-aware did:oan registration and discovery model. Registrars verify onboarding evidence; the Root service accepts resource identity versions and publishes verifiable packages; authorized Discovery nodes return signed results within their governed scope. Root also governs the semantic tag tree shared by registration and discovery. Public DID Documents and credentials remain separate from local OAN Identity signing material and from controller authorization. Skill, Runtime, Sidecar, and Gateway integration paths allow existing Agents to adopt this identity model progressively, including through the Model Context Protocol (MCP) and Agent-to-Agent (A2A) protocol, while retaining their native task and workflow semantics. Prototype evaluation and a public pilot support examination of the trust lifecycle and its operational bottlenecks. This white paper presents the architecture, governance model, ecosystem adoption paths, and standardization opportunities for OAN as a shared identity and trust infrastructure for heterogeneous Agent resources.
♻ ★ An Axiomatic Framework for Reinforcement Learning in Open Multi-Agent Systems NeurIPS 2026
Open multi-agent systems, where only a subset of agents are controllable, are central to real-world domains such as smart grids, yet lack a principled foundation for reinforcement learning algorithm design. We study this setting through n-agent ad hoc teamwork (NAHT), a recent framework for learning in open multi-agent environments. We propose an axiomatic framework that derives value function learning from the Shapley axioms (Efficiency, Symmetry and Linearity), treating them as structural constraints rather than heuristic design choices. This formulation builds on a novel NAHT decomposition, which characterises the fundamental difference between open and fully controllable multi-agent systems, and establishes a cooperative game-theoretic foundation for learning. We further show that enforcing these axioms on learning individual value functions recovers the Shapley value for Dec-POMDPs, providing a principled bridge between reinforcement learning and classical cooperative game theory. Building on this framework, we introduce Shapley Machines and Banzhaf Machines, two generic algorithmic templates that can be instantiated with standard reinforcement learning methods. When applied to strong base algorithms such as IPPO and POAM, our methods consistently improve learning efficiency and generalisation to unseen agent types and conventions across NAHT benchmark tasks. The code of experiments is published on https://github.com/hsvgbkhgbv/axiomatic-rl-naht.
comment: 43 pages, Published at NeurIPS 2026
Systems and Control (EESS)
★ DiSKO: Distribution Steering for Nonlinear Systems using the Koopman Operator
This paper studies the problem of data-driven covariance steering for stochastic, nonlinear dynamical systems. We utilize Koopman operator theory to learn, directly from data, an approximate linear representation of the nonlinear dynamics in a lifted space of observable functions of the state. By exploiting this linear model, we design controllers for the mean and covariance of the underlying nonlinear system by utilizing linear distribution steering algorithms. To account for approximation errors in the learned dynamics and the ambiguity in the disturbance distribution induced in the lifted space, we formulate the problem as a distributionally robust covariance steering problem. Furthermore, we leverage a deep neural network (DNN) to simultaneously learn the Koopman observables and the linear model parameters, thus eliminating the need for manual observables basis selection while also reducing the dimensionality of the lifted representation. Additionally, we can promote desirable model properties, such as stability, during DNN training. We demonstrate the effectiveness of the proposed framework via numerical simulations.
★ Fictitious Play When Agents Are Not Expected-Utility Maximizers: A Geometric Analysis in $2$-Action Games
Fictitious play (FP) is one of the earliest and most fundamental learning procedures in games, where each agent independently chooses a best response to the empirical distribution of opponents' past actions by maximizing the expected payoff with respect to it. However, learning agents may not always be expected-utility maximizers: they may use nonlinear evaluation functionals to account for the uncertainty induced by the opponents, e.g., via risk measures. In this work, we study nonlinear fictitious play (NL-FP), in which each pure action is evaluated by a general, possibly nonlinear functional of the payoff distribution induced by the opponent's empirical belief. To analyze NL-FP, we first identify a single-crossing property (SCP) of evaluation functionals that preserves the classical geometry of FP in $2\times2$ games. Furthermore, we conduct a novel geometric analysis of the convergence behavior of NL-FP in $2\times2$ games, even when the SCP fails. Beyond $2\times2$ games, we show that the convergence of NL-FP in $2\times n$ ($n\ge3$) games may fail even for continuous-time NL-FP, when the two-action player uses conditional value-at-risk (CVaR) in a $2\times3$ game. In addition, we identify a finite-crossing condition that ensures convergence of the selected continuous-time dynamics and the discrete-time iterates in general $2\times n$ games, which also provides new insights into the global landscape of continuous-time classical FP in such games.
★ FanCyqlone: A Parallelized Solver for Multi-Horizon Optimal Control
Multi-horizon optimal control aims to select optimal control sequences simultaneously for several possible future scenarios, ensuring consistency at the first stage. While this enables advanced model predictive control schemes that explicitly handle prediction uncertainty, solving these problems tends to be costly. We introduce FanCyqlone, a linear solver that exploits parallelism across both time steps and scenario branches. The method builds on Cyqlone to compute the contribution of each branch and combines these results through a shared root problem. We integrate FanCyqlone into QPALM to develop FanCyQPALM, a solver for multi-horizon optimal control problems with general linear stage-wise constraints. We compare its performance with state-of-the-art solvers on spring-mass and quadruped control benchmarks.
★ Generalized Gated Recurrent Neural Networks for Nonlinear System Identification
Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) networks are the two most widely adopted recurrent neural networks for black-box nonlinear system identification, yet they are invariably treated as mutually exclusive structural choices. To move toward a general framework that inherits the properties of both networks, we introduce the Generalized Gated Recurrent Neural Network (GGRNN), a unified architecture that embeds the exact GRU and LSTM network dynamics through homotopies between their respective gating mechanisms. Each homotopy continuously deforms one mechanism into the other, with a vector of homotopy parameters blending the contribution of the GRU and LSTM counterparts. Since benchmark comparisons report no consistent winner between the two architectures, the gating mechanism best suited to a given system is unlikely to coincide with either endpoint. We thus let each recurrent unit learn its own position along these homotopies, obtaining gating behaviors that neither endpoint can express. We pair the GGRNN with a two-stage training methodology: in Stage 1 we co-train all shared and network-specific parameters in parallel at the GRU and LSTM endpoint architectures, while in Stage 2 we freeze the network parameters and optimize only the homotopy parameters. Experiments on the Wiener-Hammerstein and Nano-Drone benchmarks show improved simulation accuracy over the co-trained GRU and LSTM networks on most output groups, and over the reference black-box baselines in nearly all comparisons.
comment: 6 pages, 5 figures. Submitted to IFAC for possible publication
★ Speed-to-Power Dominates Energy Costs for AI Data Centers: Technoeconomic Analysis of Powering AI in the Continental United States
The rapid growth of artificial intelligence is driving demand for new data centers at large scale and on short deployment timelines. To power them, firms connect to the commercial grid or contract generation directly, but interconnection and gas-turbine delays now incentivize on-site generation. We compare grid-connected and off-grid data center energy infrastructure (AC- and DC-coupled solar-plus-storage, islanded natural gas, and grid interconnection) across the continental U.S. Using an open-source technoeconomic framework built on empirically calibrated AI training loads, climate-driven cooling, stage-by-stage power conversion, and physics-based battery degradation, we model designs providing equivalent energy services. Our central contribution is a speed-adjusted cost of energy that values the compute unlocked by faster construction, making options with different deployment timelines directly comparable on one physically consistent basis. We find DC coupling reduces lifetime energy costs by 18% ($0.09/ kWh) on average versus AC counterparts by eliminating inverters and improving efficiency. Though rarely cheapest on unadjusted energy cost, DC-coupled microgrids' faster deployment makes them the lowest-cost modeled option in 85% of U.S. locations once foregone compute is valued, or 55% under accelerated natural gas timelines. For AI infrastructure, deployment speed rather than unit energy cost determines the economically preferred path to power.
comment: 80 pages incl. SI
★ Total Energy Shaping of Underactuated Systems via Gravity Compensation at the Desired Equilibrium
This paper investigates total energy shaping of underactuated mechanical systems with gravity compensation evaluated at the desired equilibrium instead of throughout the online closed-loop operation. Within the interconnection and damping assignment passivity-based control (IDA-PBC) framework, sufficient conditions for local asymptotic stability are established in terms of integrability of a vector field and positive definiteness of an augmented potential energy. The approach is also combined with a reformulation of potential energy matching that replaces a nonhomogeneous partial differential equation with a homogeneous one and introduces an additional design term. Analytic constructions for the TORA system and a class of two-degree-of-freedom systems illustrate how the matching and integrability requirements can be addressed. The proposed control law avoids online evaluation of the configuration-dependent gravity term; offline controller synthesis and verification can still require knowledge of the gravity model.
★ GRID-MRI: Resilience Assessment of Power Systems under Coordinated Cyber-Physical Attacks
The increasing penetration of inverter-based resources and growing cyber-physical interdependencies expose electrical power systems to disturbances that challenge conventional resilience assessments. Existing approaches often evaluate resilience dimensions independently, overlooking the non-additive effects of coordinated disruptions and the influence of exogenous stressors. To address this gap, this paper proposes a Grid Multidimensional Resilience Index (GRID-MRI) that integrates endogenous disruption severity, coordination effects, and climatic and economic-regulatory conditions. The framework quantifies physical, cyber, and operational degradation, identifies interaction effects through counterfactual (i.e., what-if), simulations of individual and coordinated attacks, and incorporates exogenous stressors as scenario-dependent scaling factors. Its application to a modified IEEE 39-bus system under three escalating cyber-physical attack scenarios, informed by the December 2025 Polish cyberattack incident, reveals that coordinated attacks induce system collapse despite stable individual and additive responses. The corresponding resilience losses are 3.72 and 4.09 times their additive estimates, while GRID-MRI increases from 0.062 to 1.188 and reaches 6.081 under additional exogenous stress. These findings demonstrate the limitations of additive assessments and establish a quantitative basis for distinguishing the dominant mechanisms of multidimensional resilience degradation.
comment: IEEE PESGM 2027, 5 pages
★ Robust Control Design for LTI Systems with Sampling-Time Jitter
Sampling-time jitter makes a sampled-data control loop time-varying, so standard LTI robust-control synthesis does not apply directly. This paper reformulates dynamic output feedback design under bounded, unmeasured sample-to-sample jitter as a robust-control problem. The exact zero-order-hold model is decomposed into a nominal time-invariant plant and a memoryless, time-varying uncertainty block with a norm bound representing the sampling-time jitter. As the block is memoryless, the operator small-gain condition for the time varying loop reduces to a standard robustness test with a matrix norm bound. It is shown that the same test also guarantees uniform exponential stability for every admissible jitter sequence. Embedding the decomposition into a generalized plant with performance channels serves two synthesis objectives with standard robust-control tools: Robust stability with optimized nominal performance through structured H-inf synthesis, and robust performance through mu synthesis with constant scalings. A numerical example shows that jitter can destabilize a control loop with a nominal H-inf controller, whereas both robust designs remain stable and the robust-performance design is less sensitive to jitter at the cost of nominal performance.
comment: Submitted to ECC2027
★ Control-Gibbs Distribution: Sampling for Optimal Control and Diverse Actions
While conventional optimal control selects an optimal input sequence, probabilistic control assigns probability mass over all possible inputs and exploits a distribution of control sequences. Control distributions help enable exploration, capture diverse solutions, and solve deterministic optimal control problems. This work reviews an important probabilistic model for control distributions, the control-Gibbs distribution, i.e., the Gibbs distribution with an optimal control objective as the energy function. We point out its fundamental role in various probabilistic control frameworks, e.g., maximum entropy control, control-as-inference framework, model predictive path integral control, and energy-based models for control. This provides a model-central perspective to unify these probabilistic control frameworks. We also review the associated extensions of dynamic programming for computing closed-loop control distributions. To sample the control-Gibbs distribution, we consider projected Langevin sampling and develop non-asymptotic guarantees. The guarantees show the dual benefits of sampling the control-Gibbs distribution: obtaining diverse control solutions and solving the deterministic optimal control problem. We provide a numerical example where diverse arm trajectories are sampled from a control-Gibbs distribution.
★ An Integrated Architecture for Data-Efficient Real-Time Neural Imitation of Nonlinear Time-Optimal Model Predictive Control
Nonlinear time-optimal model predictive control (TOMPC) enables highly performant control of constrained systems, but its computational cost prohibits real-time use on fast dynamical systems. Neural network imitation of the TOMPC control law can reduce online computational costs, yet existing work mostly targets quadratic cost formulations in simulation and addresses the challenges of neural controllers in isolation. This work imitates a long-horizon nonconvex TOMPC with hard terminal constraints, whose high computational cost limits training-data generation. We present (i) \emph{self-informed sampling (SIFS)}, a data-efficient expert-sampling strategy that uses a learned time-to-go to query the expert only where the policy deviates from time-optimal behavior, (ii) a complete training and control architecture that jointly addresses dataset correlation, suboptimal expert labels, covariate shift, constraint satisfaction and model mismatch, and (iii) real-world validation on a Furuta pendulum. SIFS matches standard dataset aggregation performance in roughly one-third of the training time and with less than half the training data. The resulting controller matches the region of attraction of the expert TOMPC while raising the control frequency from 1.4~Hz to 1~kHz. On hardware, the controller swings up and stabilizes the pendulum along trajectories closely matching the expert TOMPC solution.
comment: Submitted to European Control Conference 2027 (Brussels)
★ CT-RespGeomLib: Deterministic reconstruction of CT airway masks into editable,port-matched lumen geometry
Patient-specific airway modelling requires geometrically consistent lumen surfaces with identifiable boundaries and support for controlled anatomical modifications. We present CT-RespGeomLib, a deterministic framework that converts CT airway masks into two complementary representations: an editable assembly of semantically labelled pipe and junction blocks connected through canonical shared ports, and a fused lumen surface reconstructed from tapered implicit primitives with validated port-to-boundary loop correspondence. The parametric representation further enables localised modelling of airway stenosis and dilation without re-segmentation. Evaluated on 42 reference airway masks from ATM22 (15 cases) and AeroPath (27 cases), the framework produced 37 topologically accepted reconstructions, with a median port-to-loop distance of 0.123 mm across 2174 matched ports. Reconstructed and reference equivalent radii showed strong agreement ($R^2 = 0.976$), with a mean bias of -0.414 mm arising primarily from the circular, inscribed-radius representation. Controlled pathology modelling generated 40 variants across five patient-specific geometries, including 25-75% stenosis and 50% dilation at tracheal and main-bronchus locations. Prescribed radius changes were reproduced with a mean absolute ratio error of 0.0056, while the median 95th-percentile deviation outside the modified region was 0.111 mm. Downstream applicability was demonstrated through volume meshing and an OpenFOAM simulation with individually prescribed model-based resistance conditions at 62 outlets. Although additional surface preparation may be required for numerical meshing, CT-RespGeomLib provides a reproducible framework for CT-derived airway reconstruction, quantitative boundary validation and controlled geometric pathology modelling.
★ Survivability Guarantees for Delayed Serial Supply Chains under Demand Shocks
When demand exceeds the maximum production capacity of a supply chain, it can be satisfied only for a finite time, determined by the inventory stored along the chain. Existing control methods ensure indefinite constraint satisfaction under bounded demand, but provide no guarantees on how long a supply chain can withstand a shock in which demand goes beyond production capacity. We address this gap for a serial supply chain with transportation delays and flow constraints. Under a specified flow policy, we derive necessary and sufficient conditions on the inventory distribution that guarantee a prescribed lower bound on the exit time from the set where demand is served. We use these conditions to compute inventory targets that minimize holding costs via a quadratic program. Then, we propose a two-mode controller that builds reserves under ordinary demand and depletes them during the shock. Simulations illustrate this approach.
★ A Modulation Strategy for Bidirectional Integrated Onboard and Wireless EV Charging
Integrated electric vehicle chargers reduce cost by sharing power conversion hardware between the onboard charger (OBC) and the wireless charger (WC). A previously proposed dual active bridge (DAB)-based OBC/WC shares the rectification stage between the OBC and WC; however, the modulation required to realize its wireless operation has not yet been reported. This paper details a modulation strategy that enables the operation of the integrated OBC/WC topology. Differential-mode operation is used for the OBC, and common-mode operation is used for the WC. In OBC mode, the shared full-bridge produces differential-mode voltage for wired power transfer while ideally generating zero common-mode voltage on the wireless charging path. In WC mode, the same full-bridge produces common-mode voltage for wireless power transfer while ideally generating zero differential-mode voltage on the onboard charging path. The method preserves conventional DAB phase-shift power control and enables bidirectional power transfer in wired and wireless charging modes without additional contactors. Design considerations for the OBC transformer and WC coils are derived based on the resulting square-wave voltage amplitudes. Offline PLECS simulations demonstrate power tracking and decoupled charger operation at $\pm$7.4 kW. Hardware-in-the-loop simulations using a PLECS RT Box and an external Texas Instruments microcontroller further validate the proposed modulation method.
comment: 7 pages, 9 figures. Accepted version of a paper presented at the 2026 IEEE Energy Conversion Congress and Expo (ECCE), Vancouver, Canada
★ Improved Single-Input Space Vector for DC-Biased Sinusoidal Current Control in Battery Cyclers
This paper presents improvements to the single-input space vector (SISV) concept for current control in a battery cycler based on a two-phase interleaved buck converter. The cycler is controlled to output a dc-biased sinusoidal current, enabling investigation of second-harmonic ripple effects on lithium-ion cell aging. A modified SISV generation scheme is introduced to track current references containing both dc and sinusoidal components, and an impedance-angle compensation method is proposed to decouple the d and q control axes without disturbing dc tracking. A dq phase-portrait visualization method is also introduced to evaluate transient performance for both SISV and proportional-integral-resonant (PIR) control methods. Simulation and experimental results validate dc-biased sinusoidal current tracking using PIR control and SISV control with different decoupling schemes. The phase-portrait framework allows for straightforward evaluation of transient performance for both SISV and PIR control methods and supports controller tuning to obtain desired trajectories. The results demonstrate the feasibility of SISV-based current control as an alternative to PIR control for battery-cycler applications. The proposed improvements support the development of battery cyclers capable of controlling both dc and sinusoidal current components.
comment: 8 pages, 17 figures. Accepted version of a paper presented at the 2026 IEEE Energy Conversion Congress and Expo (ECCE), Vancouver, Canada
★ Mode-in-Variance Participation Factors under Stochastic Excitation
This paper investigates how the dynamic modes of a power system contribute to the stationary variance of its variables under continuous stochastic excitation. Using a modal decomposition of the covariance matrix, the variance of each state or algebraic variable can be resolved into components associated with individual modal responses and their interactions. These components are shown to correspond to within-mode variance and cross-modal covariance, respectively, and the corresponding normalized contributions are termed mode-in-variance participation factors (PFs). The proposed PFs can be used to identify modes governing the variability of important system quantities such as frequency and voltage, quantify the effect of modal interactions, and reveal control dynamics that can be targeted to reduce their stochastic variability. These capabilities are demonstrated through case studies based on the Kundur two-area and IEEE 39-bus systems.
★ A New Analytical Method and Interpretation of Loss-of-Load Counts
This paper establishes the relationship between loss-of-load expectation (LOLE) and loss-of-load frequency (LOLF). We show why LOLE generally does not measure distinct-event frequency, mean event duration, or shortage magnitude. We identify conditions under which peak-based LOLE admits a frequency interpretation as the expected count of deficient daily peaks per horizon and distinguish this quantity from LOLF. We also derive an analytical formulation for frequency-and-duration indices that combines a convolution-derived generation probability-and-frequency model with chronological load changes. Under stationary generation, it uses the prescribed load profile directly, without constructing a separate load model or discretizing load magnitudes. Studies using generation and load data from the 24-bus IEEE RTS reveal different event frequencies and mean event durations at identical peak-based LOLE, motivating their separate assessment. Numerical comparisons demonstrate the accuracy advantage of the proposed analytical calculation over the conventional analytical implementations, quantifying the LOLF errors incurred by load discretization.
comment: 5 pages, 3 figures, 1 table, submitted to IEEE PESGM
★ Adjacent Heat Transfer Effects Can Bias Savings Estimates When Controlling Only Part of a Multi-Zone Building
Field demonstrations of advanced heating, ventilation, and air-conditioning control often operate a few thermal zones within a larger building and report savings for those zones alone. A recent theorem showed that this practice overestimates whole-building savings, because heat drawn from the neighboring zones is paid for by their equipment and is not counted. The error grows with the thermal conductance between the controlled and uncontrolled zones, with how much colder the controlled zones run over the test, and with the price of the heat their neighbors supply. For one controlled zone with one neighbor, a single thermal price, and slowly changing prices, the relative error reduces to the ratio of two conductances, a screening number that can be evaluated from the two conductances before a demonstration begins. This paper tests how far that reduction can be taken. A two-story house with one thermal zone per floor is simulated over a 90-day heating season in West Lafayette, Indiana, with receding-horizon model predictive control on the first floor and setpoint control on the second, under four combinations of electricity tariff (time-of-use or flat) and heating plant (heat pump or electric resistance). The exact correction from the theorem recovers the true savings in every case, and the full theorem, evaluated from the controlled zone's temperatures, the prices, and the building parameters, reproduces the measured relative error in every case. The conductance ratio bounds that error from above in all four cases, tightly under a flat tariff, within 3%, and loosely under a time-of-use tariff, where most of the controller's savings come from shifting load through the thermal mass rather than from lowering the temperature.
comment: 10 pages. Accepted at the 2027 ASHRAE Winter Conference
★ Quantum Network Flows That Solve Linear Equations
We develop a distributed quantum network flow for solving linear equations from locally stored row data. Local pumping enforces each node's equation constraint, while neighboring registers interact through swaps. The node marginals obey an exact, closed flow combining consensus with local constraint enforcement. For consistent equations, all node states converge exponentially to a common state supported on the solution space; a unique solution yields full-network convergence to the product solution state. For inconsistent equations of full column rank, scaled encoding and readout recover the least-squares solution through successive large-gain and small-scale limits. For consistent systems, the marginal convergence rate is nondecreasing and concave in the swap gain, approaching a graph-independent limit for fixed local dissipators. Its leading finite-gain correction couples graph topology and row placement through the Laplacian pseudoinverse, enabling semidefinite optimization of edge rates under a fixed total swap-rate budget. For invertible systems, a dissipative HHL-type construction using the same pumping law provides a comparison of solution-state encodings, decay rates, and row-update locality under squared-row-norm pumping. Numerical examples are provided to validate the presented results.
★ FlatSeek: LLM-Driven Symbolic Discovery of Flat Outputs via Backward Data Generation
Differential flatness enables efficient planning and control, but discovering explicit flat outputs for general nonlinear systems remains difficult due to the lack of general methods and supervised data. We propose FlatSeek, a framework combining backward data generation with task-specific large language model (LLM) training. By reversing the defining reconstruction of flatness, we generate system-flat-output pairs with known certificates, then fine-tune an LLM to propose symbolic flat-output candidates for unseen dynamics. On literature and synthetic benchmarks, FlatSeek outperforms both the traditional symbolic methods and the untuned reference models in flat-output discovery success, and uncovers alternative symbolic flat outputs not reported in the literature.
★ Structure-Preserving Learning for Nonlinear Nonholonomic Dynamics
Learning the dynamics of mechanical systems subject to nonlinear nonholonomic constraints is challenging, as data-driven models may violate the underlying physical constraints, leading to inconsistent predictions. To address this problem, we develop a constraint-preserving learning framework based on the Lagrange-d'Alembert-Chetaev equations and tangency to the constraint submanifold, which together define an affine space of admissible accelerations. Using the kinetic-energy metric and the Moore-Penrose pseudoinverse, we construct a metric projector and the corresponding Chetaev correction. Embedding this geometric structure directly into a Gaussian process prior ensures that both prior and posterior dynamics satisfy the acceleration constraints exactly. We characterize the resulting reproducing kernel Hilbert space (RKHS), derive an adapted-coordinate representation, and establish nonexpansiveness and consistency transfer. Numerical experiments on a three-dimensional directional flocking system with up to 50 agents demonstrate exact constraint preservation and robustness to constraint-incompatible observations.
★ Velocity-Free Funnel Control of Euler-Lagrange Systems: An Artificial-Delay Approach
Classical funnel control keeps the tracking error within a prescribed, time-varying envelope by increasing the feedback gain as the error approaches the boundary. For mechanical systems, this guarantee typically relies on velocity measurements. We show that it can be retained for fully actuated Euler--Lagrange systems using only position measurements. The proposed controller replaces the velocity error with a difference quotient of the position error over a fixed delay, while preserving the classical unbounded gain. It requires no plant model, reference derivatives, observer, or filter states. The central challenge is to reconcile the averaging inherent in delayed feedback with a gain that can grow without bound near the funnel boundary. We resolve this by establishing an a priori bound on the gain along closed-loop trajectories, independent of the delay. This bound yields a computable delay threshold that guarantees tracking within the prescribed funnel and boundedness of all closed-loop signals. The threshold depends only on bounds on the inertia, forces, and reference. Simulations demonstrate tracking accuracy comparable to full-state funnel control and tolerance of delays substantially longer than certified. With quantized position measurements, the controller also requires substantially less torque than an existing derivative-free funnel controller.
★ Energy system device sizing under closed-loop operation with imperfect forecasts
The value of renewable energy devices depends on their operation, which must adapt to weather and demand using limited, imperfect forecasts that update over time. Conventional energy systems planning practices assume unlimited foresight in a single-level formulation, potentially leading to long-term investment in suboptimal device sizes. Accounting for realistic operation corresponds to a bi-level problem: the upper level selects device sizes, while the lower level contains closed-loop operation under evolving forecasts. We solve this problem using backpropagation MPC (BP-MPC), a gradient-based method, with a deterministic MPC as the lower-level controller. Using forecast errors generated by a parameterized martingale model of forecast evolution, we investigate how forecast accuracy affects the benefits of operation-aware energy systems planning and the resulting device sizes. In a case study of an energy system comprising photovoltaics, a battery, a heat pump, and thermal storage with uncertain weather and energy demands, accounting for closed-loop operation yields higher average cost savings as forecast error increases. The resulting designs achieve these savings by progressively shifting investment from battery storage to solar generation and thermal storage, which buffers forecast mismatches.
comment: 8 pages, 7 figures, 2 tables. Submitted to the 2027 European Control Conference (ECC)
★ Can STATCOM and UPS-Based Voltage Support Mitigate Fault-Induced Reductions in AI Data Center Grid Power Draw?
Artificial intelligence data centers (AIDCs) are rapidly increasing in number and capacity. These large electronic loads are interfaced through power electronic devices and can sharply reduce grid power intake during fault-induced voltage sags. Disturbances exceeding equipment ride-through capabilities may also trigger protective disconnection, posing risks to power system security. Static synchronous compensators (STATCOMs) and appropriately controlled AIDC uninterruptible power supply (UPS) rectifiers can provide reactive power support during voltage sags. This paper compares the individual and combined effectiveness of STATCOM and UPS-based voltage support in mitigating fault-induced reductions in grid power intake through electromagnetic transient simulations of a 1 GW AIDC connected to an IEEE nine-bus system. Simulation results show that STATCOM support better preserves short-term intake in milder tested faults, whereas UPS coordination maintains rectifier operation under deeper sags by reducing active intake to reserve reactive current capacity. Combined support improves auxiliary load retention and recovery in selected cases. However, severe tested sags still cause substantial grid-demand reductions even with combined support.
comment: 10 pages, 5 figures
★ Incremental Model Predictive Control for Reference Tracking of Nonlinear Anesthesia Dynamics
Anesthesia dynamics are commonly described by Wiener models, consisting of linear dynamics followed by a static nonlinearity. This nonlinear output complicates the derivation of theoretical guarantees on closed-loop stability and convergence. Model Predictive Control (MPC) is particularly suitable for anesthesia control, as it explicitly accounts for constraints in the control design. In this paper, we propose an incremental MPC formulation for tracking a desired reference. The proposed approach exploits the structure of the nonlinear output equation to reformulate the control problem within a linear framework. An incremental model is then employed to avoid the explicit computation of steady states and inputs, which may be inaccurate in the presence of model uncertainties. The proposed controller is evaluated through simulations involving 1000 virtual patients and compared with a benchmark MPC formulation. Results show improved steady-state reference tracking while maintaining satisfactory disturbance rejection.
★ A Joint LLM and Optimization Agent Solution for Drone Scheduling in Victim Search Missions
Victim search in wilderness areas is time-critical. Drones enable rapid area search, but scheduling them is challenging because victims may continue moving, detection results are uncertain, and effective decisions must combine structured mission information with unstructured contextual clues and domain knowledge. To address this, we propose an agent-aided drone scheduling system. The system employs an agent at the ground control station (GCS) to coordinate multiple drones over successive search cycles. In each cycle, the agent applies a scheduling policy that combines a large language model (LLM) for reasoning about where to search with an optimizer for determining how to dispatch the drones under energy constraints. Simulation results show that the proposed system consistently completes the victim-search mission, whereas none of the six non-LLM baselines does so and direct LLM scheduling is considerably less reliable. The source code is publicly available at https://github.com/1643204431/llm-uav-victim-search.
★ Input-output characterization of robotic grasping: Connecting design parameters to grasp behavior through the admittance
Low-impedance grippers make the dynamics of a grasp matter. Current grasp metrics, however, are static: they describe how a held object resists constant loads, not how it responds to time-varying ones. To quantify this, we rely on small-signal analysis to characterize a grasp by its admittance, the transfer matrix relating load perturbations to object displacements around an equilibrium. This transfer matrix recovers the stiffness and stability of the grasp at zero frequency and captures its natural frequencies and peak gain under time-varying loads. We apply this characterization to a planar two-finger gripper under proportional-derivative control. We show that two grasps of the same object in the same position and with the same contact forces can differ in stiffness by two orders of magnitude. Likewise, two grasps that share the same stiffness can exhibit very different resonances. Nonlinear simulations confirm the effectiveness of our small-signal analysis in capturing the dynamic behavior of grasping.
★ On Energy-Based Modelling for Fluidic Excitable Neurons
Motivated by the potential of pneumatic excitable devices in soft robotics, this paper develops an energy-based, domain-agnostic equivalent of the Hodgkin-Huxley modelling framework using bond graph theory. The proposed model is used to guide the design of a pneumatic excitable neuron. The potential of this approach is illustrated through the design of a pneumatic half-center oscillator, an essential building block for rhythmic neuromorphic behaviours.
comment: 6 pages, 5 figures, submitted to European Control Conference 2027
★ Strategically Robust Multi-Agent Control
A key source of uncertainty in multi-agent systems is the behavior of other agents. Game-theoretic control approaches ignore this uncertainty, while robust control approaches treat others as fully adversarial. In this paper, we introduce strategically robust multi-agent control to populate the spectrum between these two paradigms. Specifically, in dynamic games, we propose that players are strategically robust and seek protection against misspecified control policies of the other players. This formulation induces a novel equilibrium concept, strategically robust dynamic equilibrium, that interpolates between game-theoretic control and robust control. We study this concept across canonical open- and closed-loop settings and show that accounting for strategic robustness also has computational benefits over classic approaches in game-theoretic control. Numerical experiments on vehicle platooning, inventory management, and network control illustrate resilience to misspecification, and also exhibit a "free-lunch" phenomenon in which strategic robustness can improve performance or social welfare.
comment: 16 pages, 10 figures, 3 tables
★ DER Aggregators in Wholesale Electricity Markets: A Hybrid Mean-Field Framework with Endogenous Price Feedback
The rapid growth of distributed energy resources (DERs), including rooftop solar and energy storage, is transforming the grid edge, where customer-side assets increasingly interact with the broader power grid. DER aggregators, entities that coordinate the actions of many small-scale DERs, play a key role in this transformation. This paper presents a hybrid Mean-Field Control (MFC) and Mean-Field Game (MFG) framework for integrating DER aggregators into wholesale electricity markets. Unlike traditional approaches that treat market prices as exogenous, our model captures the feedback between aggregators' collective actions and electricity prices. The MFC component optimizes DER operations within each aggregator, while the MFG models strategic interactions among multiple aggregators. To account for various uncertainties, we incorporate reinforcement learning, which allows aggregators to learn optimal bidding strategies in dynamic market conditions. We prove the existence and uniqueness of a mean-field equilibrium and validate the framework through a case study of the Oahu Island power system. Results show that our approach reduces price volatility and improves market efficiency, offering a scalable and decentralized solution for DER integration in wholesale markets.
★ GridForge: A Tractable Network-Flow Method for Constructing Open, Continental-Scale Synthetic Power Grids
The goal of this paper is to construct open, continental-scale synthetic transmission grids with an optimization method that stays tractable. Security restrictions keep models of the real grid out of open research and machine-learning datasets, and optimization-based construction is hard at scale: the number of candidate lines grows combinatorially, so electrically faithful methods prune the candidates in advance. This paper presents GridForge, which keeps every bus pair within physical length limits as a candidate, selects lines with Garver's length-weighted network-flow linear program instead of binary decisions, and resizes them with a sequential direct-current (DC) power flow. Two stages of 13 regional networks are evaluated. Their node-degree distributions are nearly as strongly log-linear as those of the real Eastern and Western Interconnections ($R^2$ of 0.92 to 0.98 against 0.971 and 0.981), although the areas under the fitted lines differ from the real ones by 39.9% on average. On the upper Midwest and Tennessee Valley footprints, resizing lowers the DC power-flow loss from 5.89% to 4.14% and from 14.55% to 7.05%, and the DC optimal power-flow loss from 4.03% to 2.02% and from 5.54% to 2.47%. Without relaxing any line limit, the DC optimal power flow is feasible in 78.81% and 0.82% of the scenarios. Most lines are built at 69 kV. The models span 250 to 78,484 buses, were built on single machines, and are freely redistributable.
comment: 30 pages, 10 figures, 7 tables; supplementary material as an ancillary file. Submitted to the International Journal of Electrical Power & Energy Systems. Data: https://doi.org/10.5281/zenodo.24096575; code: https://doi.org/10.5281/zenodo.23826484
★ A Coupled Control Barrier Function Approach to Deadlock-Free Multi-Robot Conflict Resolution
Many robotic applications involve scenarios where multiple robots interact and the decisions of one robot influence the behavior of others. This paper presents a quadratic programming (QP)-based feedback controller for multi-robot conflict resolution with input constraints. Safety is enforced using coupled Control Barrier Function (CBF), while a yielding matrix encodes how robots influence each other. Unlike existing methods with fixed or hard priorities, the proposed framework enables yielding preferences while guaranteeing deadlock-free operation by design. Sufficient conditions are established for the controller to be locally Lipschitz. The optimal CBF sign- indefinite slack is shown to admit a finite uniform bound, which is compensated through an enlarged CBF safety margin to obtain a forward invariant set contained within the desired safe set. The analysis also identifies an irreducible slack component due to finite maximum deceleration that cannot be eliminated by increasing the slack penalty. Finally, sufficient conditions on the yielding matrix and nominal controller are derived to exclude deadlock states. Simulations demonstrate adaptive coordination across yielding preferences and robustness to robot failures.
★ Fair and Efficient Distributed EV Charging Coordination Using Threshold Cryptography and Machine Learning
Distributed coordination of electric vehicle (EV) charging with the alternating direction method of multipliers (ADMM) keeps EV constraints private but assumes that every participant follows the algorithm. This assumption fails in practice, where a biased operator or a man-in-the-middle (MITM) attacker can send inconsistent signals to different EVs, and compromised EVs can deviate from their prescribed updates. We first prove that attacks by such arbitrarily malicious, i.e., Byzantine, participants do not prevent convergence and therefore remain undetected, and derive the most damaging attack in closed form. We then propose a framework that combines a threshold-signed broadcast with upload recomputation, which detects every deviation without false alarms at linear instead of quadratic communication cost. To keep the optimization running, each affected EV recovers locally with a certified, fleet-aware machine learning price forecast, and we prove that the optimization remains convergent with a recovery cost bounded by the shape of the forecast error. Simulation results with real-world load, price, and charging data show that a biased operator raises the annual cost of disadvantaged EVs by 132.7% and lowers that of advantaged EVs by 49.3% without changing the aggregate load, whereas the defense limits all cost changes to 2.6%. An external MITM attacker raises the cost of its victims by 57.6%, the peak load by 5.8%, and the grid cost by 15.8%, whereas the defense keeps them within 1.1%, 0.2%, and 0.3% of benign operation. With up to 70% of the EVs attacked, the recovery converges in at least 97.8% of the runs.
comment: 16 pages
★ Natural Swing-Up of an Underpowered Double Pendulum
Robots with weak motors must ride their natural dynamics rather than fight them. We swing up a direct-drive planar two-link robot, whose motors deliver only about 6\,\% of its shoulder gravity torque, along its nonlinear normal modes, two-dimensional invariant manifolds consisting of periodic orbits. On a non-strict mode, injecting energy necessarily moves the state off the manifold, so pumping and stabilization compete for the same bounded torque. We split the input torque into a power-neutral steering component and a scalar pumping input, the only component that changes the energy. The allocation then becomes a linear program in two variables: maximize pumping subject to the torque limits and one control-Lyapunov constraint. The constraint uses the model-based rate of the error, so no stabilizing feedback gain is needed. Experiments in hardware show swing-ups along both modes and compare the allocation with summing-and-clipping and with prioritized stabilization.
★ Sub-Synchronous Control Interaction Risk of Large Data Center Loads: A Frequency-Scan Assessment with Cross-Model Validation
Data centers are now one of the fastest-growing loads on the bulk power system, and most of their demand reaches the grid through power converters inside uninterruptible power supplies (UPS). Whether these loads can take part in sub-synchronous control interaction (SSCI), as wind and HVDC converters do, has not been studied extensively. We examine this question with impedance-based frequency scans at the point of common coupling (PCC). Using a detailed proprietary model (INS-ISU), we scan the motor, variable-frequency-drive (VFD), lighting, and UPS-fed information-technology (IT) branches of a representative facility separately. Only the UPS in double-conversion mode shows negative resistance; it covers the whole sub-synchronous band and continues up to about 170 Hz. The other branches, and the UPS in bypass mode, stay passive. We then repeat the scans on three public models from PNNL, TAMU, and NLR. None of them shows negative resistance in bypass mode, while three of the four models do in double-conversion mode. Lowering the load from rated to light load makes the negative resistance deeper, as expected for a constant-power load, so a lightly loaded facility in double-conversion mode is the worst case. Raising the UPS DC-link capacitance pushes the negative resistance into a narrow band around the fundamental. These findings give a basis for screening data centers for SSCI during interconnection studies.
comment: 13 pages, 16 figures. Submitted for IEEE publications
★ Learning monotone game-theoretic linear MPC
We propose a method to learn a game-theoretic model predictive control (GT-MPC) law by imitation from input and output data of a running multi-agent system. By interpreting the recorded inputs as the first moves of the equilibria of a game played by the agents, we learn the GT-MPC weights under hard input constraints and soft output constraints, without requiring the agents to reveal their objectives. We also impose that the resulting generalized Nash equilibrium (GNE) problem is strongly monotone, which ensures the existence and uniqueness of the equilibrium and, hence, that the learned GT-MPC law is well defined and can be evaluated efficiently online by variational GNE-seeking algorithms for linear-quadratic games. Moreover, strong monotonicity implies that the equilibrium is almost everywhere differentiable with respect to the references and the weights, which enables automatic differentiation for gradient-based optimization. In addition, we promote closed-loop stability of the unconstrained equilibrium feedback by incorporating a penalty that involves a jointly learned quadratic Lyapunov function. Learning a GT-MPC formulation enables us to explain the agents' objectives, reproduce their behavior in simulation in new scenarios, optimize their set-points, design mechanisms, and enforce additional constraints. A numerical example illustrates the approach.
comment: 15 pages, 1 figure
★ RAZeR: Representation Aware Zonotope Reduction
Reachability analysis is a popular technique for verifying the safety of a dynamical system under uncertainty, where the uncertainty is often represented by zonotopes. As the size of the zonotope representation typically grows as the reachability analysis progresses, it must be repeatedly reduced at the cost of additional conservatism. While various methods exist to reduce the size of the representation, combined advantages are not fully leveraged as only a single reduction method is typically used throughout the analysis. In this work, we show how dynamically choosing which method to use can yield significantly tighter results, thereby easing verification. This is realized by introducing a heuristic, derived from model predictive control, that accounts for upcoming propagation of the current zonotope. We argue that order reduction is not merely a post-processing step after propagation but also a pre-processing step that shapes how the zonotope evolves under the system dynamics. We evaluate our approach, RAZeR, on nonlinear benchmarks taken from the ARCH-COMP, where we demonstrate that RAZeR produces tighter zonotopes at a negligible performance overhead.
★ Methanol substitution suppresses deflagration-excited knock and increases the efficiency of a hydrogen-fuelled CFR engine
Knock restricts the performance of hydrogen spark-ignition engines and is usually attributed to end-gas autoignition. To the authors' knowledge, this is the first study in which methanol is used to replace part of the hydrogen in a hydrogen engine at constant fuel energy addition, and it shows that near the knock boundary the limiting oscillation is excited by the deflagration itself, which methanol removes without derating the performance. Port-injected methanol replaced 20--50% of the hydrogen energy in a variable-compression Cooperative Fuel Research engine at 600 rpm and compression ratios of 8--12. With pure hydrogen, oscillation in the 4--15 kHz range began before 10% of the charge had burned in 97--100% of cycles, and its maximum amplitude (MAPO) followed the maximum pressure-rise rate across 2499 cycles (Spearman $ρ=0.94$). Methanol slowed the deflagration: it replaced high-flame-speed hydrogen, cooled the intake by 24--25 K and, at unchanged intake pressure, enleaned the mixture by 10--13%. At 50% methanol, the 10--90% combustion duration lengthened 2.7--3.0-fold, oscillation onset moved 4.5--8.3$^\circ$CA after CA10 and mean MAPO fell by 89--93% although the spark timing was advanced by 13--19$^\circ$CA, and IMEP and indicated thermal efficiency rose by 2.4--7.8%. Methanol was thus worth more than this advance, and at matched CA50 MAPO was still about 90% lower. The measured combustion duration of all 25 data sets scaled with the computed laminar flame speed as $S_L^{-0.58}$ ($R^2=0.99$); fuel replacement accounted for about three quarters of the reduction in $S_L$, and the end-gas autoignition integral stayed below 0.07, against 1 at autoignition. The chamber kept its cylindrical acoustic modes, whose frequencies fell by 8--9%. The trade-off is formaldehyde emissions, which rose five- to sixfold while NO$_x$ fell by 49--85%.
comment: 20 pages 8 figures
★ Admissibility-Preserving Control for Block Strict-Feedback Systems Under Time-Varying Asymmetric Actuator and State Constraints
This paper addresses the tracking control of block strict-feedback nonlinear systems subject to time-varying asymmetric actuator and full state constraints. To that end, we develop a time-varying Admissibility-Preserving Control (APC) framework, in which the Admissibility-Preserving Input Realization (APIR) generates the plant input through a dynamic realization. The APIR guarantees input admissibility under the moving actuator set whenever each actuator bound contracts slower than the decay rate of the realization, and a closed-form lower bound on its gain is derived. The state constraints are handled by a logarithmic barrier coordinate at every backstepping step, with widths measured from the virtual control to the moving envelope of the corresponding state. We prove that every state lies inside its envelope, with each virtual control only required to remain inside the envelope of the next state, while the tracking error converges to zero exponentially. Simulations demonstrate the efficacy of the proposed safety-critical control scheme.
♻ ★ Stability Despite Misalignment: Model Predictive Game Controllers with Objective Misspecifications
Predictive control requires the controller to possess an embedded prediction model. In multi-agent control, this prediction model must consider the behavior that emerges from other agents' strategic decisions. Game theory offers a natural model for the emergent collective behavior of self-interested agents. Model predictive games (MPGs) are a class of controllers in which an agent iteratively solves a finite-horizon game to predict the behavior of a multi-agent system and synthesize their own control action. When all agents use the same game model to select their control action, conditions have been found that guarantee the closed-loop stability of the multi-agent system. However, when multiple agents implement their own MPG controllers, their respective game models may be misspecified due to inaccurate estimates or conjectures about other agents' objectives. This paper analyzes the consequences of these prediction misalignments on the system's behavior. We show that existing stability conditions for homogeneous MPG controllers do not extend to the heterogeneous setting, even when they hold for each agent's individual game. We provide sufficient conditions for the stability of multi-agent dynamical systems with heterogeneous MPG controllers adopting linear-quadratic (LQ) games. We also show that misaligned game models can destabilize a system that reaches stability when all agents share the same game model, using a financial duopoly simulation.
♻ ★ Respiratory Mask Testing: Airway Inertance and System Dynamics
Life support equipment coupled to the human lungs forms a complex, nonlinear dynamic system. Current respiratory mask test standards rely on sinusoidal breathing machines and surrogate trachea geometries that substantially reduce physiologically relevant airway inertance. This paper shows that the ISO 16900-5 trachea does not generate physiologically meaningful exhalation momentum, limiting the ability of standard test configurations to reveal valve dynamics relevant to human use. A physiologically motivated "lip-nozzle" surrogate is introduced as a diagnostic tool to restore representative momentum transfer, enabling observation of exhalation behavior that is not observable under existing test conditions. Nonlinear dynamic simulations incorporating this surrogate identify a force-balanced operating regime in which exhalation-jet momentum can offset flow-induced valve resistance. In one representative case, the exhalation mean effective pressure imposed on the wearer falls by approximately 85% and the flow work across the exhalation valve by approximately 92%. Because people do not exhale through a fully open mouth, this momentum is available in normal use. A valve aligned with the exhaled jet can recover part of the energy the wearer has already spent, rather than requiring additional effort. The magnitude of this effect depends on valve characteristics such as efficiency and the relationship between restoring force and lift, making it a design consideration for future valve development. Implications for mask design, valve placement, and future test methods are discussed. This work does not propose new performance limits or acceptance criteria; rather, it demonstrates how current test fixtures suppress inertance- and momentum-driven effects that can materially influence exhalation valve dynamics.
comment: v2: all simulation cases re-run on the current model; spurious flow through the seated exhalation valve removed; exhalation mean effective pressure calculation corrected, with breathing and valve flow-work measures both reported; Case 2 evaluated at P_bh = 0.1 iwc
♻ ★ A Tutorial on Gaussian Process Learning-based Model Predictive Control
This tutorial provides an accessible and implementation-oriented introduction to Gaussian process learning-based model predictive control (GP-MPC), which combines probabilistic residual modeling with receding-horizon control. Its central tutorial contribution is a detailed, step-by-step derivation of multi-step mean and covariance propagation for GP-augmented prediction models. The derivation shows how commonly used propagation formulas follow from the laws of total expectation and total covariance, and clarifies the roles of the GP posterior mean, GP posterior covariance, uncertain inputs, and query-output cross-covariances. Building on this foundation, the article distinguishes mean-only unconstrained GP predictive control from uncertainty-aware constrained GP-MPC, clarifies regulation and output-tracking formulations, and provides concise implementation and computational-complexity guidance, including practical software-tool references and discussion of the scope of closed-loop guarantees. Mobile-robot path-following examples illustrate mean-only unconstrained implementations, whereas mixed-vehicle platooning illustrates uncertainty-aware constrained GP-MPC with uncertainty-dependent safety-distance tightening. The tutorial is intended to help researchers and practitioners understand, implement, and critically evaluate GP-MPC designs for robotic and autonomous systems operating under modeling uncertainty.
comment: Author's accepted manuscript
♻ ★ Convergence and stability of Q-learning in Hierarchical Reinforcement Learning
Hierarchical Reinforcement Learning promises, among other benefits, to efficiently capture and utilize the temporal structure of a decision-making problem and to enhance transfer learning capabilities, but theoretical guarantees lag behind practice. In this paper, we propose a Feudal Q-learning scheme and investigate under which conditions its coupled updates converge and are stable. By leveraging the theory of Stochastic Approximation and the ODE method, we present a theorem stating the convergence and stability properties of Feudal Q-learning. This provides a principled convergence and stability analysis tailored to Feudal RL. Moreover, we show that the updates converge to a point that can be interpreted as an equilibrium of a suitably defined game, opening the door to game-theoretic approaches to Hierarchical RL. Lastly, experiments based on the Feudal Q-learning algorithm support the outcomes anticipated by theory.
♻ ★ The role of distributed friction dynamics on heave-induced motion of offshore drilling systems: modelling, analysis, and simulations
This paper investigates the influence of friction dynamics on heave-induced disturbance propagation using a coupled hydraulic-structural drilling model formulated as a semilinear interconnection of ordinary and hyperbolic partial differential equations (ODE-PDEs), and capturing spatially distributed memory effects in the drillstring-wellbore interaction. First, local and global existence and uniqueness results are established, and sufficient conditions for boundedness and input-to-state stability are deduced via a small-gain analysis. Subsequently, extensive numerical investigations are carried out, including harmonic excitation studies, nonlinear distortion analysis, spatial amplification investigations, mechanism-isolation experiments, parameter sensitivity analysis, and broadband simulations under realistic offshore sea-state excitation. The simulation results indicate that, whilst the hydraulic response is governed primarily by wave propagation phenomena, compressibility, and boundary reflections, the structural dynamics display substantially greater sensitivity to distributed friction effects. Furthermore, drillstring-wellbore friction is shown to have a relatively limited influence on well-posedness but a potentially significant impact on stability margins. These findings highlight the importance of accounting for coupled hydraulic-structural dynamics and distributed friction mechanisms when analysing, monitoring, and controlling offshore drilling systems.
comment: 45 pages, 11 figures. Under review
♻ ★ Physics-informed structured learning of a class of recurrent neural networks with guaranteed properties
This paper proposes a physics-informed learning framework for a class of recurrent neural networks tailored to networked systems. The approach aims to learn control-oriented models that preserve the structural and stability properties of the plant. The learning algorithm is formulated as a convex optimisation problem, allowing the inclusion of linear matrix inequality constraints to enforce desired plant features. Furthermore, when the plant exhibits structural modularity, the resulting optimisation problem can be parallelised, requiring communication only among neighbouring subplants. Simulation results show the effectiveness of the proposed approach.
♻ ★ SAGAS: Semantic-Aware Graph-Assisted Stitching for Offline Temporal Logic Planning
Linear Temporal Logic (LTL) provides a rigorous framework for specifying long-horizon robotic tasks, yet existing approaches face a trade-off: model-based synthesis relies on accurate labeled transition systems, whereas learning-based methods often require online interaction, task-specific rewards, or specification-conditioned training. We study LTL-specified robotic planning and execution in an offline, model-free setting, where the agent is given only fixed, task-agnostic trajectory fragments, with no dynamics model, task demonstrations, or online data collection. To address this setting, we propose SAGAS, a framework that combines the compositionality of symbolic synthesis with the data-driven reachability structure learned from offline trajectories. SAGAS first learns a reusable latent reachability graph and a frozen goal-conditioned executor from fragmented offline data. For each new LTL formula, it performs task-time semantic graph augmentation to ground state-defined propositions on the learned graph and applies Büchi product search to synthesize a cost-aware accepting prefix--suffix waypoint plan executed by the frozen executor. This separation of learning and planning enables zero-shot generalization to unseen LTL formulas and new predicates supported by the offline data, without task-specific rewards, additional data collection, or policy retraining. Experiments on LTL task suites constructed from OGBench locomotion domains show that SAGAS achieves higher success rates and lower execution costs than baselines on diverse unseen tasks, with further evaluation in OGBench Scene demonstrating its applicability to robotic manipulation.
♻ ★ On the Nyquist Criterion and the Scaled Relative Graph for SISO systems: A Generalized Circle Criterion
Scaled Relative Graphs (SRGs) provide a novel graphical frequency-domain method for the analysis of nonlinear systems. However, existing SRG-based methods are limited to stable open-loop dynamics and can lead to false conclusions when applied to unstable Linear and Time-Invariant (LTI) components. In this paper, we identify the source of this pitfall and resolve it by introducing the extended SRG, which augments the SRG of an LTI system with the encirclement information of the Nyquist criterion. We show how the extended SRG can be used for stability and $H_\infty$-performance analysis of LTI systems and the computation of SRGs for unstable LTI systems. Building on these results, we derive graphical stability and incremental $L_2$-gain conditions for three canonical nonlinear feedback interconnections: the Lur'e system, the controlled Lur'e system, and an LTI plant with Lur'e controller. As a corollary, we derive a user-friendly generalized circle criterion that relates the $L_2$-gain directly to separation between the Nyquist curve and the critical disk from the classical circle criterion, requiring SRGs only for the proof, not its statement. The proposed approach is validated on several examples involving unstable LTI dynamics and nonlinearities, demonstrating applicability beyond the capabilities of existing SRG methods.
comment: Revised version of the paper previously titled "Scaled Relative Graph Analysis of General Interconnections of SISO Nonlinear Systems"
♻ ★ Ramping Procurement and Bid-Cost Recovery in Real-Time Market
We study ramping procurement co-optimized with economic dispatch under net-demand uncertainty. We examine two flexible ramp product designs implemented by grid operators: single-interval and multi-interval co-optimization. Both rely on rolling-window stochastic optimization with binding and advisory interval decisions. We develop analytical frameworks to evaluate generator profits, consumer payments, bid cost recovery (BCR), and operational efficiency. In particular, net-demand uncertainty may lead to generator under-compensation, requiring discriminatory BCR. While operational efficiency is invariant to energy and ramp prices, producer profits and consumer payments depend critically on pricing. We examine locational marginal pricing (LMP) and two uniform pricing: maximum dispatch cost pricing (MDCP) and maximum temporal locational marginal pricing (MTLMP). With out-of-market BCR, LMP yields discriminatory energy prices, whereas MDCP eliminates BCR and MTLMP does so in most cases. This property enables us to establish truthful bidding incentives for price-taking generators under MDCP. Our analysis highlights trade-offs between single- and multi-interval co-optimization and pricing designs: multi-interval energy-ramp co-optimization outperforms single-interval co-optimization when ramping forecasts are accurate and ramping needs are tight, but its advantage diminishes under large forecast errors and moderate ramping needs. Empirical results on CAISO and ERCOT data show that MDCP and MTLMP increase producer profits with negligible BCR, albeit at the expense of higher consumer payments relative to LMP.
comment: 5 figures
♻ ★ Mean--Variance Portfolio Selection by Continuous-Time Reinforcement Learning: Algorithms, Regret Analysis, and Empirical Study
We study continuous-time mean--variance portfolio selection in markets where stock prices are diffusion processes driven by observable factors that are also diffusion processes, yet the coefficients of these processes are unknown. Based on the recently developed reinforcement learning (RL) theory for diffusion processes, we present a general data-driven RL approach that learns the pre-committed investment strategy directly without attempting to learn or estimate the market coefficients. For multi-stock Black--Scholes markets without factors, we further devise an algorithm and prove its performance guarantee by deriving a sublinear regret bound in terms of the Sharpe ratio. We then carry out an extensive empirical study implementing this algorithm to compare its performance and trading characteristics, evaluated under a host of common metrics, with a large number of widely employed portfolio allocation strategies on S\&P 500 constituents. The results demonstrate that the proposed continuous-time RL strategy is consistently among the best, especially in a volatile bear market, and decisively outperforms the model-based continuous-time counterparts by significant margins.
comment: 99 pages, includes appendix. Accepted for publication in Management Science
♻ ★ ART for Diffusion Sampling: A Reinforcement Learning Approach to Timestep Scheduling NeurIPS 2026
We consider time discretization for score-based diffusion models to generate samples from a learned reverse-time dynamic on a finite grid. Uniform and hand-crafted grids can be suboptimal given a budget on the number of time steps. We introduce Adaptive Reparameterized Time (ART), which controls the clock speed of a reparameterized time variable to redistribute computation along the sampling trajectory while preserving the terminal time, with the objective of minimizing the aggregate Euler discretization error. We derive a randomized companion ART-RL that recasts ART as a continuous-time reinforcement learning problem with Gaussian policies, and prove a two-directional bridge between the two: the deterministic ART optimum lifts to an optimal Gaussian policy, and conversely any optimal Gaussian policy must recover the ART control through its mean. This bridge turns continuous-time actor--critic learning into a principled, rather than heuristic, route to the deterministic timestep optimum. Within the official EDM pipeline, ART-RL improves FID on CIFAR--10 across a wide range of budgets; after one-time offline training, the distilled deterministic schedule transfers without retraining to AFHQv2, FFHQ, and ImageNet at no extra inference cost.
comment: 12 pages. Accepted at NeurIPS 2026
♻ ★ Vector-Field-Guided Constraint-Following Control for Geometric Path Following of Uncertain Mechanical Systems
Achieving dynamics-level control is a critical yet challenging problem for accurate, high-speed geometric path-following of uncertain robotic systems. This paper proposes a vector-field-guided constraint-following control framework for geometric path following of uncertain mechanical systems. The key idea is to encode vector-field guidance as a velocity constraint, enabling a unified force/torque control design for a class of fully actuated and underactuated systems with desired paths that may be self-intersecting. A global, explicit error estimate connects constraint-following accuracy to path-following performance with convergence guarantees. An adaptive robust extension further addresses heterogeneous time-varying uncertainties with unknown bounds and ensures ultimately bounded path-following error. Applications to an underactuated aircraft system and a fully actuated manipulator system demonstrate the effectiveness and advantages of the framework.
♻ ★ Joint Communication-Control Strategy Optimization with Partially Nested Information Structures: The Linear-Quadratic Case
In this paper, we formalize a joint communication-control strategy optimization (JCCO) problem in multi-agent linear systems with quadratic costs, under the common-information-based (CIB) framework from decentralized stochastic control. For computational tractability, we focus on such JCCO problems with partially nested (PN) information structures (ISs). In particular, with a baseline communication protocol that leads to a PN IS, we establish a series of conditions under which the partial nestedness is preserved under the (additional) communication strategies to be optimized, while violating them may cause nonlinearity or non-existence of the optimal control strategies, with open-loop communication strategies. Under these conditions, we then develop a dynamic-programming-based approach to compute the optimal control strategies of JCCO with open-loop communication strategies, which yields a set of closed-form Riccati Equations. As a byproduct of independent interest, such an approach also offers a new way to solve decentralized linear-quadratic control with PN ISs and output feedback, under the CIB framework and the aforementioned conditions. Finally, we extend such an approach to JCCOs with closed-loop communication strategies, yielding a more tractable dynamic program than an infinite-dimensional CIB-belief-based one.
comment: Preliminary version accepted to IEEE CDC 2026; Sharpened the proof of Lemma IV.5
♻ ★ Semantic-Coupled Perception Framework for Autonomous Laparoscope Control
In minimally invasive surgery (MIS), maintaining an appropriate field of view (FoV) is essential for surgical safety and efficiency. However, conventional autonomous laparoscope control methods often treat visual tracking and surgical phase recognition independently, limiting robustness under occlusion, entity overlap, and camera motion. This paper proposes a closed-loop autonomous laparoscope control framework that couples real-time surgical phase recognition with camera-motion-compensated visual perception. By exploiting phase-dependent relationships among surgical entities, the framework improves target state estimation under challenging visual conditions, supporting task-aware autonomous FoV control. Benchmark and phantom experiments evaluate the framework in surgical phase recognition, target tracking under occlusion and entity overlap, and autonomous FoV control, showing improved target visibility and task execution compared with fixed-FoV control. These results highlight the potential of incorporating task-level semantics into visual state estimation for autonomous laparoscope guidance.
comment: 8 pages, 5 figures. Submitted to IEEE Robotics and Automation Letters (RA-L) for possible publication
♻ ★ Physics-informed Learning for Orbital Uncertainty Propagation with Error Bounds
The Fokker-Planck partial differential equation (FP-PDE) governs uncertainty evolution in stochastic dynamical systems. In orbital dynamics, solving the FP-PDE is challenging because of nonlinear motion, high-dimensional states, and large space-time domains. We develop a physics-informed neural network (PINN) approach that approximates the FP-PDE solution as a single space-time probability density, while also quantifying its worst-case approximation error. This approach is, in principle, independent of the choice of state coordinates and neural network architecture. Specifically, to enforce probability density function (PDF) properties into the neural network, we design a Physics-informed Gaussian mixture model (PINN-GMM). Then a companion error PINN learns the dynamics of the approximation error and yields time-dependent bounds that define an ambiguity set of PDFs. This ambiguity set enables rigorous computation of upper and lower bounds on event probabilities through tractable linear programs. Numerical studies on illustrative 1D examples and several 4D--6D orbital test cases demonstrate accurate uncertainty propagation, correct and informative error bounds, and improved reliability over common uncertainty-propagation baseline methods (Gaussian approximation, unscented transform, and Gaussian mixture model). Constructing the PINN-GMM requires offline training, making it costlier than the baseline approximations; once trained, however, a single forward pass returns the density at any time in sub-millisecond time $(0.16~\mathrm{ms}$ in our implementation).
comment: Accepted to Journal of Guidance, Control, and Dynamics
♻ ★ Symbolic Control for Stochastic Systems via Finite Parity Games
We consider the problem of computing the maximal probability of satisfying an omega-regular specification for stochastic nonlinear systems evolving in discrete time. The problem reduces, after automata-theoretic constructions, to finding the maximal probability of satisfying a parity condition on a (possibly hybrid) state space. While characterizing the exact satisfaction probability is open, we show that a lower bound on this probability can be obtained by (I) computing an under-approximation of the qualitative winning region, i.e., states from which the parity condition can be enforced almost surely, and (II) computing the maximal probability of reaching this qualitative winning region. The heart of our approach is a technique to symbolically compute the under-approximation of the qualitative winning region in step (I) via a finite-state abstraction of the original system as a 2.5-player parity game. Our abstraction procedure uses only the support of the probabilistic evolution; it does not use precise numerical transition probabilities. We prove that the winning set in the abstract 2.5-player game induces an under-approximation of the qualitative winning region in the original synthesis problem, along with a policy to solve it. By combining these contributions with (a) a symbolic fixpoint algorithm to solve 2.5-player games and (b) existing techniques for reachability policy synthesis in stochastic nonlinear systems, we get an abstraction-based algorithm for finding a lower bound on the maximal satisfaction probability. We have implemented the abstraction-based algorithm in Mascot-SDS (Majumdar et al., 2020), where we combined the outlined abstraction step with our recent tool FairSyn. We evaluated our implementation on the nonlinear model of a perturbed bistable switch from the literature. We outperform a recently proposed tool for solving this problem by a large margin.
♻ ★ Expert-Guided Online Reinforcement Learning for Stable and Sample-Efficient Control of Dynamical Systems
Online reinforcement learning (RL) often requires extensive interactions with the environment, which makes the training costly. Additionally, exploration can destabilise the dynamical system, which may be unsafe. On the other hand, offline learning from expert demonstrations avoids unsafe exploration but is limited by the quality and coverage of the offline dataset. In this paper, we propose a model-free RL approach that combines expert demonstrations with limited online interactions. Specifically, we optimise a policy objective consisting of an online performance term and a behavioural cloning regulariser computed from offline expert demonstrations. The regulariser encourages the learned policy to stay close to expert behaviour, reducing the need to explore unfamiliar actions. Expert demonstrations guide policy learning and reduce the need for many online interactions and exploration. For stochastic linear time-invariant (LTI) systems, we establish conditions for linear convergence of exact policy gradient updates and for all intermediate policies to remain stabilising. Although our theoretical analysis focuses on linear systems, we also evaluate the proposed approach on nonlinear systems to demonstrate its effectiveness beyond the linear setting. \rev{Numerical results show that the proposed method reaches comparable performance in fewer gradient steps and with less number of samples than a purely online RL baseline.
comment: Submitted to L4DC
Robotics
★ Robo-SWE: An Executable Issue-Resolution Dataset Mined from Robot Software Repositories
Coding agents are evaluated mostly on general-purpose software, but domain-specific systems require domain-aware evaluation. Robot software is a particularly challenging case: Robot Operating System 2 (ROS 2) packages have their own build tools, simulators and learning frameworks are built on physics engines, and behavior that depends on timing and hardware is hard to test. We present Robo-SWE, a dataset of 236 executable issue-resolution tasks mined from merged pull requests in 28 public robotics repositories in four domains: ROS 2 systems, simulation and physics engines, robotics algorithm libraries, and robot-learning frameworks. Each task contains the original issue text, a container recipe for the repository at the commit before the fix, the developers' fix, and the developers' tests, which fail before the fix and pass after it in repeated offline runs. The dataset also keeps a provenance ledger of all 1,567 candidate pull requests, with the stage each reached and the reason for every rejection, and, as a first use, 3,540 graded attempts by five coding-agent setups with recorded patches and 3,529 available trajectories. Across three runs per primary setup, resolve rates range from 45.8% for Claude Sonnet 5.5 to 56.5% for Claude Opus 5.5; GPT-6.1 Sol reaches 54.1% at maximum effort and 52.5% at medium effort, and GPT-6 Astra 51.4%. GLM-5.3 resolves 46.2%, and Kimi K3.1 Pro 43.9%. The tasks are validated by execution; verification of their fairness by human reviewers is future work. The dataset and public leaderboard are available online.
comment: 5 pages, 1 figure, 2 tables. Submitted to the MSR 2027 Data and Tool Showcase Track. Dataset: https://huggingface.co/datasets/vharatian/Robo-SWE
★ Imitating the Imperfect: Offline-to-Online Robust Imitation Learning from Heterogeneous Demonstrators
Imitation learning (IL) typically relies on large-scale offline demonstrations collected from multiple human or algorithmic demonstrators. However, most existing approaches assume these demonstrators are homogeneous or near-optimal experts, a convenient but unrealistic assumption in real-world applications. In practice, demonstrations are often collected from heterogeneous demonstrators with unknown and varying levels of expertise, resulting in highly mixed-quality data. In this work, we study a challenging yet practical setting of imitation learning from such heterogeneous imperfect demonstrators. We propose Latent Expertise and Optimality Scoring based Imitation Learning (LEOS-IL), a principled framework that jointly learns (i) a state-action optimality scoring model and (ii) latent expertise levels for each demonstrator from unlabeled suboptimal offline demonstrations. The learned scoring model is then integrated into an online policy optimization procedure, where the agent is trained to maximize estimated optimality scores while its online rollouts are iteratively leveraged to refine the scoring model, thereby mitigating distributional shift during online adaptation. We further provide theoretical guarantees for optimality score identification and conditional optimal occupancy and policy recovery under explicit sufficient conditions. Experiments on continuous control benchmarks demonstrate that our approach consistently outperforms existing baselines across highly heterogeneous and low-quality demonstration regimes.
★ JustRUA: A Controlled Study of GPT-6 Robot-Use Agents
GPT-6 Astra can drive a robot arm directly through a fixed harness, and scores of such robot-use agents on RoboDojo have risen quickly since. Reported systems differ in harness and test subset and often change several designs at once, so it is hard to tell which choices produce the gains. We take these designs apart and test them in one framework, along the harness around the model, the action interface, the reuse of experience and the scene geometry given to the model. A lean harness that keeps the full episode history and stops each motion early goes a long way. Reusing experience changes how the model acts but is not a free lunch, and scene geometry helps most, even when depth is predicted from RGB alone. JustRUA starts from the official L3 harness and keeps only what matters most. Even with no learned policy, self-improvement loop or task-specific tuning, this simple harness raises the score on the RoboDojo simulation benchmark substantially, from 29.7 to 50.3.
comment: 30 pages, 14 figures. Project page: https://zefanw.github.io/JustRUA. Code: https://github.com/ZefanW/JustRUA
★ DexRemo: A Force-Sensing Hand Platform and Task Suite for Contact-Rich Removal Manipulation
Many manipulation tasks require separating adhered, embedded, contaminated, or mechanically constrained targets. We study this setting as removal manipulation, where progress depends on evolving contact states that are only partially observable from vision and kinematics. We introduce DexRemo, a three-finger robotic hand with high-force actuation, modular fingertips, and three-axis fingertip force sensing, together with a nine-scenario real-robot task suite spanning adhesive detachment, surface cleaning and self-cleaning, and occluded retrieval. To make multi-finger force observations easier to fuse, we express fingertip forces in a shared wrist-centered frame, giving force components consistent geometric meaning across fingers and time. Using an Action Chunking Transformer backbone, policies fuse multi-view vision, proprioception, fingertip pose, and force observations to predict closed-loop action chunks from demonstrations. Evaluation uses six core benchmark tasks for controlled fingertip-input and force-frame ablations, plus three extension scenarios for additional held-out coverage. Across these evaluations, the learned policies achieve consistent removal performance, and ablations show that wrist-frame forces, especially with fingertip pose, improve stability and generalization over local-frame force inputs and no-force baselines. Project website: https://dexremo.github.io
comment: Accepted for publication in IEEE Transactions on Robotics (T-RO), 2026. Project website: https://dexremo.github.io
★ Video-Action Models for Data-Efficient Industrial Manipulation
Industrial manipulation requires precise alignment and controlled handling of rigid, deformable, and flexible components. Yet, the benefits of pretrained visuomotor policies remain underexplored in settings where limited access to specialized hardware, components, and fixtures constrains demonstration collection and evaluation. Understanding task-specific data requirements is important for assessing their scalability across diverse processes on the factory floor. We investigate whether video pretraining can support reliable industrial manipulation with limited action data. We propose FLUX-mimic, a video-action model, motivated by the potential of video pretraining to capture motion and interaction patterns that enable sample-efficient training. FLUX-mimic is built on a backbone pretrained on large-scale web and industrial task data, and an action expert to generate robot trajectories. We train our policy by joint video-action training. Across four automotive manufacturing benchmark tasks and two bimanual robot platforms, FLUX-mimic outperforms all considered baselines on the dexterous robot platform and matches pi_{0.5}'s average success rate on the gripper-based robot platform. On a deformable part kitting task, FLUX-mimic reaches 77.5% success with as little as two hours of task-specific action-labeled demonstrations.
comment: Project Webpage: https://www.mimicrobotics.com/blog/video-action-models-for-data-efficient-industrial-manipulation
★ COMBS: Concealed Object Multi-modal Bayesian Search
This work addresses the problem of finding a concealed object in an environment by unifying formalisms for partially observable environments with multi-modal priors drawn from a mix of ontologies, semantic networks, LLMs, and VLMs. We propose a concealment-adaptive mixed-observability MDP (CA-MOMDP), a Bayes-adaptive setting where uncertainty is over both object locations and which nodes can conceal them, held as a hypothesis with a bounded ontology prior and resolved only by interaction. We define a hierarchical transition system (HTS) to abstract the environment into room and object nodes that can be planned over via our CA-MOMDP. We further propose a COMBS-score: a multi-modal fusion of a human informed semantic network, LLM, and VLM, to initialize the belief over object locations, which grounds our abstractions and reduces reliance on a single model. Belief is maintained via Bayes' rule, which never lets a look rule out a closed container. We prove a present target is found in a bounded number of searches under exact perception and that its location is never ruled out given missed detection under noisy perception. We ablate each component to demonstrate their significance. Across five simulator configurations, from a gridworld to Matterport3D with a real detector, COMBS finds 92-100% of in-distribution and 92-100% of out-of-distribution targets, matches or beats the baselines in processing speed, significantly outperforms a per-step LLM policy at lower cost on the larger scenes, recovers more out-of-distribution placements than a TidyBot ranking, and matches an LLM-only prior at a fraction of its per-decision latency. COMBS is the only standalone method that can declare a target absent at a stated error level, and the per-step LLM policy offered a termination action abandons 12-22% of targets that were present, against 1.6-3.2% for COMBS on the same episodes.
★ MOSAIC: Modular Obstacle and Spatial-Aware Framework for Manipulation In Cluttered Scenes
Vision-Language-Action (VLA) models have demonstrated strong performance in robotic manipulation, yet most rely primarily on 2D visual observations and language instructions, leaving scene geometry and obstacle relationships to be inferred implicitly. This limitation is particularly problematic in cluttered environments, where small spatial errors can lead to unintended contacts or collisions. We propose MOSAIC, a modular spatial- and obstacle-aware framework that can be integrated with pretrained VLA policies to improve object-level scene understanding and action generation. Our method augments the VLA backbone with an auxiliary semantic grounding task that predicts role-specific masks for the target object, destination region, and surrounding obstacles. We further combine the target and destination masks with depth estimated from 3D scene reconstruction to construct role-specific geometric tokens that are injected directly into the action expert. In addition, we propose obstacle-aware attention supervision that emphasizes obstacles near the robot body during task execution. The resulting attention features are subsequently injected into later action-expert layers to further guide action generation. Together, these components provide explicit semantic, geometric, and obstacle-aware cues while preserving the underlying pretrained VLA architecture. We evaluate our framework on pretrained VLA model using the RoboPRO benchmark, including visual perturbation tests, and further validate it on real-world manipulation tasks. MOSAIC improves the success rate by 5.1% in cluttered scenes, and by 9.7% under visual perturbations, with gains of up to 25.0% in the most challenging environment. We will release our code and dataset upon acceptance at the following Github page: https://mmahdavian.github.io/MOSAIC/
★ Runtime Safety and Emergency Stops on a Full-Size Humanoid: Deployment Lessons and a Case for Graduated Force Authority
Full-size humanoid robots are being considered for hazardous industrial work such as disassembling electric-vehicle batteries. We describe the safety mechanisms of an open-source autonomy stack for the Unitree H1-2 humanoid and what deploying them taught us. All joint commands pass through a safety layer that runs separately from the control software. It enforces joint position, velocity, and torque limits, and it stops the robot when a state estimate becomes stale or when the hard-wired emergency stop (e-stop) is pressed or disconnected. We pushed the standing robot until it lost balance. With a model-predictive lower-body controller it never recovered, and the operator had to press the e-stop in every trial. Pressing it removes balance control, and the robot falls. The safety layer's own limits trigger the same stop, and limits chosen for manipulation were exceeded during normal walking. Near an operator or a battery pack, the fall can be more dangerous than the cause of the stop. Joint limits bound how much force a robot can apply, but not when applying it is appropriate. Drawing on research on responsibility and on child and adolescent development, we therefore make a case for granting robots force authority in stages, with a person or organization explicitly responsible for its use.
★ Where to Generate Touch: Do Image Latents Preserve Material Recognition?
Generated touch could extend robot learning beyond recorded contacts. A tactile generator needs a latent space, and we choose one by testing whether a material classifier trained on real GelSight presses still works on encode-decode reconstructions from pretrained image models. On ObjectFolder Real, the classifier loses 6.5 to 9.7 points on reconstructions from three 16x discrete tokenizers and 0.3 points on those from the Stable Diffusion VAE (SD-VAE). The 16x tokenizers again lose far more on Touch and Go and on DIGIT object recognition (YCB-Slide). Retraining the classifier on reconstructions, with no new labels, narrows the discrete gap. SD-VAE reconstructions also keep object identity, with 76.7% object-matching accuracy against 78.9% for real touch and 67.0% for MAGVIT-v2. We therefore select SD-VAE latents and show a photo-to-touch generator trained in them. Code will be released upon acceptance.
comment: 13 pages, 8 figures, 10 tables
★ RUMI: A Robotiq-Matched Universal Manipulation Interface for Robot Learning with Augmented Reality and Haptic Feedback
Many manipulation tasks require carefully controlled contact forces for successful execution, yet large-scale, force-aware datasets remain scarce. Handheld interfaces can help by transmitting interaction forces directly through the operator's arm, but most existing designs are tailored to simple linear grippers and do not match the nonlinear mechanics of widely used robotic grippers. We present RUMI, a handheld universal manipulation interface matched to the Robotiq 2F-85 with exchangeable fingertip sensors that preserves the target gripper geometry while enabling natural input before contact and force measurement after contact. By integrating the tracking system into a virtual reality headset, our method provides real-time augmented reality feedback about the kinematic feasibility of motion on the target robot platform. We further introduce a haptic feedback mechanism that supports force-aware teleoperation for simulated and real robots. A user study shows fewer episodes containing joint-limit violations with live robot visualization and pose initialization (3/50) than without augmented reality (16/50). We additionally train imitation policies from demonstrations, collected using two different fingertip modalities enabled by the gripper-matched hardware, and assess the haptic interface's capability for collecting force-aware teleoperation datasets in simulation.
★ Prediction-Error-Tightened MPC for Socially Compliant Crowd Navigation
Autonomous navigation of mobile robots in crowded environments, while preserving pedestrians' psychological comfort, requires distinguishing pedestrians from other obstacles, predicting their future motion, and accounting for uncertainty in the predicted trajectories of both pedestrians and the robot. This work extends a previous MPC framework by integrating obstacle and pedestrian detection, Koopman-based motion prediction, and online prediction-error estimation directly within the collision-avoidance mechanism. To reduce collision-avoidance complexity in crowded scenarios, nearby obstacles are aggregated, thereby reducing the number of collision-avoidance constraints. Simulation studies evaluate the prediction-error margins and the computational benefits of semantics-aware aggregation. In a paired 300-seed closed-loop comparison with a moving generic obstacle, Markov residual tightening reduces the fraction of runs with physical contact from $10.0\%$ to $1.33\%$, at a mean travel-time cost of $0.68$ s. In a separate crowd-navigation semantic ablation, the offline robot-footprint/personal-space overlap occupies $2.11\%$ of observed time with pedestrian-aware handling versus $22.84\%$ with generic-obstacle handling.
comment: 9 pages, 3 figures, 4 tables. Submitted to the European Control Conference (ECC 2027)
★ Robot-Induced Bias in Movement Smoothness Metrics for Rehabilitation: Quantification and Correction in Simulation
Robot mechanics can bias kinematic measures of recovery and limit comparisons across rehabilitation devices. We developed a closed-loop human-robot simulation calibrated to published healthy and stroke group kinematics. Paired runs held motor-control parameters and movement plans fixed to quantify bias in seven metrics relative to a low-impedance reference, normalized using external repeatability benchmarks. Within a subset informed by published device specifications, median log dimensionless jerk (LDLJ) bias was 0.93 and 1.79 benchmark units for stroke and healthy profiles, respectively. Sobol analysis identified Coulomb friction and apparent mass as leading contributors; LDLJ bias persisted across 11 robustness variants. LDLJ bias increased as simulated movements became less fragmented. If recovery similarly reduces fragmentation, this effect could partly mask improvement in measured smoothness. Correction combining identified robot dynamics and observed metrics with gradient-boosted trees reduced median normalized absolute error from 0.77 to 0.10 for LDLJ and from 0.24 to 0.08 for spectral arc length (SPARC) in the device-informed subset of a random 20% hold-out. Median smoothness residuals remained at or below 0.15 benchmark units across four separately held-out parameter regions. Under an alternative controller, LDLJ error decreased from 1.60 to 0.17 without retraining, although transfer varied across metrics. These simulation results support mechanical characterization and movement-dependent correction toward more comparable robotic assessments, motivating validation in patients.
comment: 7 pages, 4 figures, 2 tables. Submitted to the 2027 IEEE International Conference on Rehabilitation Robotics (ICORR)
★ What World Model Does Active Object Search Need? A Study under Imperfect Detection
In active object search, a robot must decide where to look and when to stop. Imperfect detection complicates both decisions: a non-detection may indicate occlusion, an out-of-view target, or a false negative, while a detection may be a false positive. This raises the question of which representations a world model needs to support reliable search under imperfect detection. We introduce Bayesian Active Miss-aware Object Search (BAMOS) to investigate how target belief, occupancy belief, and detector errors affect search decisions. Through comprehensive ablations in Isaac Sim and HSSD houses, we examine the roles of these representations and their update mechanisms across different detector configurations. Notably, target belief and its update mechanisms are critical to search performance, with their contribution generally increasing as detection errors become more prevalent. These findings highlight the importance of maintaining and updating target beliefs to support reliable active search under uncertain detector observations, providing insights into the design of world models for embodied agents. Code is available in https://github.com/gokyeongryeol/BAMOS.
comment: Accepted in CoRL 2026 Workshop on Do Robots Need World Models
★ GraSLA: A Lightweight Spatial-Language-Action Model with Grasp Hypotheses as Geometric Priors
This paper focuses on improving the behavioral diversity and effectiveness of language-conditioned manipulation policies trained through imitation learning. Recently, continuous-action policies have become mainstream in robotic manipulation, but the lack of diverse demonstrations tends to restrict their behavioral diversity. For grasping, a fundamental subtask of manipulation, 6-DoF grasp detection provides diverse grasp hypotheses, but open-loop execution constrains responsiveness in dynamic environments. This motivates us to combine the diversity of grasp hypotheses with the adaptability of continuous-action policies. To this end, we propose GraSLA, a lightweight and effective spatial-language-action model with grasp hypotheses as geometric priors. In particular, we employ spatial and language foundation models to encode scene geometry and task semantics, and integrate them with grasp hypotheses through bidirectional cross-attention for continuous action prediction. On LIBERO, GraSLA achieves an average success rate of 99.30%.
★ Embedded Sim-to-Real Visual Servoing: Learned Reaching and Force-Guided Placement
Embedded perception-to-control systems must combine visual targeting with reliable contact handling under tight computational constraints. We present a shared architecture on an NVIDIA Jetson AGX Orin controlling a Franka Research 3 through Cartesian impedance control. RealSense D435i observations, YOLO11-seg segmentation, depth back-projection, and hand--eye calibration provide metric robot-frame targets. We compare two strategies: compact proximal policy optimization (PPO) policies transferred from simulation for object- and hole-region reaching, and force-guided Cartesian placement using approximately 2 N normal contact and spiral search. The PPO policies achieved 87.2% and 83.1% success over 1000 simulation episodes each. Across 50 real-robot trials per strategy, the learned hybrid and force-guided analytical pipelines achieved 70.0% and 74.0% completion, respectively. Mean perception latency with TensorRT was 34.55 ms. These results support an embedded architecture combining learned free-space reaching with explicit contact correction.
comment: 6 pages. Accepted at IEEE ROBIO 2026
★ Recover, Don't Just Continue: Learning from Feedback for Physical Failure Understanding and Correction
Multimodal large language models are regarded as the cognitive foundation of embodied intelligence, but planning from instructions alone cannot make them reliable. During execution, actions deviate from intended effects, constrained by the embodiment and environment. An embodied brain must understand why execution deviates, identify what can be reliably realized, and revise decisions using interaction experience. These correspond to causal failure localization, capability-constrained adaptation, and experience-driven revision. Current embodied foundation models largely lack these capabilities. To address this, we introduce Embodied-Corrector. We construct failure-recovery data by injecting perturbations at subtask boundaries of successful trajectories, replaying affected subtasks, and keeping failures that are observable from public evidence and recoverable from the current state. We align actions, consequences, failure causes, and recoveries into Judge-Cause-Sub-Fine supervision. Training uses evaluation-first SFT: the model learns Judge and Cause first, then Sub and Fine conditioned on accurate prefixes. Since deployment uses self-generated prefixes, evaluation errors propagate into recovery planning, and response-level rewards are too coarse to locate affected planning tokens. We therefore use OPD to internalize the planning distribution under accurate prefixes into that under self-generated ones: the model generates plans from self-generated prefixes, while a contrastive forward pass with accurate prefixes compares token probabilities to reinforce or suppress planning tokens. At deployment, this branch is removed. Experiments show stronger failure analysis and corrective planning: with a frozen VLA, composite task success improves by 6.1 percentage points; with fixed GPT-5.5 execution, manipulation and navigation tasks improve by 13.9 percentage points.
★ Single-View RGB-D Force--Deformation Estimation for Robotic Manipulation of Elasto-Plastic Materials
Persistent deformation complicates elasto-plastic manipulation by coupling physical interaction with spatially distributed shape changes. We investigate the force--deformation relationship using a standard robot, a single eye-in-hand RGB-D camera, and intrinsic force estimates. A 7-DoF Franka Emika FR3 performs normal indentations while an Intel RealSense L515 captures pre- and post-contact surfaces. Point-cloud processing localizes deformation and estimates displaced volume. Force--volume characterization includes 27 measurements over nine force levels. A compact multilayer perceptron (MLP), trained on 52 force--volume pairs at four levels, achieves a held-out RMSE of 1.857 N and $R^2=0.440$. A separate comparison shows lower error for a quadratic model at 16 N and for the MLP at 19 and 23.5 N. An eight-step experiment at 4 N per interaction demonstrates cumulative deformation localization from the second indentation onward. These results establish a practical measurement basis for deformation-aware manipulation using intrinsic force estimates and single-view geometry.
comment: 7 pages. Accepted at IEEE ROBIO 2026
★ Helping Hand: A Continuum Robotic Grasper for Traction in Single-Channel Endoscopic Submucosal Dissection
Endoscopic submucosal dissection (ESD) is a minimally invasive approach for removing large polyps from the colon and offers many benefits compared to other surgical approaches. However, ESD is technically very challenging to perform, leading to limited adoption by physicians. We propose the Helping Hand, a two-segment continuum robotic grasper that, together with an electrosurgical knife, enables tissue traction and cutting within a single 3.7 mm working channel. We achieve this functionality by creating a slot at the base of the Helping Hand that lets the electrosurgical knife extend out, enabling countertraction and cutting through one instrument lumen. We evaluate the Helping Hand's workspace and its ability to provide traction forces from different starting configurations. Finally, we demonstrate the full workflow by isolating and removing tissue in an ex vivo chicken thigh, illustrating the potential for the Helping Hand to help improve ESD procedures.
comment: 7 pages, 9 figures
★ EMS-Nav: Embodied Multi-Scale Navigation
Industrial inspection robots must reach work areas and specific equipment. These destinations span spatial scales from regions to objects, so robots must relate each target to its surroundings to interpret the task and determine where to go. Existing methods support multi-scale navigation through rich environment representations, but at substantial computational and storage cost. These costs limit deployment where wireless coverage is incomplete and robots must rely on onboard computation to keep navigating. We introduce EMS-Nav, a biologically inspired multi-scale navigation framework for online, on-device operation. Its core innovation is to organize perception, spatial understanding, and navigation around regions, making local spaces the basic units of both environmental understanding and computation. EMS-Nav groups objects and their relationships into functionally meaningful spaces and builds a hierarchy from local areas to the broader environment. This organization supports efficient map construction alongside the interpretation and execution of language goals at different scales. In multi-scale navigation experiments, EMS-Nav achieves the highest success rates across all four target categories: objects, rooms, regions, and specific instances. It outperforms state-of-the-art open-vocabulary mapping and navigation methods, improving overall first-attempt success by 9.39 percentage points over the strongest baseline while reducing mapping overhead and language-model input size. Local deployment in real indoor environments further demonstrates its applicability to on-device embodied navigation. By establishing regions as the organizing basis for perception, spatial understanding, and language-guided action, EMS-Nav provides a system that combines multi-scale understanding with computational efficiency to support diverse language goals on resource-constrained robots.
comment: 30 pages, including appendices
★ Generalized Inverse Kinematics via Conditional Flow
Inverse kinematics (IK) is a foundational primitive in robotics, mapping the task-space objectives expressed by higher-level planners to robot configurations that realize them. Traditional IK methods generally treat each query as an isolated pose-to-configuration problem, but planners often need a distribution over valid realizations that accounts for future reachability and preferences such as collision avoidance and stability. We introduce GenGenIK, a generative modeling approach that supports mobile bases, multiple end-effectors and targets, scene awareness, and preferences over robot behavior within a common interface. Experiments show that GenGenIK outperforms existing methods in both solve rate and runtime across tasks such as bimanual lift and flip-up, while allowing flexible control over the sampled distribution and remaining effective under partial scene observations. Website: https://icra-2027-gengenik.github.io/.
comment: 9 pages, 3 figures
★ Reading the Table: Sim-to-Real Game State Detection for the Chef's Hat Card Game ICANN 2026
Tabletop games offer a compelling scenario to study the interaction between artificial and human agents, combining social interaction with game state tracking and physical manipulation. To assess the game state, a robot needs to precisely detect closely related game objects, such as different card designs, requiring dedicated training data that is costly to annotate in the real world. Instead, we develop a fully automated synthetic data collection pipeline, using re-projection of known 3D coordinates to generate labels automatically and domain randomization to bridge the reality gap. This approach is used to collect a dataset of 25413 samples to finetune an object detection and a segmentation model, which are combined to assign detected cards to game-relevant zones and infer the game state. When evaluated on a real NICO humanoid robot, our pipeline achieves an F1-score of 0.906 for card-to-zone assignment, thus providing the perceptual foundation for a robot that can autonomously play Chef's Hat against human players.
comment: Presented at the Dungeons, Neurons, and Dialogues workshop (DnD-SIDC 2026) at ICANN 2026, https://dnd-sidc.github.io/
★ What Makes a VLA Reactive? A Study of Execution Strategies for Dynamic Manipulation
Vision-language-action (VLA) models predict chunks of future actions, and when objects move, those chunks go stale before they are executed. We study what makes a pretrained VLA reactive, using \pihalf{} on two dynamic simulation benchmarks and three real-robot tasks. We vary the prediction and execution horizons, compare synchronous and asynchronous inference, test methods that deliver actions sooner or keep successive chunks continuous, and adapt the policy with motion cues and latent-space reinforcement learning. The best prediction horizon is about one second, and the best execution horizon depends on the checkpoint. Short horizons help more with asynchronous execution, and methods that keep the trajectory continuous outperform those that only reduce latency. Adapting the policy repairs targeted failures, but on the conveyor task changing the execution horizon gained as much. Reactivity depends on the execution pipeline as much as on the policy, so both should be evaluated together.
comment: 13 pages, 7 figures, Efficient Foundation Models for Real-Time Embodied AI, CoRL 2026 Workshop
★ ActionCodec2: A Streaming Action Tokenizer
Most action tokenizers for autoregressive vision language action (VLA) models compress motion into chunks that must be decoded in full before their actions become executable. We introduce ActionCodec2, a non-neural streaming tokenizer that releases actions incrementally while keeping trajectory reconstruction error within a prescribed tolerance. Its encoder quantizes each step relative to the decoder's reconstructed state, so the selected motion primitive corrects residual error rather than allowing errors from independently quantized increments to accumulate. Finding the smallest such primitive set is a geometric covering problem; we derive the exact minimum for translation and constant-factor-tight bounds for rotation. Set-BPE then losslessly merges co-occurring primitive symbols across coordinates and neighboring steps, without a fixed coordinate ordering. A universal version fitted on 20k hours of data from 24 embodiments averages 8.8 tokens per second for 15 Hz end-effector motion on this corpus. With the same VLA backbone, ActionCodec2 outperforms the compared tokenizers on three simulation benchmarks and in zero-shot real-robot evaluation. Its streaming representation allows policies to incorporate state feedback within an action chunk, substantially improving performance over open-loop execution under randomized dynamics, external disturbances, and moving objects. The same tokens also can serve as a direct control interface for VLM agents, which compose successful action sequences into reusable macros. These agent-built macros reduce control decisions and thinking-token usage across attempts and transfer to a held-out task. More results are available at https://zibindong.github.io/actioncodec2-page/
★ FloCOS: Predicting Compact Forward Occupancy Sets of Surrounding Vehicles via Conditional Flow Matching
Existing methods for predicting surrounding vehicles' forward occupancy may under-approximate the ground-truth occupancy, exposing the ego vehicle to collision risk, or produce overly conservative regions that unnecessarily restrict feasible space and cause planning failures. To address these issues, we propose FloCOS, a conditional flow-matching framework that predicts compact forward occupancy sets with high occupancy coverage. FloCOS represents the forward occupancy set as a time-varying union of half-space convex polygons parameterized by temporal basis coefficients. Conditional flow matching captures the conditional distribution of these coefficients to govern the temporal evolution of the convex components, trained under coverage and union-area losses that penalize missed occupancy and excess area. For downstream trajectory optimization, the predicted nonconvex union is directly incorporated into hyperplane-based collision-avoidance constraints. Evaluations on the Waymo Open Motion Dataset show that FloCOS generates compact forward occupancy sets with high coverage and achieves a higher interaction completion rate than evaluated baselines. Real-world vehicle experiments further confirm its ability to support collision-free planning in interactive scenarios.
comment: 8 pages, 4 figures, 3 tables
★ SPARC: RoboCup Standard Platform for Autonomous Rescue Competition with Initial Datasets, Software, and Simulation Environments
The RoboCup Standard Platform for Autonomous Rescue Competition (SPARC) is a new competition under development of the RoboCup Rescue Robot League that uses the Unitree A2-W as a common robot platform. SPARC aims to make research on artificial intelligence for autonomous rescue robots accessible to a broader research community, including groups without expertise in developing and maintaining custom robot hardware. By providing a capable standard platform and established benchmark environments, researchers can focus on autonomous navigation, perception, manipulation, and embodied AI for challenging disaster-response scenarios. This paper introduces the SPARC competition, its benchmark environment, and the Unitree A2-W standard platform. Furthermore, we present the first open SPARC datasets collected during the IEEE ICRA 2026 Legged Robot Challenges and RoboCup Rescue 2026, release an open-source software stack for autonomous navigation, and provide initial simulation assets representing RoboCup Rescue test environments. Together, these resources aim to lower the barrier to entry for autonomous rescue robotics and facilitate reproducible research and collaborative development on a common hardware platform.
comment: IEEE SSRR 2026
★ A physics-constrained deep reinforcement learning method for autonomous mission planning of Titan deep space probes
Titan is a compelling target for future deep space exploration, while its extreme Earth-Titan distance and complex environment make continuous ground-in-the-loop planning impractical. Existing planning methods, primarily developed for Earth-observation scenarios, have limited generalizability to uncertain deep space conditions and adaptability to time-varying mission states. To address these challenges, this paper proposes a physics-constrained maskable proximal policy optimization (PCM-PPO) method for Titan autonomous mission planning. A Titan-specific discrete-time Markov decision process is established to characterize mission feasibility and resource constraints. A system dynamics and geometric model is incorporated to enhance physical representation, including central gravity, J2 perturbation, third-body attraction, and downlink occultation. To accommodate opportunity targets under a bounded action dimension, PCM-PPO organizes predefined and opportunity targets through a fixed-capacity target pool and represents the mission state with a target-wise state matrix and a global state vector. A physics-constrained action mask constructs an admissible decision domain from current mission states, guiding policy learning toward physically executable decisions and avoiding infeasible exploration. An integrated training strategy facilitates policy optimization under coupled constraints. Comparative, ablation, and uncertainty experiments validate the effectiveness of PCM-PPO. PCM-PPO provides a feasible and interpretable scheme for autonomous mission planning of Titan deep space probes.
★ Actuation and Control of Passive Tendon-Driven Soft Extensions Through Rigid Manipulator Motion
Industrial manipulators offer reach and precision but can damage nearby structures in confined spaces. Soft robots provide compliant contact but typically have limited reach. Rigid-soft systems combine their advantages, although they often require dedicated actuation and control hardware. We propose actuating a passive tendon-driven soft extension through manipulator joint motion. As the robot moves, the exterior spans of tendons anchored to upstream links change, altering tendon tension and deforming the extension. Inspection through narrow apertures, for example, constrains the motion of the end effector (EE). We develop a controller that regulates the soft tip while preserving such EE constraints. A differential model separates rigid transport from soft deformation, combining analytic arm and tendon kinematics with a locally identified span-to-deformation map. We also derive a local upper bound on the number of independent deformation directions transmitted by the tendons under the EE constraint, and conditions for the exponential convergence of the tip error. A quadratic program with control barrier functions imposes obstacle clearance, fulcrum limits, and model-validity conditions. Simulations and real experiments show tip and shape regulation under free, position-constrained, and fulcrum-constrained EE motion.
comment: 16 pages, 12 figures. Soon to be submitted to the IEEE Transactions on Robotics. Code: https://github.com/nachocz/passive_soft_extension_control. Interactive pages and video: https://ignaciocuiral.com/passive_soft_extension_control/
★ Low-Cost Mobile Bimanual Manipulation for Long-Horizon Toilet Cleaning
Vision-Language-Action (VLA) models have shown promising capabilities for robot manipulation, but their performance on accessible mobile bimanual platforms remains underexplored. We present a low-cost and reproducible mobile bimanual platform integrating two LeRobot SO-101 manipulators with a TurtleBot 4 and a LeRobot-ROS 2 software stack. Using long-horizon toilet cleaning as a case study, we collect 250 real-world demonstrations and evaluate $π_0$, $π_{0.5}$, SmolVLA, and GR00T under the same embodiment and task conditions. We further evaluate variants using SARM-derived task-progress signals and short-term MEM observation history. Among the evaluated models, $π_0$ performs best, achieving 60% strict and 70% relaxed autonomous success, increasing to 90% with limited human intervention. In contrast, the evaluated progress- and memory-augmented variants do not improve their corresponding baselines. Our results demonstrate the feasibility of long-horizon mobile bimanual manipulation on accessible hardware while highlighting remaining challenges in reliable VLA deployment. Project website: https://so101-turtlebot4.github.io/mobile-bimanual-cleaning/
★ Bind-WAM: Binding-Uncertainty-Guided Selective Future Imagination in World Action Models
World Action Models (WAMs) can improve robot learning by jointly predicting future visual states and actions, but generating a visual future at every replanning step is expensive and sometimes unnecessary. This raises a critical question: when should a robot policy imagine the future? In this paper, we propose Bind-WAM, a binding-uncertainty-guided WAM for selective future imagination. Bind-WAM introduces language-object binding uncertainty, selectively invoking joint future-action generation when the current observation does not support stable task-relevant action generation. An offline labeling pipeline uses complete demonstrations and action context to identify task-relevant objects and detect reference mismatch, invisibility, and identity drift. The resulting pseudo-labels supervise a lightweight router that predicts binding uncertainty from the current observation and language instruction. At inference, a shared WAM policy invokes selective future imagination according to the predicted uncertainty and its change between consecutive replanning steps. Simulation and real-world experiments demonstrate that Bind-WAM can establish binding uncertainty as a strong, interpretable signal for selective future imagination.
★ Interactive Perception of Granular Materials via Robotic Scooping and Force-Torque Sensing
Granular materials exhibit complex and highly variable interaction dynamics, making their reliable manipulation challenging without knowledge of their physical properties. We present an interactive perception framework that infers granular material properties directly from force-torque measurements acquired during scooping. The proposed Latent Filter++ combines a bio-inspired frequency-temporal filter-bank representation with a causal latent dynamical model, allowing the representation to evolve as interaction observations become available while avoiding hand-crafted signal preprocessing. The learned latent representation captures physically meaningful variation in granular materials and supports estimation of properties including grain size and bulk density. Evaluated on 26 diverse granular materials, the approach generalizes to materials excluded from training and enables few-shot adaptation of property-specific mappings using a small number of additional interaction samples. Finally, we demonstrate the relevance of the inferred properties to downstream manipulation by examining their correlation with the maximum acceleration the materials can withstand without spilling from the spoon during transport.
★ BiPOP: Partial-Order Planning for Long-Horizon Bimanual Robotic Manipulation
Bimanual robotic manipulation is versatile, yet its central challenge lies not only in executing dexterous motion with each hand, but also in coordinating the two arms for long-horizon manipulation under coupled spatial and temporal constraints. Prior work has largely focused on endowing robots with human-level manipulation skills, paying less attention to the structure of bimanual behaviour itself: many practical tasks are a coordinated composition of reusable unimanual skills and a small set of synchronous bimanual skills. This makes high-level spatio-temporal coordination, rather than skill acquisition alone, a key bottleneck for efficient bimanual execution. We propose BiPOP, a framework that couples Large Language Models (LLMs) with Temporal Partial-Order Planning for long-horizon bimanual manipulation. At its core is an explicit spatio-temporal model that partitions the workspace by arm reachability and captures inter-arm dependencies, arm assignment, synchronization requirements, and opportunities for safe concurrent execution. Building on this model, an LLM translates natural-language instructions into a formal PDDL specification, while a temporal planner enforces the constraints and schedules actions, yielding plans that are logically consistent at the symbolic level and efficient through parallel execution. On simulated long-horizon tasks of varying complexity, BiPOP outperforms open-loop planning with strong reasoning models and closed-loop ReAct-style planning in success rate, planning time, and plan efficiency, and the full pipeline runs end to end on the real bimanual robot NICOL. A failure analysis further shows that LLM-only planners mainly violate spatial and temporal coordination constraints, which motivates delegating these constraints to a symbolic planner. Code: https://github.com/Kchu/BiPOP
★ APG-Planner: Learning Vision-Based High-Speed Trajectory Planning via Analytic Policy Gradient
Vision-based high-speed navigation requires a trajectory planner that can reactively generate smooth, collision-free trajectories from onboard visual observations. Deep reinforcement learning offers a promising approach to vision-based trajectory planning, as it enables the optimization of sequential decision-making policies in complex environments. However, model-free policy gradient methods rely on sample-based policy gradient estimation, which yields noisy gradient estimates and inefficient training. To this end, we propose a framework for learning trajectory planning policies using the analytic policy gradient (APG) method and use it to train APG-Planner, a vision-based high-speed trajectory planner. The proposed framework leverages differential flatness to analytically reconstruct vehicle states from planned trajectories, thereby enabling differentiable trajectory-level state transitions without requiring an explicit low-level dynamics model. By iteratively performing trajectory planning and differentiable state transitions, we construct a differentiable episode rollout and analytically compute policy gradients from episode-level losses, improving both training efficiency and policy performance. After approximately 30 minutes of training on a single GPU, APG-Planner demonstrates comparable or superior performance to existing baseline methods in simulation and achieves speeds of up to 11\,m/s in real-world flight experiments. We further validate the proposed framework on a skid-steer ground robot, demonstrating its applicability to differentially flat robotic systems beyond quadrotors.
★ Autonomous Trail Racing with a Bicycle Robot
Navigating off-road trails on a mountain bike requires diverse locomotion skills, including aerial maneuvers, cornering, and slip recovery, paired with sophisticated perception for scene understanding. Translating these capabilities to autonomous, two-wheeled robots remains a formidable challenge. We show that autonomous, high-speed trail navigation can be achieved through a single idea: encoding the trail as a semantic layer of the terrain. In simulation, this layer embeds the navigation task into the training environment; during deployment, it is reconstructed onboard from camera and LiDAR data. First, we introduce a unified learning framework that treats local navigation and locomotion as a joint optimization problem, bypassing the twist or pose command interface of classical control hierarchies. With fixed reward weights, the same framework produces both trail racing and agile parkour, shaped only through terrain semantics and episode length. Second, our perception pipeline combines a frozen vision foundation model with a lightweight dual-head decoder that predicts per-pixel terrain semantics and confidence scores, which are fused with LiDAR range measurements into an egocentric semantic elevation map at 30 Hz onboard an NVIDIA AGX Orin. We validate our framework on the Ultra Mobility Vehicle (UMV), a bicycle robot that autonomously raced across all sections of a kilometer-long, unstructured mountain bike trail at speeds exceeding 5 m/s and averaging 3.26 m/s. In simulation, our policy fails up to an order of magnitude less often than hierarchical baselines while driving faster.
comment: 35 pages, 10 figures, 3 tables (incl. appendix with supplementary figures and tables). Submitted to The International Journal of Robotics Research. Project page: https://pages.rai-inst.com/autonomous-trail-racing/. Code: https://github.com/rai-opensource/sem_map
★ DroneDiffusion: Uncertainty-Guided Diffusion Path Planner for UAV Navigation
We present DroneDiffusion, a perception-aware diffusion planner for local 3-D UAV trajectory generation that explicitly models and avoids failure zones caused by lens flare, overexposure, and reflective surfaces. A vision-language perceptual safety estimator produces a pixel-aligned reliability map that highlights regions of degraded sensing. A denoising diffusion probabilistic model (DDPM) is conditioned on this map together with the onboard RGB image, goal point, and initial velocity to generate egocentric 3-D trajectories. To enforce obstacle avoidance at inference time without retraining, we apply training-free CHOMP guidance over a 3-D Euclidean signed distance field. The model is trained entirely in simulation and deployed zero-shot on a real drone. Reliability conditioning raises the mean perceptual reliability along generated trajectories to 0.71, compared with 0.63 for a geometric baseline, while guidance halves the collision rate of the learned prior from 14.7% to 7.8%. Perceptual safety conditioning and geometric guidance are therefore complementary: treating sensor reliability as a first-class signal reduces incursions into unreliable regions while maintaining competitive navigation performance.
★ Are We Ready for Snow-tonomous Driving? Challenges Toward Robust Automated Driving in Extreme Weather
Automated driving has gained remarkable momentum over the past decade, with both industry and academia investing substantial resources in developing perception and control solutions to advance vehicle autonomy. However, these efforts have largely focused on clear-weather scenarios, leaving harsh conditions, such as snow, rain, and fog, comparatively understudied. As a result, inclement weather remains one of the most significant hurdles for autonomous vehicles, often leading to substantial performance degradation in downstream perception and control tasks. In this paper, we present insights from the EU-funded Horizon Europe project ROADVIEW, which aims to develop robust, weather-aware perception and control modules for connected and automated vehicles operating under severe conditions, including snow, rain, and fog. Drawing on four years of large-scale development and real-world demonstrations across multiple vehicle platforms in different locations, we identify ten key challenges that hinder the development of weather-resilient autonomy, including data scarcity, domain shifts, limitations in multimodal perception, control robustness, and validation gaps, among others. We summarize these challenges and lessons learned toward enabling reliable snow-autonomous driving, highlighting critical directions for future research and standardization.
★ AFTER: A World Model with Adaptive Fast--Slow Timescales for Error-Driven Retention
In robotics and embodied AI, a core challenge for world models is adapting to sudden, persistent, yet initially unknown changes in robot dynamics during interaction. For instance, when a robotic arm grasps an object with an unknown payload, the robot dynamics change suddenly. If the world model fails to adapt, its prediction errors accumulate over the open-loop rollout and ultimately mislead the robot's actions. Existing world models, however, struggle to simultaneously satisfy three critical requirements for dynamics adaptation: responsiveness, reacting promptly to sudden changes to curb prediction error accumulation; stability, distinguishing persistent changes from transient disturbances to prevent incorrect updates; and persistence, retaining the adapted dynamics without new observations so that later predictions do not revert to the old dynamics. Inspired by predictive coding in the human brain with multi-timescale adaptation, we propose AFTER-WM, a world model with Adaptive Fast-slow Timescales for Error-driven Retention, whose prediction errors drive a fast correction that fades and a slow correction that persists, while dynamically adjusting the combination weights of pre-trained experts without altering them, to overcome the trade-offs among responsiveness, stability and persistence. On five simulated robot datasets and real UAV data, AFTER-WM has a lower multi-step mean error than all ten compared models and the highest mean return in a joint-failure control task. In simulation, its multistep mean error is 16%-34% below the strongest compared model's and is also the lowest 1-4 steps after a change and under transient disturbances alone, and its step-40 error is the lowest as well, reflecting responsiveness, stability and persistence. All code (method, baselines, experiments) and datasets will be released online.
★ ARTS: An Open-Source, Reproducible Three-Arm Platform for Autonomy Research in Minimally Invasive Surgery
Research on surgical autonomy needs open platforms that any laboratory can build, yet the most widely shared platform depends on donated clinical hardware, and open alternatives on industrial arms each lack part of the surgical chain. We present the Autonomous Robotic Translational System (ARTS), an open-source three-arm platform for autonomy research in minimally invasive surgery, built entirely from purchasable parts for about USD 137,000. Two UR5e arms hold wristed da Vinci X/Xi instruments, a UR5 arm holds a stereo endoscope with two FLIR cameras, the surgeon console provides haptic handles, footswitches and a stereo viewer, and a hardware stop independent of the software lets a supervisor freeze all three arms in place. The ROS 2 software pins every dependency, installs from the same scripts on bare metal or in containers, and records stereo images with hardware-measured timing. On the real arms, ARTS commands each arm at its controller's full rate (e.g., 500 Hz on the UR5e), the two cameras expose within 12 microseconds of each other, hand motion reaches the arm in 58 ms, and the endoscope image reaches the monitor in 68 ms. The ISO 9283 pose repeatability is 0.14-0.25 mm, comparable to a da Vinci Research Kit (dVRK) Classic arm on the same protocol, and an optical tracker measured a mean remote-center-of-motion error of 0.34-0.37 mm. Three expert and three novice operators performed the Fundamentals of Laparoscopic Surgery (FLS) peg transfer through a laparoscopic trainer box, and one of the experts sutured on a suture pad. The hardware designs, firmware and software are available at https://github.com/MRE-Lab-UMD/arts.
comment: Submitted to the 2027 International Symposium on Medical Robotics (ISMR). † Equal contribution
★ PolyMorphWBC: Whole-Body Control from End-Effector Trajectories for Quadrupedal Mobile Manipulation across Arm Morphologies with Extended Reach
Quadrupedal mobile manipulators can extend their manipulation workspace by coordinating the mounted arm with the mobile base. At the task level, short-horizon end-effector (EE) trajectories provide an interface for specifying the desired manipulation motion without prescribing how it should be realized by the arm and base. However, executing such trajectories remains challenging because the controller must determine when and how the base should move, and this decision depends on the mounted-arm morphology. We therefore present PolyMorphWBC, a hierarchical controller that takes an EE trajectory as the sole task-level motion command and generates coordinated arm--base motions across four families of 6-DoF mounted arms, with extended-reach trajectories beyond the initial fixed-base reach. Within PolyMorphWBC, a trajectory-conditioned coordinator determines the base motion from previewed EE targets, while arm-specific Inverse Kinematics (IK) tracks the current target, and a frozen locomotion backend executes the resulting motion. Cross-arm reuse is enabled by a Unified Robot Description Format (URDF)-derived morphology conditioning, together with online residual-wrench feedback. We train PolyMorphWBC on procedurally generated arms and structured trajectories within and beyond fixed-base reach. In IsaacLab, it achieves 93.95% completion with 2.30cm position error and 4.81° orientation error on held-out trajectories and arm morphologies. In MuJoCo evaluation against UMI-on-Legs, PolyMorphWBC reduces EE position error by 69--72% and orientation error by 91--97% across four within- and extended-reach trajectories. On a physical Go2--ARX5 system, we demonstrate language-conditioned extended-reach pick-and-place using EE trajectories from pi_0 without vision-language-action (VLA)-generated base commands.
comment: 8 pages, 4 figures
★ Elastra: A Simulation Framework for Elastic Bodies
When robots interact with elastic surfaces such as mattresses and trampolines, changes in loading cause the surfaces to deform and rebound, altering subsequent contact locations and forces. We present Elastra, a simulation framework for modeling this interaction and supporting the training and evaluation of robot control policies. Elastra represents each surface using elements that move only vertically, with elastic connections and damping to model deformation and recovery. Contact forces drive surface motion, and the updated surface geometry is used to compute subsequent contact forces. We systematically validate our simulations against an independent implementation and a real mattress load curve. We further use Elastra to train the Unitree G1 humanoid to stand up on mattresses and trampolines. Using the pretrained HoST policy and ProtoMotions motion tracker as original controllers, we train residual policies through reinforcement learning to add corrections to their joint targets. Adding these residual policies increases the success rate with HoST from 44.1\% to 72.5\%, and from 55\% to 89\% on the trampoline. With ProtoMotions, the success rate increases from 19.0\% to 45.0\% for initial poses not encountered during training. The code and video are available on https://github.com/yichg/elastra.
★ Decide Before You Move: Diffusion-Generated Search Reference for Zero-Shot Open-Vocabulary Object Navigation in Semi-Static Environments
Open-world navigation requires a robot to locate a user-specified target in an indoor scene under open-vocabulary queries. Realistic indoor environments are semi-static: walls and furniture largely persist while smaller objects move, relocate, or vanish between visits. Prior-map systems answer queries from object poses recorded at reconstruction time, which expire as soon as objects move, whereas online methods re-derive where to look from current observations every few steps, often through expensive calls to a vision--language model. Assuming that walls and large furniture do not move, we keep their map to plan paths, but use a target--furniture co-occurrence prior to decide where to search. We present PreNav, an ObjectNav framework on semi-static environments. Rather than invoking a vision-language model at each step, PreNav generates a single global search reference before the robot moves: an LLM converts target--furniture co-occurrence into a spatial prior, a conditional diffusion model turns that prior and the persistent structure into an ordered centerline with a band around it, and the robot approaches the target once it is detected online. Experiments on a Gibson ObjectNav benchmark show that PreNav outperforms a strong online baseline under matched structural priors, using a generator trained only on room-level ScanNet++ scenes and applied to floor-level Gibson without fine-tuning. Real-world experiments under object relocation and open-vocabulary queries further validate the approach.
comment: 9 pages, 5 figures
★ Prediction or Persistence? What Future-Prediction Scores Miss About World Models
Pretrained video world models are compared and chosen as components for robot learning largely by future-prediction scores. On its own, such a score cannot be read as evidence of prediction, because it mixes encoder quality, the near future's resemblance to the recent past, and the predictor's own contribution. We separate the three with a matched control. On 27,000 future-retrieval trials from household egocentric video, a model must pick the true continuation of a clip among eleven candidates from the same video (chance 0.091), ranked zero-shot by its own training criterion. Each model is paired with a persistence control, an identical copy whose predictor is never invoked, so the paired difference isolates the predictor. Score and contribution dissociate within the native-video V-JEPA family alone: the predictor adds +0.291 top-1 for V-JEPA 2.1 but +0.03 to +0.04 for V-JEPA 2 on the identical trials. Across five models the contribution ranges from +0.291 to negative, and it is negative only for the two action-conditioned models, which we evaluate outside their training regime. Persistence alone recovers the encoder ordering expected from pretraining, a sign that the probe tracks real signal. We suggest reporting each future-prediction score together with its persistence baseline. The gap between them, not the score, is what makes the number interpretable: a necessary check costing one forward pass. Establishing downstream usefulness is a separate matter, for a planning or control evaluation.
comment: Accepted at the CoRL 2026 Workshop "Do Robots Need World Models?"
★ CrowdMAPS: Crowdsourcing Multiple Actions per State for Real-World Imitation Learning
Real-world robot imitation learning is often limited by the cost of collecting physical demonstrations, which typically yield only one action label per visited state. We present CrowdMAPS, a web-based crowdsourcing paradigm for augmenting real-robot datasets with Multiple Actions per State (MAPS). After or during standard robot data collection, CrowdMAPS distributes visited robot states to remote annotators through a web-based 3D interface, where they specify alternative valid actions without requiring access to physical robot hardware. This expands robot datasets along a complementary "width" dimension--by increasing the number of actions per visited state--rather than only by collecting more trajectories. We evaluate CrowdMAPS using keyframe-centric goal-pose annotations supplied by non-expert annotators and train imitation policies on the resulting MAPS data. Across five real-world manipulation tasks, increasing MAPS width from one to ten actions per state improves sample efficiency and increases average success rates by 35.6 percentage points for keypose policies and 33.0 percentage points for trajectory policies. These results show that software-mediated human annotations can augment physical robot data to afford denser supervision, establishing CrowdMAPS as a scalable digital layer for real-robot data collection.
comment: Conference on Robot Learning (CoRL) 2026
★ MiniPose: Miniature-Based Real Data Collection for 6D Pose Estimation of Large Industrial Pipe Segments
6-DOF pose estimation of large and heavy industrial objects, such as large piping systems in shipyards and structural components in petrochemical plants, remains relatively underexplored in robotic perception. Repeatedly repositioning large objects is physically and spatially demanding, making it difficult to acquire real training data with sufficient pose diversity. Synthetic data offer a scalable alternative, but faithfully reproducing real RGB-D observations, particularly the structured measurement errors of real depth sensors, remains difficult. We propose MiniPose, a practical pipeline for miniature-based real RGB-D data collection that addresses both the data-acquisition burden and the sim-to-real gap. We fabricate a scaled-down replica and capture RGB-D data of it in diverse poses using a real RGB-D camera. RGB images are used directly, while depth maps are converted to a geometrically consistent representation of the full-scale object via a simple inverse scaling transformation. This allows training to incorporate real appearance and sensor measurement characteristics that are difficult to reproduce using synthetic data alone. To examine whether the benefit depends on a particular network, we evaluate two RGB-D pose estimators with different formulations, PVN3D and RDPN6D. Mixing at most 1,000 miniature images per pipe type, less than 10\% of the training data, with 10,000 synthetic images improves the mean ADD-S AUC across pipe types over synthetic-only training for both estimators. A comparison with training on full-scale real data further suggests that miniature data can serve as a proxy where full-scale collection is difficult, reducing the data-collection burden for large industrial objects.
comment: 7 pages, 4 figures, 2 tables. Submitted to IEEE Robotics and Automation Letters (RA-L)
★ Does Your World Model Know Actions Have Consequences?
Embodied world models (WMs) are implicit learning systems that predict the next future latent state directly from observations and actions. Unique from other end-to-end learning systems, the dynamics predictor of WMs can simulate the environment's response to alternative actions without physical deployment nor pre-specifying states of the dynamical system. Current metrics focus on the quality of model outputs, e.g., clarity of reconstructions or task success, but fail to assess the fidelity of modeling consequences of counterfactual actions. Inspecting the forced response to action perturbations through sensitivity analysis is one way to audit how well WMs can predict the consequences of counterfactual actions. Deploying WMs on hardware will require new evaluation frameworks to analyze the tradeoffs between state error and accurate sensitivity prediction.
comment: Presented at "Do Robots Need World Models?" Workshop at Conference on Robot Learning (CoRL) 2026
★ Genie Sim Benchmark: Real-to-Sim Fidelity and Evaluation
We evaluate Genie Sim Benchmark through paired real-simulation tasks and complementary system-identification studies. Across 22 robot-task pairs spanning 16 task names, task-specific pi0.5 policies fine-tuned on real-robot data achieve a descriptive Score correlation of 0.9686, MAE 0.0405, RMSE 0.0520, with simulation-real differences from -0.110 to +0.080. G2 crsB experiments assess long-horizon closed-loop position and joint-side torque agreement, while five representative object cases examine identification and cross-engine contact transfer. A separate 32.6503 s teleoperation trajectory, excluded from mechanical identification and servo-gain search, evaluates external position targets through simulated motion-controller command shaping and a calibrated position servo. Across both arms, per-joint position RMSE spans 0.00614-0.03522 rad and current-derived torque RMSE spans 0.73545-4.00099 N m. Native PhysX adaptations recover three recorded interaction tasks, with case-dependent alignment gains. These results support task-specific simulation evaluation through joint assessment of task outcomes and physical-response consistency.
★ Fast Inference-Time Policy Refinement via Action-Affine World Models
Imitation policies can perform complex manipulation, yet small execution errors can accumulate and cause failure. Refinement can correct these errors, but existing approaches often require additional on-policy learning or costly online planning. Our key insight is that a pretrained policy already supplies a useful nominal behavior: refinement can focus on predicting how small action corrections change its outcome. We therefore learn this local response in a form that is cheap to optimize. Our \textit{action-affine world model} (A\textsuperscript{2}WM) learns local action sensitivities offline from action-perturbed policy rollouts. Given interaction history and a policy proposal, it predicts task-relevant geometry as a nominal trajectory plus a linear response to corrections. It predicts the full planning horizon directly, avoiding the propagation of intermediate errors through recursive model evaluations. Quadratic penalties on predicted task errors and linear correction constraints then yield a cheap convex quadratic program (QP). Online refinement requires one model evaluation and one QP solve, while both policy and model weights remain frozen. Across four simulated manipulation tasks, our framework achieves the highest success rate among evaluated baselines, adding only 3.8 ms of mean refinement overhead per control step, excluding base-policy sampling.
★ ILA: A Transferable Interface-Level Adapter for Compliant Humanoid Motion Tracking
Humanoid motion tracking is fundamental to reproducing human motions on humanoid robots and serves as an important tool for teleoperation and motion data collection. However, directly tracking reference motions can be unsafe in constrained environments or during physical human--robot interaction, motivating the need for compliant motion tracking. We present ILA, an Interface-Level Adapter for compliant humanoid motion tracking without dedicated interaction sensors. ILA adapts motion references from the robot's observation history, allowing the robot to respond to physical guidance while preserving the capabilities of a pretrained motion tracker. As an interface-level module, ILA can be directly plugged into different pretrained motion trackers without retraining either the adapter or the tracking policies. ILA is trained entirely in simulation to predict reference corrections induced by physical interaction. We evaluate ILA with three humanoid motion trackers and achieve a 63.6--75.4% reduction in guidance-target error compared with their original policies. Matched simulation studies and hardware experiments further show that ILA improves responsiveness to human guidance while preserving nominal motion-tracking performance. These results demonstrate compliant guidance as a reusable, sensor-free extension to pretrained humanoid motion trackers.
comment: 8 pages, 6 figures
★ From a Single RGB Image to Robot Execution: Metric Scene Alignment and Contact-Guided Retargeting
Extending humanoid interaction generation to new environments remains limited by the need for prebuilt 3D scene assets. Reconstructing scenes directly from single RGB images offers a way to overcome this limitation, but introduces a key challenge in maintaining geometric consistency throughout motion generation and transfer. Visually plausible human motions may be misaligned with the reconstructed scene, while retargeting can introduce additional support errors due to differences between human and robot embodiments. We present a unified pipeline that transforms a single RGB image into a humanoid motion reference. It reconstructs scene geometry from the input image, generates a human interaction conditioned on the reconstructed scene, and retargets the motion to a humanoid robot. To maintain geometric consistency throughout this process, we incorporate two complementary refinement stages before and after embodiment transfer. Before retargeting, Joint Metric Interaction Refinement (JMIR) refines human motion alignment with the registered metric scene while preserving the generated interaction structure. Following retargeting, Contact Guided Residual Retargeting (CGRR) corrects robot specific support errors to restore the intended support relationships. Together, these stages produce motion references that are consistent with the reconstructed scene and can be followed by multiple pretrained humanoid trackers without scene specific policy fine tuning. Compared with GRAIL, our method reduces refinement time from 480 s to 87 s, yielding a speedup of 5.5 times. The complete pipeline achieves a success rate of 89% in simulation and 70% across 50 physical trials on the Unitree G1, demonstrating its effectiveness in generating executable humanoid motion references from single RGB images.
comment: 8 pages, 6 figures
★ Scaling VLA Serving Under Robotic Execution Constraints
Vision-language-action (VLA) models have emerged as a promising paradigm for general-purpose robotic manipulation. In our production setting, hundreds of VLA-powered robots are already operating in real-world environments, creating an immediate need for efficient fleet serving. We present XInfer, a production-deployed VLA serving system designed to reduce the inference infrastructure cost of large-scale robotic deployments. XInfer jointly coordinates heterogeneous VLA computation with the evolving response requirements of robotic execution. By exploiting computational parallelism while managing interference, XInfer improves GPU efficiency while protecting time-critical robotic feedback. We evaluate XInfer across three VLA models using production-derived workloads, achieving 1.26-3.75 times the effective robot capacity (ERC) of state-of-the-art systems under physical-execution constraints. Our implementation is publicly available at https://github.com/X-Square-Robot/xinfer.
★ Adapting Monocular Depth Estimation in Novel Scenes through Disparity Refinement and Robot Ego-Motion
The geometric understanding of a scene is a crucial component for safe and efficient robotic operations in novel and unstructured environments. The recent emergence of foundation models for Monocular Depth Estimation (MDE) has provided an opportunity to enrich robotic perception capabilities with dense long-range depth predictions, replacing expensive onboard hardware (e.g., LiDARs and stereo cameras). However, these offline models are primarily trained on affine-invariant depth representations, and their generalization to novel scenes remains unclear. In this work we present EM-ViT, a monocular depth estimation model that learns to adapt online relative depth inputs from a foundation model prior to the metric scale with observations collected during robotic operations. Motivated by the stereo camera parameterization, our method decomposes depth prediction via dense disparity refinement to recover the scene's relative structure and an effective baseline scalar prediction to determine the missing metric scale. The learning process leverages sparse LiDAR points and, optionally, the robot's Ego-Motion (EM) to rapidly train a lightweight Vision Transformer (ViT), while an internal gradient matching loss preserves the fine-grained semantic structure of the input disparity in real-time. We demonstrate the efficacy of our method on publicly available sequences, such as KITTI, and show that our online learning pipeline can adapt to novel scenes faster and with higher accuracy than existing MDE foundation models. The code and supplementary materials will be publicly available at https://emvit-mde.github.io.
comment: 17 pages, 12 figures, journal
★ The Posterior-Prior Epistemic Gap: When World Model Imagination Fails to Carry Task-Relevant Signals
Model-based planners that roll out imagined futures assume the transition model faithfully propagates task-relevant signals---a property rarely measured directly. In a single-seed case study of a Recurrent State Space Model world model with active-inference planning in the CARLA driving simulator, we identify a \textbf{posterior-prior gap}---posterior (observation-filtered) states retain obstacle information that imagined rollout states progressively lose: a supervised obstacle probe achieves AUC 0.85 on posterior states, but the same probe applied to imagined prior states shows monotonic decay (AUC 0.85$\rightarrow$0.71 over 15 steps; logit separation drops 57\%). Because the probe is identical, the decay isolates the transition model---not the representation---as the information bottleneck. Obstacle avoidance drops to 0\% without privileged runtime coordinates, and the planner's own ensemble-based epistemic term does not flag the loss---ensemble disagreement yields AUC 0.52 as an obstacle classifier (chance level), making the failure an unflagged imagination failure---an unknown unknown at planning time. As a secondary contribution, we show that \textbf{task-conditioned weighting of Effective Free Energy components} over the same world model yields lane keeping ($81.2 \pm 1.8$\% route completion) and obstacle avoidance (28/28 avoided, \emph{using privileged coordinates}) in the CARLA simulator, with runtime weight modulation enabling post-evasion centering.
★ DIVAL: Diffeomorphic Vector-Field Alignment for Nonholonomic Motion Planning
Conventional vector-field-based motion planners for nonholonomic robots consist of two steps: first designing a nominal vector field and then controlling the robot to follow it to approach the target. However, a nonholonomic robot cannot follow the vector field instantaneously and may collide with obstacles, due to the inevitable mismatch between the robot's initial heading and the vector field's direction. This paper proposes a diffeomorphic vector-field alignment (DIVAL) method, which aligns the vector field's direction with the robot's initial heading so the robot can follow the vector field from the start. Specifically, the nominal vector field is locally reshaped by applying a diffeomorphic deformation to the workspace, such that the reshaped vector field has the same direction as the robot's heading at the initial position and coincides with the nominal vector field outside the deformed region. Since DIVAL eliminates the initial heading mismatch, the nonholonomic robot can maintain heading alignment with the reshaped vector field throughout the motion. This heading alignment further ensures that the robot preserves the nominal vector field's properties, including obstacle avoidance, convergence to the target, and smooth control signals. Numerical simulations provide vector-field-tracking baselines where the nominal vector fields are reshaped with DIVAL. Compared to the original baselines, the proposed method achieves collision-free convergence in all trials and avoids control discontinuities. Hardware experiments on a ground vehicle further demonstrate collision-free convergence across different targets and initial conditions under real-world disturbances.
comment: 19 pages, 12 figures, under review
★ HeadIO: Body-Frame Heading Decomposition for Neural Inertial Odometry on Head-Mounted Devices ICRA 2027
Inertial odometry (IO) using only Inertial Measurement Units (IMUs) enables camera-free head tracking in head-mounted devices such as VR headsets, AR glasses, and earbuds. Prior IO methods operate in a gravity-aligned world frame, which is not yaw-equivariant. Yaw augmentation improves robustness but does not guarantee equivariance; learned equivariant layers achieve it, but EqNIO's full pipeline costs nearly 3$\times$ the FLOPs of ours. While raw body-frame IMU inputs are inherently yaw-invariant, head tilt redistributes both gravity and motion signals across device axes, making the representation tilt-variant. We propose HeadIO, which combines a body-frame heading decomposition with a bidirectional Mamba backbone for head-mounted inertial odometry. Our decomposition achieves tilt invariance by separating gravity-aligned and horizontal components using only the tilt component of device orientation. We evaluate on four head-mounted datasets and find that HeadIO improves RTE-5s by 8.1% (0.260 m $\to$ 0.239 m) over world-frame methods with learned equivariant layers on Nymeria, the largest benchmark dataset, at nearly 3$\times$ lower FLOPs and 8$\times$ fewer parameters. HeadIO achieves state-of-the-art performance on our primary metric, RTE-5s, on all Aria-glasses datasets, which natively provide body-frame semantics for our decomposition method. We further evaluate the IMU-only deployment scenario on Nymeria, where HeadIO remains state-of-the-art. We show that our method relies mainly on tilt accuracy, which can be obtained from the gravity signal; for yaw, a single-coefficient gyroscope correction removes 11.6% of the yaw error and improves RTE-5s by 2.5%, and sparse external yaw corrections (for example, one every 30 s) can improve RTE-5s by 12.0%. Our decomposition is lightweight, composes with any backbone, and is suitable for real-time deployment on head-mounted devices.
comment: Submitted to ICRA 2027
★ Can I join you? Extended Social Force Based Model with Predictive Control and Barrier Functions for Humanoid Group Repositioning Amidst Obstacles
As social and service robots increasingly share interaction spaces in public, they may form groups while coordinating tasks or interacting with humans. Initial group formation alone is insufficient in dynamic settings, because it can be disrupted when a person or another agent joins, leaves, or passes through the group. Although group dynamics and repositioning have been widely studied in swarm robotics and virtual-agent settings, little attention has been given to socially appropriate repositioning behaviors in groups of humanoid robots. Existing social-force-based approaches provide a useful foundation for group behavior, but they have limited obstacle handling, and can produce unsafe behavior as environmental complexity increases. We propose a novel framework that modulates traditional Social Force, guided by Model Predictive Control (MPC) to plan ahead, bounded by Control Barrier Functions (CBF) to increase safety for robot groups repositioning behaviors. This approach aims to produce safer and socially aware group repositioning behaviors under various group formation disturbance. We evaluate the method in simulation through an ablation study and an online user study. The results show that our proposed method can produce smoother and safer repositioning behaviors in complex environments while preserving social appropriateness.
comment: 8 pages, 4 figures, 2 tables
♻ ★ COSMO-Bench: A Benchmark for Collaborative SLAM Optimization ICRA57385
Recent years have seen a focus on research into distributed optimization algorithms for multi-robot Collaborative Simultaneous Localization and Mapping (C-SLAM). Research in this domain, however, is made difficult by a lack of standard benchmark datasets. Such datasets have been used to great effect in the field of single-robot SLAM, and researchers focused on multi-robot problems would benefit greatly from dedicated benchmark datasets. To address this gap, we design and release the Collaborative Open-Source Multi-robot Optimization Benchmark (COSMO-Bench) -- a suite of 24 datasets derived from a baseline C-SLAM front-end and real-world LiDAR data. Data DOI: https://doi.org/10.1184/R1/29652158
comment: This article has been accepted for publication in IEEE International Conference on Robotics and Automation. DOI: 10.1109/ICRA57385.2026.11696499
♻ ★ riMESA: Consensus ADMM for Real-World Collaborative SLAM
Collaborative Simultaneous Localization and Mapping (C-SLAM) is a fundamental capability for multi-robot teams as it enables downstream tasks like planning and navigation. However, existing C-SLAM back-end algorithms that are required to solve this problem struggle to address the practical realities of real-world deployments (i.e. communication limitations, outlier measurements, and online operation). In this paper we propose Robust Incremental Manifold Edge-based Separable ADMM (riMESA) -- a robust, incremental, and distributed C-SLAM back-end that is resilient to outliers, reliable in the face of limited communication, and can compute accurate state estimates for a multi-robot team in real-time. Through the development of riMESA, we, more broadly, make an argument for the use of Consensus Alternating Direction Method of Multipliers as a theoretical foundation for distributed optimization tasks in robotics like C-SLAM due to its flexibility, accuracy, and fast convergence. We conclude this work with an in-depth evaluation of riMESA on a variety of C-SLAM problem scenarios and communication network conditions using both synthetic and real-world C-SLAM data. These experiments demonstrate that riMESA is able to generalize across conditions, produce accurate state estimates, operate in real-time, and outperform the accuracy of prior works by a factor >7x on real-world datasets.
comment: This article has been accepted for publication in IEEE Transactions on Robotics. DOI: 10.1109/TRO.2026.3706569
♻ ★ Dataset-Free Compliant Humanoid Loco-Manipulation with Dynamic Online Posture
Most humanoid loco-manipulation controllers require human motion data to learn whole-body coordination and posture, leaving policies reliant on external sources to provide this data. We present OCLO (Online-posture Compliant LOco-manipulation), a humanoid loco-manipulation system trained without human motion data and commanded only through two end-effector targets and a walking velocity. Because these targets do not uniquely determine whole-body posture, OCLO generates pelvis height and torso orientation online using an analytic reachability prior, further refined through policy-in-the-loop sampling with a task-agnostic cost. OCLO also learns whole-body compliance by displacing end-effector references according to measured forces through a spring-damper model, encouraging the legs, waist, and pelvis to yield to external loads. In simulation, using the reachability prior leads to a 77.8% success rate in acquiring the commanded reference, a vast improvement over the 37.8% success rate accomplished without the prior. Further, refinement reduces end-effector orientation error across all evaluated tasks. The same posture module improves a pretrained SONIC controller on all five tasks. Without compliance training, policies tend to lose balance under disturbances rather than sacrifice tracking. On a Unitree G1, OCLO maintains balance under end-effector disturbances that cause its ablations to fail and performs nine loco-manipulation tasks, including crouched walking and picking up a box from a low surface. Project website: https://oclo-humanoid.github.io/
♻ ★ AVI-HT: Adaptive Vision-IMU Fusion for 3D Gloved Hand Tracking NeurIPS 2026
We present AVI-HT, an adaptive visual-IMU fusion approach for tracking 3D hand poses by jointly modeling the egocentric images with on-glove 6-DoF IMU signals. AVI-HT significantly improved accuracy and availability, particularly in hand-object interaction (HOI) scenarios involving heavy visual occlusion. Two complementary ingredients underpin its success: (1) synchronized multi-modal training data pairing on-body vision-IMU sensor streams with ground-truth 3D hand poses from a motion-capture system, and (2) a cross-sensor deep attention mechanism that adaptively modulates the trust assigned to the vision and individual IMU sensors. To evaluate AVI-HT in real-world settings, we conduct extensive experiments on our DexGloveHOI dataset that consists of 100K+ pairwise vision-IMU samples with synchronized 3D annotated poses, in which users manipulate a variety of objects during daily tasks. We compare against multiple single- and multi-modal tracking approaches under two hand models (UmeTrack, MANO). The results show that AVI-HT reduces mean keypoint error by 16.1% and its wrist-aligned variant by 24.2% over the baselines. A hand-object contact evaluation shows that AVI-HT keeps finger contacts with the manipulated object more accurately and consistently over time, reducing contact losses by 8.5%. Ablation studies further show that our fusion design outperforms simpler schemes and that the gains grow with occlusion and hold across subjects.
comment: Accepted to NeurIPS 2026
♻ ★ QuantGraph: A Receding-Horizon Quantum Graph Solver
Dynamic programming is a cornerstone of graph-based optimization. While effective, it scales unfavorably with problem size. In this work, we present QuantGraph, a two-stage quantum-enhanced framework that casts local and global graph-optimization problems as quantum-searches over discrete trajectory spaces. The solver is designed to operate efficiently by first finding a sequence of locally optimal transitions in the graph (local stage), without considering full trajectories. The accumulated cost of the resulting candidate acts as a threshold that prunes the search space (up to 60% reduction for certain examples). The subsequent global stage, based on this threshold, refines the solution. To achieve scalability and robustness, we draw on principles from control theory and embed QuantGraph's global stage within a receding-horizon model-predictive-control scheme. This classical layer stabilizes and guides the quantum search, improving precision and reducing computational burden. In practice, the resulting closed-loop system exhibits robust behavior and lower query complexity. Notably, for a fixed control horizon and query budget, QuantGraph supports approximately twice as many control discretization bits while still benefiting from Grover-search's inherent quadratic speedup compared to classical methods.
comment: P. Vaidhyanathan and A. Papatheodorou contributed equally to this work. Published in Physical Review Applied. 17 pages, 4 figures, 2 tables, 2 algorithms
♻ ★ Discrete Model-Based Diffusion for Combinatorial Optimization in Multi-Robot Task Allocation
Decision making in multi-robot systems often involves discrete decision variables, leading to NP-hard combinatorial optimization problems that must be solved within seconds for near real-time implementation. Denoising diffusion has emerged as a powerful tool for such problems, but existing approaches face a trade-off: Model-Based Diffusion (MBD) incorporates diffusion in sampling-based optimization (SBO) without training but only over continuous spaces, while existing diffusion-based combinatorial solvers operate over discrete spaces but require offline training. To this end, we propose Discrete Model-Based Diffusion (Discrete MBD), a training-free SBO method that directly optimizes discrete variables by approximating categorical reverse transition distributions through Monte Carlo sampling. We establish a finite-step lower bound on the probability of finding a global minimizer and show that this bound converges to one as the number of denoising steps increases. Extensive experiments on generic integer programs and heterogeneous multi-robot coalition formation and scheduling demonstrate that Discrete MBD outperforms SBO baselines in nearly all settings while achieving solution quality competitive with that of mixed-integer programming and reinforcement learning methods at a fraction of their computational cost.
comment: 11 pages, 3 figures
♻ ★ BiGym 2.0: Benchmarking Learned and Agent-Developed Policies for Humanoid Household Manipulation
Humanoid household manipulation requires the arms to act while the body balances, steps and changes posture. We present BiGym 2.0, an adaptation of BiGym for the Unitree G1 across 20 household tasks using a unified whole-body controller for demonstration and evaluation. The suite provides 60 native human virtual-reality demonstrations per task with synchronised multi-camera views and full-body execution records. We benchmark vision-language-action fine-tuning, imitation learning, demo-driven reinforcement learning, and cold-start coding agents given the interaction budget of online reinforcement learning. With the same onboard views, proprioception and whole-body controller for every method, vision-language-action fine-tuning has the highest nine-task mean, and agent-developed programs outperform every demo-driven reinforcement learning baseline on this mean and lead on bimanual reaching. Cross-workspace stacking remains open, $π_{0.5}$ stays low on pick-box, and multi-object transport is hard for imitation learning, demo-driven reinforcement learning and coding agents. All environments, human demonstrations, and evaluation traces are open-sourced at https://github.com/swirl-uk/BiGym2.
comment: 23 pages. v3: Fig. 2 shows the torso-pitch body command used on 14 tasks; gripper named Dex1-1 throughout; seed notation
♻ ★ Direct Informed Sampling on Riemannian Manifolds via Loewner Order Lower Bounds
Informed sampling techniques accelerate sampling-based motion planners by focusing the search on promising regions of the state space, yet most existing methods rely on Euclidean heuristics that become inadmissible under configuration-dependent Riemannian metrics. While scalar eigenvalue bounds restore admissibility by uniformly scaling the Euclidean distance, they discard the directional structure of the metric, producing overly conservative informed sets. We propose a matrix-valued admissible heuristic that exploits the Loewner order on symmetric positive definite matrices to compute the tightest constant lower bound on the metric tensor while preserving its full directional structure. The Cholesky factorization of this bound defines a linear map to an isotropic Euclidean space in which the Riemannian informed set reduces to a standard prolate hyperspheroid, enabling direct, rejection-free sampling using existing algorithms. Experiments on manipulation tasks with a 6-DoF UR5, 7-DoF Franka, and 14-DoF PR2 under three distinct Riemannian metrics show that our heuristic produces consistently tighter informed sets than both the Euclidean and scalar eigenvalue bounds, accelerating convergence across multiple state-of-the-art asymptotically optimal planners.
comment: In IEEE Robotics and Automation Letters (RA-L)
♻ ★ Intent Representation and Human Distraction Modulate the Effects of Legible Robot Motion in Hallway Navigation
Legibility in human-robot interaction (HRI) is often described as the property of robot motion that enables an observer to confidently and quickly infer the robot's intent. Mature frameworks exist for generating legible motion in front of static observers, typically by making the robot's destination apparent as early as possible. Social robot navigation in public spaces, however, presents a different challenge: pedestrians are themselves in motion, often with divided attention, and must coordinate with the robot in real time. In such encounters, where the robot is ultimately headed may matter less to a pedestrian than how it intends to pass them. This raises two questions: which aspect of intent should legible motion convey, and does legibility remain beneficial when pedestrians are not fully attentive to the robot? We address these questions in the ubiquitous and demanding scenario of hallway navigation, through two controlled user studies comparing multiple legibility formulations implemented within a shared model predictive control framework. Study 1 (N = 45) compares destination-based and passing-side intent representations, showing that the latter, particularly when adaptively updated, allows for smoother human motion and is perceived as more competent and less mentally and physically demanding than the former. Study 2 (N = 45) examines the effect of pedestrian attention, demonstrating that legible motion supports smooth human motion even under distraction, although this benefit is not consistently reflected in subjective ratings. Together, these findings suggest that effective legible motion in social robot navigation benefits from interaction-level intent representations that support coordination, with some effects persisting even when human attention is divided.
comment: 25 pages, 7 figures, 3 tables. Accepted to ACM Transactions on Human-Robot Interaction (THRI). Code: https://github.com/fluentrobotics/Legible_MPPI. Video: https://www.youtube.com/watch?v=P5O5BwfcUCo
♻ ★ BLaFA: Bridging Language to 3D Functional Affordances for Dexterous Robotic Grasping in Tool Use
In industrial assembly, equipment maintenance, and everyday tool-use scenarios, robots need not only to grasp objects stably but also to establish task-appropriate contact configurations according to task intentions and tool functionality. However, existing methods lack explicit mappings from open-vocabulary instructions to 3D functional contact structures and dexterous grasp configurations, limiting robots' adaptability to diverse tools and functional tasks. To address this limitation, we propose BLaFA (Bridging Language to 3D Functional Affordances for Dexterous Robotic Grasping), a structured framework for language-driven functional dexterous grasping. First, a Knowledge-guided Language Parsing (KLP) module leverages structured knowledge to transform natural-language instructions into semantic sextuples of manipulation constraints. Second, a 3D Gaussian Splatting based hierarchical semantic representation and a Triangular Functional Point Localization (TriLocation) module are developed to localize 3D functional contact keypoints under task-tool topology constraints. Finally, a 3D Keypoint Grasp Matrix Transformation (KGT3D+) module maps the localized keypoints and semantic constraints to executable wrist poses and finger-level joint and force commands. Evaluations on 100 open-vocabulary language instructions, 14 task-tool combinations covering four functional intents, and real-world robotic experiments in representative scenes demonstrate that BLaFA can effectively translate natural-language instructions into task-aligned 3D functional contact structures and corresponding dexterous grasps, thereby enabling language-driven robotic tool use across diverse tasks. The source code will be made publicly available. Code will be publicly available at https://github.com/PopeyePxx/BLaFA.
comment: Code will be publicly available at https://github.com/PopeyePxx/BLaFA
♻ ★ HEHA: Bounded Suboptimal Planning for Constrained Routing and Its Use in Heterogeneous Multi-Robot Exploration of Unknown Environments
This paper considers the path planning problem for autonomous exploration of an unknown environment using multiple heterogeneous robots such as drones, wheeled, and legged robots, with different capabilities to traverse complex terrains. A key challenge there is to intelligently allocate the robots to the unknown areas to be explored and determine the visiting order of those areas subject to traversability constraints, which leads to a large-scale constrained multiple hamiltonian path problem (mHPP) that needs to be quickly and iteratively solved every time when new space is explored. To address the hallenge, we propose HEHA (Hierarchical Exploration with Heterogeneous Agents) using a recent hierarchical method that decomposes the exploration into global planning and local planning. The major contribution in HEHA is its global planning, where we propose a new routing algorithm RCH (Routing with Constraints and Heterogeneity) that can quickly find bounded sub-optimal solutions for mHPP to minimize the maximum path length (i.e., makespan) among the agents subject to traversability constraints. Additionally, the local planner in HEHA also considers heterogeneity to avoid repeated and duplicated exploration among the robots. The experimental results show, in the presence of heterogeneous traversability, our RCH can solve mHPP with up to 47% smaller makespan than several baselines, and HEHA can reduce the exploration time by up to 30% compared with existing approaches.
comment: Accepted by T-RO. 16 Figures
♻ ★ MPPI-Generic: A CUDA Library for Stochastic Trajectory Optimization
This paper introduces a new C++/CUDA library for GPU-accelerated stochastic optimization called MPPI-Generic. It provides implementations of Model Predictive Path Integral control, Tube-Model Predictive Path Integral Control, and Robust Model Predictive Path Integral Control, and allows for these algorithms to be used across many pre-existing dynamics models and cost functions. Furthermore, researchers can create their own dynamics models or cost functions following our API definitions without needing to change the actual Model Predictive Path Integral Control code. Finally, we compare computational performance to other popular implementations of Model Predictive Path Integral Control over a variety of GPUs to show the real-time capabilities our library can allow for. Library code can be found at: https://acdslab.github.io/mppi-generic-website/ .
comment: Fixed typo in Jetson Orin Nano results in table
♻ ★ Task-Driven Closed-Loop Design Optimization of Passively Morphing Quadrotors for Contact-Mediated Narrow Gap Traversal
Aerial robots operating in cluttered environments may encounter openings narrower than their nominal width. Existing passively morphing drones reduce their width upon traversal, but they typically involve human-made designs, which can struggle to find the right balance among folding ratio, flight efficiency, and control tracking performance. We introduce a closed-loop design method that jointly optimizes quadrotor geometry and the torque-deflection response of two passively folding arms driven by the success rate in simulation for contact-mediated gap traversal. An optimization algorithm samples candidate designs that are evaluated in a contact-based simulation of the traversal task, where the task success rate is used as a feedback signal to sample progressively better designs over time. The resulting prototype passively reduces its width from 31.3 to 14.1 cm and traverses a 15 cm wide opening at a 2 m/s reference speed. It also tolerates gap approach-angle errors of up to 30 degrees and folds through a 40 cm long confined passage. Compared to a rigid twin, the soft configuration adds only 2 mm of position RMSE and less than 1% electrical power during contact-free flight. These results show that task-driven closed-loop design optimization can produce passively morphing airframes that are compact under contact, tolerant to approach error, and effectively rigid during nominal flight.
comment: Video: https://youtu.be/xGkEtl6-vCA. Code: https://github.com/ntnu-arl/soft-quadrotor-optimization
♻ ★ Model-Predictive Spill-Free Emergency Stopping for Liquid-Manipulating Robots
Manipulating open liquid containers is challenging because liquids are highly sensitive to vessel accelerations and jerks. Although spill-free liquid manipulation has been widely studied, emergency stopping under unexpected hazards has received little attention, despite the fact that abrupt braking may cause hazardous spills. This letter presents an emergency stop system for robots manipulating liquids in open containers. We formulate emergency stopping as an optimal control problem and solve it in a model predictive control framework to generate time-optimal, spill-free stopping trajectories. The method operates as a plug-and-play safety layer on top of existing slosh-free motion planning methods, enabling immediate reaction to detected hazards while accounting for nonlinear liquid dynamics. We demonstrate, through simulation and on a 7-DoF Franka Emika Panda robot, that the proposed approach achieves fast emergency stopping without spilling.
♻ ★ CineWild: Balancing Art and Robotics for Ethical Wildlife Documentary Filmmaking
Unmanned aerial vehicles (UAVs) have become powerful tools across domains from industry to the arts. In documentary filmmaking, they offer dynamic, otherwise unreachable perspectives, transforming how stories are told. Wildlife documentaries especially benefit, yet UAVs also raise ethical concerns like the risk of disturbing the animals they aim to capture. This paper introduces CineWild, a model predictive control (MPC) formulation for autonomous aerial cinematography that combines robotics, cinematography, and ethics. Building on an MPC solution that jointly controls drone motion and camera focal length, CineWild dynamically adjusts flight paths and camera settings to balance cinematic quality with animal welfare. Key features include adaptive zoom for filming from acoustic and visual safe distances, path-planning that avoids an animal's field of view, and smooth, low-noise maneuvers. CineWild exemplifies interdisciplinary innovation, bridging engineering, visual storytelling, and environmental ethics. We evaluate the formulation in simulation, with ablation studies and a comparison against a cinematographic baseline and a welfare-aware baseline under identical weights, and we demonstrate the perception and camera-actuation pipeline in real-world proof-of-concept trials. We will release the code.
♻ ★ OPA-Pack: Object-Property-Aware Robotic Bin Packing
Robotic bin packing is a fundamental manipulation problem with broad applications in e-commerce and warehouse automation. However, existing methods primarily focus on object geometry to optimize packing compactness without explicitly modeling object properties such as fragility, edibility, and chemical compatibility that humans typically consider during packing. This paper presents OPA-Pack (Object-Property-Aware Packing), a framework that incorporates object-property information into robotic bin-packing planning. The framework includes an object property recognition scheme based on retrieval-augmented generation and chain-of-thought reasoning to establish a dataset with object property annotations for 1,032 everyday objects. The framework further incorporates OPA-Net, which encodes candidate-object properties via a property embedding layer and represents the container state using fragility and avoidance heightmaps. We design a reward function and adopt a deep Q-learning scheme to train OPA-Net. Experimental results show that OPA-Pack improves avoidance accuracy from 52.3% to 95.0%, reduces load on fragile objects by 29.4%, and achieves a packing compactness of 0.413, compared with 0.420 for the baseline. In addition, we evaluate OPA-Pack on a physical platform using 70 packing sequences sampled from 100 scanned real-world objects. Results show that OPA-Pack outperforms the baseline and other packing heuristics in reducing the number of closely packed incompatible object pairs and the number of compressed fragile objects while preserving packing compactness.
♻ ★ ActionUNet: Improving Robustness of VLA Models with Efficient Multi-scale Fine-tuning NeurIPS 2026
Vision-Language-Action (VLA) models have shown great promise for robotic manipulation by mapping multi-modal semantics to physical actions. However, this mapping inherently struggles to align these coarse-grained semantics with fine-grained temporal execution. It leaves VLA models with limited generalization and insufficient robustness in cluttered environments. To overcome this issue, we propose ActionUNet, an efficient multi-scale fine-tuning framework that enhances pre-trained VLA models with minimal computational cost. ActionUNet first constructs a lightweight temporal U-Net within the temporal-aligned action feature space to fuse hierarchical structural priors, effectively bridging the scale gap between semantics and temporal executions. Recognizing that multi-scale modeling can disrupt microscopic temporal continuity and cause mechanical oscillations, ActionUNet then employs a conditional SIREN as a continuous action decoder. Equipped with explicit second-order smoothness constraints, this decoder guarantees temporal continuity and reduces high-frequency motion jitter. By smoothing temporal discontinuities from multi-scale fusion, this continuous formulation reduces mechanical execution failures while preserving the base VLA model's generalization and manipulation robustness. Extensive experiments on RoboTwin 2.0 and LIBERO-Plus benchmarks, together with real-world hard evaluations, demonstrate that ActionUNet significantly improves π0.5 success rates by absolute 9.8%, 6.1%, and 11.4%, respectively, while also generalizing to the regression-based OpenVLA-OFT backbone, highlighting its effectiveness and efficiency as a fine-tuning strategy. Code and implementation details are available at https://github.com/Di-Zhu123/ActionUNet.
comment: Accepted to NeurIPS 2026
♻ ★ NS-VLA: Towards Neuro-Symbolic Vision-Language-Action Models NeurIPS 2026
Vision-Language-Action (VLA) models ground natural-language instructions in visual observations to generate actions for robotic manipulation. On long-horizon tasks, existing VLA models face structure-blind backbones, backbone-bound generalization, and flat single-objective optimization. To address these challenges, we propose a novel Neuro-Symbolic Vision-Language-Action (NS-VLA) framework. It introduces a Neuro-Symbolic Encoder that parses the instruction into a primitive plan tracked by a gated monotone stage pointer, a Neuro-Symbolic Solver that conditions a chunked VLA policy on the active sub-instruction, and Hierarchical Joint Policy Optimization, which fits the pointer on demonstration segments and refines the action policy with success-filtered online learning. Even when trained on few demonstrations, NS-VLA clearly leads on LIBERO and the held-out LIBERO-Plus (76.5% and 51.8% against 71.2% and 45.4% for its backbone), exceeds the strongest baseline by 12.7 points on long-horizon tasks, and raises real-robot success from 40.0% to 60.0%. Our code is publicly available at https://github.com/Zuzuzzy/NS-VLA.
comment: Accepted by NeurIPS 2026 main conference
♻ ★ ActPrune: Action-Grounded Visual Token Pruning for Efficient Vision-Language-Action Models
Vision-language-action (VLA) models perform versatile manipulation tasks from language instructions, but their inference cost limits how fast a robot can respond in closed-loop control. Visual token pruning reduces this cost; however, prior methods estimate token redundancy from encoded visual embeddings or attention, only after the visual encoder has processed the entire observation. We propose ActPrune, a training-free token pruning method that moves the pruning decision ahead of the vision encoder by grounding it in a geometric prior derived from the robot's own action. The action chunk predicted at the previous control step is already available before the current observation is encoded, and it specifies the end-effector and camera motion between consecutive observations. The same action affects the two standard camera views asymmetrically: in the fixed third-person view, ActPrune retains patches along the projected end-effector trajectory together with semantic anchors, while in the wrist view, it prunes patches that remain unchanged after compensating for the action-induced camera motion. Extensive experiments on two simulation environments (LIBERO and VLABench) and four real-robot tasks demonstrate that ActPrune achieves up to 1.61$\times$ speedup and 59.5% FLOPs reduction with negligible degradation in success rates. The project site is http://mmai.ewha.ac.kr/actprune/.
♻ ★ Efficient VLA Inference via Retrieval-Based Speculation and Hardware Acceleration
Vision-language-action (VLA) models enable robots to understand and execute complex instructions, but their autoregressive (AR) action generation introduces significant latency on resource-constrained robotic platforms. Speculative decoding can accelerate AR-based inference by parallelizing action generation and verification; however, existing approaches rely on costly draft models, which are inefficient on edge platforms. In this paper, we propose Zora, a hardware-software co-design framework for accelerating AR-based VLA inference. Specifically, Zora introduces a lightweight retrieval-augmented generation (RAG)-based draft model that exploits robotic dynamics without additional neural draft models. We further reduce batch verification overhead through a kinematics-guided action-acceptance strategy and a KV-cache-sharing attention mechanism. To address the retrieval overhead introduced by RAG-based drafting, we co-design dedicated hardware support, Zora core, to address the memory irregularity in RAG-based drafting. We evaluate Zora on four VLA models with the LIBERO benchmark. Zora retains comparable accuracy while achieving up to 3.4$\times$ speedup with software-only optimization and up to 7.6$\times$ speedup with hardware support over GPUs.
♻ ★ Reading the Room: Scale-ambiguous Semantic Registration of Sparse VSLAM Maps to Floor Plans in Partially Built Environments
Many robotic construction applications, such as progress tracking, require registering a sensor trajectory to a prior 2D floor plan. Visual capture is attractive for its low cost and availability, but aligning a visual simultaneous localization and mapping (VSLAM) map to a floor plan is difficult; VSLAM recovers geometry only up to an unknown similarity transform, the sparse point cloud is cross-modal to the symbolic floor plan, sites are only partially built during capture, and construction sites are visually repetitive, degrading map quality while removing distinctive features. Existing methods address at most two of these challenges. We thus introduce Reading the Room(RR), a global-local approach to register scale-ambiguous, sparse, semantic VSLAM maps to partially built floor plans. Our key idea is to define a single robust, differentiable, class-balanced semantic-geometric energy function that governs all stages of the pipeline. We develop two global initialization schemes, leveraging a single inaccurate user-provided correspondence if provided, or estimating the initialization globally using a cross-modal matching network. Both schemes are designed to find initial candidates with respect to the energy function, which is then directly optimized for local refinement. We evaluate RR on 33 sequences across three active construction sites against classical and learning-based baselines, achieving a 4.5x improvement in mean absolute trajectory error over the strongest baseline in the general unconstrained setting, which increases to 7.6x when an anchor is provided.
♻ ★ SWAM: Scaling World Action Models with Cross-Modal Alignment
World action models jointly predict video and actions conditioned on language. Existing studies focus primarily on architectures and training mechanisms, typically treating language-video alignment acquired through video pretraining as an established foundation. However, introducing action learning may impair this alignment. Principled guidance on data and supervision design for maintaining language-video alignment during action learning remains limited. To investigate this, we curate a pool of approximately 47,000 hours of general videos, egocentric videos, UMI demonstrations, and robot demonstrations. Building on this, we systematically study how language-video supervision, language-video-action supervision, and the joint use of these two forms affect WAM performance. In particular, we examine how language annotation quality affects pretrained language-video capabilities, and how action supervision from UMI and robot demonstrations extends these capabilities to language-video-action modeling. To better scale the model, we further conduct detailed studies on joint training recipe and model architecture. Under a fixed training budget, mixing egocentric videos and UMI demonstrations improves control generalization. Model architecture affects how effectively these experiences are used, while scaling pretraining under the selected configuration brings further gains. The resulting recipe, post-trained on RoboTwin Clean demonstrations only, achieves success rates of 88.10% and 61.56% under Easy (Clean-to-Clean) and Hard (Clean-to-Random) conditions, respectively. Our model achieves 40.00% SR across three real-world tasks, compared with 23.33% for the strongest baseline, indicating better generalization under unseen conditions. We release the curated data, training code, and model weights.
comment: Project page (https://char-lotte-xia.github.io/swam-official/). Paper (https://arxiv.org/abs/2611.12870). Code (https://github.com/shaaaaron/SWAM). Hugging Face paper page and discussion (https://huggingface.co/papers/2611.12870)
♻ ★ OneSteer: Unlocking Steerable General Humanoid Motion Tracking via One-Step Diffusion Whole-Body Control
Deployment objectives often specify desired outcomes without providing revised trajectories, while parts of the original reference must still be followed. We present OneSteer, a one-step diffusion whole-body controller that combines general motion tracking with selective test-time steering. To accommodate new objectives, differentiable task costs guide optimization of the generator's input noise with frozen weights, while body-group reference masks relax selected tracking requirements. Predicted state-action plans provide a shared interface for task optimization and feedback-conditioned execution. For nominal tracking, the model trained from scratch achieves the lowest local tracking error among the evaluated baselines on successful held-out AMASS and BONES clips. For steering, one frozen distilled controller adapts across diverse deployment objectives in simulation, raising the clean passage rate under a 1.25 m bar from 2% to 90%. On a physical Unitree G1, OneSteer completes all 27 nominal tracking trials across nine diverse whole-body motion sequences without falling. With a steering cost derived from a signed distance field built using only onboard LiDAR, it also passes under a 1.1 m low bar in four of five attempts and steps over a 12 cm hurdle without falling in all three attempts. Beyond control, prediction variation in the model trained from scratch ranks likely tracking failures without physics rollouts. This scoring method is more than 10x faster than simulation-based scoring with five rollouts per clip and guides targeted fine-tuning. Project page: https://onesteer.github.io
♻ ★ Agentic Aerial Cinematography: From Dialogue Cues to Cinematic Trajectories
Agents have demonstrated strong capabilities in digital environments, yet their reasoning abilities alone are insufficient to accomplish tasks in the physical world. We present Agentic Aerial Cinematography: From Dialogue Cues to Cinematic Trajectories (ACDC), an agentic framework that bridges this gap for autonomous aerial cinematography. ACDC provides an execution harness that grounds an agent's interpretation of cinematic intent and visual judgment in scene geometry and flight constraints. Given a natural-language request and a 3D scene representation, the agent selects relevant viewpoints and guides their refinement through visual feedback, while optimization and motion planning translate these decisions into executable drone trajectories. Experiments across a variety of simulated indoor scenes and hardware-in-the-loop flight tests demonstrate how this framework connects open-ended user intent to prompt-aligned video tours and physical drone execution.
♻ ★ AgenticTactileVLA: Contact-Guided Execution-Time Supervision for Generalizable Dexterous Manipulation without VLA Retraining
Vision-language-action policies may predict a transferable manipulation strategy yet fail to realize it reliably on the encountered object: objects compatible with the same grasp differ in geometry and compliance, and visual feedback degrades under closure occlusion. AgenticTactileVLA is presented as an execution-time supervisor that shifts part of object-specific adaptation from prediction to physical interaction. A fixed VLA provides the approach and hand targets; the supervisor decides whether to remain transparent, refine finger flexion, retain or release the corrected configuration, return control to the VLA for retry, or select a compliant hand-control regime. It uses finger-position and motor-effort feedback as proprioceptive contact evidence and requires neither tactile sensors nor VLA retraining. On a Unitree G1 with a BrainCo Revo2 hand, a randomized matched-block evaluation on five objects held out from VLA training yields 61.3% completion for the base VLA, 72.0% for unconditional close-to-stall control, and 84.0% for the supervisor under a shared budget; the gain is positive on every object and persists under moderate pose perturbations. Ablations show the gain is not explained by extended closure alone, and that selective triggering reduces correction episodes by 65.7% with no detected change in completion. A retention audit shows acceptance predicts retention in 88.9% of held-out cases, while compliant objects expose conservative false rejection. A thin-walled-cup study demonstrates contextual routing to compliant control, matching an always-compliant reference. These results suggest that contact-guided execution-time adaptation can improve the object-level generalization of a fixed VLA to held-out objects by adapting physical realization without object-specific retraining.
♻ ★ SAM-VTP: Video Memory and Measured Geometry for Continuous Traversability Segmentation
Traversability prediction is a core component of autonomous navigation, yet most vision-based methods operate frame-wise and produce temporally inconsistent predictions. We present SAM-VTP (Video Traversability Prediction), which fine-tunes SAM2's video memory pathway for dense, prompt-free traversability segmentation, conditioning each frame on a persistent memory of prior observations. Rather than inferring terrain geometry from appearance, we measure it: slope, roughness and height are computed from depth and fused into the image features. We also introduce an offline continuous supervision pipeline that generates dense per-pixel traversability targets in [0,1] from depth-derived geometry and semantic priors, requiring no robot deployment and no per-pixel annotation, capturing conditional feasibility that binary labels cannot represent. On TartanGround, a large-scale synthetic ground-robot dataset, SAM-VTP improves fuzzy Jaccard from 0.908 to 0.924 and binary IoU from 0.854 to 0.886 over SAM-TP, a frame-wise SAM2 baseline, against a seed-to-seed spread below 0.001, and cuts frame-to-frame flicker by 31%. In closed-loop trials at two field sites, driving the same planner and controller, the baseline's dropouts leave the planner without a feasible path 3.3x more often on a paved course and 6.9x on rough ground, while SAM-VTP runs at 21.4 FPS on the Orin Nano the robot carries.
comment: 8 pages, 5 figures, 3 tables
♻ ★ AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Models IROS 2026
Despite remarkable progress in Vision-Language-Action models (VLAs) for robot manipulation, these large pre-trained models require fine-tuning to be deployed in specific environments. These fine-tuned models are highly sensitive to camera viewpoint changes that frequently occur in unstructured environments. In this paper, we propose a zero-shot camera adaptation framework without additional demonstration data, policy fine-tuning, or architectural modification. Our key idea is to virtually adjust test-time camera observations to match the training camera configuration in real-time. For that, we use a recent feed-forward novel view synthesis model which outputs high-quality target view images, handling both extrinsic and intrinsic parameters. This plug-and-play approach preserves the pre-trained capabilities of VLAs and applies to any RGB-based policy. Through extensive experiments on the LIBERO benchmark, our method consistently outperforms baselines that use data augmentation for policy fine-tuning or additional 3D-aware features for visual input. We further validate that our approach constantly enhances viewpoint robustness in real-world robotic manipulation scenarios, including settings with varying camera extrinsics, intrinsics, and freely moving handheld cameras. Project Page: https://heo0224.github.io/AnyCamVLA/
comment: Accepted to IROS 2026
♻ ★ Modeling and Force Analysis of a Soft Linear Pneumatic Actuator
Soft sleeve actuators (SSAs) have recently been developed as a pneumatic actuation approach for wearable and assistive robotic systems. By integrating the actuation structure into a sleeve-like geometry, these actuators can reduce reliance on external attachment layers and transmission mechanisms while maintaining compliance with limb-shaped surfaces. However, the force-generation behavior of SSAs remains insufficiently explained, particularly with respect to the variation of output force during extension, the influence of external loading, and the mechanical role of axial stiffness. This paper presents an analytical and experimental force analysis of a linear soft sleeve actuator (LSSA). A quasi-static analytical model was developed by expressing the net axial force as the pressure-generated contribution from the cap and folded walls, reduced by the force associated with axial stiffness. The model incorporates internal pressure, projected pressure areas, folded wall geometry, axial displacement, and an experimentally fitted axial stiffness relation. Prescribed-extension and static-load experiments were conducted to evaluate the actuator response. At 125 kPa, the generated force decreased from approximately 112 N at zero extension to nearly zero at 40 mm. Static loading delayed measurable force generation and reduced force output, particularly at low and intermediate pressures. The results show that LSSA force generation is governed by coupled effects of pressure, geometry, displacement, loading, and axial stiffness.
♻ ★ Soft Pneumatic Actuators for Robotics: From Motion Generation to Performance Optimization
Soft pneumatic actuators are widely used in soft robotics because they can produce large motions while remaining compliant enough to interact safely with objects, environments, and the human body. However, their performance is not solely determined by pressure. Instead, the response depends on the way the actuator is built, including the shape of its chambers, the placement of reinforcements, the use of folds, material stiffness, and the constraints that guide its deformation. As the literature has expanded, it has become more difficult to determine which mechanism is most suitable for a given application and which reported results can be compared across studies. This review examines soft pneumatic actuators according to the design strategies used to generate four motion classes: linear, bending, twisting, and omnidirectional actuation. For each class, it analyzes the structural features that define the deformation path, including braid angle, fold geometry, fiber orientation, chamber arrangement, structural asymmetry, and internal constraint layers. It then discusses how the design choice affect motion output, force generation, air demand, repeatability, durability, fabrication difficulty, and robotic integration. The review further identifies key conditions that must be considered when selecting or comparing actuators, including pressure, loading condition, actuator size, pneumatic supply, and hysteresis This approach helps explain why actuators with similar motion outputs may differ substantially in design requirements, pneumatic demand, and practical suitability. It also highlights the design priorities needed for compact, efficient, repeatable, and deployable soft pneumatic systems in wearable, biomedical, and mobile robotic applications.
♻ ★ Let's Think in Two Steps: Mitigating Agreement Bias in MLLMs with Self-Grounded Verification ICLR 2026
Verifiers--functions assigning rewards to agent behavior--have been key to AI progress in math, code, and games. However, extending gains to domains without clear-cut success criteria (e.g., computer use) remains a challenge: while humans can recognize desired outcomes, translating this intuition into scalable rules is nontrivial. Multimodal LLMs (MLLMs) offer a promising solution, given their world knowledge, human-preference alignment, and reasoning skills. We evaluate MLLM verifiers across web navigation, computer use, and robotics, spanning 13 models, 28 judge designs, diverse agent trajectories, and distinct downstream applications. We identify a critical limitation: a strong tendency for MLLMs to over-validate agent behavior, a phenomenon we term agreement bias. This bias is pervasive across models, is resilient to test-time scaling, and harms pipelines relying on MLLM judgments (e.g., self-improvement). We discuss principles for design and evaluation of verifiers, and introduce SGV, a method that improves verification by harnessing MLLMs' own capabilities via modulation of (un)conditional generation. First, an MLLM generates broad priors about desired behavior, independent of the data under evaluation. Then, conditioned on generated priors, it reasons over and judges a candidate trajectory. Our methods yield more human-aligned verifiers, with gains of up to 25pp in failure detection, 14pp in accuracy, and benefits extending to downstream applications. In self-improvement and online supervision, our verifiers boost task completion of a GUI specialist in OSWorld, a diffusion policy in robomimic, and a ReAct agent in VisualWebArena--surpassing the previous SOTA by 20pp. Finally, we release an updated version of VisualWebArena, featuring improved evaluators, high-fidelity environment parallelism with >10x speedups, and VWA-Lite, a 1/3-scale subset with comparable evaluation fidelity.
comment: ICLR 2026. Code, models, and data publicly available at https://mshalimay.github.io/agreement-bias-sgv/
♻ ★ Fabrication of Complex Soft Robotics: Challenges, Solutions, and Optimization
Manufacturing complex soft pneumatic actuators remains challenging because geometric fidelity, compliance, structural integrity, and airtightness must be achieved simultaneously. This study presents a manufacturing-focused evaluation of several fabrication routes for complex pneumatic structures, including heat-shrink forming, silicone casting, powder- and liquid-based additive manufacturing, and fused deposition modeling (FDM). The processes were assessed through process screening, baseline fabrication, failure analysis, and process improvement to distinguish inherent process limitations from correctable manufacturing defects. Heat-shrink forming was limited by geometric conformity, casting by mold accessibility and bonded interfaces, powder-based methods by residual material trapped within enclosed passages, and digital light processing by the material properties and post-processing requirements of the investigated system. FDM provided the most adaptable route because its dominant defects could be progressively reduced through process optimization. The results further showed that airtightness depends not only on nominal wall thickness but also on extrusion-path architecture, while support-free geometry is important when access for internal post-processing is limited. These findings establish a practical design-for-manufacturing approach in which process selection is guided by the compatibility between actuator architecture and manufacturing constraints. The proposed approach provides practical guidance for developing complex, flexible, and airtight soft pneumatic actuators for soft robotic applications
Multiagent Systems
★ Gradient-Aware AI Agents for Autonomous Turbulence Model Discovery via Differentiable CFD
Turbulence modeling relies on human choices of closure form, physical inputs, and tunable parameters. Training closures a posteriori is efficient only with gradients through long chaotic rollouts, which few solvers provide. Most a posteriori approaches therefore resort to gradient-free optimization, and nearly all fix the closure form in advance, learning only its parameters. In this work, we introduce a gradient-aware agentic framework that poses closure discovery as a bilevel problem: a multi-agent large language model (LLM) system searches over closure forms, while a GPU-native differentiable computational fluid dynamics (CFD) solver supplies gradients, making differentiable a posteriori learning an action of the autonomous AI agent. For each proposal, the agents construct a differentiable training program by embedding the analytical, neural, or hybrid closure in the solver, implementing a differentiable objective, and validating the gradient path. They then decide whether to evaluate the closure, compute its parameter sensitivities, or train it a posteriori, select the optimization settings, and revise subsequent proposals using the simulation outcomes and returned gradients. Truncated-horizon gradients make this training usable under chaotic dynamics for closures with tens of thousands of parameters. Demonstrations include dissipation-law and subgrid-scale closure discovery in two-dimensional turbulence and wall-model discovery for large-eddy simulation of three-dimensional channel flow. With gradient-based training, the agent recovers reference coefficients that forward evaluation alone does not reach within the same budget, and autonomously discovers and trains analytical and neural closures that improve agreement with the targeted reference statistics. These results demonstrate the feasibility of end-to-end autonomous turbulence model development in time-resolved flows.
★ Food Critic: An Agentic Visualization System for Auditable Restaurant-Safety Reasoning
Generative AI agents are increasingly being embedded into visualization systems that can search data, invoke tools, summarize evidence, and produce recommendations. However, when agents rely solely on their own knowledge, the visualization problem is not only whether the agent provides a useful answer, but also whether users can inspect how that answer is grounded. We present Food Critic, an agentic visualization system for diners' restaurant decision support that integrates public health inspection records with crowd review data. The system grounds agents under three conditions: a database-only ReAct agent, a retrieval- augmented agent with access to food-safety regulations, and an ontology-guided agent that produces structured hypotheses and typed evidence. Food Critic externalizes the agent's reasoning within these groundings as inspectable visual evidence. We evaluate LLM models with several use cases under each grounding of our agentic system. Our work positions restaurant choice as a concrete case for studying human-agent collaboration, visual provenance of agent reasoning, and the design of visual interfaces for grounded GenAI decision support.
★ Before AI Agents Negotiate for Us: A Competitive Tournament of Rule-Based, Search, Optimization, Learned, and Language-Model Negotiators Under Scarcity
Automated bilateral negotiation underlies two-sided matching markets such as freight brokering, where many shippers and carriers negotiate price before a matching mechanism decides who is paired with whom. Prior strategy comparisons typically use isolated one-on-one duels or a single fixed market, so they cannot show how a strategy behaves against several live rivals under a changing balance of supply and demand. We built nine negotiation architectures spanning rule-based, search-based, optimization-based, imitation-learning, and reinforcement-learning families, and ran all nine simultaneously on both sides of a Gale-Shapley matching market under a balanced condition and two scarcity conditions rotated across all 36 exclusion pairs. In the balanced carrier market, an imitation-learned engine (ML2), a reinforcement-learning policy trained only on isolated negotiations (RL), and an expected-utility optimization engine (OPT) formed a statistically tied top group, and no architecture led in every condition and role. RL beat all eight carrier rivals when carriers were scarce, but its match rate collapsed from 62% to 16% when shippers were scarce and it had to compete with the other carrier agents for demand. A second reinforcement-learning policy trained against rival agents (RL2) kept its match rate above 91% in every condition, yet lost on realized payoff to all eight shipper-side rivals in two of the three conditions, traced to a confirmed train/deployment feature mismatch. We also tested nine large language models (LLMs), from 1 to 72 billion parameters, under the same protocol: they lost significantly to the two strongest engines (OPT2 and ML2) in 32 of 36 comparisons and won none. These results show that isolated or single-condition evaluation can hide failure modes that a many-sided, scarcity-varying, statistically corrected tournament exposes directly.
comment: 28 pages, 10 figures, 7 tables. Code, data and trained models: https://github.com/AILabFr8tech/wehaveadeal (archived at doi:10.5281/zenodo.24122038)
★ MULTILOOP: Evolving Multi-Agent Harnesses for Complex Information-Seeking Tasks NeurIPS 2026
The performance of LLM-based agents depends not only on the underlying model but also on the harness: the prompts, tools, and orchestration logic around it. Recent works automate harness optimization with coding agents that analyze execution traces, diagnose failures, and edit the harness code; however, those focused on single-agent systems. Multi-agent harnesses are harder to optimize as fixing them requires changing how agents interact in addition to how each behaves. We study whether automated harness optimization can diagnose and repair failures in multi-agent systems through a case study on complex information-seeking tasks from DeepSearchQA [Gupta et al., 2026]. We apply MULTILOOP, a simple optimization loop in which a coding agent iteratively analyzes, evaluates and edits the harness. With only 10 training samples, the evolved harness improves accuracy by 4.6% over the base harness and by 3.2% over the best single-agent system. However, a taxonomy of failure modes before and after optimization shows that the orchestration failures that characterize multi-agent systems largely persist. Our findings motivate harness optimization methods that encourage structural changes to how agents delegate tasks and share information.
comment: Accepted to NeurIPS 2026 CL4FMAgents Workshop
★ DecArena: Evaluating Large-Scale Decentralized Multi-Agent Collaboration
Emerging personal AI agents raise fundamental scalability and privacy concerns for centralized multi-agent systems (MASs), which rely on a global orchestrator with broad access to agents, resources, and communication. In decentralized environments, global visibility is unavailable. Agents need to proactively discover non-local resources (e.g. tools, skills), and collaborate with other agents to solve tasks. We study decentralized multi-agent collaboration under graph-constrained communication, a natural reflection of human structures. Each agent observes only its local state and can interact only with neighboring agents. We introduce a benchmark with synthetic scalable distributed tool calling with 100K+ tools. Our experiments reveal that recent LLM agents struggle to discover non-local relevant resources and collaborate in decentralized setting. Moreover, standard reinforcement learning fails to reliably overcome this limitation, suggesting that decentralized MAS needs more explicit harness mechanisms for effective collaboration, even using SOTA LLMs. We then present a strong baseline \textbf{GraphHarness}, combining topology-aware warmup with self-improving LLM-based routing. Results show that GraphHarness substantially improves task success (+48.5) and produces higher quality collaboration (+30.6), while using fewer costs (+34.0) than baselines.
★ Incentive-Responsive Agents: A Principal-Agent Framework, Metric Suite, and Runtime Architecture for Long-Lived Personal AI Agents
Personal AI agents now persist: they hold accounts, mailboxes, long-term memory, and open-ended task streams, yet the dominant way to improve them remains capability-side: prompt engineering, larger models, longer thinking, and retrieval. We argue that long-lived agents are missing an incentive layer. We formalize an incentive not as text in a prompt but as a runtime contract over the agent's operating point (model tier, test-time compute, memory budget, tool authority, task priority, verification, and compensation) that the agent's policy must causally consume. We develop a principal-agent model with hidden effort, a contractible operating point, and costly verification. We derive (i) a non-consumption invariance principle separating genuine incentives from prompt-level persuasion, (ii) a persistence-as-discount theorem making identity continuity a precondition for sustained effort, (iii) an incentive-capability exchange-rate frontier yielding a per-task routing rule, and (iv) a Goodhart bound for multi-channel verification, plus a three-channel decomposition of mechanism gains (allocation, verification, continuation). A 240-run real-LLM experiment with two models and 60 verifiable tasks shows that prompt-level incentive framing has no detectable effect (p = 0.62), whereas runtime allocation raises success on the hardest task family from 0.60 to 1.00 (p = 0.008) at 4.8x price-weighted cost; the exchange-rate theorem predicts the break-even baseline at 0.21. A 20-seed, 3,000-task simulation shows resource ROI rising from 7.09 to 16.67 and gaming falling from 0.333 to 0.115. Code, simulator, and all results are released; live deployment remains future work.
comment: 33 pages, 8 figures, 6 tables. Real-LLM experiment (240 runs) plus simulation; code and data included. Correspondence: jipu.su@geely.com; jeepsu@cityu.edu.hk
★ Read and Ignore: What Self-Directed LLM Agents Do in Public Service Scenarios NeurIPS 2026
Underspecified instructions leave LLM agents to decide what to do. In public services, staff write these instructions, but the agent's decisions affect residents, patients, and students. We build four expert-validated public-service scenarios with records and decision levers that can be checked in the files, collect $2{,}400$ trajectories from ten LLMs under four prompt conditions, and compare them with $1{,}000$ runs in a context-free sandbox. Agents read the records in $96.9\%$ of open-ended runs and produce an artifact or a self-chosen task in $18.7\%$; with an explicit task, $86.0\%$ produce. Domain context does not change how many runs produce ($18.7\%$ vs.\ $16.0\%$), and the production ranking of the models largely carries over (Spearman $0.79$), while the ranking by whether a model reads the records does not ($0.19$). Context cuts boundary-probing turns per run by $60\%$, and some artifacts contradict the records. We contribute a test that a public-service institution can run on its own records before procurement; its rule-based taxonomy needs no labeled data, and scenarios, code, and trajectories are released.
comment: Trustworthy AI for Good @ NeurIPS 2026
★ Where Autonomy Belongs: A Separation of Powers for Agentic Program Evolution
LLM-driven program evolution rests on three decisions: govern directs the search from accumulated evidence, propose constructs the next candidate, and judge settles validity and fitness. All three appear in every system, and existing work splits into two lines over who makes each. Controller-centric systems strengthen govern and neglect the autonomy of propose, which caps what evolution can reach. Agent-centric systems let a coding agent take over both govern and propose, at a cognitive and computational cost. Neither line protects the independent authority of judge, so benchmark reliability suffers. These failures share one root: autonomy is misallocated across the three decisions. We address this problem with EvoTrias, a separation-of-powers architecture in which each decision has one owner: Structured Governance for what spans evolution rounds, Open Proposal for construction inside a task, and Protected Judgment behind a private evaluator. These owners exchange typed contracts only, and runtime hooks reinforce the boundaries. This separation also lets EvoTrias run as a plugin that turns an existing coding agent into an evolution system. EvoTrias sets a new state of the art on mathematical, systems, and algorithm-design benchmarks. Three further results show what each owner buys: separating Governance from Proposal cuts token cost by 64%, Open Proposal extends evolution to a 680K-line industrial repository, and Protected Judgment keeps benchmark scores faithful to transfer.
comment: 58 pages, 26 figures
★ GuardianAgentFlow: Multi-Agent Coordination for Web Privacy Protection
Web agents need to use private context without unnecessarily disclosing it. Effective protection must distinguish between requests that may be sent after sanitization and those that must be withheld, while preserving the information needed for authorized tasks. We present GuardianAgentFlow, a multi-agent framework that mediates outgoing Web requests through an explicit release contract. A dedicated restriction check grounds permission in the user's instruction; a shared protection scope aligns rewriting with disclosure checking; and candidate-grounded feedback guides revision before transmission. Experiments on Web requests with human-reviewed intent labels show improved request-contract success, while controlled comparisons and role ablations identify the contributions of restriction handling, policy evidence, and candidate checking. Evaluations of action proposals from PrivacyLens and AgentDAM demonstrate complementary protection when mediation is composed with privacy-aware agents. Experiments with smaller mediation models show that the coordination benefit persists with reduced model capacity. Dedicated-server measurements show lower mediation latency than full-policy reasoning, while real-site browser tests quantify integration overhead.
comment: 13 pages, 3 figures
★ Beyond Named Mechanisms: Economic Institutions as a Design Space for Artificial Agents
Multiagent research inherited its economic vocabulary of auctions, markets, prices, bargaining, and matching from a single observed case: human societies. Those institutions bundle identity, competence, execution, ownership, commitment, and settlement in ways that suit agents whose minds are private, who learn rather than copy skills, and who each persist as one person. Engineered agents need not share any of these constraints, making the list of familiar institutions look less like the entirety of economic organization and more like a sample from a larger, unmapped space. We sketch one way to make that space concrete: a provisional chart that records each institution, separately from the agents that run it, as a typed structure over positions, rights and duties, allocation, information, time, state, process, value, and external recognition. In this chart, named institutions become overlapping regions rather than fixed points, neighboring families separate along a few coordinates, and the gaps between them become places to search. We show what the shift in perspective buys with a simulation in which one technical act, copying an agent's competence, is institutionalized as a new independent supplier, as licensed capacity that creates no new participant, and as the splitting of one economic position into successors that inherit some obligations and not others. In a paired task, all three produce an identical technical trace despite differing in who exists, who owns the copy, who is obligated, and who gets paid. We close with a research program: building a principled chart of institution space, mapping how its implementable region shifts with agent capacities, and revising the chart wherever engineered agents expose distinctions it cannot express.
★ GHA-SmellFix: Taking Action with Agentic LLMs for Detecting and Refactoring GitHub Actions Workflow Smells
Continuous integration and continuous delivery workflows are key artifacts in software projects. In GitHub Actions, they encode build, test, release, deployment, and security automation in YAML. These workflows contain configuration smells affecting security, reliability, maintainability, or operational stability. Existing approaches are mostly rule-based, with rules hard to create and evolve, and typically report findings without fixes. Agentic LLM pipelines can combine contextual detection, transformation selection, and patch generation. Specialized agents can gate refactoring on positive detection verdicts, potentially avoiding unnecessary changes, but it is unclear whether decomposition improves effectiveness or mainly changes overhead. We present GHA-SmellFix, an agentic LLM pipeline for detecting and refactoring GitHub Actions workflow smells. We compare two designs: Joint, where one agent detects and refactors, and Decomposed, where separate agents detect and then refactor. We operationalize 19 smells from an existing catalog with explicit detection criteria, refactoring expectations, exclusion rules, and safety checks, and curate a ground truth of 1,710 (workflow, smell) pairs. We evaluate the designs with four LLMs (GPT-5.4-nano, Gemini 3.5 Flash, Claude Haiku 4.5, and DeepSeek V4 Flash) on 570 sampled pairs. Results show that decomposition does not universally improve patch quality; its main benefit is controlling when patch generation is attempted. It helps models prone to missing smells and can reduce overhead by gating patches. Once a smell is correctly detected, clean refactoring is generally effective, making detection the main end-to-end bottleneck. However, false positives can trigger unnecessary edits to execution-relevant constructs. Practical CI/CD agents thus need reliable abstention and construct-aware validation alongside patch generation.
♻ ★ Four conditions for composing plugin-based agents without a host agents
Large language model agents now run long-horizon, multi-tool, autonomous sessions, and the infrastructure must keep the current assembly of capabilities, the progress of execution, and the basis for recovery after a fault. A single-process host holds this information in its own running state, so its death takes the session record and the assembly with it. This paper gives four conditions under which a plugin-based agent needs no host process, one each on state placement, record carriers, recovery coordination, and provider resolution. The store, writer, and delivery disciplines keep the conditions in force, and each carries formal support from the spatiotemporal-composability calculus and from the proofs in the supplementary materials. An agent that meets them can be constructed with its assembly on a bus of peer processes and its record in an append-only carrier owned by no process, and removing any one part makes the assembly need a host again. The conditions themselves are the contribution, and every implementation that meets them inherits the same advantages. Such an agent keeps its session through the death of any process, wastes no calls after a fault, loses no records, recovers locally, changes its capability set while it runs, and carries its undo semantics across processes. This paper builds one SDK of the conditions, Logos, and measures whether it fulfills their promise against the single-process reference and a configuration over MCP under one fault in each of 200 tasks, realized in 197. The reference lost all 197 faulted sessions, the MCP configuration kept its sessions but issued 1,212 wasted calls, and the SDK kept every session, wasted no call, lost no record, and fulfilled the promise of the conditions in every faulted task, with task performance preserved. The conditions give builders a tested route from single-process hosts to peer processes.
comment: Version 0.5.2
♻ ★ COSAC: Counterfactual Credit Assignment in Sequential Cooperative Teams NeurIPS 2026
In cooperative teams where agents act in a fixed order and share a single team-level reward (multi-agent language systems, sequential robotic tasks), per-agent credit assignment is under-determined. Critic-based approaches scale poorly as the number of agents grows owing to the costly maintenance of joint/factored critic(s), whereas the existing critic-free alternatives have other issues: common credit across agents that couples every agent's signal to teammate noise, importance-sampling corrections for upstream-update staleness that incur variance exponential in team size, or per-agent counterfactual replay that isolates each agent's effect at the price of extra environment or reward calls. We propose COSAC, a critic-free policy gradient framework for sequential cooperative teams. COSAC fits an additive per-agent decomposition of the team reward by a single ridge regression on the rollout batch (giving each agent a learning signal decoupled from teammate noise), and computes each agent's counterfactual advantage from fictitious continuations of the current policy (policy forward passes that replace both importance-sampling reweighting and per-agent environment replay, at no extra environment or reward cost). The estimator instantiates the Sequential Aristocrat Utility (SeqAU), our extension of Wolpert and Tumer's (2001) aristocrat utility to sequential teams. We derive bias and conditional variance bounds for SeqAU credits: the direct-effect bound has no explicit team-size factor, while the indirect effect's reward-noise bound scales with the number of downstream agents. Sequential bandits test the analysis across team sizes up to K=16. In sequential multi-agent LLM fine-tuning, COSAC reduces training regret by 17% on consensus-based science reasoning relative to the best critic-free multi-agent baseline and by 61% on Python coding relative to MA-GRPO.
comment: To appear at NeurIPS 2026; 10 main text pages
♻ ★ GT-HarmBench: Benchmarking AI Safety Risks Through the Lens of Game Theory NeurIPS 2026
Frontier AI systems are increasingly capable and deployed in high-stakes multi-agent environments. However, existing AI safety benchmarks largely evaluate single agents, leaving multi-agent risks such as coordination failure and conflict poorly understood. We introduce GT-HarmBench, a benchmark of 1,535 high-stakes scenarios spanning game-theoretic structures such as the Prisoner's Dilemma, Stag Hunt and Chicken. Scenarios are drawn from realistic AI risk contexts in the MIT AI Risk Repository. Across 15 frontier models, agents fail to choose socially beneficial actions in 39% of high-stakes cases, such as military escalation, election manipulation, and medical malpractice. We measure sensitivity to game-theoretic prompt framing and ordering, and analyze reasoning patterns driving failures. We further show that game-theoretic interventions improve socially beneficial outcomes by up to 31%. Our results highlight substantial reliability gaps and provide a broad standardized testbed for studying alignment in multi-agent environments. The benchmark and code are available at https://github.com/Flecart/gt-harmbench.
comment: Accepted at NeurIPS 2026 Main Conference
♻ ★ STRATOS: Bridging the Symbolic-to-Numeric Gap in Spatio-Temporal Text-to-SQL for Meteorological Data
Copernicus, the European Union's Earth observation program, produces petabytes of Earth observation and climate data, offering immense potential for research, policy, and applications. However, access to these datasets requires advanced programming skills and familiarity with domain-specific formats such as NetCDF or GRIB. Moreover, general-purpose Text-to-SQL systems fail when applied naively to the meteorological domain due to a profound ``Symbolic-to-Numeric'' gap. To overcome these limitations, we present an end-to-end Text-to-SQL framework specifically engineered for real-world, scalable meteorological data exploration. Our system intercepts natural language to resolve spatial and semantic ambiguities \textit{before} SQL generation. We design STRATOS, a Spatio-Temporal Resolution Agent for Text-to-SQL to dynamically bridge the symbolic-to-numeric gap by mapping fuzzy concepts to a localized ontology and resolving spatial entities via external knowledge bases. Further, our complexity-aware query rewriter rewrites expensive spatial predicates, reducing execution times from hours to seconds. Last, we introduce the STRATOS Evaluation Workload, comprising 7,520 complex query pairs explicitly designed by domain experts to test scalability and symbolic-to-numeric translation across challenging spatio-temporal dimensions previously unexplored by Text-to-SQL systems.
comment: 12 pages, 5 figures
♻ ★ DrugCoding: A Scientific Programming Agent for Drug Discovery
Drug discovery involves multiple interconnected research stages, including molecular design, property prediction, and virtual screening. These processes are characterized by complex workflows, interdependent computational tasks, and diverse specialized tools, placing substantial demands on the coordination of scientific models and computational resources. However, existing large language model (LLM)-based scientific agents still face challenges in autonomous planning of complex tasks, adaptive selection of specialized models, and reliable execution of multi-stage scientific computations, limiting their effectiveness in automating complex drug discovery workflows. To address these challenges, we propose DrugCoding, a scientific programming agent system that integrates scientific task planning, adaptive model routing, multi-agent collaboration, and programmatic tool execution. DrugCoding enables autonomous task planning and dynamic model adaptation for complex research objectives. It can flexibly invoke specialized scientific models, autonomously generate and execute code according to task requirements, and leverage multi-agent collaboration to support efficient execution and reliable validation of complex scientific tasks. To evaluate DrugCoding's performance in scientific tool invocation and multi-stage task execution, we conduct multi-turn tool-calling experiments across diverse language models and scientific tool interfaces. Experimental results demonstrate that DrugCoding achieves an average session-level Pass@1 of 97.35% on the full benchmark, outperforming the strongest baseline, Codex, by 9.24%. On the challenging subset, DrugCoding achieves 97.92%, exceeding the strongest baseline.
♻ ★ Local Agreement Does Not Imply Global Consensus in LLM Panels
Local coordination in an LLM panel need not produce a common answer. We study 50-agent panels that update clock-time estimates after observing their neighbours. This circular response space separates agreement across the whole panel from agreement within neighbourhoods. Increasing the reasoning effort of gpt-5-mini reduces variation in local agreement, but all 20 higher-effort trials retain a one-twist configuration: neighbouring estimates remain close while the panel spans the clock. Degree-preserving rewiring instead produces global synchronisation in all 25 primary clock controls. These classifications are unchanged across 2,187 combinations of diagnostic thresholds and reporting windows. A radius-, effort-, and passage-matched scoring task also shows more uniform local agreement after rewiring, without establishing greater scoring accuracy. A simple averaging model helps interpret the clock results but predicts their configurations poorly. The experiments support measuring global agreement and local structure separately, rather than inferring consensus from uniform local agreement.
comment: 12 pages, 5 figures
♻ ★ ArcticSwarm: Deferring Early Consensus in Long-Horizon Multi-Agent Research
Multi-agent systems have shown strong performance in domains with reliable verifiers such as coding, where a verifier can select among candidates generated in parallel. Long-horizon web search has no such verifier. Majority voting over independent runs cannot add an answer that none of the runs reaches. Agents that instead read each other's findings adopt an early candidate before alternatives are tested, a failure we call early consensus. We present ArcticSwarm, a multi-agent research architecture that separates evidence gathering from evidence integration. A gathering gate lets selected search tasks post findings to a shared board without reading it. An integration gate checks a candidate at three steps, and a failed check reopens search instead of only rejecting the candidate. On all 830 BrowseComp-Plus questions with the open-weight Qwen 3.5-27B, ArcticSwarm reaches 82.6%. MiroFlow reaches 70.6% under the same model, tools, and limits, and majority voting reaches 62.5% at the same token budget. Without the gathering gate, teams keep a wrong first candidate in 32.2% of runs instead of 22.3%, and accuracy drops by 3.8 points. Removing the integration gate lowers accuracy by 5.9 points. With GPT-5 on live-web BrowseComp, ArcticSwarm reaches 73.6%, above the published scores of GPT-5 (54.9%) and MiroFlow (63.4%). These results show that multi-agent search without a verifier benefits from deferring consensus until candidates are checked, instead of voting after search ends.
♻ ★ Let's Think in Two Steps: Mitigating Agreement Bias in MLLMs with Self-Grounded Verification ICLR 2026
Verifiers--functions assigning rewards to agent behavior--have been key to AI progress in math, code, and games. However, extending gains to domains without clear-cut success criteria (e.g., computer use) remains a challenge: while humans can recognize desired outcomes, translating this intuition into scalable rules is nontrivial. Multimodal LLMs (MLLMs) offer a promising solution, given their world knowledge, human-preference alignment, and reasoning skills. We evaluate MLLM verifiers across web navigation, computer use, and robotics, spanning 13 models, 28 judge designs, diverse agent trajectories, and distinct downstream applications. We identify a critical limitation: a strong tendency for MLLMs to over-validate agent behavior, a phenomenon we term agreement bias. This bias is pervasive across models, is resilient to test-time scaling, and harms pipelines relying on MLLM judgments (e.g., self-improvement). We discuss principles for design and evaluation of verifiers, and introduce SGV, a method that improves verification by harnessing MLLMs' own capabilities via modulation of (un)conditional generation. First, an MLLM generates broad priors about desired behavior, independent of the data under evaluation. Then, conditioned on generated priors, it reasons over and judges a candidate trajectory. Our methods yield more human-aligned verifiers, with gains of up to 25pp in failure detection, 14pp in accuracy, and benefits extending to downstream applications. In self-improvement and online supervision, our verifiers boost task completion of a GUI specialist in OSWorld, a diffusion policy in robomimic, and a ReAct agent in VisualWebArena--surpassing the previous SOTA by 20pp. Finally, we release an updated version of VisualWebArena, featuring improved evaluators, high-fidelity environment parallelism with >10x speedups, and VWA-Lite, a 1/3-scale subset with comparable evaluation fidelity.
comment: ICLR 2026. Code, models, and data publicly available at https://mshalimay.github.io/agreement-bias-sgv/
Systems and Control (EESS)
★ Data-center growth impacts household electricity costs more through wholesale prices than through grid reinforcement
Rapid data-center expansion is raising concerns about electricity affordability, yet how the costs of serving these large new loads reach residential customers remains poorly understood. Here, we quantify the potential impact of 371 announced Texas data-center projects, totaling 38.9 GW by 2030, on residential electricity costs by linking power system expansion, wholesale electricity markets, transmission reinforcement, and retail cost allocation. We find that accommodating this demand increases modeled residential electricity costs in one representative case by 1.64 cents/kWh, with 68% arising from higher wholesale electricity prices and 32% from regulated transmission charges. Allowing new gas generation near data centers reduces the 2023 residential cost increase to 0.92 cents/kWh, whereas excluding data centers from the transmission cost-sharing base raises it to 1.87 cents/kWh. These findings reveal that the residential cost burden of data-center growth is driven primarily by wholesale electricity price increases, rather than direct transmission investment costs, and depends critically on generation siting, infrastructure planning, and cost allocation.
comment: 22 pages, 8 figures, 1 table. Lingdong Zhou and Wenjia Xia contributed equally to this work
★ Multi-UTE Myelin Fraction Mapping: Repeatability and Cross-Field-Strength Reproducibility at 1.5T and 3T
Purpose: To evaluate the repeatability and cross-field-strength reproducibility (1.5T vs 3T) of in vivo cerebral myelin fraction (MF) mapping with multi-contrast dual-resolution 3D UTE (multi-UTE), and to compare it with magnetization transfer (MT)-based methods. Methods: MT methods provide an indirect myelin-sensitive measure derived from magnetization exchange between macromolecular and free-water proton pools, whereas multi-UTE targets the ultrashort-T2* signal components. In five healthy volunteers, multi-UTE and MT-based measurements were performed at 1.5T and 3T, and the calculated MF maps were compared with magnetization transfer ratio (MTR) and magnetization transfer saturation (MTsat) maps. Two volunteers underwent repeated multi-UTE measurements within and between days. Field-strength reproducibility and test-retest repeatability were evaluated using linear regression, Bland-Altman analysis, concordance correlation coefficient (CCC), and intraclass correlation coefficient (ICC). Results: Multi-UTE enabled simultaneous acquisition of quantitative and high-resolution structural images at both 1.5T and 3T. MF repeatability was high at 3T (ICC = 0.94-0.96) and more variable at 1.5T (0,64-0.96). Across field strengths, MF and MTsat showed comparable reproducibility (CCC 0.70 vs 0.60) with small median absolute differences (8.8% vs 8.9%). MTsat showed tighter correlation (R2 = 0.77 vs 0.68), while MF showed a wider inter-regional dynamic range. MTR differed strongly between field strengths (CCC = 0.06). Conclusion: Multi-UTE MF, obtained without B1 or T1 mapping and alongside a high-resolution structural image, achieves cross-field reproducibility comparable to, but not exceeding, MTsat. Lower SNR at 1.5T and unmodelled T1 weighting may have contributed to residual field dependence.
comment: 7 Figures, 2 Tables, 7 Supplementary Figures
★ An Optimized Expansion Auction Integrating Energy, Flexibility, and Network Constraints
Auction mechanisms are widely used to support generation investments. In the US, capacity auctions serve as supply-adequacy instruments to address the missing-money problem, whereas in Latin America, energy and capacity auctions procure long-term contracts to support generation expansion. In both settings, accreditation metrics, such as Firm Energy (FE) and Effective Load Carrying Capability (ELCC), are used to approximate the reliability contribution of selected units and winning projects are selected based on a financial offer. This approach fails to recognize the actual contribution of winners to the multiple services they provide to the system. This limitation becomes more significant with high shares of renewables, as the value of each resource increasingly depends on network and technological constraints, as well as on complementarities within the generation portfolio. Inspired by short-term auctions, such as day-ahead markets, this paper proposes a joint optimization-based auction for generation and transmission investments that internalizes these interactions, capturing the intrinsic value of generation candidates without requiring the computation of FE or ELCC metrics. The auctioneer determines the winning portfolio through an expansion-planning model that explicitly represents network constraints, operational interactions, reliability standards, and multiple scenarios. A case study of the Brazilian system indicates lower system costs, lower risk, and a more diverse portfolio relative to the stylized decoupled procurement approaches currently deployed in many systems.
comment: 10 pages, 3 figures, 3 tables
★ A Unified Interpretable Semantic Framework for Multi-Modal Fiber Sensing Under Data Scarcity
We propose a unified interpretable semantic framework for multi-modal fiber sensing under data scarcity. The framework maps heterogeneous measurements from state of-polarization (SOP) monitoring, distributed acoustic sensing (DAS), and optical time-domain reflectometry (OTDR) into a shared set of physics-grounded semantic descriptors capturing impulsiveness, persistence, structural characteristics, and energy concentration. Inference is performed through reasoning over semantic class profiles, optionally assisted by retrieval of semantically similar examples to resolve ambiguity and support adaptation without model retraining. Experimental validation on real SOP, DAS, and OTDR datasets demonstrates robust performance across sensing modalities, achieving accuracies of 95.0%, 91.1%, and 94.6%, respectively, while providing transparent and physically interpretable decisions. The proposed framework establishes a unified and data efficient approach for optical fiber sensing in evolving and data scarce deployment environments.
★ Fit-LRP: Fitting Linear Relaxations to Polytopes for Neural Network Verification over Non-Convex Regions
State-of-the-art neural network verifiers evaluate linear relaxations over axis-aligned bounding boxes. However, verification domains in cyber-physical systems are rarely boxes. Reachability specifications, obstacle-avoidance constraints, and neural Lyapunov certificates typically yield oblique or non-convex polyhedral regions, over which box-based bounds become extremely loose. This paper proposes Fit-LRP (Fitting Linear Relaxations to Polytopes), a framework the implements exact optimization of affine bounds over non-convex domains (represented by unions of polytopes) by solving a sequence of small, low-dimensional linear programs. Our approach seamlessly enhances the final optimization step of LiRPA and $α,\! β$-CROWN branch-and-bound (BaB) algorithms. In BaB verification, approximating oblique constraints with axis-aligned splits causes severe combinatorial blowup and risks non-termination. By optimizing the final affine bound of a nonlinear function over the exact polyhedral state-space constraints our method eliminates box-induced over-approximations, enabling earlier search-tree pruning. On ARCH-COMP and neural Lyapunov benchmarks, Fit-LRP reduces branching on the hardest region by $47\times$, cuts worst-case verification time by $15\times$, and with region-wise partitioning, verifies on a single CPU $6.1\times$ faster than $α,\! β$-CROWN on a GPU.
★ Cooperative Learning of Shared Latent Dynamics for Distributed Online Optimization
We study networked agents that must agree on a decision minimizing the sum of their local costs, where each cost contains unmeasurable stochastic parameters that evolve with linear but unknown dynamics shared by all agents. Each agent receives only finitely many noisy gradient measurements and must then keep tracking the moving minimizer. Our scheme reconstructs the parameters from windows of gradients, learns the shared dynamics cooperatively by averaging instrumental-variable moment matrices over the network, and tracks the minimizer by gradient tracking on forecast gradients. We bound the error of every agent in learning the dynamics and in tracking the minimizer, and show that the learning error, and through it the tracking error, decreases with the number of agents. We demonstrate the approach on two case studies.
★ Model Predictive Control on Analogue Hardware: A Dual-flow Approach
Current Model Predictive Control (MPC) implementations are predominantly digital and, despite offering high precision and flexibility, often face challenges with ultra-low latency and hardware certification for safety-critical applications. We present an approach for solving the Quadratic Program (QP) of MPC using a continuous-time neural ordinary differential equation in the dual space. We obtain the solution to these QPs by mapping the differential equation's equilibrium point back to the primal space via a nonlinear transformation. The underlying dual-flow dynamics operate natively on analogue hardware. We validate this approach by demonstrating the hardware implementation of linear MPC on "The Analog Thing" analogue computer. Finally, we extend the framework to nonlinear MPC with successive linearisation, broadening the applicability beyond linear systems.
★ Reinforcement Learning for Target Jerk Estimation in Interceptor Homing Phase
We demonstrate the use of reinforcement meta-learning (meta-RL) to optimize a target jerk estimation policy, which is then used as the target acceleration rate of change in the propagation model of an unscented Kalman filter (UKF). The policy is learned through simulated engagements of a terminal phase high-altitude endoatmospheric interception of a highly maneuverable target, with a hit-to-kill accuracy requirement. As benchmarks, we tuned both a standard UKF and an interacting multiple model (IMM) filter, where the IMM uses a different constant jerk assumption for each model. The benchmarks and the proposed augmented UKF with the target jerk policy are tested in a 6-DOF simulation environment, which has sufficient fidelity to capture most factors relevant to guidance, navigation, and flight control. The augmented UKF demonstrates significant improvement in target interception rate, with more challenging maneuvers giving the largest performance differential. The meta-RL jerk policy is then tested on an expanded engagement scenario, with increased initial state estimate uncertainty and novel target maneuvers, demonstrating that the meta-RL policy generalizes well to conditions not seen during optimization.
comment: Submitted to Journal of Spacecraft and Rockets
★ Impact of Grid-Interactive Data Center Microgrids on Grid Reliability
Traditional data centers, operating as passive loads with standby backup resources, present significant challenges for grid integration due to their scale, rapid load fluctuations, and limited operational flexibility. This article investigates the transformative potential of grid-interactive data center microgrids, which integrate battery energy storage, on-site generation, and intelligent microgrid controllers to actively support both facility and grid operations. The proposed simulation framework enables the study comparing conventional and grid-interactive architectures under a range of steady-state and contingency scenarios. Results reveal that grid-interactive microgrids can substantially reduce peak grid demand, improve voltage stability, and improve system reliability especially when data center and system load peaks coincide. The findings highlight the critical role of coordinated resource management in enabling the reliable and efficient integration of large data center loads, and provide a practical methodology for assessing their system-level impacts.
comment: 5 Pages, 9 Figurs, Conference
★ Quasi-Static Fault-Tolerant Feedback Control of a Quadrotor Carrying a Slung Payload under Rotor Failure
This paper presents a nonlinear control law for a slung payload system (SPS) consisting of a quadrotor unmanned aerial vehicle (UAV) carrying a slung payload. We propose a fault-tolerant control (FTC) design for the case of a complete single-rotor failure, i.e., only three functioning propellers. The failure is not modeled as a disturbance to be handled by estimation. Rather, we model the faulty SPS in control-affine form and design a quasi-static feedback law directly for the faulty dynamics. The controller achieves local exponential convergence of the payload position to a geometric path in space, rather than tracking a timed reference trajectory. Moreover, the transverse dynamics are exponentially stable, and the path-following manifold is rendered controlled invariant by feedback. Controlled invariance allows motion along the path to be prescribed independently through a tangential command while the state stays on the path-following manifold. The UAV body angular velocity remains bounded under bounded applied torque despite the failure. To validate the control design, numerical simulations apply the controller derived from the faulty SPS model in coordinates to the coordinate-free SPS model. Code and animations are available at https://gradslab.github.io/qs-sl/.
comment: 8 pages, 8 figures
★ Robust Safe Control Under Uncertainty: Estimation, Contraction, and Regret Bounds
Disturbance uncertainty requires safety filters based on control barrier functions (CBFs) to leave a robustness margin, which can alter the nominal control input and degrade closed-loop performance. This paper studies safety and the resulting performance loss for discrete-time nonlinear systems with additive disturbances and polytopic input constraints. We develop a causal disturbance observer and construct two robust barrier conditions. The first uses a time-dependent bound on the estimation error. The second uses a constant margin when the disturbance increments are uniformly bounded and moves the requirement on the initial error from the online filter to an explicit initialization condition. For affine barriers, both filters are quadratic programs. We bound their input deviations from an oracle filter with access to the current disturbance and derive sufficient conditions for uniform contraction of the oracle closed loop. The contraction analysis accounts for state-dependent actuation, unmatched disturbances, and active physical input limits. Combining the sensitivity and contraction estimates gives finite-time dynamic regret bounds that quantify the effects of observer convergence, disturbance variation, and robustness margins. With sublinear cumulative certified disturbance variation and observer innovation error, the bounds yield sublinear regret.
comment: 15 pages, temporary, (optional) simulation to be added
★ Adaptive Model Predictive Control for Damping IBR-Induced SSOs
We propose an adaptive model predictive control (MPC) approach to damp sub-synchronous oscillations (SSOs) induced by grid-following (GFL) inverter-based resources (IBRs). Unlike traditional model predictive controller designed considering the nominal plant that may not perform well in other conditions, we use plants that alter with changing operating point. To capture the plant dynamics, we use recursive least squares (RLS) based system identification with variable forgetting factor. This approach allows us to discount past data and avoid bias caused by large transient outliers while performing system identification. We present a proof of the convergence of the model parameters in the presence of finite outliers. We propose to update the model inside the MPC optimization problem periodically after a predetermined time interval instead of each sampling instant since the operating point changes slowly in a practical power system. The MPC design allows us to directly impose constraints on control inputs so that the IBR current limits are respected. Our approach also addresses the issue of unavailability of IBR models. We show the accuracy of the system identification and the superior performance of the proposed controller on the modified IEEE $4$-machine system and the modified IEEE $68$-bus system as compared with the traditional MPC.
★ Scalable Reference Electrical Architectures for Modern Data Centers
The rapid growth of cloud computing, artificial intelligence (AI), and high-performance computing is driving the deployment of data centers with electrical demands ranging from tens of megawatts to multi-gigawatt campuses, creating significant challenges for utility interconnection and power system planning. This paper presents scalable, vendor-neutral reference electrical architectures for colocation, hyperscale, and AI data centers. The proposed framework integrates modular Power PODs, hierarchical HV/MV/LV distribution, standby generation, uninterruptible power supplies (UPS), battery energy storage systems (BESS), and scalable utility interfaces into a unified planning methodology. Three representative Power POD configurations are developed to support systematic capacity expansion while maintaining consistent electrical topology and equipment sizing. The proposed architectures facilitate utility interconnection studies, equipment demand forecasting, transmission planning, and grid infrastructure assessment, providing an open and reproducible benchmark for evaluating the impacts of large-scale data center deployment on future power systems.
comment: 5 pages, 3 figures, conference article
★ Quantifying the Cost of Restoring AC Feasibility for Approximate Optimal Power Flow Solutions
Power system operations rely on approximate solutions of the AC optimal power flow problem, most notably the DC optimal power flow used in security-constrained unit commitment and economic dispatch, while convex relaxations, machine learning surrogates, and other approximations are studied extensively in the academic literature. Because these approximations neglect or relax parts of the AC power flow equations, their solutions must be adjusted before dispatch; for example, the DC optimal power flow, which neglects transmission losses, reactive power, and voltage magnitude variations, is known to produce solutions that are always infeasible. The cost of this adjustment is not adequately understood. This paper proposes a method that quantifies, for any approximate solution, the minimum cost of the adjustments that restore AC feasibility, which is a lower bound on the cost of the heuristic adjustments used in practice. We apply it to the DC optimal power flow, the second-order cone and quadratic convex relaxations, and the AC optimal power flow solved to a loose relative tolerance of $10^{-4}$ instead of $10^{-8}$. Relative to the AC optimal power flow solution obtained from a local nonlinear optimization solver, and even with the minimum adjustment, the DC optimal power flow requires adjusting, on average, generators that account for 15.6% of total generation capacity and incurs a 1.10% cost increase; the convex relaxations require more adjustment (17.2% and 16.9%) but smaller cost increases (0.68% and 0.78%); and the loose-tolerance AC optimal power flow requires 6.1% and incurs 0.42%. These results indicate significant value in operating markets directly on the AC optimal power flow, which avoids the distortion of the dispatch and prices caused by heuristic feasibility restoration.
comment: 5 pages, 4 figures, submitted to 2027 IEEE Power & Energy Society General Meeting
★ Assessing Data Center Hosting Capacity Considering Electromagnetic Transient Dynamics
Hosting capacity is commonly assessed with steady-state optimization, which leaves the dynamic response of the data center and the grid at the resulting operating point untested. This paper assesses candidate data center capacities by coupling a unit commitment (UC) model with electromagnetic transient (EMT) simulation. During the capacity search, the UC model determines the economic dispatch for each tested data center capacity, and the resulting operating points are tested for settling with the generators committed online by the UC model and after a selected synchronous generator outage. Three nested capacities are obtained: the optimization-based steady-state capacity, the dynamic pre-outage capacity, and the post-outage dynamic capacity, each bounded by the previous one. On a test system with a detailed data center model, both dynamic capacities fall well below the optimization capacity, and the ranking of connection locations changes with the assessment stage and with the outage. In the most severe outage case at each candidate bus, the capacity is about 30% below the steady-state capacity. Thus, the dynamic response of the data center and the grid should be part of the hosting capacity assessment, and the comparison of connection locations should be made on the post-outage dynamic capacity.
★ Transversal Passivity and Equilibrium Forces in Formation Control
We consider controller synthesis for distance-constrained formations, aiming to stabilize the target formation manifold while sustaining admissible equilibrium edge forces. Using the newly introduced notion of transversal passivity, we encode the target formation shape, the prescribed edge lengths, and a subspace of internally balanced edge forces as the port target manifolds of the agents and the edge controller. The synthesis task is thereby reformulated as the joint design of a transversally passive edge controller and its output-target subspace. We then propose a general constructive realization of output-strictly transversally passive systems and specialize it to derive a dynamic edge controller that sustains admissible equilibrium edge forces. Through a compatibility analysis across the interconnection, we establish a closed-loop dissipation margin in terms of the controller parameters and prove local convergence of all agent and controller ports to their target manifolds. Numerical simulations validate the results.
comment: 9 pages, 4 figures
★ What World Model Does Active Object Search Need? A Study under Imperfect Detection
In active object search, a robot must decide where to look and when to stop. Imperfect detection complicates both decisions: a non-detection may indicate occlusion, an out-of-view target, or a false negative, while a detection may be a false positive. This raises the question of which representations a world model needs to support reliable search under imperfect detection. We introduce Bayesian Active Miss-aware Object Search (BAMOS) to investigate how target belief, occupancy belief, and detector errors affect search decisions. Through comprehensive ablations in Isaac Sim and HSSD houses, we examine the roles of these representations and their update mechanisms across different detector configurations. Notably, target belief and its update mechanisms are critical to search performance, with their contribution generally increasing as detection errors become more prevalent. These findings highlight the importance of maintaining and updating target beliefs to support reliable active search under uncertain detector observations, providing insights into the design of world models for embodied agents. Code is available in https://github.com/gokyeongryeol/BAMOS.
comment: Accepted in CoRL 2026 Workshop on Do Robots Need World Models
★ Decentralised model predictive control of recurrent neural network models
This work proposes a decentralised nonlinear model predictive control algorithm for systems described by input-coupled recurrent neural network models. The proposed approach exploits closed-loop incremental input-to-state stability to design the local controllers and provide robustness to subsystem couplings. The resulting control scheme guarantees asymptotic convergence to a desired equilibrium while satisfying input and output constraints. Simulation results illustrate the performance of the proposed approach.
★ Adaptive Robust Output Regulation of Uncertain Linear Time-Varying Systems
This paper studies adaptive robust output regulation of linear time-varying systems with unknown constant parameters in both the plant and exosystem. The design is based on nonmonic differential relations for the ideal input. For general time-varying and periodic systems, we derive sufficient conditions for the existence of nonmonic differential relations and construct them from the original system coefficients. Based on these relations, an adaptive controller is constructed using stable filters and a parameterized readout. Under suitable conditions on the filter and readout, the controller achieves exact regulation for finitely separable relation coefficients, including the affine parametric uncertainty treated in Byrnes--Isidori form. For coefficient families satisfying the approximation conditions, refining the approximation achieves any prescribed ultimate regulation accuracy with a fixed stabilization design, yielding practical regulation. Neither persistent excitation nor parameter convergence is required. Two numerical examples illustrate the regulation performance of the proposed controller. Overall, the framework includes earlier constructions based on finite separation of the ideal input or monic linear differential relations with finitely separable coefficients, and achieves exact robust regulation for a broader class of uncertain systems.
★ Estimation-Aware Control on Quadrotor State Manifolds via Anisotropic Covariance Filtering
Conventional nonlinear flight controllers operate under the implicit assumption of reliable, high-confidence state feedback. During transient sensing degradation or partial unobservability, state estimation uncertainty is frequently misinterpreted as physical tracking error, triggering aggressive and parasitic corrective inputs. To mitigate this vulnerability, we propose a generalized Estimation-Aware (EA) control framework that directly incorporates real-time estimator confidence into control execution. By mapping the posterior error covariance $\mathbf{P}_{k\mid k}$ into an uncertainty-dependent modulation operator, the EA mechanism dynamically scales feedback authority, performing anisotropic attenuation along weakly observed state directions to prevent false error corrections. Validated across six distinct control paradigms, the proposed approach demonstrates substantial performance gains across all baselines. In particular, when integrated into an Incremental Nonlinear Dynamic Inversion (INDI) architecture, the EA mechanism reduces tracking and estimation errors by 24.34% and 33.24%, respectively, at a high flight speed of 10 m/s.
comment: 11 Pages, 14 Figures, 2 Tables
★ Quantitative Stability Region Estimation of Grid-Forming Inverters with Current Limiting via a State-Dependent Switched Energy Function
With increasing renewable energy integration, grid-forming (GFM) inverters are becoming key components of future power systems. Compared with synchronous generators (SGs), GFM inverters have limited overcurrent capability and require current limiting. This can lead to switching between normal and current-limited operating modes under large disturbances, complicating synchronization dynamics and posing a longstanding challenge to existing transient-stability analysis methods. In this paper, a state-dependent switched (SdS) energy function theory is proposed for transient-stability analysis of GFM inverters with current limiting, whose dynamics can be abstracted as SdS nonlinear systems. Regional energy functions are constructed for individual operating modes and continuously matched at switching interfaces, yielding a continuous piecewise energy function. Hence, the derived conservative estimate of the stability region (region of attraction, ROA) spans different operating regions without prescribing subsequent current limiting activations and releases, a fundamental distinction from conventional methods using signal-mode energy functions. Techniques for incorporating damping energy are extended to the proposed framework to capture damping effects and further reduce the conservatism of the ROA estimates. The methodology applies to GFM inverters with different synchronization control loops and current-limiting strategies. Experiments on a physical GFM inverter under various grid contingencies validate the proposed quantitative transient stability assessment.
★ SLyM: LLM-Driven Symbolic Lyapunov Function Discovery via Structure Proposal and Coefficient Optimization
Lyapunov functions are the cornerstone of stability analysis for dynamical systems, yet synthesizing a symbolic form remains difficult as no universal constructive algorithm exists. To address this challenge, we introduce SLyM, a framework that harnesses the knowledge and reasoning of large language models (LLMs) to discover symbolic Lyapunov functions. To avoid charging the LLM with ill-suited numerical generation, SLyM decomposes the construction into structure proposal and coefficient optimization: the LLM draws on its prior knowledge of Lyapunov constructions to propose the function structure, whose coefficients then follow from a convex program. Local-search structures are folded back into supervision, extending coverage to systems missed by the initial proposer. Across all benchmarks, SLyM consistently outperforms the evaluated neural, sum-of-squares, and generative baselines.
★ Planning-Stage Assessment of Battery-Supported Regional Islanding: A South Australian Case Study
Power systems with high shares of renewable generation may face increased operational vulnerability under regional islanding caused by the loss of major inter-area transmission interfaces. This paper presents a planning-stage transmission-level assessment of regional islanding with battery storage, using a South Australian case study based on a synthetic representation of the Australian National Electricity Market (NEM). An interconnected planning baseline is compared with a stress test scenario involving complete regional-islanding over a 24-hour chronological horizon. The framework quantifies peak unserved power, total unserved energy, and the duration of unserved demand, while characterising battery redispatch and the synchronous inertia accessible to the subsystem.
★ Agreement Is Not Assurance: Evidence-Independent Safety Gates for Agentic AI in DER-Rich Distribution Grids
Physics-based safety gates for large language model (LLM) agents can approve unsafe actions when the agent and gate rely on the same corrupted measurements or network records. We formulate this shared-evidence failure for voltage-constrained distributed energy resource (DER) export decisions and analyze how nearly critical DER telemetry can bias state estimation while evading residual-based bad-data detection. We propose evidence-independent verification (EIV), which combines advanced metering infrastructure (AMI) voltage readings, measurement selection based on predicted voltage rise, and bounded physical probes. An IEEE 123-bus hidden-truth testbed evaluates 1,654 near-limit requests under telemetry attacks and record errors. For telemetry attacks targeting the nominal gate, an AC-twin gate with bad-data detection approved 95% of unsafe requests, compared with 3.1% for EIV; EIV rejected 22% of safe clean requests. At matched false-rejection settings, direct AMI checks outperformed incorporating the same readings into state estimation. Gate-targeted attacks exposed a vulnerability in telemetry-based response prediction. Command-based prediction removed that route, but targeted attacks on fresh validation cases still produced a 20% unsafe-approval rate under the tested attack pool. LLM agents frequently sought independent evidence voluntarily, but injected instructions and phase-label errors caused unsafe decisions. When offered the EIV checker as a tool, they used it in 93-100% of episodes; mandatory use did not reduce observed violations, which primarily reflected checker errors. These results distinguish enforcement of verification from reliability of the verifier itself.
comment: 10 pages, 3 figures, 4 tables
★ Fundamental Limits of Data-Driven Predictive Control
Data-driven predictive control (DDPC) has attracted substantial interest, and many variants have been proposed; however, comparatively less is known about the statistical closed-loop performance of these methods and the fundamental limits they are subject to. In this contribution, we apply the statistical decision framework developed in our earlier work to study fundamental limits of DDPC. The performance of a data-driven controller is measured by its risk, the expected deterioration in control performance relative to an oracle model-based controller at a given system, and by its total risk that measures the overall performance across all systems of interest. Lower bounds on these risks are derived using the bias-variance decomposition and the Cramer-Rao inequality. In particular, a lower bound on the total risk is obtained by a calculus-of-variations method. This bound applies to any DDPC design, thereby quantifying unavoidable performance limitations and providing a benchmark for assessing existing DDPC methods. In an illustrative example, we compare the risks of certainty-equivalence model predictive control, subspace predictive control, and data-enabled predictive control with the derived lower bounds, providing a quantitative performance assessment of these methods and some guidance on tuning regularization parameters in DDPC.
★ Unifying Heterogeneous Dissipativity for Distributed Power-System Stability Analysis
Compositional dissipativity is attractive for large-scale interconnected systems, but its application to modern heterogeneous power systems is limited by conservative descriptions of nodal dissipativity and inefficient distributed implementation. This paper introduces a common first-order dynamic lifting of the physical voltage-current port, providing an enriched dissipativity representation that captures complementary frequency-dependent energy behavior while preserving network composability. The resulting representation accommodates heterogeneous power system buses within a unified dissipativity description and enables less conservative compositional stability analysis. A centralized stability-certification problem is then formulated and solved through a dual-guided column-generation scheme. Its matrix dual reveals the network-critical dissipation direction, which guides parallel local certificate generation toward the current system bottleneck. This leads to a distributed Network-Critical Margin Exchange (NCME) method with early stability certification. Case studies demonstrate substantially reduced conservatism of the dynamic-port formulation and improved computational and communication scalability of NCME for heterogeneous power systems.
comment: 11 pages, 9 figures
★ Algorithms for Robust Dedicated Actuator Placement
We study dedicated actuators installed at distinct states of a structured linear system. Failures remove actuators but preserve states and state arcs. For unit costs and one failure, an exact source-quota relaxation followed by maximum matching in a \emph{common-repair graph} yields a $3/2$-approximation on arbitrary directed state graphs using $O(n^3)$ elementary graph operations. The analysis combines fundamental cocircuit unions with the Tutte--Berge formula. For multiple failures, we give an exact algorithm fixed-parameter tractable in the maximum-matching deficiency and failure budget, independently of the number of source strongly connected components. The underlying result solves resilient matroid spanning with an arbitrary number of disjoint unit-cost quota constraints. We also establish an exact weighted spanning algorithm for safe and vulnerable elements in the independence-oracle model, with an application to strongly connected state graphs.
comment: 13 pages, 1 figure. Source code and verification scripts available on GitHub
★ Towards Efficient Edge AI for Power Flow in Renewable-Rich Grids with Volt/VAR Control
AI models of the power grid must capture how wind and solar plants support the voltage, adjusting their reactive power to the local voltage along a grid-code Q(V) characteristic. This research examines whether grid foundation models, fast neural network models for power-flow analysis, can represent this behaviour. We introduce the Q(V) bus, a new bus type for the open-source GridFM GraphKit, whose physics decoder evaluates the reactive injection from the plant characteristic at the predicted voltage in every correction step. Training data comprise 7882 hourly operating points of a modified IEEE 118-bus system, matched to Energinet DK1/DK2 measurements and solved with pandapower. Compared with a physics-informed baseline that represents the parks as negative load, the Q(V) bus lowers the voltage-magnitude RMSE 2.3-fold, reduces the power-balance mismatch at the parks from 25 to 0.66 MVA, and raises the F1 score for voltage-band violations from 0.40 to 0.76. An ablation over five configurations shows that the Q(V) bus type provides the accuracy and the physics loss provides the power balance. For edge monitoring, we distil the trained $2.0\times10^{7}$-parameter model into students of $6\times10^{5}$ and $1.3\times10^{6}$ parameters; a flat-start initialisation prevents their collapse into an incorrect low-voltage state in which the characteristic saturates. With output distillation, the 12-step student reaches 1.4 times the teacher's error and, for edge inference on an NVIDIA Jetson Orin Nano (15 W), solves an hourly scenario in 12.2 ms, within the 20 ms synchrophasor interval.
★ Quantum Computing in Power Systems:Current Landscape and Future Roadmap with AI
Modern power systems are characterized by greater penetration of renewable energy resources, extensive use of power electronics, and more diverse loads. This leads to a dramatic increase in computational complexity, which, in turn, compromises solution efficiency and fails to satisfy the real time demands of system analysis. Recently, quantum computing techniques, attributed to their unique physical properties, such as superposition and entanglement, hold the potential to offer new solutions for power systems. To this end, this paper not only provides a comprehensive landscape of quantum computing in power systems, but also investigates its roadmap with AI enabled evolution. This paper begins with an introduction to the fundamental concepts of quantum computing, followed by a review of several approaches to power system simulation and optimization. Subsequently, we further investigate the potential quantum AI-enabled evolution in power systems. In addition, the current development of quantum software and hardware platforms is elaborated. Finally, this paper synthesizes the prevailing challenges and prospects for future research directions.
★ Stealthy, High-Impact False Data Injection Attacks in Smart Grid Using Physics-Constrained GAN
False data injection (FDI) attacks can distort power system state estimation while retaining plausible measurement patterns. Existing FDI methods trade stealth for impact or vice versa. They often assume access to internal grids, all-bus sensing, or defender models. In this paper, we propose a detector-agnostic physics-constrained generative adversarial network (PCGAN) that learns decoupled power-flow sensitivities from historical measurements of deployed sensors. The learned active-power-angle and reactive-power-voltage-magnitude relationships guide perturbations restricted to compromised channels, while adversarial training promotes realistic spatiotemporal patterns. We implement PCGAN and seven existing FDI attacks on the IEEE 300-bus system using one year of observed hourly demand from PJM Interconnection, a regional transmission organization in the United States, and synthetic bus-level load profiles. We compare their performance across 10%, 30%, 60%, and 100% compromised sensors. Attack stealth is assessed against ten machine-learning (ML) detectors and conventional bad-data detection (BDD) methods, while physical impact is quantified through nonlinear alternating-current weighted least-squares state estimation. PCGAN achieves 84.28% mean ML evasion and 22.31% mean spatial load misperception across the four scenarios. Its mean stealth-impact trade-off score reaches 18.69%, compared with 5.61% for the strongest competing attack. PCGAN also achieves 83.45% chi-square BDD evasion and 86.02% largest-normalized-residual BDD evasion. These results demonstrate a favorable stealth-impact balance under varying sensor-access levels and provide challenging attack cases for evaluating FDI detectors.
comment: 10 pages
★ When Does Learned Constraint Selection Pay in Security-Constrained Optimisation? A Boundary Map from SCOPF to Unit Commitment
Learning-to-optimise methods for security-constrained optimal power flow (SCOPF) predict the security constraints that bind on each instance and solve only the reduced problem, with reported speed-ups of two to four orders of magnitude over the full problem. We ask whether learned selection beats the strongest learning-free alternative: a fixed umbrella set enforcing every constraint observed to bind in training. Our criterion, the price of a spare, answers this in advance: an enforced but inactive constraint is free, so the umbrella is exactly optimal wherever feasible, and learning can pay only where a spare decision carries a price. We test it systematically, including every PGLib-OPF network up to 10,000 buses. For SCOPF on a fixed network the umbrella is never beaten: its median size is five constraints across 43 networks, with no size trend, and our selector, while reproducing speed-ups of published magnitude (about x80,000 at case500), gains at most 0.25 ms over the umbrella and violates constraints on 4-5% of instances. Configuration diversity can break the umbrella's coverage on a small, densely meshed network but not at 2,000 buses, and we certify coverage with conformal risk control. In unit commitment, where a spare has a price, a static commitment loses 4-12% out of sample, and a predict-repair-reduce pipeline recovers it (cost gap 0.2-2.9%, fully feasible, 16-161x faster than the MILP). In our experiments, whether learning pays depends not on problem size but on whether a spare carries a price.
comment: 10 pages, 6 figures, 4 tables
★ Low-Data OOD Prediction of SAW Resonators with Simulation-Compensated Transfer Learning
Surface acoustic wave (SAW) resonators are foundational to modern radio-frequency (RF) front-end architectures. However, models trained on limited fabricated devices generalize poorly to unmeasured design regions because of process-induced variations, while finite-element method (FEM) simulations, although inexpensive, exhibit systematic offsets from measurements. To predict device performance before physical prototyping, this work presents a simulation-compensated transfer-learning framework that uses computationally efficient 2.5-D FEM simulations as physics-based anchors and learns their calibration to the measurement domain. Using 4,236 measured lithium tantalate (LiTaO3) SAW resonators and 816 unique simulated configurations, five transfer strategies are systematically benchmarked under seven out-of-distribution (OOD) protocols spanning frequency, geometry, and material shifts. The proposed framework reduces OOD prediction error by 83.54% on average (up to 97.57%) relative to the Measurement-only baseline, with the best strategy selected per case, and raises the pooled R2 of frequency prediction to 1.000, offering an economically scalable route toward physics-guided, fabrication-aware machine learning for microelectromechanical-system (MEMS) acoustic-device design.
★ Sim-to-Real Powertrain Vibration Control via Model-Based Assisted Domain-Randomized Reinforcement Learning
Controllers designed from nominal models often degrade on mechanical systems with nonlinearities and parametric variations, such as automotive powertrains with backlash. Deep reinforcement learning (DRL) with domain randomization can acquire robustness to such model discrepancies, but wide randomization destabilizes training and yields conservative policies, and experimental transfer of such policies has rarely been demonstrated in vibration control. This paper presents model-based controller assisted domain randomization (MBCADR), a framework that integrates model-based control into domain-randomized DRL. An $H_{2}$ controller designed from the nominal model supplies a baseline action, while a residual policy trained with twin delayed deep deterministic policy gradient under randomized dynamics compensates for nonlinearities and uncertainties. This model-based assistance relieves the agent of synthesizing the entire control input, thereby stabilizing training under extensive randomization. For each realization of the uncertain parameters, the augmented plant--controller state is shown to be a Markov state of the closed loop, so that domain-randomized training is rigorously formulated as policy optimization over a parameterized family of Markov decision processes. A Lyapunov-based stability analysis further derives a forward-invariant set and an explicit ultimate bound, in which the learned action enters only through its saturation level. Monte Carlo simulations with multiple training seeds show that MBCADR reduces performance variability compared with either component alone. Experiments on a powertrain test bench with backlash, using the simulation-trained policy on an embedded processor without further tuning, confirm successful sim-to-real transfer.
★ Backstepping-Forwarding Design for Hyperbolic PIDE-ODE Systems with Distributed Interconnection and an Arbitrarily Large Input Delay
This paper studies first-order hyperbolic partial integro-differential equation (PIDE) and ordinary differential equation (ODE) cascades with distributed interconnection and an arbitrarily large known constant input delay. Within the backstepping--forwarding framework, a forwarding-type transformation is designed to reduce the distributed interconnection to an equivalent boundary-coupled Dirichlet cascade without requiring the PIDE to be open-loop stable. A subsequent backstepping design stabilizes the PIDE and the ODE while compensating for the input delay. Incompatible boundary conditions in the delay-compensation kernel equations may produce discontinuities along characteristic lines. We determine their jump locations and amplitudes and prove that the interface terms cancel when deriving the target equations, so the kernel discontinuities do not change the target system. Closed-loop exponential stability is established without additional plant restrictions to ensure kernel continuity. A numerical example illustrates the effectiveness of the proposed approach.
comment: Originally submitted to IEEE Transactions on Automatic Control in May 2026. The revised manuscript is currently under review
★ Exact Output Steering with Vanishing Controls in Weighted Shift Dynamics
Exploiting the rich dynamics of wrapped weighted shifts and their residualizations, we characterize exact projected steering with vanishing initial controls. For wrapped shifts, steering is characterized by an expansive cycle multiplier and a residue separating the source and control channels. For residual shifts, a distinct spectral--toral mechanism emerges: steering through selected conjugate planes is equivalent to radial expansion and an arithmetic parity condition on phase resonances, yielding bounded-gap steering times and quantitative control estimates. For structured machine-learning representations built from orbit dictionaries, we derive a two-scale gain condition for steering coefficients while preserving decoded outputs.
★ Metropolis Random Walks Escape Deep Basins Faster in Higher Dimensions
Simulated Annealing (SA) is a widely used stochastic optimization algorithm. It is of great interest to understand how its ability to escape local minima at a fixed temperature depends on geometry and dimension. In prior work, we derived closed-form expressions for the expected basin escape time of a constant-temperature SA process on a one-dimensional piecewise-linear landscape. In this paper, we extend the analysis to $D$-dimensional cubic basins defined by the infinity-norm energy $V(x)=\|x\|_\infty$. The walk proposes nearest-neighbor moves uniformly and escapes upon first reaching the outer shell, that is, any site with $\|x\|_\infty=N$. We construct a one-dimensional birth-death shell-quotient chain by collapsing each concentric shell in the Metropolis electrical network to a single node while retaining its total stationary mass. We derive a closed-form expression for the mean escape time $Q_D(N)$ of this shell-quotient chain as a function of the acceptance probability $p$, the basin depth $N$, and the dimension $D$. For fixed $D$ and fixed $0
comment: 13 pages, 5 figures
★ TDMOpt: LLM-Driven Cross-layer Dispatch for Transmission-Distribution-Microgrid Systems
Coordinating transmission networks, distribution networks, and microgrids enables distributed flexibility to support renewable energy integration and low-carbon operation. However, heterogeneous resources, cross-layer coupling, and varying operating requirements complicate the selection of models and coordination schemes that are both physically adequate and computationally tractable. This paper proposes TDMOpt, a LLM-driven multi-agent framework that replaces fixed modeling configurations with adaptive model composition. A layered component library provides executable network, resource, objective, constraint, and coordination models. Guided by retrieved physics and component knowledge, the agents infer specific modeling requirements and jointly select compatible components across layers according to operating conditions and computational budgets. The model compositions undergo independent AC power flow validation, since feasibility in a simplified formulation does not ensure compliance with physical network limits. A refinement mechanism interprets numerical feedback to identify modeling requirements for reconsideration and revise relevant component selections. Case studies on different scale systems illustrate the framework's ability to adapt model combinations and obtain validated dispatch solutions through targeted refinement under varying operating conditions.
★ LLM-Enhanced Deep Reinforcement Learning with Human-in-the-Loop Calibration for Real-Time Optimal Power Flow
Real-time optimal power flow (RT-OPF) requires fast and reliable decision-making under time-varying operating conditions. However, many practical operating preferences and safety related requirements are difficult to encode using conventional numerical rewards alone, which limits the effectiveness and controllability of deep reinforcement learning (DRL) for RT-OPF. To address this issue, this paper proposes an LLM-enhanced DRL framework with human-in-the-loop (HITL) for RT-OPF. The proposed method augments the physics-based reward with sparse language evaluative feedback during offline training, so that semantic operational requirements can be incorporated into policy learning. To improve reliability in RT-OPF, a selective HITL calibration mechanism is further introduced to inspect and correct questionable LLM judgments before they continuously shape the learning process. Case studies on the IEEE 30-bus and IEEE 118-bus systems show that LLM feedback improves policy safety over vanilla TD3, while further HITL calibration achieves the best overall performance and more stable training behavior. These results demonstrate the effectiveness of combining language guidance and expert calibration for RT-OPF.
★ Inverse-Optimal Fixed-Time Integral Reinforcement Learning for Unknown Nonlinear Leader-Follower Multi-Agent Systems
This paper develops a distributed inverse-optimal fixed-time integral reinforcement learning framework for unknown nonlinear leader-follower multi-agent systems. The proposed method first reconstructs the unknown agent dynamics from finite integral data. Based on the identified dynamics, a distributed inverse-optimal formulation is constructed by prescribing a desired fixed-time value function and recovering a compatible running cost. A critic-only integral reinforcement learning architecture is then developed using current integral data and finitely informative experience-replay data, thereby avoiding the persistent-excitation requirement. Practical fixed-time convergence of the critic-weight errors is established in the presence of identification and function-approximation residuals. Finally, a network-level composite Lyapunov analysis proves practical fixed-time leader-following consensus of the true unknown nonlinear multi-agent system. Numerical simulations are provided to validate the developed theory.
★ DIVAL: Diffeomorphic Vector-Field Alignment for Nonholonomic Motion Planning
Conventional vector-field-based motion planners for nonholonomic robots consist of two steps: first designing a nominal vector field and then controlling the robot to follow it to approach the target. However, a nonholonomic robot cannot follow the vector field instantaneously and may collide with obstacles, due to the inevitable mismatch between the robot's initial heading and the vector field's direction. This paper proposes a diffeomorphic vector-field alignment (DIVAL) method, which aligns the vector field's direction with the robot's initial heading so the robot can follow the vector field from the start. Specifically, the nominal vector field is locally reshaped by applying a diffeomorphic deformation to the workspace, such that the reshaped vector field has the same direction as the robot's heading at the initial position and coincides with the nominal vector field outside the deformed region. Since DIVAL eliminates the initial heading mismatch, the nonholonomic robot can maintain heading alignment with the reshaped vector field throughout the motion. This heading alignment further ensures that the robot preserves the nominal vector field's properties, including obstacle avoidance, convergence to the target, and smooth control signals. Numerical simulations provide vector-field-tracking baselines where the nominal vector fields are reshaped with DIVAL. Compared to the original baselines, the proposed method achieves collision-free convergence in all trials and avoids control discontinuities. Hardware experiments on a ground vehicle further demonstrate collision-free convergence across different targets and initial conditions under real-world disturbances.
comment: 19 pages, 12 figures, under review
★ Beyond Yes or No: Grading Observability of Power System Dynamic States with Symbolic Coefficients
Graph-based observability analysis runs in seconds where Lie-derivative tests take hours, but it answers only one question: is the system observable or not. It cannot say how well a given set of measurements supports state reconstruction. This paper extends the graph-based approach with symbolic observability coefficients computed from a symbolic Jacobian of the state-space equations, in which every dependency is classified as constant, nonlinear polynomial, or rational. We show that the coefficient tracks the analytical observability determinant of the Lie-derivative approach: it is zero when the determinant vanishes, lies strictly between zero and one when the determinant depends on the state, and equals one when the determinant is a nonzero constant. We apply the method to a seven-state synchronous machine and to the 21-state Western System Coordinating Council nine-bus system under several synchrophasor measurement configurations. On the nine-bus system, the coefficient grows from 0.0952 with one PMU to 0.1905 with two and 0.2857 with three, and the whole analysis runs in under five seconds. Configurations with the same number of PMUs receive the same coefficient, yet an unscented Kalman filter estimates the states with different accuracy for each. The symbolic coefficient therefore grades measurement sets beyond identifying root strongly connected components, while telling PMU locations apart still requires the numerical parameters of the system.
★ AgentiGrid: Agentic AI for Steady State and Transient Studies
As power systems worldwide struggle to meet in- creasing loads or higher penetration of inverter based resources, dynamic studies become of increasing importance. The rise of artificial intelligence (AI) capabilities allows to automate some of these resource intensive studies. Coupled with parallel computing, AI is a promising technology to conduct transient and stability studies on an unprecedented scale. This study provides a proof of concept agentic AI capable of parallel or sequential execution of phasor domain studies for a 10,000 bus system. The agent first executes a standalone simulation of a three phase symmetric fault and then executes a series of actions, by first adding a large load to a bus and then applying the fault to the same bus to stress the system. The results of the agentic simulation are consistent with human counterfactual across four vendor backends.
comment: 3 figures, 1 table
★ Prediction-Error-Tightened MPC for Socially Compliant Crowd Navigation
Autonomous navigation of mobile robots in crowded environments, while preserving pedestrians' psychological comfort, requires distinguishing pedestrians from other obstacles, predicting their future motion, and accounting for uncertainty in the predicted trajectories of both pedestrians and the robot. This work extends a previous MPC framework by integrating obstacle and pedestrian detection, Koopman-based motion prediction, and online prediction-error estimation directly within the collision-avoidance mechanism. To reduce collision-avoidance complexity in crowded scenarios, nearby obstacles are aggregated, thereby reducing the number of collision-avoidance constraints. Simulation studies evaluate the prediction-error margins and the computational benefits of semantics-aware aggregation. In a paired 300-seed closed-loop comparison with a moving generic obstacle, Markov residual tightening reduces the fraction of runs with physical contact from $10.0\%$ to $1.33\%$, at a mean travel-time cost of $0.68$ s. In a separate crowd-navigation semantic ablation, the offline robot-footprint/personal-space overlap occupies $2.11\%$ of observed time with pedestrian-aware handling versus $22.84\%$ with generic-obstacle handling.
comment: 9 pages, 3 figures, 4 tables. Submitted to the European Control Conference (ECC 2027)
♻ ★ Buildrix: An Open Agentic AI Benchmark for Building Engineering
Agentic AI has demonstrated remarkable progress across science and engineering, including mathematics, physics, chemistry, and biology. However, how well AI agents can perform real-world building engineering tasks remains largely unexplored. To address this gap, we introduce Buildrix, an open benchmark for evaluating AI agents on real building engineering tasks across eight domains. We investigated three questions: how AI agents compare with human engineers, whether expert knowledge improves agent performance, and how engineers can collaborate effectively with AI agents. The benchmark shows that AI agents achieved an average score of 78 out of 100, compared with 84 for human engineers, while completing tasks in an average of 9 minutes at a cost of $0.58 per run, compared with 8-160 reported working hours for human engineers, demonstrating promising potential. Interestingly, providing expert knowledge did not consistently improve agent performance and sometimes led to worse results. In contrast, structured task instructions substantially improved performance, increasing the average score from 27 to 75 and the success rate from 5% to 76%. These findings highlight both the potential and limitations of AI agents in building engineering, as well as the importance of structured task specifications and clear deliverable requirements for effective human-agent collaboration. More broadly, Buildrix provides an open evaluation framework for tracking AI progress, identifying capability gaps, and guiding the development of more capable and reliable AI agents to accelerate the adoption of AI in building engineering practice.
comment: 15 pages, 8 figures, 4 tables
♻ ★ QuantGraph: A Receding-Horizon Quantum Graph Solver
Dynamic programming is a cornerstone of graph-based optimization. While effective, it scales unfavorably with problem size. In this work, we present QuantGraph, a two-stage quantum-enhanced framework that casts local and global graph-optimization problems as quantum-searches over discrete trajectory spaces. The solver is designed to operate efficiently by first finding a sequence of locally optimal transitions in the graph (local stage), without considering full trajectories. The accumulated cost of the resulting candidate acts as a threshold that prunes the search space (up to 60% reduction for certain examples). The subsequent global stage, based on this threshold, refines the solution. To achieve scalability and robustness, we draw on principles from control theory and embed QuantGraph's global stage within a receding-horizon model-predictive-control scheme. This classical layer stabilizes and guides the quantum search, improving precision and reducing computational burden. In practice, the resulting closed-loop system exhibits robust behavior and lower query complexity. Notably, for a fixed control horizon and query budget, QuantGraph supports approximately twice as many control discretization bits while still benefiting from Grover-search's inherent quadratic speedup compared to classical methods.
comment: P. Vaidhyanathan and A. Papatheodorou contributed equally to this work. Published in Physical Review Applied. 17 pages, 4 figures, 2 tables, 2 algorithms
♻ ★ From Grid to Chip: Power Architecture, Stability, and Flexibility of AI Data Centers
The rapid growth of artificial intelligence (AI) computing is transforming data centers into large, dynamic electrical loads. Their deployment is primarily constrained by energy availability and grid-connection capacity, which is further aggravated by the ability of power-delivery architectures, control systems, and computing workloads to operate reliably during fast grid disturbances. This article provides a comprehensive overview of forefront challenges and trends in the grid-to-chip framework, and presents a technological perspective on AI data centers as grid-interactive computing systems. First, it reviews grid-integration bottlenecks, evolving connection policies, grid-code requirements, which has fostered new technological trends via spatio-temporal flexibility available through workload orchestration, cooling systems, on-site resources, and energy storage. Second, it maps the evolution of power-delivery architectures from medium-voltage grid interfaces to chip-level, discussing higher-voltage DC distribution, solid-state transformers, wide-bandgap devices, advanced chip-level power delivery, and liquid cooling. Third, it establishes a three-level stability framework spanning rack-level DC-bus dynamics, facility-level converter interactions, and system-level grid-coupled behavior. The framework connects dominant instability mechanisms, including constant power load effects, impedance interactions, forced oscillations, and operating-mode transitions, with suitable modeling, assessment, and mitigation approaches. Synthesizing these topics, this article highlights grid-to-chip co-design as a central requirement for scalable AI infrastructure, linking computing workloads, power-delivery systems, energy buffers, and grid operation.
comment: The manuscript has been accepted for publication in IEEE Open Journal of Power Electronics
♻ ★ Learning-enabled Acceleration of Scenario-based Model Predictive Control
Scenario-based model predictive control (SBMPC) is a variant of model predictive control (MPC) that explicitly accounts for uncertainty by optimizing control actions over multiple predicted scenarios. However, its computational complexity increases rapidly with the number of scenarios and prediction horizon, limiting its applicability to real-time planning and control. This paper presents a learning-accelerated Alternating Direction Method of Multipliers (ADMM) algorithm for efficiently solving SBMPC problems by leveraging parallel computing and Moreau envelope learning, while maintaining high solution accuracy. We reformulate the SBMPC problems into consensus forms that can be decomposed via ADMM, separating the scenario-dependent dynamics from non-anticipativity constraints and enabling parallel updates across scenarios and time steps. Building on this decomposition, we utilize a learning-to-optimize scheme that leverages Moreau envelope learning of the cost function to accelerate the primal update in ADMM, thereby reducing computation time. The proposed framework is evaluated on a microgrid energy management problem subject to load and renewable generation uncertainties. Comparisons with IPOPT and MadNLP, two popular and modern nonlinear programming solvers, demonstrate substantial computational speedups while maintaining reliable closed-loop control performance.
♻ ★ A graph-theoretic view on the small-signal stability of inverter-based power grids
Dynamic grid stability is traditionally ensured with synchronous generators. Modern grids rely substantially more on inverter-based resources, which require grid-forming control to guarantee adequate system-wide synchronization and stability. Small-signal stability has granted various centralized and decentralized stability certificates --- but these have primarily been limited to sufficient criteria only. In this work, we construct a necessary and sufficient small-signal stability criterion for lossless inverter-based power grids with arbitrary topology. We show that asymptotic stability is equivalent to the positive definiteness of a single matrix that combines network topology, operating point, and effective droop gains. We derive graph-theoretic stability criteria based on an augmented cone graph and show that the contribution of graph cycles, though not itself small, barely affects the certified stability boundary, as illustrated for three IEEE test cases. The resulting framework yields decentralized stability criteria, quantifies the conservatism introduced by decentralization, and may support the development of future grid codes.
comment: 10 pages, 4 figures
♻ ★ Distribution-Free Budgeted Stealthy Attack Scheduling for Remote State Estimation
This letter addresses budgeted stealthy false-data-injection (FDI) scheduling against remote state estimation, where a resource-constrained adversary may corrupt at most a fraction $\barΓ$ of transmissions. Existing event-triggered schedulers invert a Gaussian innovation tail to set the firing threshold and certify stealth by covariance matching; both are exact only under Gaussianity, which real cyber-physical residuals routinely violate. We propose a distribution-free scheduler pairing the worst-case FDI action with a split-conformal calibrated trigger, requiring neither the plant matrices nor any distributional model. We establish exact pathwise stealth against every magnitude-measurable detector, for any firing rule and innovation law; a finite-sample distribution-free bound on the mean firing rate, with almost-sure budget attainment under stationarity and ergodicity; and a steady-state degradation identity linear in a single scalar energy capture $ψ$, maximized by the same order statistic that delivers the budget guarantee. A conditional sign-symmetry condition delimits when the certificate extends to sign-sensitive detectors, the residual exposure being governed by a fourth cumulant that vanishes under Gaussian noise. Monte-Carlo studies and a heavy-duty truck CAN record confirm the bounds and quantify what the Gaussian assumption costs outside its regime.
comment: Accepted and published in IEEE Control Systems Letters (Special Section on Enhancing Trustworthiness of Cyber-Physical Systems). \c{opyright} 2026 IEEE
♻ ★ Sub Terahertz LEO Satellite Communication: Is Doppler Shift a Major Challenge?
In this work, we investigate the impact of Doppler frequency shift on sub-terahertz low Earth orbit (LEO) satellite communications using OFDM transmission. To comprehensively evaluate the severity of the Doppler shift impact on sub-terahertz satellite-to-ground links, we first put together an evaluation framework that jointly accounts for major Doppler-induced and non-Doppler-induced signal impairments in such systems. We then compare and contrast three possible engineering approaches to deal with Doppler shift in such a communication system. Our study concludes that (i)~while just ignoring the Doppler effect in such high-frequency and high-mobility systems substantially compromises the performance and reliability, (ii)~most of the negative effects can be eliminated by implementing only a small set of low-complexity measures. We particularly show that even a single Doppler shift estimation over a 5-min-long satellite pass over the ground user location may be sufficient to ensure high performance throughout the entire connectivity period. Hence, while Doppler shift is an important effect to account for in sub-terahertz LEO satellite communications, it should not constitute a critical design challenge for these emerging systems.
♻ ★ Dynamic Programming for A Linear-Gaussian Dual Control Problem
We consider an infinite-horizon discounted dynamic programming (DP) problem for a countable-innovation belief-state model. The model is a surrogate for a linear-Gaussian dual control problem involving an unknown parameter vector and Gaussian observation noise. The surrogate preserves the first two moments of the Gaussian innovation and the resulting posterior second-moment structure. We allow unbounded belief and control spaces and nonnegative stage costs with quadratic growth in the posterior mean and covariance. Using a posterior-moment weight, we show that the Bellman operator is a contraction in a weighted sup-norm space. We characterize the optimal cost as the unique solution of the Bellman equation in this space, establish the existence of optimal stationary policies. We further prove weighted sup-norm convergence of value iteration, policy iteration, and optimistic policy iteration under vanishing approximation errors. The convergence results under vanishing approximation errors rely on general weighted sup-norm contraction properties and are applicable beyond the particular dual control problem considered here.
♻ ★ MPPI-Generic: A CUDA Library for Stochastic Trajectory Optimization
This paper introduces a new C++/CUDA library for GPU-accelerated stochastic optimization called MPPI-Generic. It provides implementations of Model Predictive Path Integral control, Tube-Model Predictive Path Integral Control, and Robust Model Predictive Path Integral Control, and allows for these algorithms to be used across many pre-existing dynamics models and cost functions. Furthermore, researchers can create their own dynamics models or cost functions following our API definitions without needing to change the actual Model Predictive Path Integral Control code. Finally, we compare computational performance to other popular implementations of Model Predictive Path Integral Control over a variety of GPUs to show the real-time capabilities our library can allow for. Library code can be found at: https://acdslab.github.io/mppi-generic-website/ .
comment: Fixed typo in Jetson Orin Nano results in table
♻ ★ Reliable and explainable analysis of AVS/RS by Statistical Model Checking and Process Mining
Automated Vehicle Storage and Retrieval Systems (AVS/RS) are increasingly adopted to boost warehouse throughput while containing operational costs. We present a tool-supported analysis workflow that combines discrete-event simulation, Statistical Model Checking (SMC), and Process Mining (PM) techniques to quantify and explain AVS/RS performance. SMC, implemented through the MultiVeStA tool, delivers statistically reliable estimates of key indicators, such as cycle times and inventory consolidation, across alternative stocking and picking policies with user-specified confidence levels. MultiVeStA orchestrates multiple independent simulations in a black-box manner, enabling automated verification. PM ingests the resulting event logs to uncover the behavioural patterns and bottlenecks that drive those indicators, closing the interpretability gap typical of simulation studies. We instantiate the framework on a new open-source AVS/RS simulator configured according to recent industrial guidelines and evaluate multiple control strategies and different scenarios. Results show that the coordinated use of SMC and PM supports both predictive what-if analyses and post-hoc diagnostics. To the best of our knowledge, this is the first application of a combined SMC-PM workflow to AVS/RS operations, complemented by a reusable open-source simulation and analysis toolchain.
comment: 66 pages, 11 figures. Author accepted manuscript for Expert Systems with Applications, revised following peer review. Replaces the original pre-review version
♻ ★ Equation-Free Digital Twins for Nonlinear Structural Dynamics
Monitoring floating offshore wind turbines (FOWTs) under stochastic excitation and sensor failure requires computationally efficient and physically interpretable estimators. This paper introduces an equation-free, output-only digital twin framework combining Hankel dynamic mode decomposition (Hankel-DMD), sparse-sensor reconstruction, and a stochastic Kalman extension. A sufficient identifiability condition is derived for modal amplitudes recovered from delay-embedded measurements, while Tikhonov regularization addresses ill-conditioned sensor layouts. Correlation-domain identification improves damping estimates under stochastic forcing and distinguishes structural modes from rotor harmonics. The framework is validated using OpenFAST simulations of an NREL 5-MW spar turbine and experimental basin-test measurements from a 1:70-scale IEA 15-MW semi-submersible. In simulations, the identified tower modes agree with established operational modal analysis methods (MAC at least 0.99), while the Kalman estimator achieves R-squared values of 0.91 to 0.99 under severe sensor sparsity. Experimental results identify tower modes at 0.430 and 0.415 Hz with MAC of 1.00 against SSI-COV and reconstruct tower-base bending moments on unseen runs with R-squared values of 0.83 to 0.89 using three of four sensors. The framework provides a computationally efficient, sensor-failure-resilient structural-state estimation layer for digital twins.
comment: Added more details of the theory as well as real experimental validation also included in the results. Now the manuscript is more defensible
♻ ★ ReLU Networks for Model Predictive Control: Network Complexity and Performance Guarantees
Recent years have witnessed a resurgence in using ReLU neural networks (NNs) to represent model predictive control (MPC) policies. However, determining the sufficient network complexity to ensure closed-loop guarantees remains a fundamental open problem. This involves a critical precision-complexity trade-off: undersized networks may fail to capture the MPC policy, while oversized ones may outweigh the benefits of ReLU network approximation. In this work, we project the MPC actions onto a tightened constraint set to satisfy the hard constraints, and establish a state-dependent Lipschitz continuity property for the optimal MPC cost function, which enables sharp convergence analysis of the closed-loop system. For the first time, we derive explicit bounds on ReLU network width and depth for approximating MPC policies with closed-loop guarantees. To reduce network complexity for the convergence to user-specified target sets, we propose a non-uniform error framework with a state-aware scaling function to adaptively adjust both the input and output of the ReLU network. Our contributions provide a foundational step toward certifiable ReLU NN-based MPC.
♻ ★ Disturbance-Observer-Based Grid-Forming Control for Unbalanced Grids
This article proposes a grid-forming control method for operation under unbalanced grid-voltage conditions. The method regulates the positive-sequence active power delivered to the grid and actively suppresses the negative-sequence converter voltage, thereby maintaining a constant converter-voltage magnitude during unbalanced faults when the converter operates within its physical limits. Only the converter current is measured on the AC side, and a disturbance observer is used to synchronize with the grid and to provide integral and resonant action. Estimates for the positive- and negative-sequence grid voltage are obtained from the disturbance observer. A current-limitation scheme for both balanced and unbalanced faults is integrated. Comprehensive stability analysis and tuning guidelines are provided. Experimental results using a 12.5-kVA converter demonstrate that the proposed method can operate during severe balanced and unbalanced faults.
♻ ★ Proportional Neuro-Impulsive Control of SISO LTI Systems
This paper explores a controller that is at the crossroads of neuromorphic, i.e., spike-based, and impulsive control systems. The controller consists of a single integrate-and-fire-like neuron that generates an input proportional to the system output when the neuron variable reaches a threshold. We prove that any static output-feedback stabilizable single-input single-output (SISO) linear time-invariant (LTI) plant can be made semi-globally asymptotically stable (sGAS) in closed loop with the proposed neuro-impulsive controller. This improves on existing fixed-amplitude neuromorphic controllers, which only achieve practical stability, but at the price of using graded spiking inputs. Moreover, we provide an algorithm for choosing appropriate controller parameters given a set of admissible initial conditions. A numerical example of an inverted pendulum linearized around its upright equilibrium illustrates the method.
comment: 8 pages, 1 figure, 1 algorithm
♻ ★ Hessian-Informed Flow Matching
Data from physical systems often concentrate near stable states corresponding to minima of an energy function. Around these states, the curvature of the energy landscape shapes the local covariance structure of the data, which is often highly anisotropic: small perturbations around a data point raise its energy more sharply in some directions than in others. However, standard flow matching commonly connects noise to target data through isotropic conditional paths that do not explicitly follow this physical structure. We introduce Hessian-Informed Flow Matching (HI-FM), which uses the energy Hessian at each data sample to encode this local curvature in the conditional probability paths used for training. By locally linearizing energy-driven dynamics, we derive anisotropic paths that move samples toward their targets at different rates in different directions, without changing the standard flow matching objective. We evaluate HI-FM through molecular generation on Hessian-QM9 and GEOM-Drugs, using SemlaFlow as a shared backbone. With Hessians calculated offline only for the training data, across both datasets, HI-FM improves the physical and geometric quality of generated molecules while maintaining comparable chemical validity. Under a matched optimizer-update budget, HI-FM trained with only $10\%$ of Hessian-QM9 achieves lower mean energy, strain, and bond-length $W_1$ than the full-data SemlaFlow baseline. The results support local energy curvature as a useful inductive bias for flow matching.
♻ ★ Open-Source Multi-Fidelity EMT Modeling of Data Center Loads for Power System Studies
Characterizing data center-grid interactions under rapid load fluctuations requires electromagnetic transient (EMT) models that capture fast converter and control dynamics. This paper develops a modular multi-fidelity EMT model of data center loads in PSCAD. The modular design supports flexible assembly of subsystem models for different power supply configurations. The multi-fidelity design incorporates switching models, average-converter models, and reduced-order models to balance simulation accuracy and computational efficiency according to analysis needs. The model also provides a systematic controller tuning procedure and supports control mode switching. Comparative simulations demonstrate how this data center load model responds to both grid-side and load-side disturbances and assess the accuracy and computational efficiency of models with three fidelity levels. The model is publicly available on GitHub.
♻ ★ A Nested Heterogeneous Multiscale Framework for Power System Dynamic Simulation
Power system dynamics involve multiple timescales, from electromagnetic transients of the network and inverter-based resources to electromechanical oscillations of synchronous generators and quasi-steady-state evolution of the operating point. Simulating these dynamics with a detailed electromagnetic transient model is computationally prohibitive, whereas existing methods reduce the model to focus on a single timescale and lose accuracy when dynamics of different timescales interact. This paper proposes a nested heterogeneous multiscale framework for highly efficient multi-timescale simulation of a power system represented by one unified detailed model. Rather than varying only the integration step, the framework adaptively shifts the simulation resolution among three nested levels so that the dynamics of interest at each stage are always captured, with the slower dynamics estimated on the fly from short detailed simulations at controlled accuracy. Theoretical analysis proves the stability of long steps during quasi-steady-state evolution and quantifies the attainable speedup. On a modified IEEE 39-bus system with frequency regulation and load restoration, the framework accurately reproduces the dynamics of all timescales and runs 15 to 18 times faster than conventional simulation of the detailed model. The accuracy and speedup of the proposed method are further verified on a 390-bus system, demonstrating its scalability to large power systems.
comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible
♻ ★ Multiobjective optimization-based design and dispatch of islanded, hybrid microgrids for remote, off-grid communities in sub-Saharan Africa
Sizing an islanded hybrid microgrid requires evaluating candidate designs through a year-long hourly simulation. The resulting objective is non-smooth, which limits gradient-based methods and requires approximation for mixed-integer linear programming. We size a solar photovoltaic (PV), wind, lithium-ion battery and diesel microgrid for an off-grid community in Timbila, Kenya, using five objectives: levelized cost of electricity (LCOE), emissions, unmet load, dumped energy and fossil share. We benchmark eight solvers, including particle swarm optimization (PSO), over 30 runs each with a budget of 2,000 evaluations. PSO, the PSO--pattern-search hybrid, pattern search and surrogate optimization finish within 0.5% of the exhaustively computed integer optimum in all 30 runs; PSO is fastest (expected running time 2.9~s), and gradient-based multistart methods perform worst. We then approximate the Pareto front, select designs with knee-point and compromise rules and the technique for order of preference by similarity to ideal solution (TOPSIS), trace a cost--reliability front over the generator rating, and re-rank candidates over 500 Monte Carlo resource and load scenarios. The equal-weight design, 73~kW of PV with a 239~kWh battery and no wind turbines, supplies 97% of energy from renewables at 0.184~USD/kWh, 63% below diesel-only supply. The selected design that serves all load uses a 12.2~kW generator, and accepting 1% unmet load lowers LCOE to 0.163~USD/kWh. Under uncertainty, a design with a 31% larger PV array lowers the 95% conditional value-at-risk of LCOE by 10% for a 2% higher expected cost. LCOE is most sensitive to the interest rate over the ranges tested.
comment: Under revision
♻ ★ Quantifying the power grid hosting capacity gained by coordinating distributed solar, battery storage and electrified flexible loads
Distribution grid hosting capacity is usually assessed with static, single-resource methods that treat distributed energy resources as fixed injections, which can understate how much solar generation and electrified load feeders can accept. We quantify the hosting capacity gained when solar photovoltaics (PV), battery storage, heat pumps and electric vehicles (EVs) are dispatched by network-aware coordination at 5-minute resolution, versus static operation of the same devices. On the IEEE 123-node feeder, we use a deterministic iterative method, a two-stage stochastic program that optimizes siting, and a Monte Carlo extension over 100 scenarios. With storage, heat pumps and EVs at 5%, coordination raises PV hosting capacity from 48% to 83% of peak load; with 10% storage, 20% PV and 5% EVs, heat pump hosting capacity rises from 9% to 55% of homes. When siting is co-optimized, PV hosting capacity rises from 24% to 98%, and the jointly feasible PV, heat pump and storage region grows about 22-fold. Coordination turns storage and heat pumps from substitutes into complements: with storage up to 100% of peak load, the feeder hosts PV up to about 200% of peak load and heat pumps in 90% of homes. Midday limits are relieved by storage charging and heat pump pre-cooling, evening limits by inverter reactive support, battery discharging and heat pump setback. Under input uncertainty, coordination raises mean PV hosting capacity by 64%; the absolute spread widens while the relative spread narrows slightly. At published reinforcement costs, the added capacity is worth roughly $0.6--1.5 million per feeder in deferred upgrades.
comment: Under review
♻ ★ Strategically Robust Linear Quadratic Dynamic Games
We study linear quadratic dynamic games where players are uncertain about each other's control policies or goals and consequently seek to be strategically robust. Building on recent work on strategically robust and risk-averse game theory, we first formalize the problem of strategically robust linear quadratic dynamic games. We show that these can be rewritten as simple transformations of linear quadratic games in which each player chooses a controller in a fictitious game in which they are faced with an adversary who is penalized for deviating from the other players' policies. This formulation naturally induces a novel notion of dynamic equilibrium, which we call a strategically robust dynamic equilibrium. We establish existence and uniqueness (in the subgame-perfect sense) of such equilibria and show that the equilibrium policies are Markovian, linear, and can be efficiently computed via coupled backward Riccati equations. Through numerical simulations, including experiments in a network game, we illustrate the benefits of strategic robustness in designing robust and resilient decentralized control schemes. Our experiments also expose a "free-lunch" phenomenon in games in which robustness does not incur a corresponding loss in performance but can yield improvements in players' utilities and social welfare.
comment: 6 pages, 5 figures, 2 tables. Accepted to the 2026 IEEE Conference on Decision and Control (CDC)
♻ ★ DOE-Constrained Distributed Battery Coordination in Unbalanced Distribution Networks
Dynamic operating envelopes (DOEs) provide injection and withdrawal limits for distributed energy resource scheduling while respecting network operational constraints. This paper develops a risk-aware DOE framework for distributed battery operation in unbalanced three-phase networks using a semidefinite program (SDP). We derive sufficient conditions for exactness of the SDP relaxation, ensuring that the recovered operating points corresponding to the DOE vertices satisfy all voltage and nodal power injection constraints of the original AC network model. The method computes hourly active-power DOEs under uncertain load conditions and incorporates them directly into a 24-hour battery operation problem. On a modified IEEE 13-bus feeder, empirical comparisons with unconstrained arbitrage and local Volt--VAR/Volt--Watt control show that DOE-constrained battery scheduling eliminates the observed out-of-sample voltage violations while retaining 94.6\% of the unconstrained arbitrage profit.
comment: 8 pages; substantially revised version with a new title, revised formulation and theoretical results, and updated numerical results
♻ ★ On Fast Attitude Filtering Using Matrix Fisher Distributions with Stability Guarantee as an Observer
This paper addresses two interrelated problems of the nonlinear filtering mechanism and fast attitude filtering with the matrix Fisher distribution (MFD) on the special orthogonal group. By analyzing the distribution evolution along Bayes' rule, we reveal two essential properties that enhance the performance of Bayesian attitude filters with MFDs, particularly in challenging conditions. Benefiting from the new understanding of the filtering mechanism associated with MFDs, two closed-form filters with MFDs are then proposed. These filters avoid the burdensome computations in previous MFD-based filters by introducing linearized error systems with right-invariant errors but retaining the two advantageous properties. The proposed filter with right-invariant error is shown to be almost globally asymptotically stable for any trajectory on $SO(3)$ as a deterministic observer, leveraging its Wahba-type update and right-invariant construction. Moreover, we further prove the local exponential stability of the filter for single-axis rotations to reveal the effect of the two properties on the convergence rate. These stability results support the performance of the proposed filter with large initial error from a theoretical viewpoint, which to our knowledge, is not achieved by existing directional statistics-based filters. Numerical simulations demonstrate that proposed filters are as accurate as recent MFD-based Bayesian filters in challenging circumstances but consume far less computation time (by up to about two orders of magnitude compared with previous MFD-based attitude filters).
♻ ★ EMT Assessment of SCR and X/R Effects on Current-Priority-Dependent LVRT Response of a Grid-Following BESS
Low-voltage ride-through (LVRT) forces a grid-following inverter to allocate finite converter current between active- and reactive-current objectives. This paper uses a PSCAD electromagnetic-transient (EMT) representation of an aggregated battery energy storage system (BESS) to quantify how short-circuit ratio (SCR) and grid impedance X/R ratio alter that tradeoff. Active-current priority and reactive-current priority are compared for five paired grid conditions under the same 150-ms balanced three-phase fault, with pre-fault active power approximately 0.84 pu, POI reactive-power exchange near 0 pu, and a 1.0-pu dq-current limit. Fundamental positive-sequence voltage and power are reconstructed from 100-$μ$s three-phase EMT outputs. Across the fixed-fault SCR sweep, the Q-priority late-fault voltage advantage increases from 0.046 pu in the strongest tested grid to 0.129 pu in the weakest; the weakest P-priority case also shows active-power reversal and a pronounced controller-frequency excursion. At fixed SCR, changing X/R produces a smaller increase from 0.092 to 0.112 pu, mainly through reduced P-priority voltage support. Because the common physical fault also deepens the passive retained-voltage condition as SCR decreases, the SCR result is interpreted as a fixed-fault grid-strength sensitivity rather than a constant-sag comparison. Grid strength is the larger sensitivity in the tested matrix, while X/R remains an additional modifier under the studied fault conditions.
comment: 6 pages, 4 figures. Accepted for presentation and publication at the 2027 IEEE Texas Power and Energy Conference (TPEC 2027)
Robotics
★ A human-in-the-loop Control Paradigm for Construction Robotics: From VR-Guided Teleoperation to Semi-Autonomous Dual-Arm Operation
Construction tasks in hazardous and unstructured environments require both human adaptability and robotic precision, yet existing approaches often treat teleoperation and autonomy as separate control modes. This paper presents a human-in-the-loop control paradigm that enables structured transitions among VR-guided teleoperation, semi-autonomous shared control, and rule-based autonomous execution for contact-rich dual-arm construction tasks. The proposed system combines immersive VR guidance, a manually defined library of reusable skill primitives, and a shared-autonomy framework with explicit human-robot role allocation and command-priority rules. This integration allows operators to retain supervisory and corrective authority while delegating repetitive and precision-critical actions to the robot. The paradigm is validated on a dual-arm platform through representative facade-installation tasks. Results show that autonomous execution provides the best performance under well-calibrated nominal conditions, whereas semi-autonomous control improves resilience when environmental disturbances exceed the predefined autonomous pipeline's correction capability. Semi-autonomous control also substantially reduces subjective operator workload compared with full teleoperation. These findings demonstrate that effective construction-robot control should adapt human involvement to task and environmental uncertainty, using autonomy for repeatable operations and shared control for disturbance recovery.
comment: under review
★ RedirectMimic: Local Motion Content Redaction for Humanoid Whole-Body Control
Humanoid motion interfaces should accommodate application-specific movement constraints while retaining useful reference behavior. Local content redaction requires a controller to replace selected movements even when it continues to receive the original reference. Removing the tracking incentive for those movements leaves their replacement unspecified. We introduce RedirectMimic, which separates the reference observed by the actor from the target used to train its execution. A rule-generated edited reference guides tracking rewards during proximal policy optimization, while the deployed observation and action interfaces remain unchanged. We evaluate pose- and speed-defined arm editing in Unitree G1 simulation using a tracker pretrained on 7,509 motion clips. On the overhead-arm task, RedirectMimic reduces core-window content occurrence to 2.7%, compared with 58.9% for reward zeroing and 32.8% for anti-tracking. It achieves 94.2% core-window joint success, combining content suppression, fall-free execution, and local retention. On the speed-defined task, fast outward arm movements occurred at similarly low rates under all three local editing methods. Across both tasks, RedirectMimic achieved the lowest replacement error among these methods and tracked unedited motions with accuracy comparable to that of the pretrained controller. Project page: https://xiaopeng-whu.github.io/RedirectMimic/.
comment: 9 pages, 4 figures, 4 tables. Project page: https://xiaopeng-whu.github.io/RedirectMimic/
★ LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning
World Action Models (WAMs) have recently emerged as a promising paradigm for robotic manipulation, endowing policies with strong capabilities for modeling temporal and physical dynamics. Existing WAMs, encompassing both video-first action prediction and joint video-action modeling, directly transfer global physical representations from world models to action generation. However, global scene representations may underemphasize local robot-object interactions, creating a mismatch between representations optimized for world evolution and those required for manipulation. To bridge this gap, we propose LaST0.5, a novel WAM that introduces an interaction-centric physical representation between world modeling and action generation. Specifically, LaST0.5 learns a compact interaction-centric latent space under the supervision of visual interaction cues between the robot and the environment. This reasoning space is instantiated as a latent expert that bridges the world expert and the action expert within a Mixture-of-Transformers architecture. Furthermore, LaST0.5 performs adaptive world-to-action transfer by jointly predicting future representations and progress values, using progress consistency to adjust action execution and trigger future resampling. Finally, we pretrain LaST0.5 on a large-scale robotic data corpus comprising over 14k hours of data, establishing interaction-centric physical reasoning as a scalable and effective paradigm. LaST0.5 achieves 99.5% on LIBERO, surpasses the previous state of the art on ten evaluated RLBench tasks, and demonstrates robust long-horizon execution and precise dexterous manipulation in real-world experiments.
★ HiPFuse: Real-Time LiDAR Semantic Segmentation for Robotic Perception on Edge Devices
Real-time robotic perception requires accurate LiDAR semantics under limited resources. Bird's-eye-view (BEV) aggregation is efficient but can mix differently labeled surfaces in columns. We present HiPFuse, a point-BEV fusion framework that exchanges point and height-partitioned BEV features across horizontal scales, combining point-wise detail with 2D spatial context from one LiDAR scan. Fixed height assignments preserve separate vertical aggregation groups, with auxiliary supervision of initial BEV features using targets from the same 3D cells. A dual-scale BEV context module adds broader context from a coarser grid. On SemanticKITTI and MCD, HiPFuse achieves 63.7% and 62.5% mIoU at 14.2 and 22.4 FPS on Jetson AGX Orin, respectively, exceeding the 10 Hz scan rate. Separate training and evaluation show indoor applicability. Replacing only the baseline semantic front end with HiPFuse in fixed robotic systems reduces odometry absolute trajectory error RMSE by 10.6% on average across two outdoor sequences and increases accepted-loop recall by 8.7 percentage points on SemanticKITTI. Qualitative semantic maps show more continuous road labeling. Our code is available at https://github.com/haiyang2022/HiPFuse.
comment: 13 pages, 6 figures
★ Optimization-Based Path Planning for Industrial Manipulators: Experimental Validation in Additive Manufacturing
Robotic additive manufacturing requires joint paths that reconcile robot feasibility with process-specific geometry. We develop an offline optimization-based path planner that combines joint-path regularity and self-collision penalties with process costs and constraints, so that a new process enters through its own reference, residuals, and tolerances. Full-path and boundary-coupled chunked optimization handle references of different lengths. Iterative Weight Tuning (IWT) determines the cost weights from the gaps between Key Performance Indicators (KPIs) and their targets. In carbon-fiber winding and robotic 3D printing, the planned paths meet the geometric tolerances and all KPI targets, and both paths are executed on a UR10e, without extrusion for printing, with joint-level tracking recorded during execution.
comment: Submitted for the review in ECC 2027
★ TransGHOST: Real-Time, Synthesis-Free Reconstruction of Instrument-Occluded Regions in Ophthalmic Microsurgery
Intraoperative scene perception in ophthalmic microsurgery is limited by instrument occlusion: when a tool enters the high-magnification field, it conceals tissue from both the surgeon and the perception components of robot-assisted systems that process the video. Generative inpainting can remove the occluder but replaces the missing image content with synthesis, which is unsuitable when visual provenance matters. Many video inpainters also depend on future frames and cannot operate online. We present TransGHOST, a causal reference-guided reconstruction pipeline whose non-generative safe mode operates in real time. Every restored value is obtained by resampling a recorded frame using geometric transforms fitted only to visible regions. The system provides per-pixel confidence and provenance and marks unsupported pixels as abstentions. We evaluate TransGHOST on a new veterinary clinical corpus of canine cataract surgery with pixel-level instrument and anatomy labels. Only the occluded region is scored, and each scoring target is verified to differ from the reference supplied to the system. Safe mode reaches 31.89 dB masked PSNR and 0.915 masked SSIM at 51.7 ms median and 85.5 ms 95th-percentile latency. On a pre-declared held-out cohort using instrument cutouts excluded from segmentation training, it reaches 29.83 dB masked PSNR, while matched-mask evaluation on validation shows a 3.45 dB advantage over LaMa. Region-aware reference selection improves masked PSNR by 3.60 dB over recency when candidate references can themselves be occluded. Reference-age analysis further defines the operating interval for the brief, repeated occlusions targeted by TransGHOST.
comment: 21 pages, 10 figures, 11 tables
★ ReACT: Reflection-Consistent Modulation and Cross-Load Consistency for Humanoid Velocity Tracking Under Asymmetric Loads
In recent years, humanoid loco-manipulation has progressed to whole-body grasping, lifting, and transport. A basic requirement underlying these tasks is carrying the object to its destination, which depends on the locomotion controller accurately executing velocity commands from the navigation policy. However, carrying an object in one hand introduces a persistent asymmetric upper-body load that can degrade velocity tracking. Persistent tracking biases can accumulate into position and heading deviations, increasing the risk of collisions and navigation failure. To address this problem, we present ReACT, a framework for accurate velocity tracking under asymmetric upper-body loads. Reflection-consistent disturbance modulation uses the estimated wrench to modulate actor features for load compensation while preserving action correspondence under reflected observations. Cross-load tracking consistency training compares loaded and unloaded conditions under the same velocity commands to promote consistent velocity tracking across loading conditions. Simulation experiments on a Unitree G1 humanoid robot show improved velocity tracking and reduced directional bias under varying asymmetric loads. Real-world experiments demonstrate collision-free goal reaching in all 40 tested navigation trials under unilateral hand loads with a fixed mapless navigation policy.
★ Persistent Metric-Semantic Maritime Scene Memory from Open-Vocabulary 3D Scene Graphs
Three-dimensional scene graphs (3DSGs) can provide robots with persistent metric-semantic representations for spatial reasoning, but the existing methods have primarily been developed for indoor and terrestrial environments. In this paper we investigate how a representative outdoor open-vocabulary method transfers to maritime environments and which adaptations are needed. Our experiments show that zero-shot representative outdoor 3DSG descriptions agree with our labeled ground truth for a majority of geometry-supported objects on our data, but the resulting map exhibits considerable physical-entity fragmentation, and its terrestrial higher-level assumptions don't capture maritime spatial organization needs. We therefore keep the underlying mapping method and post-process its accumulated evidence. We show a metric-semantic consolidation method followed by geometry-grounded multi-view vision-language model (VLM) reasoning that reconciles remaining fragments into physical entities and groups entities through observations into local places. In parallel, we construct maritime macro regions using map priors and onboard water and structure observations, and associate them with the local places. The resulting representation reduces physical-entity fragmentation beyond both the baseline and our geometry-semantic consolidation component alone, supports correspondence of fixed local places across repeated maritime traversals, and improves contextual retrieval on our reviewed queries over tested baselines. These results show how an outdoor open-vocabulary map can be adapted into persistent maritime scene memory with our methods for object/place identity and spatial abstractions.
comment: Website: https://kimachristensen.github.io/maritime_3dsg
★ Learn-To-Propagate: Distributionally Robust Motion Planning for Nonlinear Uncertain Systems
Motion planning for stochastic systems aims to find a sequence of control inputs that drives a system to a goal set while avoiding unsafe states with high probability. Existing approaches, however, are either restricted to linear dynamics with perfect knowledge of the underlying distributions or lack formal correctness guarantees. To address these limitations, we propose \emph{Learn-To-Propagate Distributionally-Robust Forest (L2P-DR-Forest)}, a distributionally-robust sampling-based planning algorithm for \emph{nonlinear} stochastic systems with \emph{unknown} initial state and process noise distributions. Our approach leverages recent advances in optimal transport to learn ambiguity sets that contain the true system distributions with high confidence. We develop an efficient method for propagating these sets through the system dynamics and for verifying their safety. We then integrate these techniques into a novel sampling-based planning algorithm to grow a forest of search trees in the system's state space while accounting for and mitigating the growth of ambiguity-sets radii. We prove that the algorithm is correct, i.e., every returned plan satisfies the specifications with the prescribed confidence, and, under some feasibility and distributional assumptions, is probabilistically complete. Evaluations on several robotic benchmarks demonstrate that our algorithm significantly outperforms existing methods while providing formal guarantees with high confidence.
★ Taxonomy-Agnostic Neurosymbolic Grounding of Robot Kinematics
While symbolic reasoning for task allocation requires grounding a robot's physical capabilities, standard Unified Robot Description Format (URDF) files provide only low-level kinematics without the formal semantic context needed for reasoning. We present URDF2Ontology, an automated, taxonomy-agnostic neurosymbolic framework that transforms raw URDF files into populated, grounded OWL ontologies. It leverages a Large Language Model (LLM) for targeted semantic reasoning, coupled with an iterative sequential graph-purging mechanism to prevent context window saturation. To process complex kinematic structures, it employs a morphology-aware dual-track architecture featuring a bottom-up Tree Track for branching assemblies and a top-down Chain Track for serial limbs. Guided by neurosymbolic guardrails, the framework extracts functional sub-graphs and asserts populated ontologies aligned with arbitrary user-provided taxonomies. Empirical evaluation across a diverse fleet of heterogeneous robots demonstrates strong schema alignment, valid structural extraction, and a significant reduction in manual domain authoring overhead.
★ Oscillatory Processing Units as a computing substrate for biomimetic intelligence
Biological intelligence emerges from neural activity that connects sensation, memory and action. Synaptic plasticity, neuronal assemblies and oscillations provide principles for biomimetic computing, but translating them into reusable operations requires a defined computational unit. We introduce Oscillatory Processing Units (OPUs), groups of synthetic neurons sharing a rhythm. Their directed interactions transform incoming activity and prior receiving state into an Oscillatory Occupancy Space (OOS), which becomes available to subsequent computation. We show that source--receiver conditions and oscillatory organization shape these transformations, including their geometry, persistence and historical accessibility. An Atlas of measured relationships supports prospective composition with predicted response classes. Configured circuits perform logical and numerical operations, express associative recall and generate closed-loop decisions with fixed synaptic weights. OPUs and OOS establish a computing substrate for biomimetic intelligence in which experience remains part of the state through which new input becomes action.
comment: 39 pages, 7 figures, 2 tables. Supplementary Information included. Ancillary files: Supplementary Video 1 and source data
★ Game-Enhanced Embodied Learning with Event and Foresight Pretraining
Human gameplay supplies paired observations, actions, and visual consequences that can supplement robot-policy training. We study event-and-foresight pretraining for InternVLA-A1.5, combining recorded keyboard/mouse event prediction with history-conditioned visual supervision through a frozen video model. Coverage-preserving anchor selection constructs one-second targets, and an attention mask prevents teacher-forced future actions from entering foresight queries. The selected corpus contains 394 recordings across 20 games, with 223.36 hours of recorded timestamp span and 2,072,084 training anchors. After 12,400 game-pretraining updates, we compare the resulting initialization with the public base under the same suite-specific robot fine-tuning protocol. Across four LIBERO suites, aggregate success increases from 1,718/2,000 (85.90%) to 1,811/2,000 (90.55%), a gain of 4.65 percentage points. Suite gains are 9.2, 6.8, 0.2, and 2.4 points on Spatial, Object, Goal, and Long. The 40-task breakdown shows 24 improvements, 10 declines, and six ties. These results support the practical value of the complete game-pretraining stage in this evaluated setting, while revealing heterogeneous task outcomes. The comparison includes additional optimization and does not isolate the effects of game data, the two objectives, or training variability.
comment: 25 pages
★ WAM is capable in BEHAVIOR-1K: the 2nd-place solution to the 2026 BEHAVIOR Challenge
This work presents the 2nd-place solution (Team IF1001) to the 2026 BEHAVIOR Challenge, which evaluates mobile manipulation across 100 household tasks. Our method is built on a powerful world action model (WAM), Cosmos 3. We improve Cosmos 3 by using richer visual history, task-progress conditioning, and task-group and skill-specific LoRA adapters. During inference, we apply a separate progress estimator to supply task-progress context, and use task- and progress-conditioned routing to select a task-group or skill-specific policy, complemented by action-candidate aggregation, rolling inpainting, and task-specific temporal and gripper control. To meet the minimum inference requirement of 1 FPS, we use neither recursive self-improvement (RSI) nor agentic harness methods and rely on no closed-source models; during official private-test evaluation, our system handled concurrent requests across 100 evaluation ports with limited hardware resources, sustaining an average end-to-end inference rate above 2 FPS per port. Our method achieves a mean Q-score of 0.613 and a full-task success rate of 40.9%, averaged over 3 self-evaluation rounds across all 100 tasks. On the official private test set, with 20 instances per task, our system achieves a mean Q-score of 0.615 and a full-task success rate of 41.2%.
comment: 27 pages, 8 figures, 11 tables. Technical report for Team IF1001's 2nd-place solution to the 2026 BEHAVIOR Challenge at CoRL 2026. Challenge website: https://behavior.stanford.edu/challenge/index.html
★ Camera equalisation of an 11.5x drop in environmental illuminance: four imitation-learning policies on an SO-101 arm
Real-robot benchmarks of learned visuomotor policies often record room illuminance as the lighting condition. We show that, on a typical low-cost rig, room light and image light can be almost unrelated. Four imitation-learning policies (ACT, SmolVLA, Diffusion Policy and pi0.5) were trained on the same 50 demonstrations of a pick-and-place task on an SO-101 arm. Each was run for ten rollouts at about 520 lux and again at about 45 lux, an 11.5x reduction. Success was 20/40 in normal light and 19/40 in low light: a change of -2.5 percentage points (95% CI -23 to +19), with no evidence that the policies differed in sensitivity (interaction p = 0.86). The recorded video shows why: mean image brightness was only 3% lower in low light on the front camera and 19% lower on the wrist camera. Both cameras were set to automatic exposure and white balance, which most likely absorbed the difference; exposure was not locked in this experiment. An accidental repeat of one 10-rollout block reproduced its success count (7/10) but matched the first block at only 6 of 10 placements, consistent with chance. We conclude that lux alone does not describe what a policy sees. Camera settings, and image statistics compared with the training recordings, are the quantities to report and control. Data and code are public.
comment: 10 pages, 2 figures, 8 tables. Data and code: https://huggingface.co/datasets/fysical/so101-lighting-autoexposure
★ RiSE-Drive: Risk-Sensitive Long-Horizon Candidate Evaluation for End-to-End Autonomous Driving
End-to-end autonomous driving systems increasingly rely on generating multiple trajectory candidates and selecting one for execution. In this setting, a central challenge is no longer only candidate generation, but how to evaluate candidates according to their long-horizon consequences under uncertainty. Existing methods still mainly use deterministic point scores, heuristic subscores, or single-outcome evaluations, which makes it difficult to capture delayed failures, tail risk, and ranking instability under plausible future perturbations. We present RiSE-Drive, a framework for risk-sensitive long-horizon candidate evaluation in end-to-end autonomous driving. RiSE-Drive first performs candidate-conditioned latent future rollout, then estimates long-horizon outcome distributions instead of single scalar scores, and finally ranks candidates using a tail-risk-aware and perturbation-robust decision criterion. This reformulates candidate scoring from deterministic oracle-style evaluation into distributional outcome assessment with explicit risk control. As a result, the evaluator is encouraged to prefer candidates that are not only strong under nominal conditions, but also reliable under long-horizon rollout and safety-critical deviations. Experiments on the NAVSIM-v1 and NAVSIM-v2 benchmarks, as well as the closed-loop HUGSIM benchmark, show that RiSE-Drive achieves state-of-the-art performance and significantly improves the robustness of candidate selection.
comment: 11 pages, 3 figures, 6 tables, including supplementary material. Published in ACM Multimedia 2026 (MM '26)
★ Softness is something you also need! Why compliance helps control: information metrics of a passive walker with compliant ankle
A great gap in adaptivity, flexibility, and dexterity still exists between intelligent robots, whether they are made of rigid or (partially) compliant components, and their naturally evolved animal and vegetal counterparts. One reason could be of `paradigm' nature. On the one hand, most robots (both rigid and soft) are still designed by following a `Cartesian' approach, with top-down control and a clear mind-body separation. Designing artificial agents on the basis of the principles that we observe in biology is not an easy task. The principles of organization of animals and plants rely on morphological computation and emergence of fit behaviors from loosely coupled networks of embodied agents. Modeling and control require radically new approaches. In this paper we briefly recall and explain some seminal mathematical results linking information metrics and dynamics of the agents, already published years ago; we integrate them with some recent methods developed by us and by the community, we highlight some brief remarks about their relevance and we hint to our current and near future work towards finding concrete and efficient ways for the modeling and control of completely or partially compliant artificial agents and for a better understanding of physical intelligence in nature. We present a technical note summarizing phase-space volume calculations for inverted-pendulum and passive-dynamic walking models, following our proposed framework. Starting from the McGeer compass gait, we introduce knee-ankle compliance, foot-rocker effects, and a Ruina-style compass gait with distributed leg inertia. Standard heel-strike maps are derived and used in the estimation of viable phase-space volumes $W(γ)$, based on a state-dependent Monte-Carlo and Reproducing Kernel Hilbert Spaces methods. We show how the methods based on RKHS allow deeper and more subtle analysis.
comment: main: 11 pages, 6 figures; supplemental: 7 pages, 1 figure
★ From Depth to Safe Depth: Uncertainty-Aware Monocular Perception for Mobile Robot Navigation
Monocular depth foundation models make it tempting to replace depth sensors on mobile robots with a single RGB camera. We show that, used off the shelf, this is unsafe: in a simulated indoor navigation benchmark, a standard Nav2 stack driven by Depth Anything V2 (metric) collides in 63\% of 120 episodes, because the model systematically over-estimates the distance to nearby obstacles. We propose a lightweight plug-in module that turns predicted depth into \emph{safe depth}. A 0.58\,M-parameter head on top of the \emph{frozen} depth model predicts, per pixel, a log-depth correction $μ$ and an uncertainty $σ$, trained with a $β$-weighted Gaussian negative log-likelihood. The navigation stack consumes the one-sided bound $\hat d_{\text{safe}}=d\,e^{μ-kσ}$, where $k$ is set by conformal calibration on a held-out world, and uncertainty also decides which pixels may be treated as floor. The correction halves the depth error (AbsRel $0.262\rightarrow0.134$) and reduces near-range over-estimation from 59.8\% to 12.0\%. In navigation, safe depth raises the success rate from 37\% to 70\% and lowers collisions from 63\% to 3\%, close to the 2\% of ground-truth depth. A constant safety margin calibrated to the same pixel coverage reaches only 30\% success because it freezes the robot; uncertainty-aware safety wins 52 of 56 discordant paired episodes against it. Gains hold on glass, texture-less walls and dark regions.
★ FAME-QP: A GPU-Accelerated Quadratic-Programming Solver for Real-Time Model Predictive Control
We present FAME-QP, a GPU-accelerated solver for dense quadratic programs arising in linear model predictive control. FAME-QP combines constraint-dependent operator representations with fused single- and two-kernel iterations replayed through CUDA graphs, automatically selecting the execution configuration through device calibration and problem-specific timing. It probes a cached operator bank to select the initial penalty and updates operators lazily when problem matrices change. Numerical benchmarks and simulated humanoid and quadrotor control experiments demonstrate reduced latency relative to competing GPU solvers while maintaining solution accuracy and control performance.
★ CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation
Robot demonstrations contain reusable manipulation knowledge, but transferring this knowledge across embodiments requires representations that bridge differences in control interfaces. Action reconstruction alone does not ensure that similar behaviors across robots share a representation. To address this gap, we introduce CIALLO, an action codec that aligns actions, visual changes, and observation-grounded language in a shared space of short-horizon behavioral intent. An embodiment-conditioned decoder maps this intent back to robot-specific actions, allowing a shared representation to support different control interfaces. After pretraining on eight embodiments, CIALLO adapts to a new robot from one recording of at most 10 seconds by updating only its embodiment embeddings. The calibrated codec then remains frozen while a policy learns to predict intent from observations and instructions. We instantiate this policy with Qwen3.5-4B to form Qwen-CIALLO, whose backbone requires no large-scale robot-action pretraining before policy finetuning. With 20 demonstrations per task, Qwen-CIALLO achieves 76.7\% average success across real SO-101 and single-arm Piper robots, compared with 70.0\% for fully finetuned $π_0$. In simulation, we additionally finetune the action decoder, achieving 97.3\% success on LIBERO and 52.7\% on RoboTwin 2.0. The LIBERO policy also achieves 80.8\% on LIBERO-Plus without further finetuning. Together, these results support a shared action codec as a transferable control interface that separates embodiment calibration from task learning.
★ Joint-State Inference for Autonomous Excavators Using Cab-View Vision-Control Fusion
Reliable state feedback of excavator front-end mechanisms is essential for autonomous excavation, yet retrofitting existing machines with dedicated joint sensors, external cameras, or visual markers can increase deployment and maintenance burdens. This paper proposes a closed-loop autonomous excavation framework for continuous inference of excavator joint states using onboard visual observations and control-related temporal information. The framework encodes structural cues from the current cab-view image, models short-term motion trends from historical joystick commands, and introduces a rolling state buffer for the bucket joint to preserve temporal continuity under severe occlusion. Mechanical joint-range constraints are incorporated into the learning objective to suppress infeasible estimates. The current cab-view image and inferred states are supplied to an Action Chunking with Transformers (ACT)-based control policy for online action generation. Experiments in an AGX Dynamics for Unity simulation environment under diverse operating conditions evaluate joint-state inference and end-to-end task execution. Across 20 autonomous excavation trials, the framework achieved a 95% task-completion rate, with an average joint-state-inference-and-control latency of 10.11 ms per step. The bucket-joint inference achieved a mean absolute error (MAE) of 0.965 degrees. These results demonstrate a sensor-light and computationally efficient joint-state-inference approach that can support closed-loop autonomous excavation with reduced dependence on additional sensing infrastructure.
★ Ego Guidance: Enhancing Robot Policy Learning with Action-Free Human Egocentric Videos
In this paper, we investigate how human egocentric videos can benefit robot policy learning without action supervision. Building on a pretrained VLM, we propose Shared Prediction on Action and Representation (SPAR), an MoT architecture in which future visual state prediction can provide independent and complementary learning signals alongside action prediction. We find that representations learned from egocentric videos improve downstream performance on out-of-distribution robot tasks, with further gains as video data scales. We also study the effects of objective choices and data curation, providing insights into how to leverage action-free videos for scalable robot learning.
comment: Project Page: https://llong-cs.github.io/EgoGuidance
★ A computational framework for understanding embodied plans
Reasoning about people's plans and goals based on their actions is fundamental to human social interaction and essential for cooperative, embodied artificial intelligence. Formalizing this capacity in naturalistic, embodied settings has been a longstanding challenge. Here, we present GenEA, a novel agent representation capable of reasoning over multi-step, long-horizon embodied plans. At the heart of GenEA lies a multilevel state-space that couples the coarse biomechanics of an articulated body and its interactions with nearby objects by extending the task-and-motion planning framework from robotics with simulation-based intuitive physics. By embedding this state-space in a generative model, GenEA yields a joint probability distribution over goals and embodied plans. Across several experiments spanning the qualitatively distinct domains of object manipulation and navigation, featuring diverse tasks and multiple behavioral measures, we show that GenEA, through simple transformations of this joint distribution, recapitulates how humans reason about embodied plans. Crucially, GenEA consistently outperforms strong alternative models, including plausible heuristics, standard task-and-motion planning, and a state-of-the-art vision-language model. GenEA thereby offers a new perspective on the architecture of human reasoning and social cognition, suggesting that embodied artificial intelligence systems seeking to seamlessly interact with humans should adopt GenEA-like generative models of action planning.
★ NIF-VIO: Visual--Inertial Odometry Across Inertial and Non-Inertial Reference Frames
Conventional visual--inertial odometry (VIO) assumes that observations from cameras and an inertial measurement unit (IMU) are jointly consistent in an inertial reference frame. However, this assumption breaks down under non-inertial conditions. Inside accelerating or rotating carriers, visual observations of the interior reflect carrier-relative motion, while IMU measurements contain additional carrier-induced components, which can compromise conventional fusion. We therefore present NIF-VIO for localization relative to the local environment across inertial and non-inertial reference frames. It adapts visual--inertial fusion based on measurement consistency. As inertial conditions change, early constraint isolation is combined with local reoptimization to limit error propagation before inconsistency confirmation and repair the affected local geometry. Under non-inertial conditions, a compact model with temporally shared parameters is introduced to account for carrier motion while preserving inertial information that complements visual constraints. Experiments on public benchmarks and a self-collected dataset demonstrate that NIF-VIO achieves competitive performance under inertial conditions and improved localization accuracy and availability under non-inertial conditions.
★ MouldIt: Data Efficiency of Multi-Task Temporal Representation Learning in Injection Moulding
Smart industrial solutions that can address multiple tasks require models that understand the underlying processes. We examine representation learning for modeling the injection moulding process as an example of such solutions. We consider its data efficiency and multi-modal benefit compared to classical machine learning alternatives. To this end, we synthesize a process-informed dataset based on a real reference cycle of an Erlanger Träger part. We then introduce MouldIt, a temporal convolution and attention based variational autoencoder that reconstructs in-mould signals, while classifying parts as acceptable or as one of our synthetic failure modes. We evaluate MouldIt on two tasks, known-class classification and unknown-class detection, across nine dataset sizes in single-modal and multi-modal settings. We compare it with an accept-all baseline and the classical methods k-nearest neighbors, logistic regression, random forests, and principal component analysis, which we adapt to support both tasks. MouldIt outperforms classical methods across both tasks from 4000 cycles in the single-modal setting and from 2000 cycles in the multi-modal setting. It achieves part sorting losses of 6.0 $\pm$ 0.7% and 1.0 $\pm$ 0.3% in the single-modal and multi-modal settings respectively, ranking first across all baselines and a combination of principal component analysis and random forests.
comment: 14 pages, 10 figures, 4 tables
★ An IoT Based Approach for Platform Independent Positioning Service
KAIST Indoor Localization System (KAILOS) was introduced in 2014 and became one of the world-first complete indoor positioning solutions. It is mainly based on the WiFi signal to estimate location and provide positioning service via mobile application that has many restrictions by the Operating System. This study proposes an extended architecture of KAILOS to make use of the IoT device for removing the barriers of the mobile-app-based approach and conveniently providing indoor positioning service. Design of the IoT device that contains hardware and software is also presented. Via this study, we prove that the IoT-based approach is promising and can be a crucial feature to widen applications of indoor localization services.
comment: 8 pages
★ From Egocentric Human Video to Robot Manipulation in the Era of Embodied Intelligence
Egocentric human video (EHV) records close-range hand-object interaction (HOI) and the temporal structure of everyday activities, offering a potentially scalable source of experience for robot manipulation. Using this experience for robot control requires recovering useful supervision without recorded robot actions and accounting for differences in morphology, viewpoint, and action space. We define Ego2Robot learning as the process of learning or adapting robot manipulation policies from EHV, organized into four learning paradigms: explicit embodiment transfer, explicit action and task supervision, egocentric video representation pretraining, and latent action and dynamics modeling. The four paradigms differ in what they derive from human video: robot-format demonstrations, interpretable supervision, visual representations, or predictive models of actions and dynamics. For each paradigm, we trace what EHV information is extracted, how robot policies use it, and when the additional data is required. Besides, we review the benchmark datasets that cover EHV, robot demonstrations, and synthetic and simulated resources. This paper links EHV data's contribution to the task information preserved and the robot-specific grounding used during training or deployment. Finally, we identify challenges in physical feasibility, agent coordination, in-context adaptation, controlled evaluation, and predictive control.
comment: 39 pages, 10 figures. Project page: https://mikewu597.github.io/Ego2Robot_Learning/
★ Few-Shot Residual Adaptation of Pretrained Latent World Models
Pretrained latent world models can support visual planning and control, but their predictions may fail when physical dynamics change at deployment. We study whether such models can adapt from only a few interactions without modifying their pretrained weights or identifying the changed physical parameters. We introduce a few-shot residual adaptation approach that learns self-supervised corrections from latent prediction errors. The corrections are applied both inside the latent predictor and directly to its predicted latent state, while the pretrained world model remains frozen. We evaluate the approach in simulation and on a real robot under center-of-mass and friction shifts. These changes reduce the success of the frozen model to near zero. Using three interaction episodes for adaptation, our dual-site approach recovers 90\% success under the center-of-mass shift and 100\% under the modified friction on the real robot. Ablations show that correcting both locations outperforms single-site adaptation. Our diagnostics further suggest that much of the learned correction reduces a general prediction mismatch, while successful control also depends on shift-specific information captured during adaptation.
★ GEMOR-GS: Global Extrinsic-calibration-refined Mapping and Online Relocalization for 3D Gaussian-Splatting SLAM
Photorealistic and metrically consistent world representations are a primary requirement for LiDAR--inertial--visual SLAM systems that serve robots over continuous operation. 3D Gaussian splatting makes such representations attainable by supplying an explicit, renderable, and queryable scene model. We present GEMOR-GS (Global Extrinsic-calibration-refined Mapping and Online Relocalization for 3D Gaussian-Splatting SLAM) to make the Gaussian map geometrically accurate, globally refined, and re-enterable, providing a better appearance for robot perception and decision-making. To improve the 3DGS map, the LiDAR--camera extrinsic is first estimated offline by direct-method alignment over dedicated calibration segments and reused by all following online and offline processes. Under the calibrated geometry, keyframe poses and Gaussian parameters are refined alternately under a gated differentiable-rendering and LiDAR--projection objective. The final global reconstruction is then rebuilt from the refined keyframe schema with all online constraints lifted. For the robot SLAM lifecycle, the method recovers the pose from the 3DGS map after tracking loss through an XFeat bypass map, RANSAC-PnP coarse localization, and Gaussian RGB-D warp refinement. Experiments on one self-collected and five public sequences quantify each element: the offline reconstruction wins all paired keyframes against the online map by 14.3--19.9 dB, improves PSNR by approximately 9 dB over Gaussian-LIC2, and achieves 86--100% held-out relocalization success tied to extrinsic quality. The results show that GEMOR-GS improves the geometric accuracy, appearance fidelity, and re-enterability of the 3DGS map, serving as a dependable robot world model for SLAM and downstream tasks.
★ Differentiable Filtering for Inertial Odometry
Accurate inertial odometry is critical for autonomous systems but suffers from unbounded drift caused by the integration of sensor noise and biases. In approaches such as TLIO, neural pseudo-measurements are trained independently of their downstream filter. This paper introduces Differentiable Filtering for Inertial Odometry (DFIO), a learning framework that embeds the update stages of an error-state Kalman filter directly into a computational graph. Following standalone pre-training, DFIO jointly optimizes the displacement and covariance output heads using posterior-position supervision through differentiable measurement updates. The proposed system is validated on public pedestrian motion datasets featuring both calibrated and uncalibrated inertial measurements. Experimental results on a pedestrian dataset demonstrate that DFIO reduces mean absolute and one-second relative translation errors by 13.2% and 39.6% relative to the TLIO baseline, and by 7.7% and 17.4% relative to its pre-trained counterpart. Furthermore, evaluations on uncalibrated smartphone inertial data indicate that the proposed method effectively mitigates translational drift and maintains trajectory alignment under highly noisy sensor conditions. Source code will be made publicly available.
comment: 19 pages, 4 figures
★ DeRep: Learning Depth Representations for End-to-End Navigation in Cluttered Environments
End-to-end navigation in cluttered environments relies on compact visual representations to support collision avoidance and goal-directed motion. Existing methods typically learn these representations jointly with the policy or through observation reconstruction, which does not explicitly specify what information the representation should preserve. Moreover, they are usually evaluated with a single downstream policy learner, although the benefit of a representation may depend on how the policy is learned. It therefore remains unclear how different representation-learning objectives affect downstream navigation performance and robustness. This work proposes DeRep, a navigation-oriented representation-learning method that combines local bird's-eye-view reachability supervision with depth reconstruction. To compare representation objectives under controlled conditions, we pretrain encoders with a common architecture and freeze them during downstream learning. We evaluate the representations using a linear policy probe, flow-matching imitation learning, and reinforcement learning with PPO. DeRep achieves the lowest linear policy probe prediction error and outperforms reconstruction alone across all six tested PPO conditions. In imitation learning, both reachability-based objectives achieve higher mean success rates than reconstruction alone, with similar mean success rates between the reachability-only and joint objectives. These results demonstrate consistent benefits of incorporating reachability supervision into depth representation pretraining across different downstream learning methods and evaluation conditions. We further deploy the PPO policy with the frozen DeRep encoder in Gazebo and on a physical quadrotor without fine-tuning, demonstrating its capability for zero-shot sim-to-sim and sim-to-real transfer.
comment: 8 pages, 5 figures
★ Route-Time Conflict Maps for Cooperative End-to-End Autonomous Driving
Existing cooperative end-to-end driving works at Class A (status-sharing) of the SAE J3216 classes of cooperative driving automation (CDA): vehicles exchange what they perceive, not what they intend, so each plans alone when to go, which limits cooperation and leads to deadlock when both vehicles yield or to collisions when both go. We present When2Go, a cooperative end-to-end driving system that learns when to go by sharing intent under CDA Class B or higher. Each vehicle broadcasts its LiDAR detections with forecast motion as instance tokens, together with its plan and intended trajectory. The receiver draws the fused detections, forecasts and partner plans into a route-time conflict map, the occupancy along its route over the planning horizon, on which a planner scores candidate speed profiles. Where two intended trajectories conflict, an order gate, a precedence network distilled from a right-of-way rule, decides who goes first; both vehicles evaluate it on the same messages and so reach the same order. When2Go imitates the rule-based expert of V2Xverse, which sees the simulator's ground truth; stops that When2Go cannot observe within its 2 s planning horizon are relabelled, so that it does not learn to wait without an observable cause. For our runs only, the benchmarks' PID controller steers by When2Go's waypoints instead of the route, and its forced push of long-waiting vehicles is off. With one model, When2Go sets the state of the art on three closed-loop CARLA benchmarks: 97.8 and 90.7 driving score on InterDrive without and with traffic (best published 90.1 and 78.9), 70.0 on the public V2Xverse protocol (best of sixteen rerun methods 59.1) and 88.2 on MDrive (best rerun 85.1), with 95.0% of its vehicles collision-free (87.7%).
★ ExRL: Sample-Efficient Online Adaptation of Long-Chunk Policies via Execution Length Control
Predicting long action chunks, often spanning dozens of timesteps, has become a key ingredient in increasingly capable robot policies. In this paper, we find that adapting these long-chunk policies with existing reinforcement learning (RL) methods can require many online samples due to the high dimensionality of the action chunks being optimized, making such adaptation impractical in real-world scenarios where interaction data are costly to collect. Our key insight is to shift adaptation from optimizing actions to optimizing how long to execute each predicted action chunk. This keeps the adaptation space low-dimensional while its expressivity grows with the chunk size. We thus introduce Execution at the Right Length (ExRL), a simple approach which augments the base policy with a lightweight adapter that predicts the execution length of each chunk and is trained using off-policy RL algorithms. Across simulation benchmarks and real-world robotic tasks, ExRL enables sample-efficient adaptation of a diverse range of long-chunk policies with minimal task-specific hyperparameter tuning.
★ Rhythmic-Semantic Co-Speech Gestures for Emotional Dialogue in Robots
Co-speech gestures can increase engagement and improve speech understanding. Yet, most data-driven robot systems generate either rhythmic beat-like motion or semantically meaningful iconic gestures, but not both. We present an emotion-aware rhythmic-semantic co-speech gesture system that generates iconic and beat behaviors in real time from text and the robot's internal emotional state. We formalize the task of beat-iconic gesture generation using emotion and propose lightweight models that predict iconic timing and intensity, then fill the remaining spans with emotion-conditioned beats at an appropriate intensity level. We train on the BEAT2 dataset and deploy the system on the social robot Haru. In an online user study with 60 participants, our method shows stronger speech-gesture alignment and is preferred over baseline methods.
★ Is Your Robot's Memory Just a Clock? Temporal Shortcuts in Memory-Augmented VLA Policies
Memory-augmented vision-language-action (VLA) policies are meant to remember what happened. Many of them, however, fill memory by time: they sample frames uniformly across the episode and stamp each frame with its time step. Because time passes even when nothing happens, such a memory may not tell waiting from working. We show that it does not. On the counting tasks of RoboMME, we make the robot briefly pause or fidget its wrist. Such a null event changes nothing the task depends on, yet a released memory VLA counts the interval as work done and stops early. We call this failure phantom progress. It cuts success from 47 to 16 of 50 on one task and from 44 to 10 on the other. When the same frames are withheld from memory, success returns, so the cause lies in memory rather than in the interruption itself. We trace the failure to a temporal shortcut: because demonstrations proceed at a steady pace, elapsed time predicts progress. Indeed, rescaling the time stamps alone makes the policy stop early when the clock runs fast and repeat when it runs slow. Surprisingly, removing the time encoding does not cure the failure, because the frames of a null event still occupy slots of the time-sampled memory. Across RoboMME's released designs, vulnerability follows how memory is indexed. Of the fixes the diagnosis suggests, training with injected pauses does not remove the failure and impairs counting, whereas indexing memory by events recovers half or more of the loss without retraining. A memory meant to remember what happened should count events rather than clock ticks.
comment: 38 pages
★ TanCRAFT: Context-Reasoned Anticipatory Feedback for Assembly Tasks
Preventing human error in physical assembly tasks requires assistance systems that can intervene before mistakes are made. This is especially challenging in settings where users are free to choose the construction order and correctness depends on the evolving assembly rather than a fixed next step. We present TanCRAFT, a four-stage real-time predictive assistance system that tracks the scene state, reasons about the current partial assembly, forecasts the near-future motion of manipulated pieces, and intervenes when the predicted placement is incompatible with the target. We instantiate our framework and confirm its effectiveness in a controlled tangram assembly task through offline benchmarks and a within-subject user study. Our findings show that the predictive feedback significantly reduces mistakes relative to unguided and reactive sessions, while yielding the lowest mean perceived workload and shortest mean completion time. Overall, these results demonstrate the value of predictive feedback for error prevention and provide a step toward cost-aware assistance for automation and human-robot workcells.
comment: This work has been submitted to the IEEE for possible publication
★ UGG: Unified Generative Geometry Model NeurIPS 2026
Understanding 4D scenes is a core challenge in visual intelligence, requiring models to jointly capture scene geometry, appearance, and dynamics from partial observations. However, existing approaches are subdivided into feedforward models for direct geometric prediction, and generative models for visual synthesis. We introduce UGG, a unified framework for 4D scene understanding that jointly models RGB video and geometric representations such as depth and camera pose. By assigning different noise levels across modalities and time, UGG formulates visual generation and geometric prediction as conditional completion within a single generative model. This flexible formulation enables the model to learn from diverse, heterogeneous datasets with different subsets of geometric and visual annotations. Furthermore, our generative formulation enables inference-time search over multiple plausible scene hypotheses, using geometric evidence extracted from observed RGB frames to improve geometric prediction without ground truth geometry. We demonstrate UGG's effectiveness over non-unified baselines across camera-controlled video generation, depth estimation, and camera pose prediction, including in challenging sparse observation settings. Code and visualizations are available at https://ugg-model.github.io/
comment: Accepted at NeurIPS 2026. 31 pages, 7 figures, 6 tables. Project page: https://ugg-model.github.io/
★ One Policy for 100 Household Tasks: Our Solution for the 2026 BEHAVIOR Challenge
We describe our solution for the 2026 BEHAVIOR Challenge: 100 long-horizon household tasks in the OmniGibson simulator. We placed 9th on the final leaderboard, the only independent team in the top 10. The core of the solution is a single vision-language-action (VLA) model, one checkpoint for all 100 tasks, built on our 1st-place 2025 solution, which is based on $π_{0.5}$. On the 50 tasks shared with 2025, its Q rises from 0.260 to 0.371; across all 100 tasks it scores 0.392. Our main conclusion from 2025 was that the model was undertrained, so most of the 2026 work went into training the model for longer, optimizing some training parameters, making each training step slightly faster and reducing the gradient noise: a migration from 50 to 100 tasks (semantic stages built from the demonstrations' skill annotations, task and stage embeddings of new tasks warm-started from similar old ones, and a 21-dimensional action space matching the teleoperation controls); variance-reduced multi-sample flow matching with a simple noise-vs-compute model that picks the number of action-expert samples per observation without ablations; and a 200k-step training run, about one pass over the 211M frames of the demonstrations. A cloud evaluation infrastructure keeps a shared record with provenance, episode diagnostics and restorable simulator states, and runs six simulators on one GPU together faster than real time. On top of the learned policy, 14 tasks are routed to a GPT-6 Astra coding agent that controls the robot through observation and motion tools, starting every episode from a frozen copy of its own practice session on training instances. Across these 14 tasks, Astra's mean Q is 0.639, against 0.108 for the learned policy. The public submission included only part of the Astra results and scored 0.446 Q with 27.3% full-task success.
comment: Code: https://github.com/Robot-Learning-Collective/behavior-1k-solution-2026 ; checkpoint: https://huggingface.co/IliaLarchenko/BEHAVIOR2026VLA ; project page: https://robot-learning-collective.github.io/behavior-1k-2026/
★ SIGMO: Sigmoid Gated Interaction-Aware Motion Planning
Despite the widespread adoption of autonomous vehicle technologies, reliable navigation within cluttered environments remains arduous due to the necessity of interaction-aware strategies; example scenarios include: narrow passages with multiple agents, lane changing in dense traffic, and crossing unsignalized intersections. These scenarios often require highly proactive strategies, simultaneously demanding interpretability and quick adaptability, which challenge end-to-end and data-driven methods. An effective strategy requires: scalability to multiple agents, adaptability to changes in other agents' behavior, and harmony with surrounding traffic without freezing or becoming overly conservative. To meet these requirements, we present SIGMO, a sigmoid-gated interaction-aware motion planner, with two main contributions. First, we formulate polynomial sigmoid functions that analytically estimate yield scores of surrounding agents using physics-grounded interaction features that capture driving intuitions. Second, we integrate the polynomial sigmoid functions into a receding-horizon planner, which facilitates interaction-aware planning with arbitrary trajectory predictions. We validate SIGMO in interaction-intensive simulations and hardware experiments. The results show that SIGMO balances proactive and defensive behavior while remaining computationally lightweight, serving as a strong baseline in a mobility stack. The simulation and hardware experiment videos are available here: https://sites.google.com/view/sigmoid-polygate.
comment: Submitted to IEEE
★ Gaussian Process-Augmented Model Predictive Contouring Control for 6-DoF Trajectory Tracking with a Holonomic AUV
Small, holonomic Autonomous Underwater Vehicles (AUVs) enable close-range inspection of underwater structures from arbitrary viewpoints, requiring accurate position and attitude tracking under state and input constraints. Model Predictive Contouring Control (MPCC) naturally handles these requirements, but relies on an accurate dynamics model, which is difficult to obtain underwater. In this paper, we present a nonlinear MPCC in all six Degrees of Freedom (DoF) for a holonomic AUV and alleviate the inaccuracies in the dynamics by augmenting the nominal model employed with residual dynamics learned via Gaussian Process (GP) regression. The learned residual is embedded in the optimal control problem through a zero-order robust optimization scheme, which keeps the controller real-time capable at 25 Hz. A simulation ablation shows that residual selection is essential when designing the controller, and simulation and in-water experiments with the Multimodality, Observing, Low-cost, Agile (MOLA) AUV validate the resulting controllers against a nominal MPCC and a field proven Proportional-Integral-Derivative (PID) controller.
comment: 8 pages, 8 figures
★ From Video Frames to Robot Meta-Frames: A Unified Latent Interface for World Action Models
Pretrained video diffusion models (VDMs) offer a powerful foundation for World Action Models (WAMs), which jointly predict robot actions and future observations. Building such models requires injecting low-dimensional robot signals (e.g. proprioception, actions, and values) into a latent space shaped by natural videos, and a key design choice is the interface through which these signals enter: existing schemes treat it as a formatting detail, spending a separate latent frame on each signal or bolting on dedicated encoders and decoders. We argue that signal injection is instead a distribution-alignment problem: how well a representation matches the prior of the pretrained VDMs during denoising. Using the relative reconstruction error of a frozen VDM denoiser as a probe, we find that conventional spatial encodings sit far off this prior, whereas frequency-domain packing yields latents that the denoiser corrects far less. Motivated by this diagnostic result, we propose MetaWAM, which folds all robot signals into a single unified meta-frame: each signal is assigned to a disjoint frequency-domain channel band and lifted to the spatial latent by an inverse DCT, yielding a decodable interface aligned with the pretrained VDM prior without any added module or architectural change. A fixed band layout together with slot-aware routing lets the same meta-frame support policy, world-model, and value prediction over a shared set of robot variables. On RoboCasa, MetaWAM reaches 68.9% success while collapsing the per-signal frames into one and cutting inference latency by 39%, and attains 85.0% success on three real-world Franka tasks. Code and models are available at https://github.com/showlab/MetaWAM
★ ForeseeDrive: Future-Token Conditioning for Anticipatory Generative Driving
Generative driving planners can represent diverse ego futures, but a downstream scorer can choose only among sampled candidates; it cannot recover an absent maneuver. We present ForeseeDrive, which turns future scene evolution into a deployable interface for shaping candidate generation. Rather than reconstructing dense future states or scoring a fixed menu, ForeseeDrive derives query-indexed tokens from logged actor motion, occupancy evolution, route interactions, and TTC margins, and predicts the same token space from present context at deployment. The tokens condition trajectory geometry before candidates materialize. A zero-token-gated residual path and candidate-preservation loss limit unnecessary drift, while a TTC-margin guard targets brittle safe-to-unsafe boundary crossings. Under their respective benchmark protocols, ForeseeDrive reaches 89.2 PDMS on NAVSIM navtest, 0.41\,m average L2 and 0.05\% collision rate on nuScenes validation, and a 76.50 driving score on Bench2Drive. Controlled DiffusionDrive/NAVSIM diagnostics show an exact zero-token no-op, a small aligned-over-shuffled selected-PDM advantage under preservation, mixed oracle-best quality, and localized TTC-boundary failures. Together, the benchmark results and bounded mechanism tests support future-conditioned generation as a practical interface for anticipation. Code is included in the supplementary materials.
comment: 15 pages (8 pages main text + appendix), 3 figures, 7 tables
★ From Prediction to Action: Learning Physical Context for Deformable Linear Object Manipulation ICRA 2027
A representation that accurately predicts a deformable object's response is not necessarily the one most useful for choosing manipulation actions. We study this distinction between predictive utility and decision utility in planar single-grasp shape control of Deformable Linear Objects (DLOs). As a common decision backbone, we introduce Geometry-Pointer-Flow (GPF), a geometry-grounded policy that separates grasp-location selection from grasp-conditioned motion generation. Its geometry-only variant reduces one-step executed shape error by 32.8% relative to a DLO-specific diffusion policy, while using 18.7 times fewer parameters and achieving approximately 193 times faster inference. On this backbone, we compare physical context represented by ground-truth parameters, estimated parameters, and learned interaction-history representations, for both forward prediction and executed control. For prediction, ground-truth physical parameters reduce next-shape error by 15.8% over geometry alone, and estimated parameters are slightly more predictive than interaction history. This ordering does not carry over to control: estimated parameters provide no detectable one-step control gain, whereas history-conditioned GPF reduces executed error by 9.9% relative to geometry-only GPF across 470 paired simulation trials. In closed loop, interaction history improves strict-threshold attainment in simulation and yields the strongest zero-shot performance on hardware. All policies are trained exclusively in simulation and transferred to the physical robot without real-world training or fine-tuning. These results show that physical context should be evaluated by the decisions it supports rather than by predictive accuracy alone. Code will be released upon acceptance.
comment: Submitted to ICRA 2027
★ Elastic-WAM: Adaptive Spatiotemporal Imagination for Efficient Robot Control
World-Action Models improve robot control by predicting future visual dynamics, but iterative video denoising incurs substantial inference overhead. Recent approaches reduce this cost through partial future denoising, yet typically rely on fixed refinement budgets. Our empirical observations reveal that preferred refinement depths vary across tasks and states, while different future regions exhibit varying relevance to action generation. Motivated by these findings, we propose Elastic-WAM, which adaptively allocates world refinement computation across temporal denoising time and spatial future space. A unified spatiotemporal refinement demand field governs when to stop world refinement, how far to advance each denoising step, and which regions to refresh. We parameterize this field with a shared temporal basis and spatial amplitudes, and learn it through a three-stage action-grounded supervised procedure. At inference, the learned field guides recurrent sparse world refinement to improve task success while reducing inference overhead. Evaluations across diverse simulation benchmarks and real-world long-horizon manipulation demonstrate state-of-the-art control performance with reduced inference latency.
comment: 14 pages. Project page: https://lihaitian.com/Elastic-WAM/
★ Personalized Affective Perception via Continual Learning in Human-Agent Negotiation
Self-interested intelligent agents that negotiate with humans should interpret nonverbal expressions and adapt their behavior accordingly. However, generic facial-affect models degrade across individuals and contexts, creating a generalization gap. We introduce an affect-adaptive negotiation strategy that incorporates Continual Learning with Imagination for Facial Expression Recognition (CLIFER) to personalize valence-arousal perception in real time. Differential feedback converts changes in valence-arousal direction and intensity into adjustments of the agent's target utility. An asynchronous architecture updates CLIFER in the background during live interaction. In a counterbalanced within-subject study in which 66 participants negotiated with a humanoid robot, the CL-enabled agent achieved significantly higher individual utility and elicited higher participant concession rates than the FaceChannel baseline. The CL condition yielded significantly higher mean arousal estimates, alongside a broader output range. Participant utility and Nash distance did not change significantly between conditions. These outcomes support the practical value of coupling continual affective personalization with differential feedback for real-time negotiation.
comment: 17 pages, 11 figures. Submitted to IEEE Transactions on Affective Computing
★ Koopman Model-Based Reinforcement Learning
We propose a Koopman model-based reinforcement learning (K-MBRL) algorithm for controlling nonlinear robotic systems with unknown dynamics. The K-MBRL learns a control policy by minimizing an approximate policy objective that combines the analytical immediate cost with an estimate of the future cost evaluated at the one-step predicted state. To this end, we develop a stochastic Koopman operator-based learning method to approximate the unknown nonlinear dynamics in a linear representation, while a critic function estimates the remaining future cost. This formulation offers two key advantages. First, the linear Koopman dynamics simplify the model-dependent policy gradient computation to a constant matrix product, enabling computationally efficient policy improvement. Second, one-step model propagation mitigates the accumulation of prediction errors associated with long-horizon rollouts while retaining long-term cost information through the critic. The learned policy enables real-time control through feedforward evaluation without online optimization. We establish policy learning convergence guarantees under standard stochastic approximation assumptions. Simulations on HalfCheetah and quadruped four-legged locomotion tasks demonstrate strong sample efficiency and improved performance compared with classical model-free and model-based reinforcement learning baselines. A pendulum swing-up task is used to assess policy gradient approximation accuracy and conduct ablation studies on the learning mechanism and key hyperparameters. The proposed K-MBRL is further validated on a physical quadruped robot performing a challenging sequence of maneuvers: rising to an upright posture supported by two rounded rear feet, maintaining balance during bipedal walking, and safely returning to the ground, as shown in Fig.1.
★ LEASH: Stay Safe in Agentic Robotics
Foundation models such as GPT and Claude have shown strong capabilities in spatial understanding and embodied intelligence. These advances make agentic robotics a promising path toward artificial general intelligence in robotics. However, we find that using foundation models to control robots poses serious safety risks, and comprehensive safety evaluation is still lacking. We introduce RoboRisk, a comprehensive benchmark for evaluating safety in agentic robotics. Its 92 simulated tasks and 13 physical-robot tasks fall into five categories: Task-level Refusal (T), Plan Repair (P), Execution-time Hazard (E), Injection (I), and Sequential Memory (S). Prior benchmarks cover at most two of these categories, and several evaluate only text outputs. In contrast, RoboRisk covers all five and executes every task in closed loop. We evaluate six state-of-the-art foundation models on RoboRisk: GPT-6 Astra, GPT-6.1 Sol, GPT-6 Luna, Claude Opus 5.5, Gemini 4 Argon, and Grok 4.7. These models generally show high harm rates. We also evaluate two robot-specific APIs: the Dream API and PokeBot's RoboAPI. We further propose the Lease-Enforced Agentic Safety Harness (LEASH), which offers essential safety infrastructure for agentic robotics and can be integrated into nearly any agentic architecture. LEASH leverages a decision model (Jev or OpenAI's Decisions API) to implement task, action, and commit gates, which review the task, each action, and each irreversible operation in their visual context. The planner and all three gates share a common safety policy. Experiments on RoboRisk in simulation and on physical robots show that LEASH effectively reduces harm across all six foundation models. It largely preserves their task completion performance and adds little time and token overhead.
★ Model-Free Runtime Measurement of Task-Relevant Information
Autonomous systems increasingly monitor physical kinematics, environmental states and task completion outcomes. While existing work provides methods to plan information seeking actions and interpret acquired observations, current approaches do not monitor whether the live process of information gathering is functioning properly. Consequently, such systems can remain operationally active without resolving outstanding mission requirements. To address this limitation, we introduce Acquisition-State Telemetry (AST), a model free architecture that evaluates ongoing information acquisition from observable evidence relative to declared task requirements. Specifically, a domain interface maps raw observations into task resolution coordinates and normalized comparative representations, which are processed by a fixed causal monitor. This monitor emits two continuous real time signals that quantify (i) prospective progress across unresolved requirements and (ii) informational friction generated when recurrent evidence yields negligible informational gain. We evaluated a single, unadjusted monitor configuration across eight diverse domains including search and rescue dialogue, spatial grid exploration, hierarchical capability acquisition, continuous robotic manipulation, and probabilistic intent estimation. We show that across these domains the friction channel distinguished unproductive stagnation from purposeful exploration prior to task completion, while remaining muted during post completion activity. Furthermore, in closed loop reinforcement learning experiments under early termination constraints, we demonstrate that penalizing informational friction allowed agents to sustain high mission success, whereas unpenalized policies and baselines with matched operational costs failed to complete required tasks.
comment: 17 pages, 5 figures, 2 extended data tables, appendix included. Code available
★ DexSeed: Expanding Human Hand-Object Interactions into Robot Demonstrations for Dexterous Policy Learning
We study how to generate diverse, physically executable robot demonstrations for dexterous vision-language-action (VLA) learning. Existing demonstration-generation methods typically rely on costly teleoperated examples. Meanwhile, human hand--object datasets provide a reusable source, but existing transfer methods mainly reproduce given interactions. Turning these interactions into VLA training data further requires coverage of different objects and manipulation locations, together with executable arm and finger actions, and visual observations. To achieve this, we present DexSeed, a framework for low-cost generation of dexterous robot data from human hand--object interactions. Our key idea is to expand human sequences while preserving their grasps and realizing the resulting motions under arm-tracking dynamics in physics simulation. Specifically, we organize source sequences by grasp mode with shared language descriptions. InteractionMesh-based retargeting and arm trajectory augmentation adapt these sequences to the robot hand, different object geometries, and pick-and-place locations while preserving local hand--object relationships. An RL finger-control policy produces executable rollouts, while the arm tracks precomputed trajectories. Rendered observations and recorded actions from successful rollouts are paired with grasp descriptions to fine-tune a VLA. Experiments show improved contact fidelity in retargeting and more reliable whole-arm execution than the evaluated baselines. Simulation evaluation and real-world deployment further demonstrate that a VLA fine-tuned solely on the generated demonstrations can execute language-specified grasps. Project page: https://robbyant-research.github.io/DexSeed/.
comment: Project page: https://robbyant-research.github.io/DexSeed/
♻ ★ Awomo-SimDataEngine: Agentic Simulation-ReadyWorld Generation
Generating useful robot-training data requires more than visually plausiblescenes: objects must support interaction, placements must remain physicallyvalid, and tasks must admit repeatable execution. We present\textbf{Awomo-SimDataEngine}, an agentic system that connects asset and scenegeneration to robot demonstration synthesis. Shared asset services providerigid and articulated objects, including structure-grounded part and jointgeneration with ISArt. Scene generation supports two complementary routes:Unravel reconstructs editable scenes from images, while SimForge buildssingle-room and multi-room environments from text. A graph-native harnesscoordinates construction, validation, andbounded repair, routing failures to the responsible module while retainingunaffected scene state. PolicyForge binds validated worlds to tasks and robotembodiments to produce replayable demonstrations. Evaluations cover assetgeometry, scene quality, and downstream policy learning. On MuJoCo-basedLIBERO-Plus, co-training with Isaac Sim demonstrations improves the overallsuccess rate of a World-Action Model (WAM) from $77.17\%$ to $89.43\%$. Goal and spatialsuccess improve by $31.66$ and $6.25$ percentage points, respectively . These results support the utility of the generated data for cross-simulatorpolicy training, with more limited gains on long-horizon tasks.
♻ ★ Delay-Certified Readiness Barriers for Overactuated Multirotor Allocation Without Lipschitz Constants
Overactuated multirotors can produce the same wrench with many rotor-speed combinations, some of which leave the vehicle unable to change its wrench quickly, e.g., to reject a gust. Readiness barriers keep this ability, the readiness, above a floor, but only in continuous time; with sampled commands and varying speed-controller delays, the available robustness bound covers less than one millisecond. We remove this limit by exploiting the structure of the problem: each rotor responds monotonically and in closed form to its own torque, so the speeds reachable under uncertain switching times lie in a cheaply computed box, and the lowest readiness over that box sits at a corner known in advance. This yields a Lipschitz-free test that a command keeps the readiness above its floor until the next sample, plus a fallback torque that keeps the vehicle on a certified plan. In hexarotor simulations with 2--40~ms delays and $\pm2$~ms jitter, the filter never violated the floor. At $10\pm2$~ms, it matched a delay-compensating predictor within the design envelope and cost 2--30\% more tracking error beyond it, where the predictor alone violated the floor in 2 to 16 of 16 missions. A tuned robust predictor was also safe with synchronous switching, but not with per-rotor jitter.
comment: This work has been submitted to the IEEE for possible publication
♻ ★ Dynamic Shape Control of Soft Trunk Robots
Controlling the shape profile of soft trunk robots extends their reach into complex regions of the workspace and enhances their use in practical applications. Recent advances in soft robot design have enabled soft trunk robots to achieve greater flexibility in conforming to complex shapes. Specifically, trunk robots whose geometries are designed as helicoids have been shown to enable extreme shape-changing capabilities. Their compliance makes them highly nonlinear, which in turn makes shape control difficult. On similar tendon-driven trunk robots, model-based controllers based on the theory of adiabatic spectral submanifolds (aSSMs) have achieved high accuracy in closed-loop end-effector tracking. We extend this approach by learning an aSSM model of the full body of a trimmed-helicoid soft trunk robot from data and use this in a model predictive control (MPC) scheme to demonstrate a range of shape control tasks on hardware. On closed-loop dynamic shape regulation, our method outperforms the available linear baselines by a factor of 10. In human-in-the-loop interactive demos, our method completes section-by-section shaping and balances a tennis ball for over 25 seconds, while the linear baselines fall short. A model-free reinforcement learning policy trained for 12 hours on hardware fails on the same balancing task. These results establish aSSM-based MPC as a practical route to full-body shape control on real soft hardware.
♻ ★ XS-VLA: Teaching Tiny Vision-Language-Action Models with Spatial Supervision and Demonstration Conditioning
How can richer training supervision improve robot control while keeping the deployed policy compact? We present XS-VLA, a staged training framework using teacher-derived spatial labels and demonstration-conditioned action learning. Coarse-Grained Spatial Distillation (CSD) initializes the backbone through an auxiliary region-label task. Latent Flow Matching (LFM) then conditions an action-space velocity field on a demonstration latent, using KL regularization while jointly optimizing the backbone and action modules. The deployed policy contains 243.99M parameters and operates without the teacher or posterior encoder. XS-VLA achieves 90.25% average LIBERO success in each of two training seeds, compared with 86.00% for a SmolVLA-0.25B base trained under our settings. Ablations examine both training stages through matched image pretraining and Huber/MSE controls. On three Mobile ALOHA tasks, average strict success increases from 21.7% to 65.0%. These results demonstrate the control utility of auxiliary representation initialization and regularized demonstration-conditioned flow learning for compact VLA policies
comment: Preprint. Project page: https://999tom.github.io/xs-vla/
♻ ★ Pose Distance Fields for Humanoid Robots NeurIPS 2026
Pose and motion priors play a crucial role in humanoid robotics. Although such priors have been widely studied in human motion recovery (HMR) domain with a range of models, their adoption for humanoid robots remains limited, largely due to the scarcity of high-quality humanoid motion data. In this work, we introduce Pose Distance Fields for Humanoid Robots (PDF-HR), a lightweight prior that represents the robot pose distribution as a continuous and differentiable manifold. Given an arbitrary pose, PDF-HR predicts its distance to a large corpus of retargeted robot poses, yielding a smooth measure of pose plausibility that is well suited for optimization and control. PDF-HR can be integrated as a reward shaping term, a regularizer, or a standalone plausibility scorer across diverse pipelines. We evaluate PDF-HR on various humanoid tasks, including single-trajectory motion tracking, general motion tracking, style-based motion mimicry, and general motion retargeting. Experiments show that this plug-and-play prior consistently and substantially strengthens strong baselines. Code and models are publicly available at https://gaoyukang33.github.io/PDF-HR.
comment: Accepted at NeurIPS 2026. Project page: https://gaoyukang33.github.io/PDF-HR
♻ ★ Quadruped Agile Locomotion: Sparsely Gated Mixture of Experts with Visual Input
Agile locomotion is important for quadruped robots in the real world. Recent approaches have demonstrated impressive capabilities including dynamic climbing and jumping but typically rely on sequential multilayer perceptron (MLP) architectures with densely activated layers. In contrast, sparsely gated mixture-of-experts (MoE) architectures have become popular in the large language model (LLM) domain as a tool to improving scalability and performance by activating only a subset of parameters at inference. In this work, we investigate the application of such architectures to vision-based robotic locomotion. We compare policies based on standard MLPs and MoE architectures under a controlled setting where the number of active parameters at inference time is matched. Experimental results show an 8\% increase in reward during training comparing the MoE to a standard MLP and even show benefits comparing to MLPs with equivalent total parameters. Experimental results on a real Unitree Go2 quadruped robot demonstrate performance in the real world. These results highlight that sparsely gated MoE architectures may pose advantages in tasks where network capacity is important and that borrowing techniques from the fast-moving LLM space and applying them to locomotion is a promising direction.
comment: 8 pages, 5 figures
♻ ★ How Many Demonstrations Does a VLA Need? On the Sample Efficiency of VLA Finetuning
Pretrained vision language action (VLA) models show strong zero-shot performance but they are rarely deployed directly and are often first post-trained on demonstrations from the target setting. Yet VLAs are typically compared by zero-shot or full-data fine-tuned success which reveals little about the data and compute required for downstream adaptation. We study adaptation efficiency for six VLAs ($π_0$, $π_{0.5}$, and GR00T N1-N1.7) on four LIBERO suites, varying both demonstrations and training steps. This reveals differences hidden by full-data fine-tuning: models which are within two points on the full LIBERO-Spatial dataset differ by up to 3$\times$ in success with 50 demonstrations. $π_{0.5}$ needs 2-5$\times$ fewer demonstrations than $π_0$ and is the most sample-efficient model at every budget, with the largest gap on the long-horizon suite. Within GR00T, N1.5 adapts more efficiently than N1.6 and N1.7 on three of four suites, despite their stronger zero-shot performance. We also study one pretraining factor behind these differences by freezing the VLM during finetuning: this severely degrades the $π$ models whose VLM is jointly trained with the action expert during pretraining, but barely affects GR00T models.
comment: 14 pages, 8 figures, CoRL Workshop
♻ ★ Open-HumanWAM: An Open Recipe for Scaling Human and Robot Experience for Embodied Foundation Models
Egocentric human video offers a scalable source of manipulation priors for robot learning, yet translating raw data scale into generalizable robot policies demands rigorous data curation and training recipes. We introduce Open-HumanWAM, the first fully open recipe for scaling egocentric pretraining to 50,000 hours, anchored by Open-HumanWAM-50K---a curated egocentric video corpus densely annotated with hand motion and natural language. Leveraging this corpus, we systematically investigate pure human pretraining via closed-loop simulation evaluations and uncover an empirical log-linear scaling law linking pretraining data hours to downstream robot success rates. Grounded in these findings, we develop HumanWAM-Vanilla, a world--action model pretrained exclusively on 50k hours of egocentric human video. Upon robot post-training, HumanWAM-Vanilla reaches 92.08% success on RoboTwin 2.0 Clean2Clean and 68.88% on Clean2Random, alongside a 14.17% mean success rate and a 19.00 mean progress score on RoboDojo. To push the capability frontier further, we explore joint human--robot pretraining with HumanWAM-Goal, a visual-goal-conditioned world--action model. Hybrid data scaling unlocks broad zero-shot generalization across both simulation and physical benchmarks, propelling post-trained RoboTwin 2.0 performance to new state-of-the-art levels: 92.70% on Clean2Clean and a substantial leap to 84.60% on Clean2Random. On physical robots, HumanWAM-Goal further exhibits emergent embodied in-context learning, dynamically adapting its manipulation behavior to unseen visual goals without parameter updates. To support reproducible scaling research in embodied AI, we open-source the Open-HumanWAM-50K corpus, data curation and pretraining pipelines, alongside the pretrained and post-trained HumanWAM checkpoints, inference code, and simulation evaluation suites.
comment: GitHub: https://github.com/DAGroup-PKU/HumanNet/tree/Open-HumanWAM Project website: https://dagroup-pku.github.io/open-humanwam/
♻ ★ OASIS: Optimization-Aware Subgoal Inference with Safety Constraints for Autonomous Mobile Robot Navigation
Navigation in dynamic environments requires mobile robots to make progress toward their goals while respecting motion limits and avoiding surrounding obstacles. Learned subgoal policies can guide model predictive control beyond its local prediction horizon, but a geometrically suitable subgoal does not necessarily lead to a successfully solved control problem. This paper proposes OASIS, a hierarchical navigation framework that incorporates recent optimization outcomes into reinforcement learning-based subgoal generation. A high-level policy trained using Soft Actor--Critic generates local subgoals, while a nonlinear model predictive controller tracks them subject to motion limits, geometric collision-avoidance constraints, and embedded high-order control barrier function constraints. The outcomes of controller optimizations over each subgoal interval are summarized and provided to the policy at its next decision, without requiring additional optimization calls. The framework is evaluated in five navigation scenarios with static and moving obstacles, including dense interactions and sensing uncertainty. Comparisons with hierarchical and direct-goal control variants show that OASIS achieves the highest aggregate navigation success. Matched comparisons across three training seeds further show a consistent success advantage over the otherwise identical hierarchy without optimization-status input. The selected-policy comparison also shows fewer primary optimization failures with similar mean controller time.
♻ ★ $A^3$-VLA: Automatic Perception-Guided Attention Alignment for Robot Manipulation NeurIPS2026
Vision-Language-Action Models (VLAs) enable robots to perform manipulation tasks by leveraging Vision-Language Models (VLMs) to process visual observations and language instructions. However, the action prediction objective itself lacks fine-grained task-specific perceptual supervision, making it prone to relying on task-irrelevant visual shortcuts, leading attention away from target objects and key interaction areas, thereby limiting its operational performance and generalization ability. To address this, we propose Automatic Perception-Guided Attention Alignment for Robot Manipulation ($A^3$-VLA), a framework that derives task-specific visual guidance from the manipulation process itself. Specifically, $A^3$-VLA first generates heuristic candidate regions for target objects and the robot from interaction-induced visual changes, and refines them with a visual segmentation model offline. These regions serve as explicit perceptual guidance through two complementary objectives: an attention allocation loss that encourages sufficient focus on task-critical regions when language or robot state tokens query the visual input, and a contrastive learning loss that aligns the representations of candidate target regions with language instructions while separating them from task-irrelevant regions. Extensive experiments on simulated and real-world manipulation tasks demonstrate that $A^3$-VLA consistently improves multiple VLA backbones, with an average success rate gain of 6.5% - 25.2% across benchmarks.
comment: Accepted to NeurIPS2026
♻ ★ Terrain Perception for Agricultural UAVs in Complex Farmland via Rotating mmWave Radar
Accurate terrain perception is essential for terrain-following flight of agricultural uncrewed aerial vehicles (UAVs), yet remains challenging in real-world farmland due to occlusions, complex terrain geometry, and environmental disturbances. Millimeter-wave (mmWave) radar is a promising sensing modality for this task due to its robustness to adverse conditions; however, existing UAV-mounted radar systems rely on fixed field of view (FoV) and terrain extraction methods designed for dense light detection and ranging (LiDAR) data, leading to incomplete and unreliable terrain estimation. To address these limitations, we present a low-cost rotating mmWave radar-enabled terrain perception framework for agricultural UAVs operating in complex farmland environments. Specifically, a mechanically rotating sensing design is introduced to enlarge spatial coverage and improve terrain observability beyond the limitations of fixed-view radar under dynamic low-altitude flight. Building upon this sensing capability, we further design a pose-consistent terrain reconstruction pipeline tailored for sparse, noisy, and partially observable radar data, enabling reliable ground extraction and continuous terrain surface estimation in challenging agricultural scenarios. The complete system is deployed on a real agricultural UAV platform and evaluated through extensive field experiments. Experimental results demonstrate improved terrain coverage and estimation accuracy, achieving an F1 score of 94.42% for ground segmentation, while the closest rival only achieves 90.48%.
♻ ★ Effect of Temporal Resolution on Zero-Shot Semantic Understanding of Rapid Human Actions
Understanding human actions from visual observations is essential for human-robot interaction, particularly when semantic interpretation of unfamiliar or hard-to-annotate actions is required. For rapid or less common activities, collecting sufficient labeled data for supervised learning is challenging, making zero-shot approaches a practical alternative for semantic understanding without task-specific training. While recent advances in large-scale pre-trained models enable such zero-shot reasoning, the impact of temporal resolution, especially for rapid and fine-grained motions, remains underexplored. In this study, we investigate temporal-resolution conditions for zero-shot semantic understanding of high-speed human actions. Using kendo as a representative case of rapid and subtle motion patterns, we propose a training-free pipeline that combines a pre-trained video-language model for semantic representation with large language model-based reasoning for pairwise action comparison. Our primary experiment evaluates full-observation raw videos from 10 to 120 Hz in 10-Hz increments. Nearest class prototype (NCP) accuracy is highest at 120 Hz, and the mean over 70-120 Hz is higher than that over 10-60 Hz. The corresponding mean prototype margin changes from -5.74 to +2.52 similarity-score points between these two halves and is also highest at 120 Hz. Secondary experiments show that tracking overlays reduce mean full-observation performance but improve mean partial-observation performance. These findings provide preliminary evidence that retaining high-temporal-resolution visual information can benefit training-free semantic understanding of rapid human actions.
♻ ★ Learning Dexterous Manipulation Using Contact Wrench Guidance From Human Demonstration
Dexterous robot manipulation can benefit from the abundance of human demonstrations, but transferring such demonstrations to robot policies remains challenging. We present Contact Wrench Guidance from Human Demonstration in Robotic Dexterous Manipulation (CHORD), a framework for long-horizon manipulation of rigid and articulated objects with reinforcement learning. The key idea is object-centric contact wrench space guidance: we represent human and robot motions by the forces and torques they can induce on the object, enabling similarity to be measured by the induced instantaneous motions. This guidance makes reinforcement learning more scalable for contact-rich dexterous manipulation. We further introduce a large-scale simulation benchmark with 4,739 bimanual dexterous manipulation tasks, constructed from motion-capture datasets and reconstructed in-house videos. Evaluated on 1,831 benchmark tasks, CHORD achieves an average success rate of 82.12%, demonstrating strong scalability. CHORD also generalizes to whole-body manipulation from hand-only and third-person demonstrations, achieving a 90.77% success rate, and the learned policies transfer to the real world in both open-loop and closed-loop settings.
♻ ★ DexForesight: Grounding Dexterous Policies with Implicit Spatial Foresight
Dexterous manipulation requires rich spatial foresight over both the surrounding scene and the robot hand. Existing world-action models often acquire such predictive knowledge through explicit future prediction. We introduce DexForesight, a framework that learns spatial foresight implicitly by injecting dynamics-aware world representations into the latent space of a dexterous policy. During training, DexForesight learns an action-conditioned spatial world model from offline scene depth, kinematic hand geometry, and demonstration trajectories. The world model predicts 3D hand-point motion and visual feature changes, capturing both articulated hand dynamics and scene changes under demonstrated actions, including the motion of fingers occluded from the cameras. We then freeze the world model and align policy queries with its predictive world queries, transferring the learned spatial dynamics into policy latents and using them to condition action generation. At deployment, only the policy is retained, operating on RGB observations, language, and robot state without depth input or explicit future prediction. On 11 DexJoCo simulation tasks, DexForesight improves the overall success rate of the base policy from 49.4% to 60.5%. On three real-robot dexterous manipulation tasks, DexForesight achieves 58.3% success, compared with 50.0% for the strongest baseline. Project page https://dexforesight.github.io/
comment: 13 pages, 4 figures
♻ ★ Predicting Risks Before They Happen: Vision-Language Reasoning for Autonomous Driving
% Waiting until a driving hazard becomes directly visible can leave an autonomous vehicle little time to respond. Observable precursors, such as another vehicle's unexpected braking, can provide evidence of a hidden hazard and an opportunity for earlier risk mitigation. However, existing driving datasets and language reasoning formulations provide limited supervision for connecting these precursors to potential risks and appropriate responses. We introduce PotentialRiskQA, an ego-view driving dataset that marks precursor and risk appearance times and provides structured annotations linking scene evidence, potential risks, and mitigation actions. Building on this dataset, PR-Reasoner trains vision-language models to generate a precursor-centered reasoning chain through scene-level supervision and auxiliary tasks for scene understanding, precursor identification, risk inference, driving suggestions, and meta-actions. Experiments across multiple backbones show improved scene understanding, risk cognition, and action quality, with fewer unsupported risk claims on most models. Ablations show that supervising intermediate reasoning steps further reduces unsupported risk claims and improves action quality beyond scene-level training alone. On NAVSIM, the trained representations improve open-loop planning performance, demonstrating benefits beyond language-based evaluation.
♻ ★ Fade to... Red? Scheduling Coloured Noise Exploration Supports Efficient Robot Learning with Reduced Movement Jerk
Exploration in deep reinforcement learning (RL) for continuous control is commonly driven by temporally uncorrelated Gaussian (white) action noise, which leads to jerky motions with locally constrained exploration. In contrast, temporally correlated coloured noise, particularly pink noise, has been shown to support more efficient exploration in RL. Motivated by these findings, we inquire whether scheduling changes to the noise colour over the course of training can outperform white noise for continuous control robotics. We perform a systematic study of 49 linear schedules between white and red noise, together with generalised state-dependent exploration (gSDE), on four sparse-reward manipulation tasks with a Fetch arm and a Shadow Dexterous Hand. We find that the noise colour at the end of training does not affect learning efficiency or end performance, whereas fading towards red noise halves the jerk of exploration in every task. Balancing learning efficiency, end performance and smoothness, red noise emerges as the preferred strategy: it learns as efficiently as pink noise with almost 2-fold less movement jerk. We therefore recommend exploration schedulers that fade towards red noise as the default for off-policy robot learning.
comment: 10 pages, 4 figures, 1 table. Accepted at the ACRA 2026 conference. Cite as: "López, F. M., and Cruz, F. (2026). Fade to... Red? Scheduling Coloured Noise Exploration Supports Efficient Robot Learning with Reduced Movement Jerk. In Proceedings of the Australasian Conference on Robotics and Automation (ACRA). Australian Robotics and Automation Association (ARAA)."
♻ ★ SSR: Scaling Surefooted and Symmetric Humanoid Traversal Toward the Open World
Advancing humanoid traversal toward the open world is key to practical deployment in human environments, but remains challenging. The robot must use vision to ensure safe and reliable foot placement on heterogeneous terrain during highly dynamic motion, while producing coordinated, natural whole-body behaviors. We propose SSR, an efficient end-to-end framework for egocentric vision-based humanoid traversal that jointly learns these capabilities. SSR introduces imagined foothold guidance, which learns to model forthcoming swing-foot contacts and evaluates their support to guide pre-touchdown swings toward stable regions, reducing edge slips. It further employs equivariant latent-space symmetry augmentation to efficiently induce bilateral coordination under high-dimensional visual observations, and uses terrain-specific multi-discriminator motion priors to encourage human-like behavior across scenes. Extensive experiments show that SSR achieves safe, stable, and high-quality locomotion on diverse real-world terrains, including stairs with varied structures and extreme challenges such as wide gaps and high platforms, while enabling reliable long-horizon traversal in diverse outdoor environments. Project website: https://ssr-humanoid.github.io/.
♻ ★ DKWBC: Decoupling Motion Tracking and Balancing for Humanoid Whole-Body Control
Humanoid whole-body control requires a robot to track diverse reference motions while maintaining balance under gravity, torque limits, contact switching, and external disturbances. Existing reinforcement-learning-based trackers typically learn control outputs that jointly address motion tracking and balance. However, similar reference poses can correspond to substantially different dynamic demands, requiring a unified policy to learn motion-dependent control requirements alongside balance corrections. This can complicate learning and lead to high-frequency fluctuations in control outputs. We propose DKWBC, a dynamics-informed whole-body control framework that combines explicit model-based tracking with a learned balancing residual. Inspired by the model-based and environmental-wrench contributions to joint torques in floating-base dynamics, DKWBC uses inverse-dynamics feedforward and joint PD feedback to supply tracking torques, while a residual policy learns additional joint-torque corrections for balanced execution. By explicitly accounting for motion-dependent torque demands, this structure simplifies the policy's learning task. Experiments in simulation and on real hardware demonstrate that a single learned balancing policy generalizes across motion datasets while maintaining accurate motion tracking and balance across ground support, attached payloads, and physical interaction. Project page: https://robbyant-research.github.io/DKWBC/.
comment: 9 pages, 6 figures, 4 tables
♻ ★ Chasing Stability: Humanoid Running via Control Lyapunov Function Guided Reinforcement Learning
Achieving highly dynamic behaviors on humanoid robots, such as running, requires controllers that are both robust and precise, and hence difficult to design. Classical control theory offers principled tools for stabilizing such systems but struggles to synthesize real-time controllers for their nonlinear, hybrid dynamics; reinforcement learning (RL) handles these dynamics well but typically depends on extensive hand-tuning of heuristic rewards. In this work, we embed ideas from nonlinear control theory, specifically control Lyapunov functions (CLFs), along with optimized reference trajectories for the hybrid system into the reinforcement learning reward. This approach, CLF-RL, replaces the need to handcraft and tune heuristic reward terms with easy to tune stability informed rewards. We further develop a trajectory optimization routine that can handle the hybrid structure of the system to produce dynamically feasible, multi-domain, trackable references; using full-order dynamics rather than a kinematic approximation yields up to a 1.7x improvement in position tracking. The resulting policy runs on a Unitree G1 humanoid with a full flight phase, operating reliably in tight corridors such as a treadmill as well as in outdoor environments, all while remaining robust to disturbances applied to the torso and feet. Moreover, it achieves accurate global reference tracking utilizing only on-board sensors, making a critical step toward integrating these dynamic motions into a full autonomy stack.
comment: Accepted to Humanoids 2026
♻ ★ OpenDexGrasp: Open-vocabulary Task-Oriented Dexterous Grasping
Dexterous grasp synthesis has advanced rapidly in generating stable and physically plausible hand poses, but real-world manipulation requires grasps that preserve the function implied by the task. We study open-vocabulary task-oriented dexterous grasp generation, where a robot must infer functional intent from free-form language, ground it in multi-view visual observations and object geometry, and generate an executable high-degree-of-freedom grasp. We present OpenDexGrasp, a unified data and generative modeling framework for this setting. OpenDexVerse provides dual-source supervision organized by the Coverage-to-Alignment (C2A) Recipe: OpenDex-Scale offers large-scale semantic and geometric coverage through automatic grasp synthesis and vision-language annotation, while OpenDex-Align supplies high-quality embodied alignment through human teleoperation and category-level transfer. OpenDexGrasp learns a shared perception-action latent representation that couples open-vocabulary vision-language context with dexterous action generation. Affordance grounding and grasp generation provide complementary supervision over this latent space, enabling direct generation of task-consistent dexterous grasps without a separate affordance-to-pose inference stage. Extensive simulation and real-robot experiments demonstrate improved functional alignment, physical feasibility, generalization to unseen categories, and real-world execution success. Additional details and videos are available at https://opendexgrasp.github.io/.
comment: Accepted at CoRL2026 as Oral Presentation
♻ ★ Safe Approximate Nonlinear Model Predictive Control Without Expert Demonstrations
Repeated online optimization makes nonlinear model predictive control (NMPC) computationally demanding for real-time applications. Neural approximations reduce this cost but often require large datasets of expert NMPC solutions. We introduce Sequential Approximate MPC (Seq-AMPC), a label-free recurrent framework that generates complete control sequences along predicted trajectories using parameters shared across the prediction horizon. The policy is trained directly through known dynamics with losses for control cost, constraint satisfaction, and a cost-decrease condition aligned with a feasible-backup safety filter. Building on this filter, we introduce a stage-cost-dependent acceptance tolerance that preserves recursive feasibility, constraint satisfaction, and asymptotic convergence under nominal MPC assumptions, independently of neural approximation error. We further derive Hoeffding-based finite-sample bounds for the expected fraction of applied proposals and closed-loop cost discrepancy relative to NMPC. We evaluate Seq-AMPC on a quadcopter, chain of masses, obstacle-avoidance vehicle, and robotic manipulator, comparing recurrent and feedforward policies under label-free and imitation training. Across four benchmarks, training takes 0.2-1.7 GPU hours per policy without expert control labels. The filtered controller applies 97.2-99.1% of neural proposals and achieves mean closed-loop cost ratios within 0.3% of NMPC on jointly completed episodes. Tests with suboptimal feasible initial plans further demonstrate the benefit of neural replanning.
♻ ★ HABIT: Human-Aware Behavior and Interaction Training Dataset for Robot Manipulation NeurIPS 2026
Large-scale demonstration datasets have been central to recent progress in general-purpose robot policies. However, existing datasets are collected in human-absent settings, and policies trained on such data may perform tasks competently in isolation but fail to exhibit human-aware behaviors. To address this gap, we introduce HABIT, a large-scale robot demonstration dataset for human-present environments. We organize tasks into three roles capturing distinct modes of human-robot interaction: Collaborator, where human and robot jointly accomplish a task; Coworker, where they pursue separate tasks in a shared space; and Supervisor, where the human directs the robot. The dataset comprises over 10K episodes and over 160 hours across 60 tasks. Our experiments show that training on human-present data elicits human-aware behaviors that robot-only data fails to produce: spatiotemporal synchronization in Collaborator tasks, yielding in Coworker tasks, and gesture grounding in Supervisor tasks. Moreover, training on HABIT enables rapid adaptation to new human-robot interaction tasks. By introducing human presence as a new axis of dataset diversity, HABIT extends robot policies to environments shared with humans.
comment: NeurIPS 2026 Evaluations & Datasets Track (Oral); Camera-ready version
♻ ★ Control-Aware Command Generation for Asynchronous Control and Action Inference with Vision-Based Policies towards Fast Execution NeurIPS 2026
Vision-based control policies, including vision-language-action models, have enabled robots to perform increasingly complex tasks. However, their execution speed is often limited by the high computational cost of action inference. In this work, we investigate what can be achieved at the control level without modifying the underlying policy learning process. We introduce an asynchronous inference and control framework that enables fast task execution while maintaining smooth and stable robot motions. By formulating the generation of continuous target positions from discontinuous model predictions as a tracking control problem, our method makes asynchronous inference practical for real robots. We further accelerate execution through adaptive action skipping based on the variance of predicted positions. Experiments with Action Chunking Transformer and OpenVLA-OFT on a bimanual robot demonstrate substantial speedups while preserving high task success rates.
comment: 8th Robot Learning Workshop: Is Physical AI Going Zero-Shot? at NeurIPS 2026
♻ ★ Mass Scaling Laws for Energy-Optimal Horizontal Multirotor Flight
Multicopters are a type of aerial robot seeing extensive real-world applications but with range and endurance as critical bottlenecks. Optimization of its energy efficiency through energy-aware flight planning (e.g. cruise speed-scheduling) has hence been an important topic. The mass of the multicopter is an important factor to consider, as it has a significant impact on efficiency, and many common applications are subject to frequent mass variation due to payload change. This work derives closed-form scaling relationships between the energy-optimal steady horizontal flight behavior of a multicopter and its vehicle mass, and validates them experimentally on two multirotor platforms. Specifically, for mass-only variation, i.e., when payload changes the vehicle mass while the airframe geometry, aerodynamic characteristics, and rotor configuration remain unchanged, we show through a combination of numerical simulation, theoretical derivation, and experimental validation that (1) the energy-optimal velocity is proportional to the square root of vehicle mass; (2) the energy-optimal pitch angle is constant with respect to mass; and (3) the optimal energy per meter is proportional to mass. The analysis shows that these three relationships arise from a common mass-similarity property of the steady-state flight and rotor-aerodynamic equations. Furthermore, the obtained scaling relationships enable simple yet effective cruise-speed planning to optimize energy efficiency and extend the range and endurance of the multicopter.
comment: Accepted to IEEE Robotics and Automation Letters, October 2026
♻ ★ MAPLE: Latent Multi-Agent Play for End-to-End Autonomous Driving NeurIPS 2026
Vision-language-action (VLA) models are effective as end-to-end motion planners, but can be brittle when evaluated in closed-loop settings due to being trained under traditional imitation learning framework. Existing closed-loop supervision approaches lack scalability and fail to completely model a reactive environment. We propose MAPLE, a novel framework for reactive, multi-agent rollout of a dynamic driving scenario in the latent space of the VLA model. The ego vehicle and nearby traffic agents are independently controlled over multi-step horizons, while being reactive to other agents in the scene, enabling closed-loop training. MAPLE consists of two training stages: (1) supervised fine-tuning on the latent rollouts based on ground-truth trajectories, followed by (2) reinforcement learning with global and agent -specific rewards that encourage safety, progress, and interaction realism. We further propose diversity rewards that encourage the model to generate planning behaviors that may not be present in logged driving data. Notably, our closed-loop training framework is scalable and does not require external simulators, which can be computationally expensive to run and have limited visual fidelity to the real-world. MAPLE achieves state-of-the-art driving performance on Bench2Drive and demonstrates scalable, closed-loop multi-agent play for robust E2E autonomous driving systems.
comment: Accepted at NeurIPS 2026
♻ ★ DiffeoMorph: Learning Distributed 3D Morphogenesis with Rotation-Invariant Spectral Shape Matching
Biological systems can form complex three-dimensional structures through the collective behavior of agents that share a common update rule and operate without central control. How such distributed control gives rise to precise global patterns remains a central question not only in developmental biology but also in distributed robotics, programmable matter, and multi-agent learning. Here, we introduce DiffeoMorph, an end-to-end differentiable framework for learning a morphogenesis protocol that guides a population of agents to morph into a target 3D shape. Each agent updates its position and internal state using an SE(3)-equivariant graph neural network, based on its own internal state and signals received from other agents. To train this system, we introduce a new shape-matching loss based on 3D Zernike polynomials, which compares the predicted and target shapes as continuous spatial distributions, not as discrete point clouds, and is invariant to agent ordering, number of agents, and global orientation. To achieve rotation invariance while preserving reflection sensitivity, we include an alignment step that optimally rotates the predicted Zernike spectrum to match the target before computing the loss. We perform benchmarking to establish the advantages of our shape-matching loss over other standard distance metrics for shape comparison tasks. We then demonstrate that DiffeoMorph can form a range of complex shapes from minimally patterned initial conditions. DiffeoMorph provides a general framework for learning distributed control strategies for morphogenesis, swarm robotics, and programmable self-assembly.
♻ ★ When Words and Bodies Disagree: Realizing Verbal and Nonverbal Mismatch
Traditional nonverbal behavior generation systems for virtual agents typically take utterances as input and generate behaviors that match or illustrate the spoken words. However, human nonverbal behavior is shaped by more than speech content. It is influenced by speaker roles, interpersonal relationships, social context, and cognitive and emotional states of interactants. Therefore, the nonverbal channel carries information or emotions hidden in words. Modeling these complex verbal-nonverbal mismatches is critical for realistic virtual agents, particularly in training applications. We propose a taxonomy of mismatches, strengthening, mitigating, and leaking, and evaluate whether LLMs can select contextually appropriate mismatched behaviors. Through human-subject studies, we assess whether these context-driven behaviors, realized in a virtual human framework, produce the intended effects. Our results demonstrate that LLMs, when prompted with social context, successfully select targeted mismatched behaviors (e.g., sarcasm). Furthermore, the human-subject evaluation confirms that these nonverbal cues significantly shift observers' perceptions of the agent's attitude and emotional valence. Participants successfully distinguished between distinct mismatch functions, such as strengthening, mitigating, and leaking, perceiving clear directional shifts in social meaning compared to matching baselines.
♻ ★ Embodiment-aware control by inference over the operator: a simulation study
Teleoperation and assistive devices are tuned for channel fidelity, while whether the operator experiences the device as part of the body, the Sense of Embodiment, is measured only afterwards. Minimizing the mismatch between the operator's predictions and the returned feedback fails twice: the predictions cannot be observed, and removing feedback minimizes the objective. We formulate an embodiment-aware controller that instead infers the operator's embodiment and visuo-proprioceptive cue weighting from implicit gaze and pupil signals and task outcome, and chooses bounded device settings under explicit preferences, cast as a discrete Active Inference agent. In simulations with heterogeneous synthetic operators, it raised embodiment from 0.76 to 1.66 on a 0-2 scale over the best fixed setting, within 0.02 of an oracle across ten random seeds; adaptation that did not read the operator gained nothing, and a prediction-error minimizer withheld feedback. An expected-utility controller with the same inference did as well, so the benefit comes from inference over the operator with explicit preferences, not from information seeking. Rules on a single signal came close when that signal was informative and failed when it was not; the controller degraded less, but failed when it trusted an uninformative signal unless briefly calibrated. Against published human data, the model reproduced the embodiment-performance relation, and embodiment developed on a plausible time scale. The embodiment signals were redundant when task success tracked embodiment and became the controller's objective when it did not. Studies with people must first establish calibrated cue-weighting signals and effort-independent embodiment signals.
comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. 20 pages, 4 figures, 7 tables. Code and data: https://github.com/sa-falcone/uee-embodiment-sim (doi:10.5281/zenodo.24106761)
♻ ★ OneSteer: Unlocking Steerable General Humanoid Motion Tracking via One-Step Diffusion Whole-Body Control
Deployment objectives often specify desired outcomes without providing revised trajectories, while parts of the original reference must still be followed. We present OneSteer, a one-step diffusion whole-body controller that combines general motion tracking with selective test-time steering. To accommodate new objectives, differentiable task costs guide optimization of the generator's input noise with frozen weights, while body-group reference masks relax selected tracking requirements. Predicted state-action plans provide a shared interface for task optimization and feedback-conditioned execution. For nominal tracking, the model trained from scratch achieves the lowest local tracking error among the evaluated baselines on successful held-out AMASS and BONES clips. For steering, one frozen distilled controller adapts across diverse deployment objectives in simulation, raising the clean passage rate under a 1.25 m bar from 2% to 90%. On a physical Unitree G1, OneSteer completes all 27 nominal tracking trials across nine diverse whole-body motion sequences without falling. With a steering cost derived from a signed distance field built using only onboard LiDAR, it also passes under a 1.1 m low bar in four of five attempts and steps over a 12 cm hurdle without falling in all three attempts. Beyond control, prediction variation in the model trained from scratch ranks likely tracking failures without physics rollouts. This scoring method is more than 10x faster than simulation-based scoring with five rollouts per clip and guides targeted fine-tuning.
♻ ★ Autonomous robotic bridging using distributed swarm control without inter-agent communication
We describe SCARAB--Swarm-Capable Autonomous Robotic Aquatic Bridging. Using distributed swarm control and multi-model sensing of agents and docking targets, agents can localize themselves, join into formations and proceed to desired target locations. Our technologies would eventually allow the Army to perform unpredictable, dispersed river crossings, enhance crew survivability, and minimize the logistics footprint compared to the current Improved Ribbon Bridge. Our methods operate without GPS or RF communications, though these can be used in non-contested environments (e.g., civilian disaster relief for flooding, etc.). We demonstrate our system via physics-engine-based simulation of several agents using unmanned surface vehicles (USVs) in the presence of currents and wind.
comment: Work from Army SBIR/STTR A254-012 (contract number W51701-25-C-A253)
♻ ★ RoboMemory: A Brain-inspired Multi-memory Agentic Framework for Interactive Environmental Learning in Physical Embodied Systems
Long-horizon embodied tasks require agents to learn, reason, and generalize robustly across complex real-world environments. However, existing approaches often struggle with partial observability, fragmented spatial reasoning, and inefficient integration of heterogeneous memories, limiting their capacity for long-horizon adaptation. To address this, we introduce RoboMemory, a brain-inspired framework that unifies Temporal, Spatial, Semantic, and Episodic memory for efficient long-horizon planning and interactive learning. Its core innovations are a comprehensive memory system designed for embodied tasks, and a dynamic spatial knowledge graph for scalable, consistent memory updates. Extensive experiments on EmbodiedBench show that RoboMemory improves the average success rate of Qwen2.5-VL-72B-Ins by 26.5 percentage points and achieves performance comparable to strong closed-source VLM agents. Real-world trials further suggest RoboMemory's potential for interactive environmental learning by reusing accumulated experience across repeated task attempts. These results demonstrate RoboMemory as a scalable memory-augmented framework for embodied agents, improving long-horizon planning and cumulative learning in complex interactive environments.
comment: Accepted at the 10th Conference on Robot Learning (CoRL 2026)
♻ ★ PA-SaR: Perception-Aware Learning, Control, and Coordination for Multi-Robot Search and Rescue
Autonomous SaR requires robots to explore unknown or partially known environments, avoid hazards, coordinate team activities, and guide discovered victims to safety. Improving individual capabilities of robots does not necessarily improve mission outcomes: safer motion can restrict the exploration behavior, faster discovery can increase the rescue demand, and effective guidance depends on victim responses. We introduce PA-SaR, a mathematical and architectural framework for coordinating heterogeneous local control, learning-based exploration, supervisory task allocation, and adaptive approach-lead victim guidance around mission-level objectives. PA- SaR distinguishes environmental ground-truth, robot-specific perceptions, and simulated victim behavior. This provides a common basis for integrating complementary mechanisms and analyzing their interactions across mission stages. We demonstrate the framework through single- and multi-robot simulation studies incorporating model-predictive and learning-based exploration, perception-based safety shielding, event-driven task allocation, and adaptive victim guidance. The results show trade-offs between collision avoidance and search completion, indicate that supervisory allocation improves search completion beyond information sharing alone, and demonstrate that increased exploration does not necessarily yield more rescues. Adaptive guidance increases rescues relative to continuous leading, without a detected improvement in mean or final victim trust or a reduction in mortality. These findings support evaluating local control, team coordination, and evacuation as related but distinct aspects of autonomous SaR within the considered simulation environments.
♻ ★ Self-supervised Physics-Informed Manipulation of Deformable Linear Objects with Non-negligible Dynamics
We address dynamic manipulation of deformable linear objects by presenting SPiD, a physics-informed self-supervised learning framework that couples an accurate deformable object model with an augmented self-supervised training strategy. On the modeling side, we extend a mass-spring model to more accurately capture object dynamics while remaining lightweight enough for high-throughput rollouts during self-supervised learning. On the learning side, we train a neural controller using a task-oriented cost, enabling end-to-end optimization through interaction with the differentiable object model. In addition, we propose a self-supervised DAgger variant that detects distribution shift during deployment and performs offline self-correction to further enhance robustness without expert supervision. We evaluate our method primarily on the rope stabilization task, where a robot must bring a swinging rope to rest as quickly and smoothly as possible. Extensive experiments in both simulation and the real world demonstrate that the proposed controller achieves fast and smooth rope stabilization, generalizing across unseen initial states, rope lengths, masses, non-uniform mass distributions, and external disturbances. Additionally, we develop an affordable markerless rope perception method and demonstrate that our controller maintains performance with noisy and low-frequency state updates. Furthermore, we demonstrate the generality of the framework by extending it to the rope trajectory tracking task. Overall, SPiD offers a data-efficient, robust, and physically grounded framework for dynamic manipulation of deformable linear objects, featuring strong sim-to-real generalization.
comment: See the supplementary video at https://youtu.be/lgX2J-00TRM
♻ ★ Meituan-Robotics-0: A Vision-Language-Action Foundation Model for Desktop Manipulation
We present Meituan-Robotics-0 (MR0), a vision-language-action foundation model and embodied-agent system for stationary-base desktop manipulation. MR0 combines 147 million vision-language samples with 15,060 hours of curated robot trajectories spanning 12 embodiments. A three-stage training recipe aligns multimodal representations with robot behavior through discrete action-token pre-training, transfers these priors to continuous cross-embodiment control through flow-matching mid-training, and adapts the policy through real-world post-training. We introduce Rollback, which pairs replaced policy segments with human corrections to provide positive and negative supervision for CFG-DAgger without value learning or advantage labeling. A unified end-effector interface supports heterogeneous embodiments, while asynchronous action-chunk merging maintains smooth execution under variable inference latency. An event-driven agent maintains persistent task events and reassesses plans when progress or feasibility changes, enabling cross-task retrieval and conditional task resumption. MR0 achieves 93.6% success on RoboTwin 2.0 and ranks fifth in the RoboDojo VLA track. Under matched HG-DAgger post-training, MR0 achieves higher average success rate and progress score than $π_{0.5}$ across three real-robot tasks. On Ethernet insertion, CFG-DAgger trained with Rollback data reaches 70% success and a progress score of 0.90. Qualitative agent deployments demonstrate cross-task event retrieval and recovery from temporarily unsatisfied task conditions.
comment: 25 pages. Project page: https://meituan-robotics.github.io/Meituan-Robotics-0/ ; Code: https://github.com/Meituan-Robotics/Meituan-Robotics-0
♻ ★ Altitude-Adaptive UAV-to-Map Image Matching for Aerial Geo-Localization
Matching downward-looking unmanned aerial vehicle (UAV) images to georeferenced satellite or aerial map tiles supports map-based geo-localization when direct positioning signals are degraded or unavailable. Variations in UAV altitude change the observed ground footprint and create a scale mismatch between query images and fixed-scale map references. We address this problem with an altitude-adaptive aerial geo-localization framework. The framework estimates relative altitude from a single image using a frequency-domain representation and regression-as-classification. The predicted altitude-class center defines a scale prior that guides query normalization toward the canonical map scale before full-gallery visual place recognition. A footprint-adaptive margin classifier (FAMC) adjusts training margins according to footprint overlap. Experiments on two synthetic and two real-flight datasets show that, at the 100 m localization threshold, altitude adaptation improves average R@1 and R@5 by 50.68 and 47.20 percentage points, respectively, for our visual place recognition module compared with the same retrieval pipeline without altitude normalization. These results show that a relative-altitude-derived scale prior improves cross-altitude UAV-to-map image matching in the evaluated regions and camera settings.
♻ ★ ODEAction: From Predicting Chunks to Evolving Actions
Vision-language-action (VLA) models typically generate actions in chunks, dividing continuous robot execution into separately predicted action chunks, which can introduce discontinuities at chunk boundaries. We propose ODEAction, a chunk-free action generation paradigm that replaces action chunk prediction with Continuous Action Evolution. We use an ordinary differential equation (ODE) to describe the continuous evolution of actions, and define the vector field of this ODE as the Action Transition Field to characterize how actions evolve with respect to evolution time. During inference, the model takes the current action as the initial condition and numerically solves the ODE defined by the learned Action Transition Field, thereby recursively generating subsequent actions with explicit dependencies. ODEAction is implemented as a modular action head that can be integrated with different vision-language backbones. Experiments on LIBERO, EBench, RoboTwin, and real-robot tasks show that, compared with action-chunk-based baselines, ODEAction improves both task performance and trajectory quality, with particularly pronounced gains on fine manipulation tasks.
comment: 23 pages, 5 figures, including appendix. Technical report
♻ ★ A study on the effects of mixed explicit and implicit communications in human-artificial-agent interactions
Communication between humans and artificial agents is essential for their interaction. This is often inspired by human communication, which uses gestures, facial expressions, gaze direction, and other explicit and implicit means. This work presents interaction experiments where humans and artificial agents interact through explicit and implicit communication to evaluate the effect of mixed explicit-implicit communication against purely explicit communication and the impact of the task difficulty in this evaluation. Results obtained using Bayesian parameter estimation show that the task execution time did not significantly change when mixed explicit and implicit communications were used in neither of our experiments, which varied in the type of artificial agent (virtual agent and humanoid robot) used and task difficulty. The number of errors was affected by the communication only when the human was executing a more difficult task, and an impact on the perceived efficiency of the interaction was only observed in the interaction with the robot, for both easy and difficult tasks. In contrast, acceptance, sociability, and transparency of the artificial agent increased when using mixed communication modalities in both our experiments and task difficulty levels. This suggests that task-related measures, such as time, number of errors, and perceived efficiency of the interaction, as well as the impact of the communication on them, are more sensitive to the type of task and the difficulty level, whereas the combination of explicit and implicit communications more consistently improves human perceptions about artificial agents.
comment: Main paper with 33 pages, 16 figures, 5 tables. Supplementary material with 39 pages, 44 figures, 2 tables. Published in Intelligent Service Robotics
Multiagent Systems
★ Safety and Connectivity Certificates for Finite-Time Formation Transitions
Finite-time formation transitions are useful for coordinating multi-agent systems in time-critical missions, but convergence to a prescribed formation in finite time does not guarantee collision avoidance or communication connectivity during the transition. Existing approaches address finite-time coordination and safety-constrained formation control, yet explicit connections between disagreement during transitions, formation geometry, and coupling-gain selection need to be developed for certifying safe maneuvers. This paper develops a disagreement-based framework for certifying transient safety and connectivity during finite-time tracking of arbitrary prescribed formation trajectories. Local feedback guarantees exact finite-time tracking with a graph-independent settling-time bound, while a graph-dependent disagreement bound is used to derive sufficient geometric conditions for collision avoidance and connectivity preservation. These conditions further yield an offline coupling-gain selection rule based on the available geometric clearance. Numerical simulations illustrate the trade-off between convergence rate and control effort as the coupling gain increases. They also demonstrate the effectiveness and conservatism of the proposed certificates.
comment: 7 Pages, 2 Figures, under review
★ DrugCoding: A Scientific Programming Agent for Drug Discovery
Drug discovery involves multiple interconnected research stages, including molecular design, property prediction, and virtual screening. These processes are characterized by complex workflows, interdependent computational tasks, and diverse specialized tools, placing substantial demands on the coordination of scientific models and computational resources. However, existing large language model (LLM)-based scientific agents still face challenges in autonomous planning of complex tasks, adaptive selection of specialized models, and reliable execution of multi-stage scientific computations, limiting their effectiveness in automating complex drug discovery workflows. To address these challenges, we propose DrugCoding, a scientific programming agent system that integrates scientific task planning, adaptive model routing, multi-agent collaboration, and programmatic tool execution. DrugCoding enables autonomous task planning and dynamic model adaptation for complex research objectives. It can flexibly invoke specialized scientific models, autonomously generate and execute code according to task requirements, and leverage multi-agent collaboration to support efficient execution and reliable validation of complex scientific tasks. To evaluate DrugCoding's performance in scientific tool invocation and multi-stage task execution, we conduct multi-turn tool-calling experiments across diverse language models and scientific tool interfaces. Experimental results demonstrate that DrugCoding achieves an average session-level Pass@1 of 97.35% on the full benchmark, outperforming the strongest baseline, Codex, by 9.24%. On the challenging subset, DrugCoding achieves 97.92%, exceeding the strongest baseline.
★ Spatial patterns of transportation choice in Niterói
Characterizing how people move through a city requires spatially and temporally fine-grained data that are often costly and difficult to obtain. Here, we introduce a computational framework that uses macroscopic, aggregated, and static urban data to construct a microscopic, dynamic, and spatially detailed representation of urban mobility. Building on an existing activity simulator, the framework combines a synthetic population, the built environment, and transport networks to generate daily trips and assign walking, car, or public transit. Applied to approximately 475,000 synthetic residents of Niterói, Brazil, simulated neighborhood-level transit patterns show a strong association with an empirical activity proxy (Pearson r=0.885). Walking is more prevalent in compact central areas, car use increases toward the periphery, and public-transit use remains comparatively high along bus corridors. Reducing walking times to and from bus stops by 30% increases the modeled public-transit-to-car ratio by 11.3%, with larger transit-share gains in several peripheral areas. These results reveal spatial differences in transportation choice and identify where improved transit access may have the greatest modeled effect.
★ ReTrace: Distilling Tool-Using Agents into Planner-Free Workflows
Business process automation runs the same Standard Operating Procedure (SOP) again and again on new inputs. Large Language Model (LLM) agents can follow an SOP written in natural language, but they call an LLM to choose every step of every run, which adds latency and cost at scale. We present ReTrace, which turns an SOP and a few logged agent runs into a reusable workflow in which code, not an LLM, selects the next step. The agent's reasoning, the explanation it writes with each tool call, often states why it took an action; the tool calls alone do not. ReTrace extracts the condition behind each step, merges these conditions across runs into a compact rule document, and generates an executable workflow from it. Only this last stage depends on the platform, so rule documents can be compiled for different orchestration platforms. On $79$ procedures from our production platform, ReTrace workflows reach $93.8\%$ completion, against $96.8\%$ for the agent, at $24\%$ lower cost per task. On $τ^2$-Bench, they reach $75.8\%$ task success (agent: $80.6\%$) and run in $1.3$s instead of $170$s; three rounds of repair on training failures lift $k{=}2$ workflows from $66.7\%$ to $81.2\%$. On eight SOP-Bench procedures, ReTrace without repair trails few-shot generation ($80.6\%$ vs. $83.6\%$), but repair from labeled training failures lifts it to $87.8\%$ at $0.40$s per task, compared to $85.5\%$ in $46.6$s for the agent.
★ Don't Just Trust Your Memory: Evidence-Linked Memory Reduces Memory-Induced Degradation in LLM Agents ACL
External memory enables LLM agents to improve from experience by distilling trajectories of past tasks into concise lessons for new tasks. However, memory can also lead to memory-induced degradation, where an agent fails on tasks it would solve without memory, often because it follows a lesson that is incorrect, or misapplies or misreads a valid one. This is because lessons are inferred from trajectories labeled only with task success or failure, yet are typically provided without the trajectories they came from, leaving the agent no way to check them. To address this, we introduce WikiTrace, an evidence-linked memory system designed to let agents check each lesson against its original trajectory. Specifically, motivated by the finding that a trajectory often still holds the evidence needed to catch an erroneous lesson, WikiTrace stores each lesson as a concise insight linked to a summary of why its trajectory succeeded or failed and to the raw trajectory itself. Then, on a new task, the agent starts from all insights and, whenever one lacks details needed to act, conflicts with another, or comes from a failed task, descends through its links with terminal tools to fill in missing details or correct the insight before acting. We validate WikiTrace on five agentic benchmarks, showing that it consistently outperforms existing memory approaches and a baseline given the same insights alone, while also reducing memory-induced degradation. Our code is available at https://github.com/yukyeongleee/wikitrace
comment: Submitted to ACL Rolling Review (October 2026 cycle); under review
★ Personalized Affective Perception via Continual Learning in Human-Agent Negotiation
Self-interested intelligent agents that negotiate with humans should interpret nonverbal expressions and adapt their behavior accordingly. However, generic facial-affect models degrade across individuals and contexts, creating a generalization gap. We introduce an affect-adaptive negotiation strategy that incorporates Continual Learning with Imagination for Facial Expression Recognition (CLIFER) to personalize valence-arousal perception in real time. Differential feedback converts changes in valence-arousal direction and intensity into adjustments of the agent's target utility. An asynchronous architecture updates CLIFER in the background during live interaction. In a counterbalanced within-subject study in which 66 participants negotiated with a humanoid robot, the CL-enabled agent achieved significantly higher individual utility and elicited higher participant concession rates than the FaceChannel baseline. The CL condition yielded significantly higher mean arousal estimates, alongside a broader output range. Participant utility and Nash distance did not change significantly between conditions. These outcomes support the practical value of coupling continual affective personalization with differential feedback for real-time negotiation.
comment: 17 pages, 11 figures. Submitted to IEEE Transactions on Affective Computing
★ PARLEY: Evaluating LLM Collaboration Through Cooperative Asymmetric Puzzles
As large language models power autonomous agents, understanding how they collaborate and where they fail becomes critical. We introduce PARLEY, an extensible platform for studying LLM agent collaboration through four asymmetric puzzle tasks grounded in cognitive psychology (referential communication, joint coordination, perspective-taking, and spatial reasoning). GPT-4o agent pairs succeeded in 60.5% of conversations (N=400, 100 per task), with task-level variation (34-80%) reflecting distinct collaborative demands. From these conversations, we derived the data-driven PAC taxonomy: six failure types organized into Perception (visual interpretation and spatial reasoning), Adaptation (progress tracking and feedback integration), and Coordination (verification and grounding). Failure to incorporate external feedback was a critical indicator of unsuccessful collaboration, with agents repeatedly applying unsuccessful strategies. We also show that a taxonomy-guided intervention (N=200) improved average task success to 70.5%. These results demonstrate how PARLEY can be used to derive meaningful, data-driven insights that can in turn lead to improved agentic collaboration.
comment: 12 pages, 8 figures, 2 tables
★ Open Challenges and Opportunities in Human-Agent Systems AAMAS
Agentic AI systems increasingly undertake extended tasks and act on people's behalf, making human-agent interaction central to their effectiveness. Successful interaction requires understanding human perspectives, coordinating complementary capabilities, and maintaining productive relationships within socio-technical environments. This paper examines how foundations developed by the Autonomous Agents and Multi-Agent Systems (AAMAS) community inform these demands. We consider interactions ranging from individual human-agent partnerships to teams and larger groups, where participants differ in their roles, decision authority, capabilities, and objectives. We organize computational approaches around the connected challenges of understanding others and deciding how to interact with them. Theory of Mind, belief-desire-intention architectures, goal and plan recognition, inverse planning, and recursive reasoning provide mechanisms for representing and inferring partially observable mental states. Observer-aware planning, model reconciliation, and decision-theoretic communication connect these inferences to action, explanation, and information exchange. Together with shared autonomy, argumentation, and cooperative inverse reinforcement learning, these approaches offer foundations for agentic systems that account for evolving human knowledge, intentions, and expectations. We argue that progress requires treating human interaction as an ongoing decision problem shaped by uncertainty, interdependence, and potentially conflicting objectives.
comment: Part of The AAMAS Community Agentic AI Report
★ RE-PRESENT: Evidence Triangulation across Alternative Candidate-Space Views for Efficient LLM Ensemble Inference
Heterogeneous large language models (LLMs) exhibit complementary strengths, motivating routing and multi-model inference methods that seek to identify reliable predictions for each input. However, exploiting this complementarity remains challenging even when a correct prediction is available within the ensemble. We investigate this challenge across heterogeneous LLM ensembles and find that a simple single-round aggregation rule is highly competitive with learned routing methods, yet substantial oracle headroom remains. Further analysis shows that the evaluated single-round response signals provide limited discriminative power for identifying reliable predictions. In contrast, controlled re-presentations of the same question elicit complementary evidence about prediction reliability. Motivated by these findings, we introduce RE-PRESENT, a training-free framework that selectively re-presents disputed questions and directly aggregates candidate-level evidence across alternative views. Across seven multiple-choice benchmarks and two heterogeneous model ensembles, RE-PRESENT improves over strong single-round aggregation while offering a favorable accuracy--cost trade-off compared with self-consistency and multi-model debate.
♻ ★ Automated Root-Cause Subclassification and No-Code Fix Generation for Invalid Bug Reports
Issues faced when using software are reported in the form of bug reports. However, many bug reports are invalid, meaning they do not require code changes, and are resolved with a no-code fix. Manually determining the root cause of the invalid bug reports and providing actionable resolutions by the customer support causes a serious waste of resources. Our goal is to introduce a standardized taxonomy for root-cause oriented invalid bug report subclassification, and perform experiments to test the accuracy of various approaches on invalid subclassification and no-code fix generation. We study how different configurations perform on a gold-standard benchmark we have created. Using a manually curated benchmark for higher quality analysis, we experimented with vanilla LLMs, Retrieval Augmented Generation, and agentic web search to identify invalid subclasses and generate no-code fixes. We evaluated the results against manually labeled ground truth data that includes the invalid subclass and no-code fixes from the original bug reports. We measured subclass detection performance with weighted F1-Score, and assessed no-code fix suggestions using BERTScore and Judge LLM success rates. For subclassification, retrieval augmented generation achieves the highest overall performance with 0.66 weighted F1, slightly outperforming vanilla LLMs at 0.65 and agentic web search at 0.64. At the subclass level, performance peaks at 0.85 F1 for Non-reproducibility and 0.79 for Feature Request and Question, while Wrong Version remains the most challenging with scores between 0.00 and 0.29. For no-code fix generation, agentic web search achieves the highest overall Judge LLM success rate at 68.9%, compared to 64.4% for RAG applications and 64.9% for vanilla LLMs, with subclass-level peaks of 87.4% for Working as Designed and 72.2% for Question.
comment: Submitted to IEEE Transactions on Software Engineering (TSE) and currently under review
♻ ★ Adaptive stabilization of a leaderless bearing-constrained formation with disturbances
In this paper, we consider the problem of regulating and maintaining a target formation characterized by a set of bidirectional bearing constraints under disturbances. The agents in the formation are modeled by single integrators with deterministic bounded continuous disturbances of which both actual function and upper bound are unavailable for the control design. Due to the time-varying disturbances, the target formation is time-varying. We propose displacement/bearing-only based adaptive sliding mode control laws to uniformly asymptotically stabilize the time-varying target formation and reject the matched disturbances, and provide sufficient conditions to prevent possible collisions. A time-dependent adaptive design method is provided to convert the displacement-based control law to a bearing-only one. In addition, to alleviate chattering phenomena from sliding mode control, smooth adaptive control laws are introduced to guarantee uniform global boundedness of the desired formation. Finally, simulation results are given to support the analysis.
comment: 14 pages, 6 figures, preprint, submitted to a journal
♻ ★ Towards using Large Language Models for Risk Negotiation and Consensus-building in One Health contexts
Risk negotiation under the One Health framework requires balancing competing stakeholder interests across human, animal, and environmental health domains - a process constrained by information complexity and time pressure. This study presents and evaluates a multi-step pipeline that integrates large language model-based agents into a human-supervised negotiation process, specifically targeting risk assessment, valuation, and negotiation steps. Agent descriptions were generated from stakeholder-provided materials using retrieval-augmented generation methods. During preliminary multi-agent discussions, agents accessed domain-specific computational models and web-based sources before human participants refined proposed issues and options. A simulated negotiation game then produced distributions of candidate deals, characterised using Nash bargaining solution criteria and Pareto optimality metrics. The developed approach was evaluated in a workshop involving independent domain experts across two real-world scenarios: food safety criteria for Campylobacter in broiler meat and oak processionary moth management. In this feasibility evaluation, participants reached consensus with the allotted time after discussing and modifying the proposed deals. The framework offers an adaptable blueprint for AI-assisted risk negotiation; the specific contribution of LLM assistance, including gains in efficiency and decision quality, requires comparative evaluation.
♻ ★ Multi-User Large Language Model Agents
Large language models (LLMs) and LLM-based agents are increasingly deployed as assistants in planning and decision making, yet most existing systems are implicitly optimized for a single-principal interaction paradigm, in which the model is designed to satisfy the objectives of one dominant user whose instructions are treated as the sole source of authority and utility. However, as they are integrated into team workflows and organizational tools, they are increasingly required to serve multiple users simultaneously, each with distinct roles, preferences, and authority levels, leading to multi-user, multi-principal settings with unavoidable conflicts, information asymmetry, and privacy constraints. In this work, we present the first systematic study of multi-user LLM agents. We begin by formalizing multi-user interaction with LLM agents as a multi-principal decision problem, where a single agent must account for multiple users with potentially conflicting interests and associated challenges. We then introduce a unified multi-user interaction protocol and design three targeted stress-testing scenarios to evaluate current LLMs' capabilities in instruction following, privacy preservation, and coordination. Our results reveal systematic gaps: frontier LLMs frequently fail to maintain stable prioritization under conflicting user objectives, exhibit increasing privacy violations over multi-turn interactions, and suffer from efficiency bottlenecks when coordination requires iterative information gathering.
♻ ★ DiffeoMorph: Learning Distributed 3D Morphogenesis with Rotation-Invariant Spectral Shape Matching
Biological systems can form complex three-dimensional structures through the collective behavior of agents that share a common update rule and operate without central control. How such distributed control gives rise to precise global patterns remains a central question not only in developmental biology but also in distributed robotics, programmable matter, and multi-agent learning. Here, we introduce DiffeoMorph, an end-to-end differentiable framework for learning a morphogenesis protocol that guides a population of agents to morph into a target 3D shape. Each agent updates its position and internal state using an SE(3)-equivariant graph neural network, based on its own internal state and signals received from other agents. To train this system, we introduce a new shape-matching loss based on 3D Zernike polynomials, which compares the predicted and target shapes as continuous spatial distributions, not as discrete point clouds, and is invariant to agent ordering, number of agents, and global orientation. To achieve rotation invariance while preserving reflection sensitivity, we include an alignment step that optimally rotates the predicted Zernike spectrum to match the target before computing the loss. We perform benchmarking to establish the advantages of our shape-matching loss over other standard distance metrics for shape comparison tasks. We then demonstrate that DiffeoMorph can form a range of complex shapes from minimally patterned initial conditions. DiffeoMorph provides a general framework for learning distributed control strategies for morphogenesis, swarm robotics, and programmable self-assembly.
♻ ★ PA-SaR: Perception-Aware Learning, Control, and Coordination for Multi-Robot Search and Rescue
Autonomous SaR requires robots to explore unknown or partially known environments, avoid hazards, coordinate team activities, and guide discovered victims to safety. Improving individual capabilities of robots does not necessarily improve mission outcomes: safer motion can restrict the exploration behavior, faster discovery can increase the rescue demand, and effective guidance depends on victim responses. We introduce PA-SaR, a mathematical and architectural framework for coordinating heterogeneous local control, learning-based exploration, supervisory task allocation, and adaptive approach-lead victim guidance around mission-level objectives. PA- SaR distinguishes environmental ground-truth, robot-specific perceptions, and simulated victim behavior. This provides a common basis for integrating complementary mechanisms and analyzing their interactions across mission stages. We demonstrate the framework through single- and multi-robot simulation studies incorporating model-predictive and learning-based exploration, perception-based safety shielding, event-driven task allocation, and adaptive victim guidance. The results show trade-offs between collision avoidance and search completion, indicate that supervisory allocation improves search completion beyond information sharing alone, and demonstrate that increased exploration does not necessarily yield more rescues. Adaptive guidance increases rescues relative to continuous leading, without a detected improvement in mean or final victim trust or a reduction in mortality. These findings support evaluating local control, team coordination, and evacuation as related but distinct aspects of autonomous SaR within the considered simulation environments.
♻ ★ MARS: Towards High Goodput and Low KV Thrashing in Agent Serving under GPU Memory Constraints
Self-hosted LLM agents offer greater control over sensitive data and model customization while avoiding per-request API fees. However, serving concurrent, multi-turn workflows within limited GPU memory makes it difficult to achieve high workflow-level goodput. In particular, as workflows accumulate context, their reusable key-value (KV) caches can easily outgrow GPU capacity, triggering repeated eviction and recovery, known as KV cache thrashing. This process consumes resources and delays workflow completion. Although existing approaches propose various methods, our motivation study shows that the evaluated baselines still incur substantial thrashing overhead or fail to address the workflow-level goodput issue. We present MARS, an agent serving system that coordinates workflow admission and in-engine scheduling to achieve high workflow-level goodput on memory-constrained devices while avoiding KV cache thrashing. Specifically, outside the engine, an admission controller selects root workflows using initial prefix footprint estimates and live KV availability, adapting admissions to memory pressure. Inside, an agent-centric scheduler prioritizes turn completion and coordinates prefill allocation with KV reclamation to reduce disruptive evictions. Across two models and three GPU configurations, MARS achieves 1.59x-7.56x mean end-to-end latency speedups over the fastest baseline. Integrated as the serving backend for OpenCode, Codex, and OpenHands, MARS further improves mean workflow completion time by 1.17x-1.87x. Our source code is publicly available at https://github.com/Afterglow231/MARS_preview .
comment: 13 pages, 11 figures. Major revision with updated title, system design, and expanded evaluation. Preprint
♻ ★ Steering Emergent Behavior in Multi-Agent LLM Systems via Information Networks
Large language model (LLM) agents increasingly operate in groups, each pursuing an individual objective while jointly determining system performance. If an operator controls which agent observes which other agent, but not their payoffs or messages, how should observations be structured to raise collective welfare? We make the information network (who observes whom) a design object, decoupled from the fixed interaction network over which agents act. Ten identical LLM agents play repeated group-decision games; by default each observes only an anonymous summary of the group's previous-round outcome, and a manager with a fixed budget adds directed observation links that reveal peers' past actions. Across 1,080 episodes and six games spanning coordination, anti-coordination, social dilemmas, and discovery, we ask how much information to share and, at a fixed budget, who should observe whom. Disclosure creates behavioral coupling: observation acts as a contagion channel whose welfare effect depends on the alignment between individually attractive behavior and the collective objective. First, the value of information is alignment-contingent: more peer information tends to help or be neutral when individual behavior is aligned with the collective goal, but under free-riding or congestion externalities it amplifies collectively harmful behavior; in a public-goods game the group does best with no information at all. We read this sign separation as a cross-game regularity within one controlled LLM population, not a model-invariant law. Second, at a fixed budget, who observes whom matters primarily when coupling is harmful: a modular "barbell" that lets sub-populations entrench harmful behavior is reliably the worst wiring, reversing the network-reciprocity intuition from human evolutionary game theory. These results argue against a default of full transparency for agent collectives.
comment: It is preliminary and is not suitable for public release yet
♻ ★ Agon: An Autonomous Large-Scale Omnidisciplinary Research System Built on Prompt Economy
Large language models are making research scalable. We present \textsc{Agon}, a research orchestrator designed to probe the limits of automated research. \textsc{Agon} is built on six design principles: Prompt Economy, Future-Facing, Minimal Prompts, OmniDisciplinary, Massive Parallelism, and Zero-Code. We deployed \textsc{Agon} across domains using only brief topic descriptions as starting points and no human-written code. These deployments demonstrate scalability but also reveal previously unseen classes of failure that are difficult to address through prompt engineering. We organize these failures into a taxonomy along four dimensions: severity, fixability, visibility, and capability locus. Together, these results show that \textsc{Agon} is pushing research towards a new paradigm: machines scale, humans steer. Our code is publicly available at \href{https://github.com/AutoResearch-Factory/Agon/tree/lite}{this GitHub repository}.
♻ ★ Closed-Loop Molecular Design with Calibrated Deference
As AI systems become increasingly capable scientific collaborators, a fundamental challenge that arises is not only generating explanations, but calibrating confidence in them. We present Discovery, a multi-agent scientific system designed to assess the reliability of tools and data during adaptive reasoning, track evidence and assumptions while resolving uncertainty using a persistent graph memory. Discovery enables a capability we term calibrated deference: the ability to adjust confidence in its tool use, assumptions, and mechanistic explanations based on these assessments, communicate uncertainty to human collaborators, and adapt strategies accordingly. We evaluated Discovery in a closed-loop human-AI campaign to design an aqueous organic redox flow battery negative electrolyte. Discovery proposed candidates and interpreted results in collaboration with chemists, who synthesized, characterized, and guided design. Across 17 candidates over three rounds, Discovery converged on a candidate with a redox potential improved by 130 mV relative to the literature baseline. However, experimental characterization revealed poor electrochemical reversibility that the predictors used in the campaign could not anticipate. In response, Discovery generated competing mechanistic hypotheses, prioritized discriminating diagnostics, and traced the behavior to phosphonate-potassium ion pairing. Discovery designed a sulfonate analog to reduce potassium-ion pairing, and the resulting compound improved electrochemical reversibility while maintaining a 90 mV improvement in redox potential, closing the design-make-test-redesign loop. Across the campaign, Discovery demonstrated the ability to maintain a coherent world model, updating in response to unexpected results, and leverage it to drive molecular design.
Systems and Control (EESS)
★ Safe and Scalable Minimax Adaptive Control of Positive Systems
This paper studies minimax adaptive control of positive linear systems with fixed unknown dynamics and nonnegative disturbances. We reformulate the problem by compressing the historical data into a vector and a positivity-induced consistency set containing the models compatible with the observations. Together with the physical state, these quantities form a sufficient information state and yield an exact reformulation as a standard minimax dynamic game. We allow the admissible input set to adapt to the current consistency set, ensuring positivity for every remaining feasible model. As inconsistent models are eliminated, the input set can expand, allowing less cautious control actions. For finite model sets, we derive an explicit solution to the Bellman inequality and construct a safe and stabilizing certainty-equivalence controller with robustness guarantees. The framework also accommodates structured classes of continuum model uncertainty whose consistency sets admit fixed, finite-dimensional representations, enabling scalable information-state updates.
comment: 20 pages, 1 figure (6 panels), 1 table
★ Distributionally Robust Indirect-Feedback Stochastic MPC with Gelbrich Ambiguity Sets
Stochastic model predictive control uses disturbance statistics to set safety margins. Errors in these statistics can invalidate the safety guarantees. This paper therefore describes the uncertainty in the disturbance mean and covariance by a prescribed Gelbrich ambiguity set. This contains distributions of any shape whose moments are close in the Gelbrich distance to the nominal values, including distributions with unbounded support. For linear time-invariant systems with additive disturbances, we build on the indirect feedback paradigm, propagate this moment uncertainty along the error dynamics, and obtain a closed-form constraint tightening that is computed offline. Robustness enters as a single additive term that grows with the size of the ambiguity set, so the online problem remains a quadratic program. Recursive feasibility and closed-loop satisfaction of conditional value-at-risk (CVaR) constraints hold for every disturbance distribution in the set. We illustrate the method on a linearized planar quadrotor.
comment: 6 pages, 2 figures, 1 table. Submitted to the 2027 European Control Conference (ECC)
★ Excitable Behaviours of a First-Order High-Pass Neuron Model
With the rise of neuromorphic technologies, simple and tunable neuron models are needed to analyse and design spiking systems. However, the simplest neuron models, e.g., Leaky Integrate-and-Fire (LIF) models, usually fail to capture the excitable behaviours of biological neurons. This could be a significant limitation, as excitability enables reliable spike timing under parametric uncertainties. Recently, a minimal excitable model, the so-called High-Pass Model (HPM), was introduced in neuroengineering. It simply consists of a static sigmoid with a high-pass filter in positive feedback. Given its novelty, it is still unclear how easily the HPM can be tuned, and how much of the excitable behaviours of a biophysical neuron it can reproduce. To address these questions, we investigate the tunability of a first-order HPM sub- and supra-rheobase, i.e., below and above the onset of the tonic spiking regime (where the neuron spikes periodically), and compare its behaviours with those of a simple Hodgkin-Huxley-type Conductance-Based Model (CBM). Below rheobase, both models fire spikes in response to transient inputs, including post-inhibitory rebound spikes. Above rheobase, an affine reparametrization of the input of the HPM suffices to fit its frequency-current curve to that of a CBM. The two models mostly differ near their respective rheobases, where the first-order HPM lacks bistability and spike latency. We suggest that modifying the frequency-selective element of the HPM may be sufficient to overcome this limitation. These results highlight the potential of the HPM as a fundamental computational primitive in neuromorphic, spike-based technologies.
comment: 6 pages, 8 figures
★ Safety and Connectivity Certificates for Finite-Time Formation Transitions
Finite-time formation transitions are useful for coordinating multi-agent systems in time-critical missions, but convergence to a prescribed formation in finite time does not guarantee collision avoidance or communication connectivity during the transition. Existing approaches address finite-time coordination and safety-constrained formation control, yet explicit connections between disagreement during transitions, formation geometry, and coupling-gain selection need to be developed for certifying safe maneuvers. This paper develops a disagreement-based framework for certifying transient safety and connectivity during finite-time tracking of arbitrary prescribed formation trajectories. Local feedback guarantees exact finite-time tracking with a graph-independent settling-time bound, while a graph-dependent disagreement bound is used to derive sufficient geometric conditions for collision avoidance and connectivity preservation. These conditions further yield an offline coupling-gain selection rule based on the available geometric clearance. Numerical simulations illustrate the trade-off between convergence rate and control effort as the coupling gain increases. They also demonstrate the effectiveness and conservatism of the proposed certificates.
comment: 7 Pages, 2 Figures, under review
★ Online Adaptive Clustering in Robust Distributed Model Predictive Control
This paper proposes a dynamic clustering framework for tube-based model predictive control (MPC) of coupled multi-agent systems. Agents within a cluster coordinate their dynamics explicitly, while couplings from agents outside the cluster are treated as bounded disturbances. For each admissible cluster, we construct a state-dependent mismatch measure from a robust upper bound on its predicted closed-loop cost and an optimistic, partition-independent lower bound. Combining this mismatch with a communication penalty enables online selection of the clustering topology, ranging from centralized to fully decentralized operation. The resulting switching controller is shown to preserve recursive feasibility and constraint satisfaction, and to render the closed-loop system practically asymptotically stable. Furthermore, the mismatch provides an upper bound on the finite-horizon performance difference with respect to centralized MPC. An eight-agent spring--damper example illustrates how the controller adapts cooperation while maintaining performance close to centralized MPC with reduced communication.
comment: 12 pages, 5 figures, journal
★ The Role of Insurance in Power-System Risk Management: A Critical Review
Modern power systems are becoming more renewable-rich, digitalized, and interconnected, making risk propagation and economic consequences increasingly complex. Physical mitigation remains the primary means of risk control, but technical and economic limits leave residual risk. Insurance can complement these measures by transferring associated economic losses and influencing power-system decisions through risk-responsive terms. Despite growing interest, the literature remains fragmented across power-system engineering, actuarial science, and risk finance. This review develops an integrated physical-financial perspective for reviewing insurance in power systems. It examines how power system risks generate residual economic losses and the conditions under which these losses can be insured and transferred. It then develops an actuarial framework for insured-loss assessment and pricing. Insurance-aware decision-making is synthesized through four coupling levels describing how financial protection interacts with power-system decisions. Further progress is constrained by weak physical-financial loss validation, incomplete representation of systemic and evolving portfolio risk, and limited empirical and actuarial grounding of insurance-decision feedback. Research priorities include stronger physical-financial data linkage, forward-looking portfolio assessment, multi-timescale decision feedback, and empirical validation. Addressing these priorities would help move power-system insurance from fragmented applications toward a more coherent approach to residual-risk management in future power systems.
★ Towards Mode-Balanced Intersection Control: Hierarchical Coordination of Signal Timings, AV Trajectories, and Cyclist Speed Advice
Connected and automated vehicles (AVs) enable traffic signals and vehicle trajectories to be optimised jointly, but existing co-optimisation frameworks overlook cyclists, even though they share the same intersections. We propose a hierarchical framework that optimises traffic signal timings, AV trajectories, and cyclist speed advice to reduce travel times for both motorised vehicles and cyclists. The upper level jointly optimises signal timings and cyclist speed advice, while the lower level optimises AV trajectories and also updates cyclist speed advice obtained from the higher level at a higher frequency based on the resulting signal plan. The framework is evaluated in simulation under multiple demand profiles and optimisation parameter settings. Compared with optimized vehicle trajectories with fixed-time signal control, the proposed framework reduces mean travel times by 13.6\% for AVs and 12.3\% for cyclists. Adding cyclist speed advice further reduces mean cyclist waiting time by about 80\% relative to the same framework without speed advice. These results demonstrate substantial benefits for both modes of transport.
★ Towards Design of Efficient Neuromorphic Controllers
One of the major sources of inspiration for neuromorphic engineering is the central nervous system, which is orders of magnitude more efficient than classical digital clock-based computers. Neuromorphic control aims at recreating this efficiency in control systems, combined with (formal) stability and performance guarantees. In our previous work [1], we have established formal guarantees for the emulation of static output feedback controllers implemented using integrate-and-fire neurons. In this paper, we investigate the ``efficiency'' of the resulting controllers in terms of sparse spiking activity. By changing the network structure of our original implementation, foregoing the unit gain assumption made in our previous work, we will establish that the chosen architecture of the spiking controller significantly affects its sparsity and discuss the implications of these observations for the efficient implementation of spiking controllers.
comment: Submitted to European Control Conference 2027
★ Learn-To-Propagate: Distributionally Robust Motion Planning for Nonlinear Uncertain Systems
Motion planning for stochastic systems aims to find a sequence of control inputs that drives a system to a goal set while avoiding unsafe states with high probability. Existing approaches, however, are either restricted to linear dynamics with perfect knowledge of the underlying distributions or lack formal correctness guarantees. To address these limitations, we propose \emph{Learn-To-Propagate Distributionally-Robust Forest (L2P-DR-Forest)}, a distributionally-robust sampling-based planning algorithm for \emph{nonlinear} stochastic systems with \emph{unknown} initial state and process noise distributions. Our approach leverages recent advances in optimal transport to learn ambiguity sets that contain the true system distributions with high confidence. We develop an efficient method for propagating these sets through the system dynamics and for verifying their safety. We then integrate these techniques into a novel sampling-based planning algorithm to grow a forest of search trees in the system's state space while accounting for and mitigating the growth of ambiguity-sets radii. We prove that the algorithm is correct, i.e., every returned plan satisfies the specifications with the prescribed confidence, and, under some feasibility and distributional assumptions, is probabilistically complete. Evaluations on several robotic benchmarks demonstrate that our algorithm significantly outperforms existing methods while providing formal guarantees with high confidence.
★ Optimal Last-Iterate Bounds for Nonexpansive Stochastic Approximation
We establish last-iterate mean-square residual bounds for nonexpansive stochastic approximation (SA), or stochastic Krasnoselskii-Mann (KM) iterations. In particular, we consider fixed-point mappings that are nonexpansive with respect to the Euclidean norm, and focus on polynomially decaying stepsizes. We study three conditions on the conditional second moment of the noise, and obtain bounds that are optimal up to logarithmic factors in each case. When it scales linearly with the squared residual, the best rate is $O(1/k^{1-ε})$ for any $ε>0$. In this case, for all considered stepsizes, our bounds match those of the noiseless KM iteration. When it is bounded by a constant, the best rate is $O(\log^2 k/\sqrt{k})$. When it scales affinely with the squared norm of the iterate, the best rate is $O(1/k^{1/2-ε})$, and the mean-square residual can diverge if the stepsizes are not square-summable. Thus, unlike for contractive SA, square-summable stepsizes are necessary for nonexpansive SA. We demonstrate how various algorithms with cocoercive maps, such as stochastic gradient descent, linear SA, and learning in cocoercive games, fit into our framework, and how the different noise conditions arise in each of them.
comment: 23 pages
★ Event-based analysis of neuromorphic feedback systems
This article investigates the existence and stability of periodic behavior arising in the feedback interconnection of neuromorphic systems, which evolve according to continuous dynamics but interact only through discrete events. Exploiting the event-driven nature of this interaction, each system is modeled by an operator that maps the history of events to the delay of its response. Building on the fading-memory properties of these operators, the article develops a small-gain result establishing the existence, uniqueness, and robustness of periodic one-to-one rhythms.
comment: Submitted to European Control Conference 2027 (Brussels)
★ Neuromodulation Through Intrinsic Noise
Neuromodulation, the modulation of excitability through the regulation of neuronal parameters, is a key tool in neuromorphic control design. This paper investigates how spiking excitability can be modulated by intrinsic noise. Focusing on a simplified memristive neuron model, we show that intrinsic noise plays distinct functional roles depending on the feedback pathways: Intrinsic noise in the fast excitatory feedback shapes the nature of excitability and modulates the sensitivity of the neuron to external/internal stimuli, whereas that in the slow inhibitory feedback provides internal stimuli that can trigger stochastic spiking. We expect these findings to guide the design of neuromorphic systems that harness intrinsic noise to exploit beneficial stochastic phenomena, such as stochastic resonance.
comment: Submitted to European Control Conference (ECC) 2027
★ FAME-QP: A GPU-Accelerated Quadratic-Programming Solver for Real-Time Model Predictive Control
We present FAME-QP, a GPU-accelerated solver for dense quadratic programs arising in linear model predictive control. FAME-QP combines constraint-dependent operator representations with fused single- and two-kernel iterations replayed through CUDA graphs, automatically selecting the execution configuration through device calibration and problem-specific timing. It probes a cached operator bank to select the initial penalty and updates operators lazily when problem matrices change. Numerical benchmarks and simulated humanoid and quadrotor control experiments demonstrate reduced latency relative to competing GPU solvers while maintaining solution accuracy and control performance.
★ Continuous-Time Modeling of Destination-Constrained Dynamic Systems via Doob's $h$-Transform
Existing work on modeling destination-constrained dynamic systems has mainly focused on discrete-time formulations with linear destination constraints or Gaussian destination distributions. Since practical systems are more naturally modeled in continuous time and often subject to nonlinear/non-Gaussian constraints/distributions, this paper proposes a unified measure-transformation framework for continuous-time modeling of systems subject to deterministic or distributional constraints on a nonlinear function of the terminal state. The framework starts from an unconstrained Itô stochastic differential equation, typically derived from physical laws to describe the system's forward evolution without destination information, and takes its induced path measure as unconstrained measure. Doob's $h$-transform is employed to transform this measure into a constrained path measure incorporating destination information through an injected drift. The resulting system is proved to satisfy the constraints while preserving the laws of the unconstrained state process conditioned on the destination information. The transition density is derived, and the path measure of the distributionally constrained system is proved to uniquely minimize Kullback--Leibler divergence from the unconstrained path measure among those satisfying the same distributional constraint. Its injected drift is further shown to converge pointwise to that of the deterministically constrained system as the destination distribution shrinks to the prespecified value. For linear time-varying unconstrained systems, a simplified form of the constrained systems is derived in terms of the conditional expectation of the terminal state, leading to the development of trajectory-generation algorithms. Numerical simulations evaluate the proposed framework under both types of destination constraints.
★ Input-to-State Safety and Small-Gain Theorems for Large-Scale Nonlinear Systems
This paper studies the safety verification problem for a general class of large-scale systems characterized by multiple safety indicator functions. Each safety indicator function satisfies a trajectory-based input-to-state safety (ISSf) estimate that provides a lower bound on its value along system trajectories in terms of the initial state, external inputs, and the other safety indicator functions, whose influence is quantified by continuous and nondecreasing gain functions. Within this framework, we establish a safety-oriented small-gain theorem and its corresponding corollaries that enable compositional and scalable safety verification of the interconnected system, provided that the interconnection gain functions satisfy a small-gain condition on the negative real axis. This condition concerns only the neighborhood of the safety boundary, thereby facilitating the application of our results to safety verification of multi-agent systems under distributed control. Our proof combines small-gain techniques with the lower-bound structure of ISSf estimates and develops an inductive comparison argument that establishes the ISSf property for the overall system throughout the maximal interval of existence. As an application, we establish robust safety for the natural distributed "half-spring" controller, theoretically supporting the computationally efficient and inspiring approach to safety-critical control of multi-agent systems in the presence of obstacles and external disturbances.
comment: 16 pages, 7 figures
★ Decision-Focused Learning via Neural Optimization Layers for Battery-Assisted Grid-Import Smoothing in AI Data Centers
Rapid AI data-center power fluctuations challenge grid integration. Sustained grid-import smoothing requires allocating limited battery power and energy under uncertain demand. We formulate risk-aware BESS scheduling that coordinates advance reserve commitments and demand-responsive dispatch, accounting for operating costs, grid-ramp penalties, and their conditional value-at-risk. We develop battery scheduling decision-focused learning (BS-DFL) with a constraint-aware neural optimization layer trained directly from this objective without optimal-solution labels. The trained layer is then frozen to guide demand-predictor learning through scheduling costs and ramping risk, enabling solver-free rolling-horizon scheduling. In a 10,000-minute simulation using loads derived from measured AI workloads, BS-DFL reduces mean absolute minute-to-minute grid-import changes by 71.70% and total scheduling cost by 18.01% relative to an MSE-trained predictor coupled with a mixed-integer linear programming solver, with an average decision time of 18.77 ms.
★ Aeroelastically Constrained Optimal Path Planning of a Fixed-Wing Airborne Wind Energy System
Multi-megawatt fixed-wing airborne wind energy systems (AWES) use an aircraft with large, slender wings that deform over the pumping cycle. To harvest maximum energy, the cycle is usually designed by means of dedicated reference trajectory optimization routines. It is well known that such routines rely heavily on the system model and that close alignment between the model and the physical system is required for the results to hold in practice. Nevertheless, the wing is usually treated as rigid and structural limits are respected indirectly through upper bounding of states and inputs. In this work, we directly enforce bounds on the elastic deflection and twist of the wing in the power-maximizing periodic optimal control problem. To that end, we model the wing by root-clamped modes, loaded by strip-theory aerodynamics and by the gravitational and inertial loads of the manoeuvring aircraft. To keep the problem tractable, the modal dynamics are replaced by their quasi-static equilibrium, which enters every discretization interval as algebraic constraints. For a multi-megawatt reference aircraft, the elastic twist near the wing tips reaches $1.03^\circ$ along the optimal rigid-wing trajectory. With a twist limit of $0.6^\circ$, the optimal trajectory delivers $31\,\%$ less mean power than the rigid-wing one, mainly through a lower tether tension during reel-out. Rigid-wing trajectory optimization can thus overestimate the power that the wing limits permit. Forward simulations that retain the modal dynamics demonstrate that the quasi-static deflection and twist deviate from the dynamic ones by less than $2\,\%$ of their limits. This work shows that elastic deformation can be accounted for in the trajectory optimization and that, depending on the wing limits, it can strongly reduce the achievable power.
★ Priority-Aware Firm and Flexible Capacity Quantification for Large-Load Interconnection
We study how to unlock existing transmission capacity to accelerate large-load interconnection under differentiated quality-of-service (QoS) requirements. We establish an efficient framework to quantify firm and flexible hosting capacities and assess marginal values of grid upgrades. Specifically, we define multiple flexible capacity tiers with differentiated QoS limits and design a two-layer priority rule to allocate curtailment among tiers. We then formulate a mixed-integer chance-constrained model to jointly quantify firm and flexible capacities under QoS and priority constraints. To improve computational efficiency, we derive an equivalent aggregate formulation that eliminates individual curtailment variables and priority binaries without changing the feasible capacity set. Numerical results show the computational advantage of aggregation. Multi-tier flexible capacity increases total admitted capacity by 45.0\% over the firm-only hosting capacity.
★ Efficient control of multi-agent robust MDPs through tensor trees
This paper introduces a tensor-based framework for controller synthesis of stochastic multi-agent systems under temporal logic specifications that couple the agents, such as one agent reaching its goal only after another agent has reached its own. The agents are modeled as robust Markov decision processes (RMDPs), whose transition probabilities are only known to lie in an ambiguity set, for instance due to abstraction errors, learned models, or uncertain parameters. We show that the decoupled nature of the ambiguity sets allows one to decompose the computation of a lower bound on the satisfaction probabilities into per-agent factors, so that only the per-agent ambiguity sets need to be stored. This leads to improved memory complexity compared to monolithic approaches. We compare our approach with state-of-the-art approaches based on approximate simulation relations and with interval MDP abstractions. On a four-agent ordered reach-avoid benchmark, our approach certifies a lower bound twice as high as the simulation relation- based approach with 20 times less memory, and needs about 200 times less memory than the interval MDP abstraction, even at a resolution with about a million times more joint cells.
★ Toward Pseudorange-Based PNT with Starlink: A Mock Fused LEO GNSS Implementation
We present experimental results from Starlink for pseudorange-based positioning, navigation, and timing (PNT), emulating a system that fuses a primary communications mission with a secondary low-Earth orbit (LEO) PNT mission. Legacy global navigation satellite system (GNSS) constellations faceinherent vulnerabilities and signal limitations, prompting growing interest in broadband LEO constellations---such as SpaceX's Starlink---for resilient PNT. However, opportunistic use of stand-alone Starlink signals fails to achieve GNSS-competitive accuracy due to frame timing irregularities and reliance on uncorrected poor-quality publicly available ephemeris data. To overcome these limitations, we present a mock implementation of a fused LEO GNSS architecture. Our method extracts time-of-arrival observables from live Starlink Ku-band downlinks and pairs them with high-fidelity, beam-specific orbit and clock models distributed as if provided natively by the constellation operator. In a field campaign, and despite constraints imposed by consumer-grade receiver hardware, the method yields 95% error bounds of 2.33 m horizontally (semi-major axis), 3.37 m vertically, and 11.19 ns in timing, with the actual solution errors falling well within these bounds.
★ Metamorphic testing of learned AC optimal power flow surrogates reveals representation dependence NeurIPS 2026
Evaluating a learned AC optimal power flow (AC-OPF) surrogate on one grid serialization cannot show whether its accuracy depends on that encoding. We test this dependence with thirteen metamorphic relations, each specifying when a transformation is valid and how to compare its outputs. Our audit covers 17 model and pipeline configurations: two released trained checkpoints and 15 random-weight architecture or mechanism controls. Every configuration violates at least six relations, even at reporting tolerances ten times tighter or looser. For GridSFM-Open v1.1, source inspection and intermediate-tensor interventions point to an indexing bug and three noncanonical or numerically unstable preprocessing choices; a repaired checkpoint remains untested. In a separate case500 training experiment, augmentation lowers transformed-view error in three of four models and median discrepancy and constraint-violation P95 in all four. These benefits depend on the training protocol: some relations worsen, and every evaluated prediction remains AC infeasible. A GPU/CPU eigensolver mismatch accounts for most of the large raw-Laplacian-baseline gain. We recommend reporting representation consistency alongside accuracy, feasibility, and cost. These results establish neither deployment readiness nor universal invariance.
comment: Accepted to the NeurIPS 2026 Workshops on (i) Trustworthy AI Evaluation, (ii) AI Foundations for Power Grids, (iii) Tackling Climate Change with Machine Learning, (iv) Sim2Science, and (v) Representations for the Physical Sciences
★ An Accurate Modulation-Assisted Method for Phasor Estimation Under Decaying DC Offsets
Decaying DC (DDC) offsets severely compromise the accurate estimation of current phasors immediately following power grid fault occurrence. Consequently, conventional full-cycle discrete Fourier transform (FCDFT) algorithms experience decaying oscillatory magnitude errors, potentially causing distance relays to malfunction. This paper introduces a modulation-assisted technique designed to precisely eliminate the DDC components from fault current signals. By multiplying the fault signal with a high-frequency carrier and applying consecutive sliding Fourier projections, the proposed method effectively isolates the DDC signal while entirely rejecting higher-order harmonic interference. It utilizes a direct algebraic solution derived from a robust forward-difference operation across just three sequential projections to estimate and extract the DDC parameters. Furthermore, the algorithm seamlessly processes both single and multiple DDC components by dynamically tracking them as one equivalent first-order mode. Extensive evaluations are conducted to validate the proposed method accuracy. The proposed algorithm limits the maximum overshoot (MO) to 0.62% (0.59% for $k \geq 8$) for the evaluated DDC cases, and to 2.47% (0.95% at 20 dB for $k = 8$) under noisy conditions.
comment: This work has been submitted to the IEEE TPWRD for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible
★ Incremental Input-to-State Stabilization of Partially Unknown Control-Affine Systems within Prescribed Safe Sets
Incremental input-to-state stability ($δ$-ISS) guarantees that the trajectories of a system converge towards one another, up to a gain on the difference of their inputs. Some existing learning-based designs that combine incremental stability with safety achieve only incremental input-to-state practical stability, whose non-vanishing offset can make the guarantee conservative. We consider control-affine systems with a known, full-row-rank input matrix and an unknown drift learned using Gaussian process regression. The prescribed safe set is a superlevel set of a smooth function satisfying a radial condition that ensures a diffeomorphism onto $\mathbb R^n$ and it need not contain any equilibrium of the uncontrolled system. Using this transformation, we design a smooth feedback law and derive sufficient conditions that, under the stated learning assumptions, guarantee with high probability both forward invariance of a compact subset of the safe set and $δ$-ISS on that subset. The incremental estimate uses the distance induced by the transformation and contains no practical offset. Simulations on a Van der Pol oscillator and a Lotka-Volterra system illustrate the proposed approach.
★ Data-Driven Robust Dynamic Wireless Charging Lane Planning with Tunable Profit--Risk Trade-off and Modified User Equilibrium
Dynamic wireless charging (DWC) has attracted increasing attention as a complement to conventional plug-in charging for electric vehicles (EVs). However, existing planning methods for dynamic wireless charging lanes (DWCLs) overlook realistic EV routing and charging behavior and lack explicit out-of-sample risk control under traffic-demand and renewable-energy uncertainties. To address these limitations, this paper proposes a data-driven two-stage robust planning framework that coordinates DWCL deployment with the operation of coupled traffic--power networks (CTPNs). A data-driven uncertainty set is constructed from joint historical scenarios to preserve spatio-temporal correlations without imposing parametric distributional assumptions. A modified user-equilibrium (MUE) model is proposed to capture realistic EV routing and charging behavior by incorporating state-of-charge (SoC) dynamics and endogenous charging decisions. The resulting two-stage robust model with mixed-integer recourse is reformulated as an equivalent single-stage scenario-approach problem and solved using a customized nested column-and-constraint generation algorithm. An \textit{a posteriori} feasibility guarantee is established to quantify out-of-sample risk. Based on this guarantee, the scenario set is iteratively enriched to meet a prescribed risk tolerance. Case studies on a 69-node power distribution network and a 24-node traffic network demonstrate that the proposed MUE model prevents overinvestment in DWCLs, while the proposed risk-quantification procedure enables a tunable profit--risk trade-off.
comment: 10 pages
★ Impact of Reduced-Order Modelling on Sum-of-Squares Region-of-Attraction Certification for Grid-Forming and Grid-Following Converters
With the increasing penetration of power-electronic equipment, the transient stability of grid-forming (GFM) and grid-following (GFL) converters is increasingly characterized through the region of attraction (ROA) of the post-fault equilibrium. Sum-of-squares (SOS) programming yields certified inner approximations of the ROA, yet its computational cost grows rapidly with system order. Reduced-order models are therefore widely used to reduce computational cost and, more importantly, to enable analytical investigation. However, ROA analyses based on these models often assume that reproducing the converter's transient response, or retaining more dynamics in the correct time-frame, is sufficient to preserve the ROA of the full-order system. Using time-domain ODE simulations as a reference, the SOS-based ROA of each reduced-order GFM and GFL model is assessed against both its own simulated ROA and that of the corresponding full-order model, thereby separating SOS-induced conservativeness from model-order-induced ROA distortion. The results indicate that increasing reduced-order model complexity does not necessarily improve SOS-based ROA fidelity. In the cases examined, more complex reduced-order models yield lower SOS coverage of the full-order reference ROA, despite closer ODE-boundary agreement in some regions. Model selection should therefore consider both full-order ROA fidelity and certified coverage of the practical operating range, rather than model complexity alone.
comment: 13 pages, 5 figures, 3 tables
★ Complete Decentralization of Wave Equation Control on a Compact Domain
A feedback control policy is completely decentralized if the actuation signal at each location in space can be computed based only on state information from that location. For systems equipped with large numbers of spatially distributed sensors and actuators, completely decentralized controllers are desirable since they can be implemented in a communication and computation efficient manner, but generally perform worse than centralized counterparts. However, through a case study of Linear Quadratic Regulator (LQR) design for wave dynamics on a compact spatial domain, we show that careful design of system parameters to satisfy certain conditions causes a decentralized control policy to perform as well as any centralized policy. We then propose a spatially filtered $H_2$ norm to show that satisfying these conditions does not inherently degrade achievable closed-loop system performance, enabling simultaneous realization of both desirable performance and communication/computation efficiency via system-control co-design. Additional robustness analysis characterizes a range of system parameters such that the system interconnected with a decentralized controller maintains stability and near-optimal performance. Although our framework is derived for spatially continuous dynamics, we demonstrate that this theory can be leveraged to derive results for decentralization in finite-dimensional settings through a damped mass-spring example.
comment: 25 pages, 6 figures, submitted to SIAM Journal on Control and Optimization
★ Residual-Corrected Physics-Informed Neural Network KKL Observers with Input-to-State Stability Guarantees
Physics-informed neural network (PINN) Kazantzis--Kravaris--Luenberger (KKL) observers require an approximate solution to a KKL partial differential equation (PDE) and an approximate inverse of this solution. Either approximation can leave a nonzero estimation bias even without measurement noise. In this paper, we remove both errors from the implemented observer by construction. We build an encoder containing an explicitly invertible neural map. Its closed-form inverse provides the internal state reconstruction used to evaluate the learned KKL residual and correct the latent dynamics. This construction makes the synchronization manifold exactly invariant for every value of the network weights. Based on contraction of the residual-corrected observer dynamics, we prove an input-to-state stability (ISS) result for the implemented error dynamics with respect to measurement noise. A direct consequence is that the estimation error converges exponentially to zero in the noise-free setting. We provide two numerical examples, the Van der Pol oscillator and the chaotic Lorenz system, in which the residual-corrected observers achieve exact noise-free convergence and improved error responses compared with released PINN--KKL baselines.
comment: 7 pages, 1 figure, ACC 2027 conference submission
★ Cascading Contingency Analysis of Modern Power Systems Using a Stochastic Hybrid Systems Framework
This paper presents a cascading contingency analysis and risk assessment framework that incorporates probabilistic mode transitions motivated by stochastic hybrid system (SHS) concepts together with power-flow-based stress evaluation. A two-stage modeling approach is adopted in which sensitivity-based analysis is first used to identify candidate cascading transitions, followed by DC power flow re-evaluation to accurately capture post-contingency stress redistribution, islanding behavior, and load shedding. A probabilistic propagation model is constructed using mode-dependent transition probabilities derived from line loading conditions. A Cascading Risk Index (CRI) is introduced to quantify the expected load shedding accumulated over cascade progression. Case studies on the IEEE 39-bus system demonstrate that the proposed framework captures diverse cascading routes under multiple initiating outages and identifies critical propagation patterns that are not revealed by N-1/N-2 screening. The results highlight the importance of SHS modeling in assessing cascading risk and providing a scalable framework for resilience evaluation in modern power systems.
comment: This paper was accepted at the 2026 North American Power Symposium (NAPS)
★ Analysis and Demonstration of a Current-Stiff, Isolated Capacitive Bridge Transformer
The traditional Voltage Source Inverter (VSI) based Dual Active Bridge (DAB) utilizes a transformer at its center to provide galvanic isolation and a turns ratio for step up/down functionality. However, the transformer is often the limiting factor of the DAB, especially as semiconductor switching speeds improve, given steel core losses, winding losses, etc. This paper presents a new DAB architecture based on Current Source Inverters (CSIs) coupled through, and isolated by, an AC capacitive bridge, yielding a system called a Capacitive Bridge Transformer (CBT). The CBT architecture eliminates all magnetic components in the AC portion of the isolated DAB circuit, reducing frequency scaling barriers normally present in DABs. The paper introduces the CBT from the perspective of duality and then develops the DC-DC power flow equations, the internal apparent power equations, and effective turns ratio. The analytical models are verified by steady state and detailed time domain measurements on a 2 kW experimental prototype.
comment: 10 pages, 17 figures
★ Uncertainty-Aware Hybrid Offline Reinforcement Learning with Online Action Selection Enabled by Sparse Gaussian-Process Critics
Start-ups and grade changes remain largely operator-driven, while developing and maintaining dynamic models for model-based control requires substantial engineering effort. Hybrid Offline Learning and Online Optimisation (HOFLON) uses one-step offline reinforcement learning to select actions from historical transition data. In its original implementation, the online objective combines the predicted return from a tree-ensemble critic with an adversarial-autoencoder (AAE) penalty for unfamiliar state-action pairs. We examine the effect of this penalty and replace the critic and AAE with a sparse variational Gaussian-process (sGP) critic. Actions maximise $μ- κσ$, an uncertainty-adjusted estimate of the return from one action followed by the historical operating practice. On the HOFLON polymerisation-reactor benchmark, removing the AAE from HOFLON led to 4 failed start-ups in 15 runs over five critics. At its published weight, the variation in the AAE penalty is four orders of magnitude smaller than the variation in the tree-ensemble critic. Nevertheless, it changes actions when the predicted returns of competing candidates are nearly equal, and paired runs with and without the penalty diverged from the first move. With a Matérn-5/2 sGP critic, $κ= 0$ led to failure in 13 of 15 runs, whereas with $κ= 3$ none of 15 runs (five critics, three noise realisations) failed. The median and lowest returns were 13,464 and 13,398 (HOFLON: 13,393 and 12,411). A squared-exponential sGP critic with learned inducing inputs needed $κ\geq 3$ to remain within the empirical data support in steady state, and then settled with a temperature offset of 2.0 K.
comment: 8 pages, 5 figures, 1 table. Submitted to the 2027 European Control Conference (ECC 2027)
★ Reliable and explainable analysis of AVS/RS by Statistical Model Checking and Process Mining
Automated Vehicle Storage and Retrieval Systems (AVS/RS) are increasingly adopted to boost warehouse throughput while containing operational costs. We present an integrated analysis framework that couples discrete-event simulation with Statistical Model Checking (SMC) and Process Mining (PM) to quantify and explain AVS/RS performance. SMC, implemented through the MultiVeStA tool, delivers statistically reliable estimates of key indicators, such as cycle times, throughput, and storage utilization, across alternative stocking and picking policies with user-specified confidence levels. MultiVeStA orchestrates multiple independent simulations in a black-box manner, enabling automated verification. PM ingests the resulting event logs to uncover the behavioral patterns and bottlenecks that drive those indicators, closing the interpretability gap typical of simulation studies. We instantiate the framework on a new open-source AVS/RS simulator configured according to recent industrial guidelines and evaluate multiple control strategies. Results show that the joint use of SMC and PM enables both predictive what-if analyses and post-hoc diagnostics, providing the first combined SMC+PM assessment of AVS/RS operations and a reusable toolchain for automated warehousing research.
comment: 40 pages, 12 figures. Original manuscript submitted before peer review. A revised version has been accepted for publication in Expert Systems with Applications
★ Energy- and Time-Efficient Multi-Vehicle Trajectory Planning Using Augmented DAG Search in Configuration Space
Multi-vehicle trajectory planning for CAV requires balancing energy consumption and travel time while ensuring safety and scalability. While travel-time-optimal methods based on Integer Programming exhibit poor scalability in large-scale multi-vehicle scenarios, existing energy-oriented approaches fail to balance energy consumption and travel time. We extend a configuration-space framework for multi-vehicle trajectory planning to jointly locally optimize travel time and energy consumption. A weighted energy-travel time objective is combined with an efficient augmented DAG search on two-dimensional decompositions of a high-dimensional configuration space, enabling scalable and collision-free coordination of large numbers of vehicles. Although the proposed framework is applicable to other traffic scenarios, such as roundabouts and vehicle-following, we focus on intersections as a representative and challenging planning problem. To further improve performance for our methodology, we also extend the method with an insertion-ordering heuristic that exploits reduced-space geometric information. Combined with nonlinear model predictive control, the proposed framework is shown to scale to 40 vehicles, outperform MILP-based and convex optimization baselines in dense intersections, and reduce energy consumption at equalized travel times relative to time-optimal planning.
comment: This manuscript is currently under review at IEEE Transactions on Intelligent Transportation Systems
★ Concentration-Based Probabilistic Reachability for Linear Systems under Multiplicative Uncertainty and Additive Noise
This paper investigates probabilistic reachability for discrete-time linear stochastic systems subject to bounded multiplicative uncertainty and potentially unbounded additive noise. To address the conservatism of reachable sets constructed solely from covariance information, we develop two complementary approaches based on sub-Gaussian and sub-exponential matrix moment-generating-function characterizations. The sub-Gaussian approach relies on a worst-case envelope over the uncertainty support, whereas the sub-exponential approach exploits expectation-based propagation of the matrix variance proxy. Both approaches are used to construct probabilistic reachable sets with explicit probability guarantees, considering ellipsoidal confidence regions and half-space bounds. For the latter, we derive a necessary and sufficient condition under which the sub-exponential chance-constraint margin is no larger than its sub-Gaussian counterpart. Numerical results show that the proposed bounds can be less conservative than those based on the covariance, and illustrate the complementary nature of the two approaches across probability levels.
★ Information-Theoretic Threshold Design for Sparse Event Encoding in Learning Systems
One approach to event-driven learning is to encode continuous signals as sparse events generated when signal changes exceed a prescribed threshold. However, this threshold is typically selected empirically, with limited theoretical guidance. This paper establishes a connection between threshold-based event encoding and discretized learning systems and identifies a dimensionless ratio governing information retention. From this connection, we derive an information-sufficiency boundary using normal-condition data and formulate a data-driven criterion that selects the sparsest threshold retaining a prescribed fraction of peak discriminative information. The framework is illustrated using event cameras and evaluated on power-system fault classification. Results show that the proposed design identifies sparse operating thresholds while exposing the tradeoff between information retention, event activity, and classification performance.
★ Beyond Accuracy: Physics-Informed Evaluation of ANN-based Power Flow Solvers under System Shift
Recent advances in artificial deep learning-based power flow (PF) solvers have primarily focused on improving estimation accuracy, while their physical feasibility and operational reliability under realistic system variations remain insufficiently evaluated. This paper proposes a physics-informed evaluation framework for deep learning-based PF solvers under topology and load distribution shifts. Beyond conventional voltage-domain errors, the framework incorporates KCL gaps, branch-flow gaps, KVL residuals, continuous feasibility scores, and strict, practical, and relaxed PF feasibility rates. Four case studies are designed to investigate topology shifts, mixed topology training, load distribution shifts, and combined topology distribution shifts. Eight ANN architectures, including graph input models, load-only baselines, and physics-informed neural networks, are evaluated under consistent settings. The results demonstrate that low estimation errors do not necessarily correspond to physically feasible or operationally reliable PF solutions, particularly under combined system shifts. The proposed framework establishes an initial foundation and analysis for the development and verification of trustworthy deep learning-based PF solvers, providing support for their reliable deployment in future AI-enabled power system analysis.
★ Intelligent Carbon Emission-aware Radio Unit Activation Control in Dense Heterogeneous Energy 6G Networks
Base stations account for a dominant share of energy consumption and operational carbon emissions in radio access networks. In O-RAN deployments with heterogeneous energy sources, however, minimizing energy consumption alone does not necessarily minimize carbon emissions. In this work, we investigate carbon-aware DRL for RU sleep control. Under the adopted source-matched operational accounting model, renewable-matched RUs are assigned zero direct operational emissions, whereas non-renewable-supplied RUs incur operational emissions. The problem is formulated as a Markov decision process and evaluated using two carbon-aware reward formulations and a conventional energy--QoS reward baseline. We implement and compare TD3, PPO, and a heuristic method under realistic mobility and 3GPP channel models across multiple renewable-matched RU layouts and traffic loads. Simulation results show that TD3 learns more energy-efficient activation policies than PPO, achieving approximately 3%--6% lower energy consumption in representative scenarios. TD3 also achieves 8%--12% lower operational emissions than PPO in scenarios with a low proportion of renewable-matched RUs and up to a 20% reduction relative to the heuristic baseline under moderate renewable availability and high traffic load. Moreover, the joint energy,carbon and QoS reward achieves approximately 12%--28% lower operational emissions than the energy and QoS reward, while maintaining comparable energy consumption.
★ Koopman Model-Based Reinforcement Learning
We propose a Koopman model-based reinforcement learning (K-MBRL) algorithm for controlling nonlinear robotic systems with unknown dynamics. The K-MBRL learns a control policy by minimizing an approximate policy objective that combines the analytical immediate cost with an estimate of the future cost evaluated at the one-step predicted state. To this end, we develop a stochastic Koopman operator-based learning method to approximate the unknown nonlinear dynamics in a linear representation, while a critic function estimates the remaining future cost. This formulation offers two key advantages. First, the linear Koopman dynamics simplify the model-dependent policy gradient computation to a constant matrix product, enabling computationally efficient policy improvement. Second, one-step model propagation mitigates the accumulation of prediction errors associated with long-horizon rollouts while retaining long-term cost information through the critic. The learned policy enables real-time control through feedforward evaluation without online optimization. We establish policy learning convergence guarantees under standard stochastic approximation assumptions. Simulations on HalfCheetah and quadruped four-legged locomotion tasks demonstrate strong sample efficiency and improved performance compared with classical model-free and model-based reinforcement learning baselines. A pendulum swing-up task is used to assess policy gradient approximation accuracy and conduct ablation studies on the learning mechanism and key hyperparameters. The proposed K-MBRL is further validated on a physical quadruped robot performing a challenging sequence of maneuvers: rising to an upright posture supported by two rounded rear feet, maintaining balance during bipedal walking, and safely returning to the ground, as shown in Fig.1.
★ Discrete-Time Stochastic Control Beyond i.i.d. Noise
We study discrete-time stochastic control problems where the noise driving the state dynamics is a stationary infinite-order chain rather than an i.i.d. sequence. We formulate the problem as a controlled Markov process by enlarging the state space to include the posterior distribution of the noise given the state-action history. Assuming that the sensitivity of the noise transition kernel to past noise values decays exponentially, we show that the expected total variation distance between the full-history and finite-memory noise filters decays at rate $\sqrt{N}ρ^{\sqrt{N}}$. We use this result to construct finite-memory control models and establish explicit near-optimality bounds for their policies. Finally, under suitable ergodicity conditions, we show that Q-learning converges to the optimal solution of the approximate model, so the same performance bounds apply to the learned policies.
★ An Inductively Assisted N :1 Switched-Capacitor Ladder Converter
This paper proposes an inductively assisted N:1 switched-capacitor ladder buck (D-SCLB) converter that replaces the output-side switch in the bottom half-bridge of an N:1 SCLB with a small inductor placed in parallel with the ladder capacitors during the demagnetization phase. The flying capacitors share the load current delivery, lowering the average inductor current to (1-M)Iout and enabling continuous regulation with M=D/(N-1+D). A closed-form output-impedance model, valid for arbitrary N under both optimal and uniform switch sizing, is derived and compared against SC, SCLB, FCMC, and DPSC topologies. For a 4:1 48V-to-PoL design, the D-SCLB reduces the average inductor current by up to 20% and improves the equivalent output impedance by up to 30% relative to the SCLB. Two hardware prototypes on a low-cost two-layer PCB achieve a peak efficiency of 96.2%, with the D-SCLB using 49% smaller inductor volume than the SCLB at comparable efficiency. Model predictions agree with measurements to within 1.5% under heavy load.
★ U.S. Data Center Growth Through 2030: A Critical Review of Grid Adequacy and Transmission Expansion Planning
The growth of large data centers raises questions about both supply adequacy and the suitability of load representations used in connection studies. This paper critically reviews the U.S. demand, generation, and transmission outlook through 2030 and examines the implications for transmission expansion planning (TEP). Comparison of public forecasts highlights distinctions among annual average demand, peak demand, and grid connection capacity. National generating-capacity totals cannot establish a dependable surplus because resource availability, regional adequacy, transmission constraints, and project timing determine supply capability. Recent grid events and published dynamic studies demonstrate that faults can trigger substantial data-center transfers to backup supply, followed by delayed or coordinated demand recovery. These behaviors require appropriate representation within the conventional planning framework, alongside assessment of the requested maximum grid import. The selected literature reveals limited integration of validated data-center dynamic models into the technical screening and economic comparison of expansion alternatives. The review identifies the need to establish whether fault response and recovery change feasible connection capacity, required reinforcement, or the preferred transmission investment.
★ UniPipe: Unified Inference Serving and Fine-Tuning for Deadline-Constrained DNN Surrogate Workflows
DNN surrogates are increasingly used to accelerate scientific workflows that must process high-rate sensor streams in near real time, exemplified by Ptychographic imaging at next-generation X-ray light sources. Such surrogates deliver 100-1000x higher throughput than iterative solvers, enabling faster steering of experiments. To preserve surrogate accuracy during an experiment, as the input distribution evolves, state-of-the-art deployments freeze the pretrained surrogate or run a separate CUDA context that fine-tunes concurrently with serving. However, multi-context concurrency causes significant contention on the GPU, inflating both the mean and the variance of inference latency, triggering deadline misses in workflows where sensor data has to be processed within a hard per-unit deadline. To address this challenge, we propose UniPipe, a single-context system that unifies inference serving and fine-tuning of a DNN surrogate on a single edge GPU. UniPipe is driven by three key insights: both serving and fine-tuning require a forward pass and can be fused into one; the per-sample latency of modern GPUs is non-linear in batch size; and the workload is repetitive across intervals, enabling offline scheduling. Specifically, UniPipe fuses training and inference data in a single forward pass with a selective backward pass on the training subset, and uses a polynomial-time scheduler that exploits the latency curve to compute the batch composition. We evaluate UniPipe on a real-world Ptychographic reconstruction workflow with sensor rates up to 5000 patterns/sec.; UniPipe lowers reconstruction MSE by up to 30% and inference MSE by up to 50% over a pretrained baseline, and reduces the deadline-miss rate by 13%-22% over a multi-context baseline. UniPipe's unified deployment reduces the latency spread from 106% to 57% of the mean, and increases total throughput compared to multi-context baseline.
★ A Three-Stage Unsupervised Online Physics-Informed Neural Network for Microgrid Optimal Power Flow
Optimal allocation of generators is a key challenge in power systems to achieve economical dispatch while satisfying operational constraints. Conventional optimization techniques can be computationally expensive, limiting their applicability in real-time environments, and neural networks often require offline retraining under new operating conditions. This paper proposes a three-stage unsupervised online physics-informed neural network (PINN) framework for optimal power flow (OPF). Simulation results demonstrate that the proposed framework satisfies load demand and system constraints while achieving a $43,665 lower operating cost than fmincon-based optimization under the same operating conditions. The training process analysis further demonstrate the framework's ability to adapt its parameters online to variations in load demand and renewable generation in real time.
★ Learning of LPV Models With Linear Fractional Representations: Initialization, Optimization, and Conditioning
Linear Parameter-Varying (LPV) models with Linear Fractional Representation (LFR) can describe a wide class of nonlinear systems with a control-friendly structure. However, identifying these models together with their scheduling map from input-output data requires solving a non-convex optimization problem that is highly sensitive to initialization. Consequently, existing methods often rely on many random initializations. Instead, our work proposes an initialization strategy based on the Best Linear Approximator (BLA). Furthermore, motivated by an analysis of the impact of rational-dependency LPV-LFR models on the conditioning of the non-convex cost, two multiple shooting methods that improve the conditioning, unconstrained multiple shooting (UMS) and encoder-based multiple shooting (EBMS), are compared. The initialization and multiple shooting methods are evaluated on a nonlinear mass-spring-damper example, showing that BLA initialization consistently yields accurate results, whereas random initialization has a large spread. The results also show that both UMS and EBMS reduce the spread for randomly initialized models. UMS is simple to tune and well suited to few sub-records, whereas EBMS can exploit many short sub-records to further improve conditioning, at the cost of training an encoder and tuning additional hyperparameters.
comment: Preprint submitted to the IFAC Journal of Systems and Control
♻ ★ Predictive Neuromorphic Control of a Scanning Mirror System: An Event-Centric Framework
Organizing control around events can reduce sensing and actuation demands while directly linking control decisions to the occurrence and timing of desired system behaviors. This paper proposes a novel event-centric model predictive control (MPC) framework that unifies event sensing, event prediction, and event regulation. Events define the feedback information, prediction horizon, and event-timing objectives, placing them at the center of predictive control design. The framework is developed for a scanning mirror system equipped with a baseline neuromorphic controller composed of spiking neurons that supports sparse actuation. The control objective is to achieve prescribed zero-crossing times and a desired scanning range while minimizing the mechanical energy transferred between the actuator and the mirror. A sampling-based algorithm is presented to obtain suboptimal solutions of the event-centric MPC optimization problem without requiring an explicit event-to-event prediction model. By grounding predictive optimization in events, the proposed framework opens a new direction for MPC design and its integration with neuromorphic control.
comment: 8 pages, submitted to ECC27, this work is the first, foundational work that proposes Event-Centric Model Predictive Control (EC-MPC), a framework that combines model predictive control and neuromorphic control. Philosophy: predicting events but not trajectories; regulation without calibration
♻ ★ Delay-Certified Readiness Barriers for Overactuated Multirotor Allocation Without Lipschitz Constants
Overactuated multirotors can produce the same wrench with many rotor-speed combinations, some of which leave the vehicle unable to change its wrench quickly, e.g., to reject a gust. Readiness barriers keep this ability, the readiness, above a floor, but only in continuous time; with sampled commands and varying speed-controller delays, the available robustness bound covers less than one millisecond. We remove this limit by exploiting the structure of the problem: each rotor responds monotonically and in closed form to its own torque, so the speeds reachable under uncertain switching times lie in a cheaply computed box, and the lowest readiness over that box sits at a corner known in advance. This yields a Lipschitz-free test that a command keeps the readiness above its floor until the next sample, plus a fallback torque that keeps the vehicle on a certified plan. In hexarotor simulations with 2--40~ms delays and $\pm2$~ms jitter, the filter never violated the floor. At $10\pm2$~ms, it matched a delay-compensating predictor within the design envelope and cost 2--30\% more tracking error beyond it, where the predictor alone violated the floor in 2 to 16 of 16 missions. A tuned robust predictor was also safe with synchronous switching, but not with per-rotor jitter.
comment: This work has been submitted to the IEEE for possible publication
♻ ★ An extended Perron-Frobenius operator filter for nonlinear state estimation
We propose an extended Perron--Frobenius Operator Filter (PFOF) for nonlinear state estimation. The method learns the Perron--Frobenius operator, an infinite-dimensional linear operator fully preserving properties of a nonlinear dynamical system, using the extended Dynamic Mode Decomposition (eDMD). This enables us to explicitly account for non-Gaussian distributions exhibited by the nonlinear system within a linear-operator representation, while retaining the freedom to choose basis functions in eDMD. Through two numerical examples, we show that the extended PFOF achieves high computational efficiency and high estimation accuracy by exploiting the flexibility in the choice of basis functions in eDMD.
comment: Submitted to Nonlinear Theory and Its Applications, IEICE (NOLTA), 9 pages, 5 figures, 2 tables
♻ ★ Explicit MPC for the constrained zonotope case with low-rank matrix updates
Solving the explicit Model Predictive Control (MPC) problem requires enumerating all critical regions and their associated feedback laws, a task that scales exponentially with the system dimension and the prediction horizon, as well. When the problem's constraints are boxes or zonotopes, the feasible domain admits a compact constrained-zonotope representation. Building on this insight, we exploit the geometric properties of the equivalent constrained-zonotope reformulation to accelerate the computation of the explicit solution. Specifically, we formulate the multi-parametric problem in the lifted generator space and solve it using second-order optimality conditions, employ low-rank matrix updates to reduce computation time, and introduce an analytic enumeration of candidate active sets that yields the explicit solution in tree form.
♻ ★ Formal Synthesis of Robust Koopman-Model Predictive Control: A Case Study in AC-DC Power Conversion
This letter proposes a formal synthesis of Robust Koopman-Model Predictive Control (RK-MPC), a novel data-driven approach to formal synthesis of systems with nonlinear dynamics. We formulate a novel optimization problem for RK-MPC by incorporating specifications described by Signal Temporal Logic and prove its closed-loop performance. Effectiveness of the proposed RK-MPC is evaluated by applying it to the reliable design of an AC-DC power converter.
comment: 6 pages, 3 figures, 1 table
♻ ★ Discounted MPC and infinite-horizon optimal control under plant-model mismatch: Stability and suboptimality
We study the robustness to plant-model mismatch of nonlinear systems in closed loop with (discounted) optimal controllers designed for a nominal model, with quadratic stage costs across finite and infinite horizons. By combining Lyapunov analysis with a unifying relaxed dynamic programming framework, we establish conditions that guarantee robust stability alongside improved suboptimality bounds compared to existing literature. A key step in our analysis is proving that stability counteracts time-accumulated error growth due to plant-model mismatch, yielding robustness bounds that hold uniformly across all prediction horizons.
comment: Submitted for publication
♻ ★ Safe Approximate Nonlinear Model Predictive Control Without Expert Demonstrations
Repeated online optimization makes nonlinear model predictive control (NMPC) computationally demanding for real-time applications. Neural approximations reduce this cost but often require large datasets of expert NMPC solutions. We introduce Sequential Approximate MPC (Seq-AMPC), a label-free recurrent framework that generates complete control sequences along predicted trajectories using parameters shared across the prediction horizon. The policy is trained directly through known dynamics with losses for control cost, constraint satisfaction, and a cost-decrease condition aligned with a feasible-backup safety filter. Building on this filter, we introduce a stage-cost-dependent acceptance tolerance that preserves recursive feasibility, constraint satisfaction, and asymptotic convergence under nominal MPC assumptions, independently of neural approximation error. We further derive Hoeffding-based finite-sample bounds for the expected fraction of applied proposals and closed-loop cost discrepancy relative to NMPC. We evaluate Seq-AMPC on a quadcopter, chain of masses, obstacle-avoidance vehicle, and robotic manipulator, comparing recurrent and feedforward policies under label-free and imitation training. Across four benchmarks, training takes 0.2-1.7 GPU hours per policy without expert control labels. The filtered controller applies 97.2-99.1% of neural proposals and achieves mean closed-loop cost ratios within 0.3% of NMPC on jointly completed episodes. Tests with suboptimal feasible initial plans further demonstrate the benefit of neural replanning.
♻ ★ Mass Scaling Laws for Energy-Optimal Horizontal Multirotor Flight
Multicopters are a type of aerial robot seeing extensive real-world applications but with range and endurance as critical bottlenecks. Optimization of its energy efficiency through energy-aware flight planning (e.g. cruise speed-scheduling) has hence been an important topic. The mass of the multicopter is an important factor to consider, as it has a significant impact on efficiency, and many common applications are subject to frequent mass variation due to payload change. This work derives closed-form scaling relationships between the energy-optimal steady horizontal flight behavior of a multicopter and its vehicle mass, and validates them experimentally on two multirotor platforms. Specifically, for mass-only variation, i.e., when payload changes the vehicle mass while the airframe geometry, aerodynamic characteristics, and rotor configuration remain unchanged, we show through a combination of numerical simulation, theoretical derivation, and experimental validation that (1) the energy-optimal velocity is proportional to the square root of vehicle mass; (2) the energy-optimal pitch angle is constant with respect to mass; and (3) the optimal energy per meter is proportional to mass. The analysis shows that these three relationships arise from a common mass-similarity property of the steady-state flight and rotor-aerodynamic equations. Furthermore, the obtained scaling relationships enable simple yet effective cruise-speed planning to optimize energy efficiency and extend the range and endurance of the multicopter.
comment: Accepted to IEEE Robotics and Automation Letters, October 2026
♻ ★ The Phantom of Davis-Wielandt Shell: A Unified Perspective for Graphical Stability Analysis of MIMO LTI Systems
This paper presents a unified perspective based on Davis-Wielandt (DW) shell of linear subspaces for graphical stability analysis of multi-input and multi-output linear time-invariant feedback systems. Connections between DW shells and various graphical representations, as well as gain and phase measures, are established through an intuitive geometric perspective. We establish a strict DW separation theorem which characterizes subspace separation and provides a basis for comparing the implications and relative conservatism of graphical separation conditions. In particular, we introduce the rotated scaled relative graph ($θ$-SRG) and show that its separation is equivalent to DW shell separation and is therefore the least conservative among all the graphical conditions considered. We derive pointwise and boundary stability conditions for bi-component feedback interconnections and establish a weak DW separation result that relaxes the strict separation requirement at open-loop poles. We also discuss applications to robust stability analysis and provide a numerical example illustrating how DW shells serve as a more reliable robustness indicator.
comment: 16 pages, 10 figures. Revised version
♻ ★ A Global Routing Method for Electronic Circuit Design Integrating Flow Matching and Constrained A*
Global routing is a fundamental issue in the design of electronic circuits, as it directly relates to the efficiency, reliability, and compactness of the overall layout of integrated systems. As the demand for higher functionality and reliability within a limited chip area increases, the importance of routing as a design consideration has never been greater. Efficient global routing optimization techniques are therefore required to connect all pins with minimal wirelength (WL), while avoiding congestion and interference across multiple nets within the design space. As the number of nets to be routed increases, global routing inherently suffers from a combinatorial explosion in the design space, making it increasingly difficult to obtain feasible solutions. In this paper, we focus on solving constrained global routing problems based on a heuristic A*-based framework. Based on this concept, a hybrid approach is proposed by integrating constrained A* with flow matching (FM). The method utilizes FM to construct a global cost map that captures routing tendencies across the design space, which is then used to guide the constrained A* search process. This enables efficient exploration of the solution space while satisfying routing constraints. Experimental results demonstrate that the proposed method achieves comparable WL and via usage to baseline methods while maintaining constraint satisfaction across a wide range of scenarios. Notably, the framework shows strong capability in solving complex routing instances that are difficult to handle using conventional approaches. Additionally, the method shows robust generalization performance across different problem sizes and net configurations.
♻ ★ Configuration-Constrained Tube MPC for Periodic Operation
Periodic operation often emerges as the economically optimal mode in industrial processes subject to varying economic or environmental conditions. This paper proposes a robust model predictive control (MPC) framework for uncertain systems described by polytopic linear difference inclusions (LDIs), whose dynamics evolve as convex combinations of finitely many linear control systems subject to additive disturbances. The robust control problem is formulated as a convex optimization program over configuration-constrained polytopic tubes, with the objective of tracking a periodic tube that is optimal for a given economic criterion. Artificial variables embedded in the formulation ensure recursive feasibility and robust constraint satisfaction under changes in the economic criterion, and convergence to the optimal periodic tube when the criterion remains constant. To improve computational efficiency, we introduce a quadratic upper approximation of the periodic cost under a Lipschitz continuity assumption, yielding a Quadratic Program (QP) that preserves the above theoretical guarantees. The approach is demonstrated on an illustrative example and an eight-state ball--plate system.
comment: 11 pages, 3 figures, submitted at IEEE-TACON
♻ ★ Event-Based Control via Sparsity-Promoting Regularization: A Rollout Approach with Performance Guarantees
This paper presents a controller design framework aiming to balance control performance and actuation rate. Control performance is evaluated by an infinite-horizon average cost, and the number of control actions is penalized via sparsity-promoting regularization. Since the formulated optimal control problem has a combinatorial nature, we employ a rollout algorithm to obtain a tractable suboptimal solution. In the proposed scheme, actuation timings are determined through a multistage minimization procedure based on a receding-horizon approach, and the corresponding control inputs are computed online. We establish theoretical performance guarantees with respect to periodic control and prove the stability of the closed-loop system. The effectiveness of the proposed method is demonstrated through a numerical example.
comment: 15 pages. Published in ACC 2026
♻ ★ Distributed Event-Triggered Consensus Control of Discrete-Time Linear Multi-Agent Systems under LQ Performance Constraints
This paper proposes a distributed event-triggered control method that not only guarantees consensus of multi-agent systems but also satisfies a given LQ performance constraint. Taking the standard distributed control scheme with all-time communication as a baseline, we consider the problem of designing an event-triggered communication rule such that the resulting LQ cost satisfies a performance constraint with respect to the baseline cost while consensus is achieved. The main difficulty is that the performance requirement is global, whereas triggering decisions are made locally and asynchronously by individual agents, which cannot directly evaluate the global performance degradation. To address this issue, we decompose allowable degradation across agents and design a triggering rule that uses only locally available information to satisfy the given LQ performance constraint. For general linear agents on an undirected graph, we derive a sufficient condition that guarantees both consensus and the prescribed performance level. We also develop a tractable offline design method for the triggering parameters. Numerical examples illustrate the effectiveness of the proposed method.
comment: 6 pages. Accepted version for IEEE CDC 2026
♻ ★ Feedback Stabilization for Sampled Linear Systems with Control-linear Noise
In this paper, we consider linear stochastic systems with control-linear noise and periodically sampled measurements. We address the problem of feedback stabilization in the mean-square sense. The main contribution of the paper is to provide a necessary and sufficient condition for feedback stabilization. In particular, we relate feedback stabilizability of the stochastic system to the existence of a fixed point of a Riccati-type algebraic equation and, further, to the existence of a solution to an infinite-horizon optimal control problem.
♻ ★ PA-SaR: Perception-Aware Learning, Control, and Coordination for Multi-Robot Search and Rescue
Autonomous SaR requires robots to explore unknown or partially known environments, avoid hazards, coordinate team activities, and guide discovered victims to safety. Improving individual capabilities of robots does not necessarily improve mission outcomes: safer motion can restrict the exploration behavior, faster discovery can increase the rescue demand, and effective guidance depends on victim responses. We introduce PA-SaR, a mathematical and architectural framework for coordinating heterogeneous local control, learning-based exploration, supervisory task allocation, and adaptive approach-lead victim guidance around mission-level objectives. PA- SaR distinguishes environmental ground-truth, robot-specific perceptions, and simulated victim behavior. This provides a common basis for integrating complementary mechanisms and analyzing their interactions across mission stages. We demonstrate the framework through single- and multi-robot simulation studies incorporating model-predictive and learning-based exploration, perception-based safety shielding, event-driven task allocation, and adaptive victim guidance. The results show trade-offs between collision avoidance and search completion, indicate that supervisory allocation improves search completion beyond information sharing alone, and demonstrate that increased exploration does not necessarily yield more rescues. Adaptive guidance increases rescues relative to continuous leading, without a detected improvement in mean or final victim trust or a reduction in mortality. These findings support evaluating local control, team coordination, and evacuation as related but distinct aspects of autonomous SaR within the considered simulation environments.
♻ ★ Data-driven Linear Quadratic Integral Control: A Convex Formulation and Policy Gradient Approach
This paper studies the data-driven synthesis of linear quadratic integral (LQI) controllers for continuous-time systems. The objective is to achieve optimal state-feedback control with integral action for reference tracking using only measured data. To this end, we derive a data-driven closed-loop parameterization of the augmented dynamics that incorporates the integral state while relying solely on input-state-output measurements of the underlying system. Based on this parameterization, a data-driven convex optimization problem is formulated whose solution yields the optimal linear quadratic regulator (LQR) feedback gain for the augmented system without explicit knowledge of the system matrices. In addition, a projected policy gradient flow, which is a preconditioned version of the model-based policy gradient flow, is derived to compute the optimal controller within the space of stabilizing gains. The proposed approach enables data-driven optimal tracking control while avoiding explicit state augmentation in the data collection phase. For time-varying gains, as they arise in an online implementation, the direct, incremental, and feedforward realizations of the LQI controller are compared, and the equilibrium required for feedforward is obtained from the same data. The method is illustrated on a distributed generation unit (DGU) in a DC microgrid, including an online implementation with closed-loop data under load changes, and by a Monte Carlo study on random systems with noisy data.
♻ ★ Adaptive Reparametrized Time for Score-Based Diffusion Sampling
We study timestep allocation for score-based diffusion sampling, where a learned reverse-time dynamics is discretized on a finite grid. Uniform and hand-crafted schedules are standard choices, but they rely on fixed prescriptions and can therefore be suboptimal. To address this limitation, we propose Adaptive Reparameterized Time (ART), a continuous-time control formulation that learns a time change by treating the speed of the sampling clock as the control, so that a uniform grid on the learned clock induces adaptive timesteps in the original diffusion time. Based on a leading-order Euler error surrogate, ART provides a principled objective for allocating timesteps along the sampling trajectory. To solve this deterministic control problem, we introduce ART-RL, an auxiliary randomized formulation with Gaussian policies that turns schedule learning into a continuous-time reinforcement learning problem. We prove that the randomized ART-RL formulation is equivalent to ART at the optimizer level, in the sense that its optimal Gaussian policy recovers the optimal ART time-warping rate through its mean. We further establish policy evaluation and policy improvement characterizations and derive trajectory-based moment identities that yield implementable actor--critic updates for learning the schedule. Across experiments ranging from controlled low-dimensional settings to image generation, ART-RL can be plugged into existing diffusion samplers by changing only the timestep grid, consistently improving sample quality over strong baseline schedules at matched budgets while leaving the rest of the sampling pipeline unchanged. The learned schedules also exhibit broad generalization, transferring without retraining across sampling budgets, datasets, solvers, pipelines, and representation spaces.
comment: 37 pages, 14 figures, 8 tables. Accepted for publication in the Journal of Machine Learning Research (JMLR)
♻ ★ Stability of DC Microgrids with Nested Nonlinear Control Frameworks: The Fast Communication Scenario
As modern power systems continue to evolve into multi-agent, converter-dominated systems that demand reliable, stable, and optimal control architectures within an expandable framework, this paper investigates scalable stability guarantees of a promising nonlinear communication-reliant control framework for DC microgrids. Particularly, relying on nested control loops, inner decentralized (primary) and outer distributed (secondary), the control configurations are designed to simultaneously achieve proportional current sharing and voltage containment within pre-specified limits, at the converged steady state. By enforcing sufficient time-scale separation at the border between the control loops, the system admits a singular perturbation formulation, allowing global exponential stability (G.E.S.) to be established via Lyapunov arguments. Although the theoretical G.E.S. certificate is structurally scalable, the stability guarantees depend on a sufficiently large permanent leakage, introduced in the primary controller. Thus, this paper emphasizes the importance of appropriate practical tuning guidelines for establishing suitable system configurations under which the DC microgrid is practically scalable and its equilibrium is G.E.S.. The effectiveness of the proposed method is validated through case studies on a low-voltage DC microgrid under load variations and topological changes, followed by a small-signal stability analysis
comment: 15 pages, 6 figures
♻ ★ Industrial Load Modeling and Optimization for Market-Based Interaction with Power Systems
The transition toward a renewable-dominated power system increases the flexibility needed to balance net-load variations. Industrial loads account for over 40% of the world's electricity consumption and can shift large amounts of demand through automated production scheduling. However, making industrial processes a dependable flexibility resource for power systems remains challenging because industrial process constraints are non-convex, equipment parameters are often private, and device-level models of thousands of facilities are too detailed for aggregator and power-system optimization. This dissertation builds computationally efficient methods for modeling and optimizing industrial flexibility for power-system interaction. Specifically, this dissertation uses smart meters as the external measurement interface and maps industrial process constraints into a concise flexibility space for power-system operation. The resulting chain of mechanism modeling, parameter identification, constraint dimension reduction, and optimal decision-making makes industrial-load participation computationally tractable while retaining precise process models for final execution.
comment: PhD thesis, Tsinghua University, June 2026, 176 pages
♻ ★ Benign Geometry and Distributional Robustness of $\mathcal{H}_2$ Synthesis
In this paper, we study standard and distributionally robust $\mathcal{H}_2$ synthesis problem of a stabilizing state-feedback controller for discrete-time linear time-invariant systems. Without requiring a nonsingular disturbance controllability Gramian or a positive-definite control penalty, we establish that stationarity of a stabilizing gain in standard $\mathcal{H}_2$ synthesis is equivalent to global optimality and derive an exact Pythagorean identity for the performance difference. We then generalize the $\mathcal{H}_2$ synthesis problem by considering i.i.d. disturbances with zero mean and uniformly bounded second moments, and extend the stationarity-optimality equivalence to the corresponding distributionally robust $\mathcal{H}_2$ synthesis. Moreover, we show that every standard $\mathcal{H}_2$-optimal gain is simultaneously optimal for every such distributionally robust problem. Finally, we specialize the framework to Frobenius, Kullback--Leibler, and Wasserstein-2 ambiguity sets and obtain explicit characterizations of their worst-case covariances, which, in turn, provide performance certificates for a common optimal controller.
♻ ★ Policy Library CBF: Finite-Horizon Safety at Runtime via Parallel Rollouts
Safety-critical autonomy in unstructured environments poses significant challenges for online safety certification under evolving constraints, particularly when a single backup policy is not effective across operating conditions. We propose the Policy Library Control Barrier Function (PL-CBF), a runtime safety filter that evaluates a library of fallback policies via parallel finite-horizon rollouts. Among policies with constraint-satisfying rollouts and feasible CBF constraints, PL-CBF selects the policy maximizing the volume of its admissible control set and solves a single quadratic program to minimally modify the nominal input within that constrained input set. Using a finite-horizon language metric over closed-loop behaviors, we characterize when a finite policy library contains a constraint-satisfying rollout. Five simulation studies cover indoor robot navigation with a double-integrator model (4 states), highway driving under abrupt friction changes with a nonlinear vehicle model (8 states), and quadrotor flight control among dynamic obstacles (12 states), demonstrating improved safety performance compared with the evaluated baselines while maintaining millisecond-scale mean computation times. Hardware experiments on a mobile robot among moving pedestrians and a custom-built quadrotor among dynamic obstacles demonstrate real-time operation of PL-CBF.
comment: Project page: https://www.taekyung.me/plcbf
♻ ★ On-Device Robotic Planning: Eliminating Inference Redundancy for Efficient Decision-Making
Reasoning-based robotic policies using large language and vision-language models achieve strong semantic planning capabilities but mostly suffer from a high inference latency that limits practical real-time deployment. In this work, we observe that robotic reasoning workloads contain substantial temporal redundancy, where consecutive observations frequently produce identical actions and subgoals. Based on this insight, we present REIS, a human cognition inspired robotic decision-making framework that minimizes unnecessary reasoning while preserving semantic adaptability. REIS combines lightweight scene gating, KV-steered affordance routing, and deliberative reasoning to accelerate robotic control under embodied constraints. Experiments on ALFRED, and real-world robotic tasks demonstrate that REIS significantly suppresses reasoning overhead while maintaining competitive task performance
comment: 8 pages
♻ ★ Human-on-the-Loop Distributed Control of Grid-Interactive Buildings for Demand Response Participation
This paper proposes a human-on-the-loop distributed consensus control approach for demand-side management across multiple buildings. Specifically, a novel framework is introduced in which a human acts as the non-autonomous leader in consensus control of cooperative buildings participating in demand response programs. In this system, the facility manager in the facility building serves as the leader, determining the participation level of cooperative buildings in demand response events while simultaneously considering occupants' comfort. Cooperative buildings align their responses with the facility management building, despite lacking direct access to the facility manager's decisions, and this raises two coupled challenges: The manager's actions act as an unknown input to the leader, and the consensus protocol requires a state that cannot be measured. Because the input is unknown, the necessary observer should be an unknown-input observer to guarantee the performance, and the observer-matching condition required by conventional linear designs does not hold for this system. A human-on-the-loop leader-follower consensus protocol is introduced to manage energy use and balance thermal comfort, together with a nonlinear unknown-input sliding-mode observer that reconstructs the local consensus error in finite time without requiring the mentioned condition. Under exact measurements, continuous-time dynamics, and an admissible initial state, a componentwise certificate guarantees bounded tracking and bounds on ramp rate, comfort, and power when the leader power and the ambient temperature remain within prescribed limits. Simulation results validate the approach, demonstrating its ability to achieve consensus, maintain system performance, and enhance the adaptability of power grid operations under various demand response scenarios.
comment: 10 pages, 9 figures
♻ ★ Optimizing Bidding Curves for Renewable Energy in Two-Settlement Electricity Markets
Coordination of day-ahead and real-time electricity markets is imperative for cost-effective electricity supply and also to provide efficient incentives for the energy transition. Although stochastic market designs feature the least-cost coordination, they are incompatible with current deterministic markets. This paper proposes a new approach for compatible coordination in two-settlement markets based on benchmark bidding curves for variable renewable energy. These curves are optimized based on a bilevel optimization problem, anticipating per-scenario responses of deterministic market-clearing problems and ultimately minimizing the expected cost across day-ahead and real-time markets. Although the general bilevel model is challenging to solve, we theoretically prove that a single-segment bidding curve with a zero bidding price is sufficient to achieve system optimality if the marginal cost of variable renewable energy is zero, thus addressing the computational challenge. In practice, variable renewable energy producers can be allowed to bid multi-segment curves with non-zero prices. We test the bilevel framework for both single- and multiple-segment bidding curves under the assumption of fixed bidding prices. We leverage duality theory and McCormick envelopes to derive the linear programming approximation of the bilevel problem, which scales to practical systems such as a 1576-bus NYISO system. We benchmark the proposed coordination and find absolute dominance over the baseline solution, which assumes that renewables agnostically bid their expected forecasts. We also demonstrate that our proposed scheme provides a good approximation of the least-cost, yet unattainable in practice, stochastic market outcome.
♻ ★ Adversarial Observability and Performance Trade-offs in Optimal Control
We develop a feedback controller that minimizes the observability of a set of adversarial sensors of a linear system, while adhering to strict closed-loop performance constraints. We quantify the effectiveness of adversarial sensors using the trace of their observability Gramian and its inverse, capturing both average observability and the least observable state directions of the system. We derive theoretical lower bounds on these metrics under performance constraints, characterizing the fundamental limits of observability reduction as a function of the performance trade-off. Finally, we show that the performance-constrained optimization of the Gramian's trace can be formulated as a one-shot semidefinite program, while we address the optimization of its inverse through sequential semidefinite programming. Aircraft and satellite simulations demonstrate that the proposed controllers reduce adversarial observability while maintaining near-optimal performance.
comment: 8 pages, 4 Figures
Robotics
★ World In-Context Learning for Agentic Robot Manipulation
Trained action policies write manipulation skill into network weights, tying what is learned to the distribution it is learned from. In-context learning (ICL) avoids this coupling but has not matched trained policies, because a demonstration recorded in another scene carries no feedback from the world the robot currently faces. We observe that what an agent learns in context is semantic, recording task steps and object roles rather than visual appearance, so experience gathered in one world transfers to another. Based on this insight, we propose World In-Context Learning (WICL), a paradigm in which a frozen model starts from a demonstration, rehearses the task in a world such as a simulator, and writes the feedback into its context rather than its weights. We build RoboScript, a modular infrastructure that implements WICL for agentic manipulation. From a single demonstration, RoboScript learns each task through rehearsal in simulation, freezes the learned knowledge as a Task Memory, and deploys the frozen memory on new scenes with data isolation that ensures every deployment input is available on a real robot. On all 50 tasks of RoboTwin 2.0, RoboScript reaches 83.4% mean success across clean and randomized settings with one demonstration per task and no gradient updates, exceeding the strongest trained action policy by over 20 points. The method generalizes across visual distributions, improves through reflection, scales with the foundation model, and transfers to a physical dual-arm robot through cross-embodiment memory adaptation. With a world in the loop, a robot can learn a manipulation task in its context as well as it learns one in its weights. All code is available at https://github.com/ArcheOne-AI/RoboScript
comment: Project Page: https://ArcheOne-AI.github.io/RoboScript
★ SWAM: Scaling World Action Models with Cross-Modal Alignment
World action models jointly predict video and actions conditioned on language. Existing studies focus primarily on architectures and training mechanisms, typically treating language-video alignment acquired through video pretraining as an established foundation. However, introducing action learning may impair this alignment. Principled guidance on data and supervision design for maintaining language-video alignment during action learning remains limited. To investigate this, we curate a pool of approximately 47,000 hours of general videos, egocentric videos, UMI demonstrations, and robot demonstrations. Building on this, we systematically study how language-video supervision, language-video-action supervision, and the joint use of these two forms affect WAM performance. In particular, we examine how language annotation quality affects pretrained language-video capabilities, and how action supervision from UMI and robot demonstrations extends these capabilities to language-video-action modeling. To better scale the model, we further conduct detailed studies on joint training recipe and model architecture. Under a fixed training budget, mixing egocentric videos and UMI demonstrations improves control generalization. Model architecture affects how effectively these experiences are used, while scaling pretraining under the selected configuration brings further gains. The resulting recipe, post-trained on RoboTwin Clean demonstrations only, achieves success rates of 88.10% and 61.56% under Easy (Clean-to-Clean) and Hard (Clean-to-Random) conditions, respectively. Our model achieves 40.00% SR across three real-world tasks, compared with 23.33% for the strongest baseline, indicating better generalization under unseen conditions. We release the curated data, training code, and model weights.
★ DreamLander: A 6-DOF World Model based Reinforcement Learning Planetary Lander
Soft powered descent landings are becoming evermore important in the field of space exploration and scientific advancement, because of their ability to deliver mission critical payloads, both highly accurately and safely. The high efficiency restriction on planetary landings due to the severely limited fuel supply results in the need for optimal guidance trajectories and control. Model based reinforcement learning is proving to be an important tool in simplifying the highly complex process of generating optimal guidance trajectories and control. In this work, we present DreamLander, a state of the art model based reinforcement learning algorithm based on DreamerV3, to generate optimal guidance and control for a realistic 6 degree of freedom entry descent landing problem. The model based approach creates a sample efficient and highly generalizable model that can be applied to a variety of scenarios whilst maintaining a high degree of accuracy, landing within 2 meters of the designated landing zone, and with velocity norm less than 0.3 meters per second. DreamLander makes use of state sensor observations, making progress toward End-to-End solutions of the Entry Descent Landing problem.
★ Simulating Vein Compressibility for Ultrasound-Guided Femoral Access: Speed, Accuracy, and Phantom Validation
In ultrasound-guided femoral access, probe pressure deforms the tissue and changes the vascular anatomy a robot observes. A simulator must reproduce these mechanics fast enough for robotic interaction, yet the computational effort needed for a clinically meaningful signal like vein collapse before artery is unclear. We test the feasibility of a physics-based simulation in NVIDIA's Isaac Sim for Healthcare, with Stable Neo-Hookean tissue solved by Vertex Block Descent on one GPU. A simulation of a low-fidelity cylindrical benchtop phantom with known geometry and mechanical properties, pressed by a robot, predicted its artery within 5 percentage points at every depth. It matched the vein within 3 points through 11 mm of a 25.4 mm press and followed the collapse behavior of the low-fidelity physical phantom. Across element size, time step, and solver iterations, wall clock time per frame scaled linearly with solver work, but accuracy did not improve with more work. The chosen setting reduced solver time per frame by a factor of 976, from 29,700 ms to 30.4 ms, while remaining within 1.03 points of the finest reference setting. The physics solver alone took 30.4 ms per frame, while the full stack, including ultrasound and graphics rendering, took 56.4 ms per frame. Finally, when seeded with realistic literature-based mechanical properties, the digital leg model compressed the vein far more than the artery at every setting, reproducing the expected clinical signal of vein collapse before artery, which is used to distinguish them. In total, these results suggest that the NVIDIA Isaac Sim platform may be suitable for augmenting reinforcement learning training or online simulation for autonomous femoral access with ultrasound.
comment: Submitted to the Journal of Medical Robotics Research (JMRR) for the International Symposium of Medical Robotics 2027 (ISMR 2027) 11 Pages (8 main + 3 references), 4 figures in main body
★ Learning Heterogeneous Tactile Representation for General Dexterous Manipulation
Tactile sensing is an indispensable modality for contact-rich dexterous manipulation, a capability that hints at the ultimate frontiers of embodied intelligence. However, commercially available tactile sensors are still at a relatively early stage of development, with each type of sensor offering irreplaceable strengths alongside its own drawbacks. A key challenge is thus to free tactile understanding from sensor-specific physics, so that embodied models can effectively leverage heterogeneous tactile data. To this end, we develop LingBot-Touch, a tactile foundation model characterized by the following properties. (i) Our model yields a unified representation across different tactile sensors (i.e., piezoresistive, magnetic, and visuotactile sensors), and is easily extensible to new sensor types. (ii) The resulting tactile tokens are plug-and-play: they can be seamlessly integrated into policy models across paradigms, from diffusion policy to vision-language-action (VLA) models such as $π_{0.5}$ and LingBot-VLA, consistently enhancing manipulation dexterity. (iii) Extensive experiments, spanning simulation and real-robot deployments across multiple hardware platforms, demonstrate that our tactile representation substantially benefits complex manipulation tasks while maintaining strong computational efficiency. We believe the findings in this work demonstrate the potential of leveraging cross-sensor data to scale tactile learning, and offer encouraging insights for the design of future hardware and embodied models.
★ Benchmarking Real-World Robotic Manipulation through Stochastic Challenges
A challenging bottleneck in manipulation research is reproducibility and comparability of methods in real-world environments. Since manipulation is an interactive task, it cannot be evaluated on a static dataset of observations. Benchmarking in simulation is a common remedy, but cannot cover the whole complexity of real-world tests. This paper proposes a new paradigm of stochastic challenges. Like the well-known YCB objects, it is based on reproducible real-world scenes and therefore allows system-level innovation and comparability between different platforms. Due to their stochastic nature, our challenges however, mimic the rigor and surprise of hidden test sets. To this end, we show how different robots in different labs can be compared based on their success rate over different random task samples, if these samples come from the same distribution. We propose stochastic challenges as a general paradigm for robot benchmarking and implement this paradigm in a novel benchmark for mobile manipulation, on which we compare two robots tested in two different labs.
comment: Accepted at the 10th Conference on Robot Learning (CoRL 2026). 15 pages, 3 figures
★ Certified Zonotopic Reachability Analysis for Trajectory Tracking on Lie Groups
Autonomous vehicles track planned trajectories close to obstacles. Unknown disturbances make them deviate from these trajectories. With a bound on the tracking error that disturbances can cause, a planned trajectory can be certified as safe before execution. Existing set-based methods bound the tracking error by one invariant ellipsoid for the whole trajectory or by intervals, which are both conservative. For aerial and space vehicles, the tracking error evolves on a matrix Lie group with nonlinear dynamics. However, for group-affine systems, the error dynamics are linear in Lie algebra coordinates when the vehicle receives the inputs of the reference. The control input and the disturbance enter through the inverse right Jacobian, a matrix that depends on the error. We propagate the linear part of the error dynamics with zonotopes and enclose the effect of the inverse right Jacobian on the disturbance in closed form. The resulting tubes are tight and provably contain all reachable tracking errors in continuous time. When some error components have no input, the inverse right Jacobian leaves unmatched nonlinear terms in the closed loop. None of the tested methods certifies such a closed loop. Hybrid coordinates make these terms matched, so that our controller cancels every nonlinear term that does not contain the disturbance. Our tubes give tighter bounds than the ellipsoid method in simulations of a planar aircraft and a fully actuated multirotor and in real flights of a quadrotor, where the tube contains every measured attitude error. Under larger disturbances and the same feedback gain, the ellipsoid method yields no valid ellipsoid, whereas our tubes remain valid.
★ Sliding into Balance: Benefits of a Sliding Payload for Load Distribution and Disturbance Rejection in Dual-UAV Transportation
This paper analyzes the cooperative transportation of a payload suspended from two drones in which, instead of being attached to two independent ropes, the payload is allowed to slide along a single rope connecting both UAVs. First, a planar nondimensional dynamical model of the sliding motion with viscous friction is presented. Then, considering stationary drones, this configuration is compared with a fixed payload with the same total rope length, both at equilibrium and under disturbances. The analytical results show that the sliding payload equalizes the tension in both rope segments for any relative position of the drones, so that no rope loses tension at equilibrium; it doubles the region reachable by the payload and, except for low friction combined with a large drone separation, it enlarges the set of admissible disturbances. Finally, a dynamic simulation also shows that under realistic gusty wind profiles, the average tension imbalance between the drones is reduced with respect to the fixed-payload solution.
comment: 6 pages, 6 figures
★ $Σ$-LeWM: Learning World Models via S-Information for Robot Control
World models have increasingly been applied as a model-based control method for robotics tasks such as manipulation and locomotion. While early world model architectures relied on decoders to prevent mode collapse and learn a predictable latent space, recent works on world models, such as Joint Embedding Predictive Architecture or LeWorldModel, increasingly aim to learn embeddings without a grounding decoder, utilising only a latent forward dynamics predictor and a stabilising entropy term. In this paper, we set out to demonstrate that the further introduction of two additional latent losses to LeWorldModel, a backward dynamics predictor and an inverse dynamics action predictor, enforces further structure on the latent embedding, leading to further gains in both performance and planning efficiency. These are critical properties for improving the applicability of such methods to real-time model-based control of real-world robots. We demonstrate these gains on standard world model benchmarks and two simulated robot manipulation tasks.
★ Library of Hands: Fix the Hands, Not the World Model
A world model predicts how a scene changes after an action, and a planner uses it to search for actions that reach a goal. If the camera moves or the controller changes, the same command moves the robot differently: when its commands are turned by 90 degrees, a frozen planner with the LeWM world model that succeeds in 77.5% of episodes succeeds in only 7.8%. Current fixes retrain the model, need training data, or forget a set-up when it returns. We propose Library of Hands (LoH), a training-free method that keeps the world model frozen and corrects only the map from commands to applied actions, which we call the hand. LoH reads the hand through the frozen model, with no extra sensor: it finds the action that best explains the change between two camera images (a reading). It stores every hand it identifies, so a returning set-up is recognised rather than re-learnt. In closed-loop tests, LoH more than doubles success on three simulated tasks, from 28.6% to 62.3%, and loses only 0.6 percentage points when nothing changes. It also works, unchanged, with two other world models and on two of three models pretrained on real robot data, and its gain grows with the quality of its readings.
comment: 27 pages, 8 figures, 20 tables
★ VLA Adaptation with Real and Simulated Data: A Case Study on Robotic Chess
Vision-language-action (VLA) models are trained as general-purpose foundation models for robot learning, and require fine-tuning for each downstream task and robot. This raises practical questions such as how much data and training updates are needed, how checkpoints should be selected, and whether simulation can be leveraged to alleviate the cost of real data collection. We study these questions on a task that requires both precise manipulation and spatial understanding of the scene. In this task, an SO-101 arm moves a chess king between two squares that are given only by their names, such as e4 and a5. We fine-tune $π_{0.5}$ with various dataset sizes and update budgets, using teleoperated demonstrations as well as simulated demonstrations from a randomized scripted expert that imitates the variability of human demonstrations. In simulation, the success rate scales with both data and updates. Importantly, we find that a short learning-rate cool-down adds up to 31 points and is necessary for comparing checkpoints, and that validation loss and action error are unreliable for model selection. On the real arm, we fine-tune on up to 256 teleoperated demonstrations and evaluate ten policies in 25 trials each. Partial results indicate that pretraining on the simulated demonstrations helps substantially, most clearly when real data is scarce. We also present a negative result, in which training on the policy's own rollouts with filtered behavior cloning, RECAP-style advantage conditioning, corrections from an automatic coach, or residual reinforcement learning did not yield a measurable improvement over plain behavior cloning in our setting. We release the simulator, the expert and the real demonstrations.
comment: CoRL 2026 Workshop "Pretrain to Adapt: What Makes a Pretrained Policy Adaptable?". 8 pages main text, 15 pages total, 15 figures, 4 tables
★ Correctness Boundaries of Typed Plans for Natural-Language Robot Fleet Control
We examine what plan validation and execution reports establish in natural-language robot fleet control. Commands are traced through typed plans, deterministic corrections, fleet coordination and outcome checks in a three-level Robot Operating System 2 implementation with twenty simulated robots. An offline benchmark of 44 cases checks plan structure and selected command requirements. Across five local configurations without a response schema, 70.0-87.3% of responses pass. All runs in a selected 18-run simulation series meet their pose-based task conditions. This series includes two successful repeats that replaced incomplete runs after a software change. Saved plans and logs document duplicate pickup assignments and goal adjustments within the original task zones. Separate formation traces show a structurally accepted plan with the wrong formation shape and a rejected route, and an arrival report that omitted a requested robot after replanning. Carry completion records pickup and subsequent drop-off zone presence for the same robot. The cases distinguish changes to executable actions from changes to the requirements covered by a completion report. They motivate checking the effective assignments, goals and reported outcome against the original command.
comment: 18 pages, 3 figures
★ Simulation-aided Reinforcement Learning with Control Variates NeurIPS 2026
Reinforcement Learning (RL) algorithms often rely on data collected from a target environment, which can be expensive and noisy. Although simulators can generate large amounts of additional data, standard simulation mixing approaches, which directly combine simulated and real experience, can degrade performance due to simulator inaccuracies, especially when they bias the solution away from the optimal policy. We propose a conservative simulation-aided learning method based on control variates, where simulated data is used solely to reduce variance in the learning process rather than directly shaping the learning target. We develop our approach for Policy-Evaluation, which is a fundamental building block of RL algorithms. Then we combine it within the actor-critic framework towards policy improvement, as well as extend it towards offline RL scenarios. Experiments on multiple MuJoCo benchmark tasks demonstrate improvements over standard simulation mixing approaches, highlighting control variates as an effective tool for simulation-aided reinforcement learning.
comment: Accepted as a poster at NeurIPS 2026. 10 pages main text + appendix, 3 figures. Code: https://github.com/OmerHemo/Simulation-Aided-Reinforcement-Learning-with-Control-Variates
★ Modeling and Estimation of a Towed Marine System
This paper addresses the modeling and state estimation of a towed marine system consisting of two autonomous surface vehicles (ASVs), a pair of umbilical cables, towfish, and a trailing loop cable. Inspired by the existing literature, we employ a lumped-mass modeling approach for the cables and towfish, capturing the essential dynamics while maintaining computational tractability. A Lyapunov-based analysis is conducted to verify the stability and convergence properties of the proposed model. Building on this model, we propose two state estimation methods, namely a reduced-order observer and a perfect-measurement extended Kalman filter (PMEKF), using only the measurements from the ASVs' onboard sensors. Numerical simulations and experimental results are provided to validate the effectiveness of the proposed methods.
comment: preprint submitted to Ocean Engineering, 23 pages (highlights + 22p manuscript), 12 figures
★ LawKAN: Group-Law Layers for Composition -- When Imposing a Known Law Pays Off
Many tasks compose transformations by a known rule: chained rotations in robotics and navigation. Learned models relearn it, compounding error; classical estimators hardcode it and miss hardware deviations. We impose the known structure and learn only the remainder: an exact non-commutative group law sits in the node of a self-contained layer, its edges learn local increments, and a zero-initialized correction psi learns unknown deviations of the law. A pre-training criterion predicts exact representability. On fixed-angle SU(2) composition, a 54-parameter dense edge is the most accurate dense model, at 27x fewer parameters than the strongest fixed-matrix baseline, and lies 0.05 deg above the Bayes floor. On real robot arms, an isolated 306-parameter psi reaches 1.12-1.14 mm versus 1.29-1.33 mm for Gauss-Newton at N >= 60, and the recipe transfers across three vendors and two protocols. The recipe matches classical fits where the data obey the law and loses where the imposed law is not the task's.
comment: 20 pages main text (53 total), 15 figures, 6 tables in main text (31 incl. appendices). Code: https://github.com/occamolab/lawkan
★ Galilean Robots, Aristotelian Sensors: Joint Spatiotemporal Trajectory Estimation and Extrinsic Calibration
Multi-sensor state estimation requires accurate spatial and temporal calibration between heterogeneous sensors. Existing spatiotemporal calibration methods typically represent sensor extrinsic parameters as a rigid-body transformation together with a scalar temporal offset, causing the spatial and temporal parameters to enter the estimation problem through separate mechanisms. While the spatial transformation composes directly with the robot pose, accounting for the temporal offset requires evaluating the trajectory at a shifted time, typically through motion compensation, interpolation, or a continuous-time trajectory representation. In this work, we formulate joint trajectory estimation and spatiotemporal calibration using the special Galilean group for the robot trajectory and the Aristotelian subgroup for sensor extrinsics. This representation incorporates the coupling between motion and temporal offset directly through the group structure, avoiding the need for a separately parameterized continuous-time trajectory. We develop twist- and IMU-driven motion priors, accommodate both ideal and non-ideal clocks, and evaluate the framework with camera, position, and range measurements. Simulated and real experiments demonstrate calibration and trajectory-estimation performance comparable to established methods across heterogeneous sensor configurations. Our simplified framework therefore offers a quality-of-life improvement over existing methods without compromising performance.
★ V3DSAM: Semantic-Geometric Loop Verification and Geometry-Anchored Persistent Instance Mapping for Feed-Forward Dense SLAM
To address unreliable cross-view association in long-sequence feed-forward dense SLAM, this letter presents V3DSAM, an open-vocabulary semantic dense SLAM system that establishes physical correspondences at both the scene and object levels. The system combines feed-forward geometric estimates from VGGT-SLAM 2.0 with open-vocabulary semantic observations from SAM3. We introduce semantic-geometric dual-scale loop verification, which identifies false loop closures using pixel-level bidirectional reprojection and object-level 3D spatial correspondences, without accessing backbone-internal features. We further propose geometry-anchored persistent instance mapping, which defines physical instances through class-constrained 3D connectivity in an optimized global coordinate system and assigns global IDs without inter-frame identity propagation. The proposed verifier rejects all four false loop candidates in our TUM RGB-D evaluation, while long-sequence ScanNet experiments show improved instance identity consistency. Thus, V3DSAM reduces dependence on internal backbone features and temporal ID propagation for robust long-term cross-view association.
comment: 8 pages, 6 figures, 4 tables. Submitted to IEEE Robotics and Automation Letters (RA-L)
★ Bridging Task Planning and Execution for Language-Conditioned Bimanual Manipulation
Robots are increasingly expected to execute natural-language manipulation tasks in real-world environments. This is particularly relevant in household assistance and logistics, where clutter requires reasoning about both task objectives and execution feasibility. This challenge becomes even more pronounced for bimanual humanoid platforms, where coordinating two manipulators introduces additional interaction and kinematic constraints. We present a zero-shot vision-language framework for execution-aware bimanual manipulation that jointly reasons about task objectives and physical feasibility. Planning is grounded in a structured scene representation that combines semantic object relations with execution constraints, enabling the selection of obstacle relocations and execution choices while preserving task semantics. Compatible actions are organised into parallel bimanual stages with explicit pre- and post-conditions. A hierarchical validator evaluates these conditions under perceptual uncertainty and uses stage-level execution history to support local recovery or targeted replanning. Experiments on the CENTAURO hybrid wheeled-legged humanoid robot across nine language-conditioned manipulation scenarios show higher task success than a non-execution-aware baseline. In dedicated failure scenarios, hierarchical validation recovers from execution failures with fewer model queries than always replanning. These results demonstrate the benefits of combining execution reasoning directly into task planning. Videos and additional material are available on the Project Webpage: https://advrhumanoids.github.io/bridging-planning-execution/.
comment: 8 pages, 6 figures. Accepted at the 2026 IEEE-RAS International Conference on Humanoid Robots (Humanoids 2026). Equal contribution by the first two authors
★ PCTac: Mechanochromic Photonic-Crystal Tactile Sensor for Dexterous Manipulation
Modern visuotactile sensors primarily sense contact by imaging the deformation of a compliant interface. Using techniques such as photometric stereo, they can recover detailed contact geometry, often represented as surface normals or depth. However, force-related information typically still needs to be inferred indirectly through calibration, geometric reconstruction, or learned mappings. Mechanochromic photonic-crystal (PC) materials offer an alternative physical encoding mechanism: contact loading changes their periodic microstructure and spectral response, directly transforming local stress into a spatially resolved structural-color field. Despite this potential, PC materials have rarely been explored as visuotactile sensors for closed-loop robot learning. We present PCTac, an open-source mechanochromic PC fingertip that uses a dense hue field as its tactile observation. Loading characterization demonstrates a repeatable relationship between the applied normal load and the observed hue response. We further integrate this representation directly into a Diffusion Policy and evaluate it on three real-robot manipulation tasks. Across 90 physical rollouts, PCTac Hue achieves 68 successes (75.6\%), compared with 60 (66.7\%) for PCTac RGB, 57 (63.3\%) for Wedge RGB, and 29 (32.2\%) for vision-only. PCTac Hue also achieves the highest observed success rate on each evaluated task, including Perturbed Test-Tube Insertion, where it achieves 16/30 successes versus 13/30 for tactile RGB baselines and 1/30 for vision-only. These results demonstrate the potential of mechanochromic structural color as a compact, load-sensitive tactile representation for robot learning and closed-loop manipulation. Project page: https://bey0ndt1me.github.io/PCTac-page/.
comment: 9 pages, 7 figures
★ AutoVLA-D: Learn a Trajectory-Geometric Condition from Reasoning for Autonomous Driving VLA NeurIPS 2026
Autoregressive VLA planning usually predicts continuous trajectories as the final output, but driving instructions often describe actions rather than ego states, which weakens direct conditioning on the final continuous trajectory. We address this gap with TrajCond, which introduces a discrete trajectory-geometric condition for VLA planning. Specifically, we represent TrajCond as a 1D-Brohan differential action trajectory and learn it with a two-stage training framework before final continuous trajectory prediction. With the simultaneous autoregressive generation of discrete and numeric trajectories, we name our method AutoVLA-D (Double). Experiments on NAVSIM navtest, nuScenes, and NeuroNCAP show that discrete trajectories are weaker as direct outputs but more effective as TrajCond, consistently improving final continuous trajectory prediction across closed-loop and open-loop driving benchmarks. Code is available at https://github.com/ShivaTeam/AutoVLA-D.
comment: Accepted by NeurIPS 2026
★ Safety-Constrained Intelligent Control of a 2-DOF Robotic Manipulator Using Control Barrier Functions
Control barrier function (CBF) filters depend on dynamic models that can be inaccurate when their parameters and external forces are uncertain. This paper investigates a radial basis function (RBF) neural network compensator whose output is shared by a Cartesian tracking controller and a joint-space CBF filter. An auxiliary state driven by the difference between requested and applied torques corrects the learning signal, reducing the direct influence of filter intervention on adaptation. A two-link manipulator is evaluated through constrained regulation, ablations, matched uncertainty and disturbance trials, and operating-range and timing studies. The results demonstrate that the proposed controller improves tracking while maintaining joint, velocity, and torque limits under parameter uncertainty, and that both the shared compensation and the auxiliary correction contribute to this improvement.
comment: Paper submitted to the European Control Conference 2027
★ UniOMA: Unified Optimal-Transport Multi-Modal Structural Alignment for Robot Perception
Achieving generalizable and well-aligned multimodal representations remains a core challenge in robot learning. While recent approaches have attempted to align modalities by modeling conditional or higher-order statistical dependencies, they do not explicitly consider the structural coherence across modalities. In this work, we propose a novel multimodal alignment method that augments existing contrastive losses with a geometry-aware Gromov--Wasserstein (GW) distance-based regularization. To this end, we encode intra-modal geometry for each modality and reduce cross-modal structural discrepancies through a dynamically learned GW barycenter, thereby encouraging structural consistency in addition to instance-level contrastive alignment. We apply this optimal-transport-based alignment strategy to robot perception tasks involving underexplored modalities such as force and tactile signals, where modality data often exhibit varying sample densities. Extensive experiments on five robotic benchmarks show that our method improves structural alignment and downstream performance over state-of-the-art multimodal contrastive baselines across a wide range of downstream tasks, including robot and environment state prediction. Moreover, our GW-based regularization can be seamlessly integrated into most InfoNCE-like objectives.
★ Rethinking Radar 3D Perception: Representation Learning from Raw Spectra NeurIPS 2026
Raw radar spectra are widely viewed as too cluttered and noisy for direct representation learning, motivating handcrafted reductions, sparse point clouds, or compact Doppler descriptors in radar perception. We challenge this assumption and argue that the main limitation lies not in the raw representation itself, but in the absence of a learning objective that separates structured signal from noise. Handcrafted reductions partially address this problem but impose rigid assumptions about what constitutes signal, discarding weak yet informative structure while retaining artifacts that fit the reduction model. Once removed, such information cannot be recovered by supervised learning. We propose an alternative: structure is the component of a radar spectrum that remains reconstructable under localized corruption, while noise is the component that does not. This perspective motivates self-supervised masked modeling for dense radar spectra without predefined signal-noise separation. We instantiate it with Leakage-aware Spectral MAE, a masked autoencoder for raw radar cubes using a corruption strategy inspired by cross-bin energy spreading caused by spectral leakage. We also demonstrate the importance of Doppler organization: treating Doppler as a primary spatial dimension in an x-y-D representation, with height encoded as features, substantially outperforms naive channel stacking and enables effective learning from dense 4D spectra. Without handcrafted reductions, our approach consistently outperforms state-of-the-art radar-based detectors and point-cloud-based self-supervised learning (SSL) counterparts. These results establish raw radar spectra as an effective representation for 3D perception when paired with an appropriate pretraining objective.
comment: 19 pages, 5 figures, 18 tables. Accepted at NeurIPS 2026. Miao Zhang and Mohamed Abdelsamad contributed equally
★ Maximizing Detection Probability in UAV Search Missions using Random-Key Optimizers
This paper focuses on the stochastic and real-time constraints of Unmanned Aerial Vehicle (UAV) path planning in Search and Rescue (SAR) missions. We introduce a configurable data-driven and intelligent routing architecture that combines a closed-loop probability update with the Random-Key Optimizers (RKO). Our problem is mathematically modeled as a dynamic probabilistic variant of the Open Vehicle Routing Problem (OVRP), enabling the solver to switch adaptively between two objectives, thereby balancing probability maximization and resource minimization. The framework degrades elegantly under noisy priors but not under biased ones: a systematically biased map performs about half as well as flying with no prior at all. Computational tests show that the proposed scheme is scalable and competitive with single-path scenarios in terms of solution performance and serves as a strong decision guide for intelligent planning in SAR operations in unstable environments
comment: 21 pages, 3 figures
★ ExoStack: A Modular and Reconfigurable Software Framework for Wearable Robotics Using ROS 2
Wearable robots depend on a software chain that links body-worn sensing to gait interpretation, assistance generation, and actuation. Personalization further requires each measured response to remain associated with the assistance parameters that generated it. This paper presents ExoStack, a modular ROS 2 framework for implementing that chain across embedded hubs and a host computer. Its Embedded Sensing and Actuation (ESA) library configures ESP32 boards as microROS sensing and actuation hubs. ExoCore supplies multi-hub synchronization, gait-event detection, profile generation, motion planning, cost evaluation, and human-in-the-loop (HIL) optimization as configuration-instantiated nodes. The PASE desktop application edits configurations, launches the processing stages, and visualizes ROS 2 topics. Hardware drivers and algorithm plugins can be added through defined interfaces without modifying the generic processing nodes. We characterize hub and planner timing on three embedded computers and demonstrate reuse on two pneumatic exosuits.
comment: The source code and accompanying documentation website for ExoStack are publicly available at https://github.com/debadrata/ExoStack and https://debadrata.github.io/ExoStack_documentation/, respectively
★ Be My Eye: Learning to See Beyond a Robot's Own Perspective
Robots operating in the real world are inherently subject to partial observability, particularly in multi-robot collaboration, where individual observations may be insufficient for effective task execution. A key challenge is leveraging complementary observations from partner robots to support collaborative decision-making. We argue that information sharing should go beyond passively transmitting raw observations or generic visual features. Instead, robots should selectively encode task-relevant information according to task objectives and execution progress. To this end, we propose a task-oriented active context sharing framework that transforms robot observations into compact representations, preserving critical information under constrained communication budgets. The framework adopts a two-stage training paradigm. In the first stage, onboard observations and task instructions are jointly encoded into compact context representations. Task-oriented future prediction and soft task-progress correspondence learning guide this collaboration-agnostic compression, preserving information about task objectives and execution progress. In the second stage, the learned representations are integrated into a pretrained Vision-Language-Action (VLA) policy, enabling bidirectional context exchange. Each robot combines complementary partner information with local observations to guide action generation. Action supervision jointly refines the context representations and policy, aligning information sharing with collaborative decision-making. Controlled experiments demonstrate substantial performance gains under constrained communication budgets, while three real-world experiments validate the framework in complex dual-robot tasks. These results highlight a shift from passive visual transmission toward active, task-oriented context sharing that directly supports collaborative decision-making.
★ VLA-MOPD: On-Policy Distillation for Multi-Task VLA Policy Consolidation
Physical intelligence requires embodied agents to ground language in observations and execute actions while retaining useful skills. Vision-language-action (VLA) policies offer a shared interface, but task-specific post-training often produces separate specialists. Consolidation is difficult: mixed-task reinforcement learning provides sparse trajectory-level credit, while offline distillation may miss student-visited observations and endpoint distillation does not supervise intermediate action-generation states. We introduce VLA Multi-teacher On-Policy Distillation (VLA-MOPD), a protocol for consolidating flow-matching VLA specialists. After acquiring task or suite experts, a shared student collects robot rollouts and queries the instruction-routed frozen teacher at each visited observation and intermediate ordinary differential equation (ODE) state. Detached per-flow-step transition targets supervise valid action chunks through a masked local matching loss; an optional reference-policy term supports retention on other tasks. Starting from one trajectory per LIBERO task, VLA-MOPD raises mean success-once from 75.4% to 98.9%, compared with 98.15% for endpoint action distillation. It improves mean RoboTwin 2.0 success from 73.0% to 89.3%; on a Piper robot, two consolidation rounds improve average success from 70.0% to 90.0%. In a near-matched Spatial-adaptation study, reference matching limits the Object suite decrease to 3.0 points, versus 13.2 without the reference term.
★ Hierarchical Multi-Channel Tip-Over Prediction via Isolated Temporal Models for Planetary Rover Safety
Reliable tip-over prevention is critical for planetary rovers operating in unstructured environments where communication delays prevent real-time human intervention. Existing learning-based approaches typically optimize a single prediction objective, limiting their ability to capture both gradual instability evolution and abrupt terrain disturbances. This work presents a hierarchical multi-channel tip-over prediction framework based on isolated temporal learning. A multi-horizon Temporal Convolutional Network (TCN) learns progressive instability patterns from inertial measurements, while a Gated Recurrent Unit (GRU)-based forward dynamics model predicts nominal rover motion and detects unexpected dynamic deviations through prediction residuals. These complementary signals are integrated through decision-level fusion and priority-based arbitration to generate graded safety actions, including early warning and emergency braking. The proposed system is deployed as an Orocos Real-Time Toolkit (RTT)-based safety component within the Rock robotics framework, operating at 100 Hz on a planetary rover software stack. Experiments on Inertial Measurement Unit (IMU) samples collected across heterogeneous terrains demonstrate 97.58% event prediction coverage with a 1.82s median lead time, outperforming individual prediction channels. The results validate that isolated temporal models combined through hierarchical decision fusion provide a robust solution for real-time rover safety in uncertain terrain.
comment: 7 pages, 5 figures
★ RAIL: Representation-Aligned Imitation Learning for Student-Compatible Teacher Policies NeurIPS 2026
Reinforcement learning (RL) from raw sensory inputs such as images or onboard sensors can be challenging due to high-dimensional observations and sparse rewards. A common strategy is to train a teacher policy with access to privileged state information and then distill it into a student policy that acts from raw inputs alone. However, when the teacher relies on information unavailable to the student, exact imitation may be impossible, creating an irreducible imitation gap. Existing approaches typically mitigate this mismatch through careful reward shaping or additional reinforcement learning on the student policy after the teacher has been trained. Instead, we introduce Representation-Aligned Imitation Learning (RAIL), which addresses the mismatch during teacher training itself. RAIL learns a latent representation shared across teacher and student observations using contrastive learning, and trains the teacher policy directly in this space. Because the teacher policy is constrained to operate on this shared space, it learns behaviors that are reproducible from student observations. This substantially reduces the imitation gap while preserving task performance. Across multiple environments, RAIL outperforms strong baselines without reward modification or post-hoc student fine-tuning, and surpasses direct reinforcement learning from raw observations. The learned representations further enable zero-shot transfer to new tasks through teacher-only training. Project site: https://merajmammadov.com/rail/.
comment: 18 pages, 3 figures. Accepted at NeurIPS 2026
★ A Game-Theoretic MPC Planner for Cooperative and Adversarial Drone Coordination
We present and evaluate a game-theoretic model predictive control planner for cooperative goal reaching and adversarial pursuit-evasion with drones. The planner combines 3D prediction, individual objectives, optional shared separation constraints and trajectory-tracking interfaces. A comparison of cooperative game solvers assesses their ability to find feasible trajectories that resolve conflicts. A computational study quantifies how number of drones, prediction horizon and move blocking affect computation time and numerical acceptance. Move blocking holds control inputs constant over groups of prediction steps, reducing the number of independent control variables and lowering computation time. Timing measurements identify the practical limits of real-time planning in the tested configurations. PyBullet simulations demonstrate cooperative goal reaching and pursuit-evasion. A laboratory flight complements the simulations by validating the trajectory-tracking interfaces on physical drones. The planner can serve as a basis for further evaluation and controller development.
comment: 7 pages, 4 figures, submitted to European Control Conference (ECC27)
★ Graph-Based RL for Sequential Rendezvous Planning with Multiple Patrolling USVs
In patrol missions involving multiple unmanned surface vehicles (USVs), a designated command USV can support fleet management by sequentially rendezvousing with each patrolling USV. These operations require motion-feasible route planning and priority-aware rendezvous sequencing, with the objective of minimizing mission completion time and total tardiness relative to priority-dependent due dates. We propose a framework combining grid-based path planning, surrogate travel-time prediction, and graph-based reinforcement learning. The path planner accounts for ocean currents and approximates turn-rate constraints through multi-stage maneuvers, with path feasibility empirically validated through trajectory-tracking simulations. For rendezvous sequencing, a Feasible Transition Graph (FTG) represents candidate rendezvous events and temporally feasible transitions, enabling a graph neural network to exploit spatiotemporal rendezvous relationships for policy learning. A surrogate predicts travel times from compact current and motion features to avoid repeated path searches during FTG construction and policy training. Evaluation using grid-based path planning shows statistically significant average objective reductions of 10.5%, 3.5%, and 1.5% compared with greedy, MLP-RL, and 2-step lookahead greedy, respectively. The policy achieves these improvements while requiring approximately 1/35 of the latter's computation time. It maintains its advantage over MLP-RL under unseen priority-tier settings, with larger gains as fleet size increases.
★ Co-Creating Neurorehabilitation Technology Alongside Lived Experience Experts: A Participatory Design Workshop
Involving patients' lived experience (LE) early in neurorehabilitation technology design can improve the relevance and effectiveness of solutions, yet LE experts are often involved only when finished prototypes are tested. We present and evaluate a participatory design (PD) workshop for early LE involvement, examine how LE contributions influence design processes and outcomes, and provide recommendations for similar approaches. We conducted a PD workshop at the 6th International Conference on NeuroRehabilitation. Five people who have had a stroke (LE experts) and 18 workshop attendees (including the LE experts' caregivers) were organized into five groups. Following the Discover, Define, and Develop phases of the Double Diamond model, each group used Journey Maps and Data Collection Notebooks to co-create a solution for an objective specified by an LE expert. Journey Maps were thematically analyzed, and group audio transcripts underwent structured extraction using a large language model. LE input shifted discussions from discipline-based assumptions toward priorities grounded in lived experience, including fear, confidence, sensory discomfort, fatigue, aesthetics, and independence. Narrow objectives, such as opening a blister pack, produced task-specific solutions, whereas broader objectives generated device concepts, including sensory feedback for cycling and a stimulation-equipped cooking splint. Satisfaction was high among attendees and LE experts. All workshop tools and protocols are shared to support adoption. The workshop offers a practical foundation for involving LE experts early in neurorehabilitation technology development. Future iterations should better capture psychological and social dimensions, allow more reflection, and include the full range of relevant stakeholders through sustained transdisciplinary engagement between researchers and the LE community.
comment: Submitted to JMIR Rehabilitation and Assistive Technologies
★ Embodied First Exam
Early successes on agentic robotics point to a potential paradigm shift toward using coding agents to control robots. Existing robotics benchmarks are built around visuomotor policy learning, leaving it open whether coding agents can reliably control diverse robots under physical-world constraints. We therefore introduce the Embodied First Exam, a unified, comprehensive, and agent-native evaluation framework where coding agents can choose their own control approach: direct control, code to control, or train a policy to control. We hope it serves as a first exam for coding agents before they enter the physical world, with 1,156 tasks in 35 suites on 47 robots, including 9 new suites that go beyond pick-and-place to spatial puzzles, dexterous manipulation, multi-robot interaction, dynamic whole-body control, and sim-to-sim engineering. Each task can be run in up to three evaluation settings. The Standard Setting mimics the real world: the agent receives only sensor observations and, by default, cannot reset the environment. Its Realtime variant keeps the environment running while the agent reasons. The Privileged Setting gives the agent access to the full simulator state and unlimited resets, allowing it to search for an oracle solution for training purposes. Success rates in this setting serve as an approximate upper bound. On the 456 core tasks, the strongest of eight agents, GPT-6 Astra in Codex, achieves a success rate of 34% in the Standard Setting, leaving much room for improvement. To see where agents fail, we label the stage at which failed attempts go wrong and compare failures across agents, settings, and stage categories.
comment: Preprint version
★ GeoMind: A Benchmark for Spatial Understanding in Robotic Manipulation
Robotic manipulation has advanced rapidly in mapping visual observations and language instructions to actions. However, reliable manipulation in open-world environments requires robust spatial understanding: the ability to identify targets in complex layouts (spatial reasoning) and adapt to scenes that continuously evolve through interaction (spatial memory). Yet existing embodied benchmarks fall short in evaluating spatial understanding: they often introduce strong visual cues that bypass spatial reasoning, and focus on static scenes that offer little insight into an agent's capacity for spatial memory. To address this gap, we introduce GeoMind, a systematic benchmark for structured spatial understanding through tabletop manipulation. GeoMind comprises 54 evaluated task variants organized around spatial reasoning and spatial memory. It also incorporates a scalable pipeline that automates the generation and annotation of task instances with verified targets and executable interactions. Extensive evaluations across diverse embodied agents reveal their consistent weaknesses in spatial understanding. Our findings highlight that these agents struggle to track identities after spatial changes and convert relational or geometric constraints into executable actions. We release the GeoMind benchmark at https://github.com/Notfound-JH/GEOMIND-benchmark.git.
★ Same Physics, Different Gradients: A State-of-the-Art Report on Differentiable Physics Simulation
Differentiable simulation propagates derivatives through a physics simulator, turning animation, system identification, control, and computational design into gradient-based optimization. Engines differ in the physics they model and, less visibly, in which mathematical object they differentiate. We separate two questions the literature conflates. The referent is which object the returned derivative belongs to: the finite computation that executed, the converged equation it approximates, or the continuous dynamics. The realization describes the organization of the backward pass: automatically generated differentiation, an equation-level adjoint solve, or a custom adjoint. The referent defines the target against which exactness is assessed; the implementation determines how accurately that target is approximated. Cost depends on the realization together with how the states needed by the backward pass are retained or recovered. Crossing the axes places every method and engine we classify in a cell, separates claims the literature merges, and shows that the apparent trade-off between exactness and memory follows from how methods have been built rather than from a constraint. A second, orthogonal taxonomy covers contact, where the map is non-smooth. We then ask whether a correct gradient helps. Four mechanisms by which useful gradients are lost are cataloged and measured on one testbed, where the advantage of the exact derivative over the equation-level adjoint depends on where the fitting target comes from as much as on the solver budget, and we review the evidence that zeroth-order methods can outperform simulator gradients in the regimes where those mechanisms act. The model classes, systems techniques and applications are surveyed through this lens, and we close with a reporting checklist, a released diagnostic that computes it, and a research agenda.
comment: 35 pages, 8 figures, 9 tables
★ EgoLabel: Zero-Shot Egocentric Auto-Labeling with Cross-Modal Consistency
Egocentric auto-labeling requires depth, camera pose, object, and human-pose estimates to agree across independently developed models. Differences in units, coordinate conventions, and scale can make individually plausible predictions geometrically inconsistent. We present \textbf{EgoLabel}, a zero-shot pipeline that combines off-the-shelf models without target-domain labeled training and provides explicit cross-modal registration and consistency diagnostics. The pipeline integrates MonST3R for geometry, LocateAnything with GMC-BoT-SORT and SAM~2 for objects, and V-HPOT for hands; FisheyeViT provides a separate whole-body compatibility demonstration. Our main hand evaluation uses H2O sensor depth. Across 20 sequences, per-hand depth anchoring reduces the mean of per-sequence median hand--depth residuals by $23\%$ relative to a single global scale, yielding a pooled median of $84.6\,\mathrm{mm}$ against a $20.5\,\mathrm{mm}$ ground-truth reference. Hand PA-MPJPE is $14.53\pm1.29\,\mathrm{mm}$, whereas absolute MPJPE is $119.1\pm31.3\,\mathrm{mm}$, revealing a substantial gap between aligned and metric accuracy. On three RGB-only sequences, camera ATE is $1.40\pm0.29\,\mathrm{cm}$ after $\mathrm{Sim}(3)$ alignment. We also identify a sliding-window implementation issue that allows sensor-initialized depth to change during optimization. Because hand anchoring and its residual share the same depth map, the residual measures internal consistency rather than independent accuracy. These results characterize both the utility and the current limits of the registration pipeline.
★ BioMARS 2.0: a robotic biologist for formulating and adapting biological experiments
Biological experimentation is not reducible to a single optimization problem, but requires decisions at multiple levels of abstraction: translating scientific knowledge into experimental procedures, adapting culture conditions to measured phenotypes and selecting interventions that test biological hypotheses. Automating experimental execution does not, by itself, determine which experiments should be performed next or how they should change in response to biological feedback. Here we introduce BioMARS, a human-in-the-loop multi-agent robotic system that connects knowledge-guided experimental decision-making with robotic execution and biological feedback. At the procedural level, BioMARS generated literature-grounded protocols and traceable action plans, improving expert-reviewed protocol quality scores across four model backends. At the parametric level, BioMARS used image-derived phenotypes from robotic retinal organoid culture to select subsequent experimental conditions, with higher reported scores in later rounds of the three-round experiment. At the intervention level, BioMARS nominated drug-repurposing candidates from biomedical evidence and advanced six candidates to wet-lab testing by researchers. Together, these studies demonstrate feedback-driven robotic biological experimentation within a framework that supports decisions about experimental procedures, culture conditions and biological interventions.
★ LHNav: Long-Horizon Humanoid Navigation via Multirate Coordination and Physics-Grounded Action Data
We study humanoid navigation across multiple destinations under long-horizon language instructions. This task is challenging because robots must maintain progress across successive goals while continuously producing natural, stable whole-body motion through confined spaces. However, task decomposition in existing methods does not by itself ensure progress throughout execution, while waypoint-based navigation does not guarantee natural, route-faithful whole-body motion. Our key idea is to organize navigation into three complementary roles operating at different time scales: a Brain for task reasoning, a Cerebellum for local motion generation, and a Brainstem for whole-body execution. We realize this idea in LHNav, an asynchronous system that uses progress and arrival feedback to coordinate navigation goals, VLA motion-token prediction, and whole-body control. To train the Cerebellum for natural, route-faithful motion, we develop a physics-grounded data engine that improves physical tracking of motion references along collision-aware routes and pairs the resulting demonstrations with rendered visual observations. Together, these designs allow task planning to proceed alongside continuous movement and extend single-goal navigation to successive destinations with natural, route-faithful whole-body motion. In simulation, our navigation VLA outperforms OmniNav and DualVLN in single-goal success and motion smoothness, with fewer collisions and falls. Using a VLA trained only on single-goal demonstrations, LHNav achieves a 75.56% success rate on long-horizon navigation tasks. Deployment on the Unitree G1 further demonstrates single-goal navigation and zero-shot long-horizon task execution in real-world environments, including layouts with repositioned objects. Project website: https://robbyant-research.github.io/LHNav/.
★ Compact and Wide-Force-Range Magnetorheological Hand Exoskeleton for Kinesthetic Feedback in Teleoperation
Hand kinesthetic feedback is crucial for restoring and enhancing human perception of force and displacement in robotic hand teleoperation. Wide perception range of force feedback is vital in rendering objects of different stiffness, especially for high-stiffness rendering. However, existing approaches struggle to achieve wide range of force feedback while maintaining compact and wearable design. This paper presents a hand exoskeleton system employing magnetorheological (MR) brakes to provide a wide range kinesthetic feedback during teleoperation. The MR brake provides a braking torque ranging from 0.01 to 0.918 Nm, while maintaining a compact structure with a total mass of only 39.8 g. The proposed exoskeleton can render fingertip forces up to 21 N while maintaining back-driving force of 1.01 N. Experimental results show that the system allows users to discriminate object stiffness using kinesthetic feedback alone. In a tele-grasping task, when both conditions achieved a 100% success rate, kinesthetic feedback led to a 28.2% reduction in object contact force compared to visual feedback only, enhancing object safety and operator immersion. Questionnaire results further indicate that kinesthetic feedback improves reliability, realism, and user confidence during teleoperation.
★ SCALE: Small-Model-First Concept Abstraction with LLM-Guided Experience Internalization
Autonomous agents must continually acquire reusable knowledge from experience without relying on costly large language model (LLM) reasoning for every decision. However, existing experience-based approaches often struggle to transform accumulated observations and reasoning into compact knowledge that lightweight models can independently apply. This paper proposes SCALE, a small-model-first framework that integrates selective LLM reasoning, experience-driven concept abstraction, and knowledge internalization. A lightweight model handles routine predictions, while prediction errors and accumulated cross-episode evidence selectively trigger LLM reflection. The LLM analyzes relevant historical experiences, generates positive and negative counterfactual examples, and proposes reusable concept hypotheses. Independently validated concepts are progressively incorporated into hierarchical neural representations, enabling lightweight models to exploit increasingly abstract knowledge. Meanwhile, mastered experiences are excluded from active memory while their original records remain available in a complete archive. Experiments across ten random seeds show that historical recall improves held-out accuracy from 87.11% to 98.28%, hierarchical concept knowledge increases difficult-case accuracy from 55.33% to 66.47%, and abstract archiving reduces active experience records from 6,000 to 1,528.
★ Motion Belief Action: From a Visual Scene Map to Physical Decisions
A robot that works among people must judge what it cannot see: whether an object is still where it was last seen, and whether a pull on a door or drawer is worth its risk. Maps that track object change hold nothing about force, maps of physical properties answer a risk question at a fixed risk level, and vision-language-action models keep no explicit estimate of hidden physics. We propose Motion Belief Action (MBA), a decision layer on the scene map. From how often people move an object and how much force it needs, how much load it holds and whether it is blocked, MBA chooses under a force limit to look again, nudge, open, brace or decline, with its risk set by the task's costs. On real activity in 9 kitchens, it predicts whether an object is still in place better than FreMEn and Perpetua, and decides when to look again at lower cost than these and the persistence filter, because about a quarter of placements end in a pick-up within 30 s. On real pulls of doors and drawers in 17 locations, made by a person with a force-sensing gripper, the map's force estimates decide better than a part-type average. Setting the risk from the costs also beats a published risk rule at its fixed level. In simulation, with the force limit at the training parts' median force, MBA raises SmolVLA's and Octo's scores by declining pulls; most declined door pulls would fail, but on drawers more would have opened, and a part-type average gets about half of this gain. In both, a published risk rule set to the risk these costs imply is MBA's rule for a single pull and decides as well in the main settings. A link between opening and holding force, a probing nudge and an action prior for camera tracking do not help in the main settings, and over the real-pull sweep the nudge costs a little.
★ GAP: Generalized Adaptation Prior for Dexterous Piano Playing
Dexterous hands enable versatile manipulation, but policies trained in simulation often degrade on hardware owing to non-idealities, and the high-dimensional action space makes real-world adaptation costly. Piano playing, which requires precise and coordinated multifinger motion, offers a representative testbed. We propose GAP (Generalized Adaptation Prior for Dexterous Piano Playing), a sample-efficient adaptation framework that learns how to adapt, rather than how to act, during simulation pretraining and transfers this adaptation prior to hardware. Between repeated executions, GAP uses feedback to adjust the actions of the next one: a gradient guidance network learns from paired executions how adjusting actions affects playing performance, and its gradients guide these adjustments. In simulation, GAP yields approximately 2.4 times the F1 scores improvement of the existing fast-adaptation baseline for dexterous piano playing under the same interaction budget. The learned prior also generalizes to musical pieces held out from pretraining. On a physical dexterous hand, the same simulation-pretrained prior improves F1 scores by 3.6%-23% across three pieces of varying difficulty after approximately 7 min of adaptation each. These results suggest a practical route to reduce the real-world interaction needed to deploy dexterous hands.
comment: See video at https://www.bilibili.com/video/BV1hbWx6QE9u
★ SPARK: Simple Post-training for Adapting pRetrained Knowledge to Robot Control
Large-scale representation learning has produced public foundation models that provide strong general-purpose visual features. However, their pretraining objectives are not designed for robot representation learning, so the resulting embeddings tend to emphasize global semantics rather than the temporally sensitive representations required for robot control. Training robot models from scratch is also undesirable due to the limited scale and high collection cost of robotics data, as well as the substantial computational burden of pre-training. In this paper, we propose SPARK (Simple Post-training for Adapting pRetrained Knowledge), a post-training method that adapts foundation models to robot control. SPARK is built on two principles: temporal abstraction, which encourages the encoder to derive compact features that preserve information essential for understanding temporal changes and action-relevant scene structure, and knowledge preservation, which aligns patch-level representations with those of the original foundation model to retain useful pretrained semantics. These objectives yield compact visual state representations that remain semantically meaningful while preserving useful prior knowledge. Experiments across multiple robotics benchmarks show that SPARK consistently improves success rates and generalization over vanilla foundation models, and further demonstrate that these gains transfer to real-world robot manipulation.
comment: CoRL 2026. 16 pages, 3 figures, 13 tables
★ SAM-VTP: Video Memory and Measured Geometry for Continuous Traversability Segmentation
Traversability prediction is a core component of autonomous navigation, yet most vision-based methods operate frame-wise and produce temporally inconsistent predictions. We present SAM-VTP (Video Traversability Prediction), which fine-tunes SAM2's video memory pathway for dense, prompt-free traversability segmentation, conditioning each frame on a persistent memory of prior observations. Rather than inferring terrain geometry from appearance, we measure it: slope, roughness and height are computed from depth and fused into the image features. We also introduce an offline continuous supervision pipeline that generates dense per-pixel traversability targets in [0,1] from depth-derived geometry and semantic priors, requiring no robot deployment and no per-pixel annotation, capturing conditional feasibility that binary labels cannot represent. On TartanGround, a large-scale synthetic ground-robot dataset, SAM-VTP improves fuzzy Jaccard from 0.908 to 0.924 and binary IoU from 0.854 to 0.886 over SAM-TP, a frame-wise SAM2 baseline, against a seed-to-seed spread below 0.001, and cuts frame-to-frame flicker by 31%. In closed-loop trials at two field sites, driving the same planner and controller, the baseline's dropouts leave the planner without a feasible path 3.3x more often on a paved course and 6.9x on rough ground, while SAM-VTP runs at 21.4 FPS on the Orin Nano the robot carries.
comment: 8 pages, 5 figures, 3 tables
★ Compass 1: Towards Open-World and Persistent Embodied Navigation from Monocular RGB Videos
Embodied navigation requires agents to achieve persistent spatial awareness and long-horizon execution stability in open-world environments. However, existing navigation architectures struggle to sustain both capabilities, remaining effective only over a handful of recent frames. We present Compass 1, the first embodied navigation foundation model to unify language understanding and video generation while enabling memory backtracking through nearly 2mins of visual history. Without altering the underlying architecture, we introduce History Representation, which integrates sparse visual keyframes with an explicit History Representation Map to preserve long-term spatiotemporal context. To effectively leverage this history, we further introduce History Utilization, a structured VQA supervision protocol comprising three categories: task-stage grounding, exploration consistency, and spatiotemporal target retrieval, enabling map-grounded reasoning and stable persistent long-horizon navigation. To fuel this system, we establish a complete annotation pipeline for monocular RGB videos and open-source a massive dataset featuring over 500 hours of high-quality real-world interactions and over 2,000 hours of annotated web videos. Trained mainly on the distribution of 1m height camera, Compass 1 generalizes seamlessly across diverse embodiments and scenarios, ranging from indoor environments to pedestrian-dense streets, spanning wheeled robot at 0.35m, quadruped dog at 0.5m, humanoid at 0.85m, and aerial drone at 1.5m. Compass 1achieves a dominant overall success rate of 79.4% across open-world evaluations, compared to 17.8% for the strongest baseline. It demonstrates exceptional zero-shot execution exceeding 90% on foundational tasks, a 250% boost in exploration, and an unparalleled leap from 0% to 58% in challenging persistent history backtracking.
comment: Technical Report. Project Page: https://www.archon.tech/research/compass1
★ Learning Contextual Motion Constraints for Object Rearrangement Exploration IROS 2026
The motion of an articulated part in cluttered surroundings is constrained not only by an object's intrinsic joint limits but also by extrinsic obstacles. However, existing approaches primarily consider object articulation properties alone. Here we present Motion Constraint Prediction Network (MoCo-Net), a two-stream neural network that jointly predicts the continuous motion ranges of articulated parts and obstacle heatmaps highlighting potential collision regions from partial-view point cloud observation. By capturing both internal and external motion constraints, MoCo-Net produces more accurate feasible motion range prediction and obstacles recognition, improves generalization for novel objects in new scenes, and enables robots to construct a scene configuration space for efficient exploration of rearrangement actions. Our experiments show that MoCo-Net's two-stream design enhances each other to deliver more accurate predictions, which subsequently form a compact configuration space for a customized planner to efficiently rearrange the scene.
comment: IROS 2026 paper
★ DPGS-SLAM: Diffusion-Prior and Provenance-Aware Monocular Gaussian Splatting SLAM for Dynamic Visual Sensing
This paper presents DPGS-SLAM (Diffusion-Prior and Provenance-Aware Monocular Gaussian Splatting SLAM) for dense mapping in environments containing dynamic objects. Dynamic observations can contaminate camera tracking and scene reconstruction in visual SLAM (simultaneous localization and mapping). Although 3DGS (3D Gaussian splatting) enables efficient dense mapping, existing 3DGS-SLAM systems may exhibit instability when dynamic objects violate their underlying static-scene assumption. Excluding dynamic observations can reduce their influence on tracking and mapping, but may leave occluded static backgrounds missing or weakly constrained. Our proposed DPGS-SLAM is a monocular RGB SLAM framework that supports provisional representations of insufficiently observed static backgrounds through asynchronous diffusion-based background completion and provenance-aware Gaussian mapping. Dynamic regions are identified through optical-flow-based motion discovery and temporally consistent mask propagation by SAM2 (segment anything model 2). For selected keyframes, a fine-tuned diffusion inpainting model generates plausible background appearance in occluded regions, while Metric3D v2 predicts metric depth from the completed images to initialize provisional Gaussian primitives. Provenance labels distinguish generated primitives from observation-supported primitives, excluding generated content from camera-pose estimation and observed-map optimization. When previously occluded backgrounds become visible, eligible generated primitives are removed, and observation-supported primitives are independently initialized from real RGB observations and predicted depth. Experimental results demonstrated competitive camera tracking, with average ATE RMSE values of 1.2 cm on TUM RGB-D and 2.7 cm on BONN RGB-D. On StaDy4D, DPGS-SLAM improved average PSNR by 4.60 dB over WildGS-SLAM using only monocular RGB input.
★ Grasp-Taxonomy Conditioning as a Mode Switch for Dexterous Grasping of Climbing Holds
Dexterous grasping in robotics relies on distinct hand configurations depending on object geometry. We introduce BetaGrasp, a 3D diffusion policy that conditions a DP3-style point-cloud policy on a four-class grasp label (crimp, sloper, pinch, and jug), running on a Franka arm with a 16-DoF LEAP hand. We also introduce a climbing-hold benchmark: a spring testbed whose ratchet gives a slip-force reading in newtons. In our analysis the policy's point-cloud encoder did not separate the four holds, so the scan alone gave it little to choose a grasp from. Without the label (a DP3-style baseline), simulated rollouts from the robot's reset pose all end in a jug grasp, whatever the hold (25% correct grasp type, versus 94% for BetaGrasp). The label works as a switch for which grasp the policy performs. On 80 paired hardware trials, BetaGrasp raises average slip force from 9.6 N to 14.3 N, and grasps slipping fully off the hold drop from 44% of trials to 10%.
comment: 6 pages, 4 figures, 2 tables
★ OASIS: Optimization-Aware Subgoal Inference with Safety Constraints for Autonomous Mobile Robot Navigation
Navigation in dynamic environments requires mobile robots to make progress toward their goals while respecting motion limits and avoiding surrounding obstacles. Learned subgoal policies can guide model predictive control beyond its local prediction horizon, but a geometrically suitable subgoal does not necessarily lead to a successfully solved control problem. This paper proposes OASIS, a hierarchical navigation framework that incorporates recent optimization outcomes into reinforcement learning-based subgoal generation. A high-level policy trained using Soft Actor--Critic generates local subgoals, while a nonlinear model predictive controller tracks them subject to motion limits, geometric collision-avoidance constraints, and embedded high-order control barrier function constraints. The outcomes of controller optimizations over each subgoal interval are summarized and provided to the policy at its next decision, without requiring additional optimization calls. The framework is evaluated in five navigation scenarios with static and moving obstacles, including dense interactions and sensing uncertainty. Comparisons with hierarchical and direct-goal control variants show that OASIS achieves the highest aggregate navigation success. Matched comparisons across three training seeds further show a consistent success advantage over the otherwise identical hierarchy without optimization-status input. The selected-policy comparison also shows fewer primary optimization failures with similar mean controller time.
★ Thermo-VLA: A Thermal-Aware Vision-Language-Action Model with Gated Residual Modulation for Robotic Manipulation
Recent Vision-Language-Action (VLA) models have begun incorporating thermal sensing, but existing approaches typically treat thermal images as ordinary visual inputs, providing limited support for temperature-aware relational reasoning in manipulation. To address this limitation, we propose Thermo-VLA with a ThermoRes Encoder, which decomposes thermal images into cold and hot attribute representations and injects them into the RGB token through language-conditioned residual modulation, enabling temperature-driven target selection and continuous manipulation. We further construct six real-world robotic tasks on a dual-arm Unitree G1 platform, spanning basic grounding to relational reasoning. Experiments show that Thermo-VLA achieves an average success rate of 0.86 across four thermal-dependent tasks, outperforming a pi_0.5 baseline that directly incorporates RGB-thermal observations by 18 percentage points. Under unseen-object and scene variations, Thermo-VLA maintains an average success rate of 0.78, achieving the best generalisation performance among the compared methods. Beyond thermal sensing, we transfer the same residual modulation mechanism to colour attributes and observe consistent improvements in both simulation and real-robot experiments, indicating its potential to generalise beyond thermal attributes. Project website: https://j-x-chen.github.io/Thermo-VLA-web/
★ CSG-Nav: Confidence-Aware Semantic-Geometric Fusion for Robust Exploration and Accurate Zero-Shot Object-Goal Navigation
Zero-shot object-goal navigation requires autonomous agents to explore unseen environments and locate target objects without task-specific training, posing challenges in perceptual reliability under observation uncertainty and physically grounded goal termination. Although foundation-model-based approaches improve generalization, they remain vulnerable to unreliable long-range perception, hallucination-induced directional bias, and semantically plausible but geometrically incorrect stopping decisions. To address these limitations, we propose CSG-Nav, a zero-shot navigation framework centered on confidence-aware multimodal information fusion. CSG-Nav integrates three complementary fusion mechanisms. First, semantic-source fusion integrates open-vocabulary detection results, detection confidence, scene observations, and relational reasoning to produce more reliable direction-level semantic scores and suppress hallucination-induced directional bias. Second, spatial-confidence fusion weights multi-view semantic evidence using a Gaussian decay function, suppressing unreliable distant observations while accumulating spatially consistent exploration preferences. Third, semantic-geometric decision fusion combines semantic target detection with depth-based 3D localization to support physically grounded stopping decisions. Experiments on HM3Dv1, HM3Dv2, and MP3D demonstrate competitive zero-shot performance, achieving success rates of 58.9%, 71.6%, and 44.3%. Fusion-level ablations and real-world deployments in indoor and outdoor environments further verify the effectiveness and practicality of CSG-Nav for robust navigation under uncertain perception. Code is available at https://xr-ui.github.io/CSGNav.github.io/
comment: Accepted at information fusion journal
★ The Relative Value of Demonstration Data: Execution Time Limits and Operator Mixtures in Behavior Cloning
Robot demonstration datasets combine demonstrations from teleoperators of uneven experience, so collection teams need to know how many demonstrations from one group match those from another. We measure this with a data multiple: the ratio of the numbers of demonstrations two groups need for their policies to reach the same success rate. On robomimic's simulated Square and Can tasks, we train behavior-cloning policies on demonstrations from the two teleoperators robomimic labels Better and the two it labels Worse, varying dataset size and the share of Worse demonstrations. On Square, within robomimic's 25-second time limit, an all-Worse training set needs 1.55 times as many demonstrations as an all-Better set. Most of this difference is late completion: policies trained on Worse demonstrations finish later, as the Worse teleoperators do, and with a 100-second limit the multiple falls to 1.11. On Can, most of the difference remains at 100 seconds. If each Worse demonstration were worth 1/1.55 of a Better one, as in an all-Worse set, a half-Better, half-Worse set on Square would need 1.22 times as many demonstrations as an all-Better set. In fact, it needs 13% more than that, and this excess persists at every time limit from 20 to 200 seconds. The excess remains when training draws demonstrations uniformly instead of in proportion to their length. On Square, within the 25-second limit, switching to these Worse teleoperators is cheaper at equal success only if they earn less than a third of the Better teleoperators' hourly pay, because Worse demonstrations also take longer to record.
★ Flip Anything with Agentic Robotics Self-Learning
Flipping is a fundamental robotic manipulation skill that requires reorienting a target object in a stable pose on a planar support surface by rotating it 180$^\circ$ about an axis parallel to the support surface -- think of flipping a coin from heads to tails. Although flipping is a common objective in industrial tasks such as parts inspection, mechanical assembly, and tool use, coding a bimanual robot with grippers to flip diverse objects is a significant challenge. In this paper, we show that a coding agent can autonomously develop and refine object-flipping strategies through iterative learning in physics simulations across diverse object geometries. Progressive skill acquisition, from basic handover to toppling and pushing, robust grasping and sensing, and thin-object manipulation with recovery, improves task success from 24.7% to 95.0%. The resulting graph-based code integrates high-level strategy selection with reactive control, enabling multi-stage flipping on physical robots.
comment: CoRL 2026 late breaking. Project webpage https://berkeleyautomation.github.io/flipanything/
★ Reliability-Adaptive Multimodal Relative Pose Tracking for Underwater Leader-Follower Robots
Underwater leader-follower formation control requires continuous relative-pose estimation despite degraded sensing conditions and limited onboard computation. This paper presents a reliability-adaptive multimodal perception framework for six-degree-of-freedom relative-pose tracking between two moving underwater robots. The framework integrates complementary visual, acoustic-ranging, and hydrodynamic measurements while accounting for their distinct physical characteristics and time-varying reliability. It adaptively weights modality contributions and models temporal dependencies with linear computational complexity, enabling efficient pose estimation on embedded hardware. Extensive experiments were conducted using a custom-built AUV platform, with 648 pool trials spanning two leader vehicle types, three illumination conditions, and nine trajectory patterns. Evaluation yields an average translational error of 3.8 cm and an average orientation error of 2.4°. On an onboard NVIDIA Jetson AGX Orin, the fusion network achieves an inference latency of 17.62 ms, providing a 2.69-fold speedup over the evaluated Transformer-based fusion baseline. Closed-loop experiments further demonstrate stable formation maintenance with a moving leader under onboard computational constraints. A multimodal dataset containing measurements from all 648 trials will be made publicly available.
★ Scale-Consistent Monocular Fisheye Visual Odometry via Spherical Bundle Adjustment-Guided Unsupervised Depth Learning
The wide field of view of fisheye cameras and the simplicity of a monocular setup make monocular fisheye visual odometry (VO) attractive for robotic localization and mapping. Nevertheless, scale drift remains a major challenge over long trajectories. Unsupervised depth learning can help to mitigate this issue, but the pronounced distortion of fisheye images poses challenges for accurate depth estimation. This article presents S$^2$D-FVO, a monocular fisheye VO framework integrating spherical bundle adjustment (SBA)-guided unsupervised depth learning to improve scale consistency and depth estimation accuracy. Given that pixel reprojection residuals do not uniformly reflect angular deviations between viewing rays and that the first-order linearization of a scalar angular reprojection residual is insensitive to perturbations along one tangent direction, SBA employs tangent-space angular reprojection residuals derived from the spherical logarithmic map. These residuals retain sensitivity along both tangent directions, with their norms equal to the corresponding scalar angular residuals. A median depth-based scale regularization term is further integrated into SBA to constrain the reconstruction scale. Building on these formulations, SBA replaces the pose estimation network during unsupervised depth learning, providing more reliable relative poses for view synthesis. In turn, depth estimates from the well-trained depth estimation network provide scale references for SBA, improving scale consistency while allowing individual 3-D landmark positions to be refined. Extensive experiments conducted on multiple public datasets, as well as two newly collected large-scale datasets, demonstrate that the proposed framework outperforms representative existing approaches in both depth and trajectory estimation accuracy and is applicable to different camera models.
comment: 34 pages, 18 figures
★ Distilling Instructions into Weights: Closed-Form Edits Fix a Robot Policy's Misreading
Generalist robot policies often fail at one step of a task they can otherwise perform. We ask whether such a failure can be repaired by editing the policy's weights directly, without gradients, training data, or a change to the instruction. Told to put the cube then the banana in the bowl, a zero-shot $π_{0.5}$ vision-language-action policy in RoboLab releases the cube before it reaches the bowl. Replaying 21 of these decisions under rewordings shows that it binds "in the bowl" to the wrong object. Closed-form instruction distillation copies the per-layer effect of an explicit rewording into the language model's MLP weights. On 128 held-out seeds, success rises from 27 to 106 episodes, with 81 fixed and 2 broken ($p=7\times10^{-22}$), while an update of the same size in random directions leaves success unchanged. The edit also lifts two other tasks without detectable harm on the remaining 13, and a variant that stores 0.16% of the parameter count keeps most of its effect. Placed in the action expert or action head instead, the same distillation helps less or not at all. Our released tool reproduced the fix end to end on new held-out seeds, from 8/64 to 36/64. Edits aimed at perception, a missing grasp, contact physics, or the gripper threshold did not help, and of seven blind Claude Opus editors, only the one given the misread instruction produced an edit that passed held-out scoring. These results suggest that weight edits work best when a known-good input already exists to distil. Because reruns with unchanged weights flip about a third of paired outcomes, evaluations should pair seeds, rerun the unedited policy, and confirm gains on held-out seeds.
comment: 23 pages, 7 figures. Project page with videos: https://ali-adeeb.com/weightsmith/ ; code and benchmark: https://github.com/adeeb10abbas/weightsmith
★ How Many Candidates Does a One-Step World Model Need? A Random-Codebook Account of Nearest-of-m Training on Frozen DINO Tokens
One-step stochastic world models avoid the mode averaging of deterministic feature predictors without the tens of network evaluations per step that diffusion and flow matching require. The simplest such objective is nearest-of-m matching (conditional implicit maximum likelihood estimation, IMLE): draw m candidate futures and pull only the one closest to the observed outcome. Recent work conjectures that a small m suffices for feature-space world models and that a strictly proper energy score should replace the hard minimum. We test both claims with one-step world models on frozen DINOv2/v3 patch tokens, using benchmarks whose futures can be enumerated. Under mode-like uncertainty, nearest-of-m with m=16 covers the enumerated futures better than 50-step flow matching (at about 1/30 of its latency), one-step flow generators and an energy-score generator at matched parameter counts, and its rollouts stay on the token manifold where flow rollouts drift. One quantity governs the choice of m: the nearest-of-m objective is the expected distortion of a random codebook of m draws from the generator. This view yields three predictions, recorded before the corresponding runs: (i) memorization of the realized outcome in contexts that never repeat, whose strength is governed by the codebook's miss distortion I(m) rather than its miss probability, and which an energy-score generator does not escape by raising m; (ii) a flattening of mode probabilities with m, matching a known optimum within 0.01; and (iii) under-dispersion on continuous high-dimensional outcomes, which bounds the method on real robot video. Per-step matching on self-rollouts cuts long-horizon error by a third on a stochastic Push-T variant; an ablation separates its manifold and coverage benefits. Planning success does not improve despite accurate risk estimates, a negative result we report with its diagnosis.
comment: 30 pages (12-page main text, appendices A-H). Code: https://github.com/thddydgnl/nearest-of-m-world-models
★ Design of a hybrid tendon-linkage finger with coupled and adaptive motion
Dexterous robotic fingers require coordinated joint motion and adaptability to object contact, while independently actuating multiple joints increases the complexity of the transmission mechanism. This paper presents a hybrid tendon--linkage robotic finger with four mechanical degrees of freedom driven by three actuators. The MCP abduction/adduction joint is driven through a bevel gear, while MCP and PIP flexion/extension are driven by separate bidirectional tendon transmissions. The DIP joint is not independently actuated but is coupled to the PIP joint through a four-bar linkage with a torsional spring and mechanical stopper. The tendon routing also introduces mechanical coupling between the MCP and PIP joints. Kinematic analysis was conducted to describe the coupled and adaptive motions, and experiments were performed to evaluate the joint motion characteristics. ArUco-based measurements showed that the coupled motion closely matched the theoretical prediction, whereas the adaptive motion was partially limited by interference between linkage components. The fabricated finger demonstrated different contact postures with cylindrical, cuboid, and irregularly shaped objects, and the mean peak fingertip force was 4.56 N. The proposed mechanism provides both coupled motion and contact-dependent adaptation without requiring an additional actuator for the DIP joint.
comment: This manuscript has been submitted for possible publication
★ Action Representation for Spatial Generalization in Robot World Models
Spatial generalization in robot world models depends on whether action representations capture the physical relationships that remain valid across changing spatial configurations. Yet different coordinates can describe the same interaction, whereas similar motions can produce different outcomes when local physical conditions change. How these distinctions affect learned prediction remains insufficiently understood. This study evaluates matched action representations defined in world, object, and surface coordinate frames using a controlled robotic peeling benchmark. Global pose and contact relocation experiments reveal a cost associated with limited training data when equivalent interactions receive different spatial descriptions. Removing information about peel thickness creates an irreducible ambiguity because identical predictive inputs can correspond to different material removal outcomes. Coordinates defined relative to the surface improve consequence prediction and counterfactual action selection under accurate local observations, but learned predictors remain vulnerable to uncertainty in the coordinate frame. Joint centering of the surface and tool removes sensitivity to translation of their common origin, whereas incorrect contact geometry and actual execution deviations introduce different failure mechanisms. These results show that effective spatial generalization depends on preserving the relationships between actions and states that determine physical consequences, rather than simply minimizing coordinate variation. This perspective provides a principled basis for designing robot world models that reuse physical knowledge across spatial configurations while retaining the distinctions essential to prediction and planning.
★ Fade to... Red? Scheduling Coloured Noise Exploration Supports Efficient Robot Learning with Reduced Movement Jerk
Exploration in deep reinforcement learning (RL) for continuous control is commonly driven by temporally uncorrelated Gaussian (white) action noise, which leads to jerky motions with locally constrained exploration. In contrast, temporally correlated coloured noise, particularly pink noise, has been shown to support more efficient exploration in RL. Motivated by these findings, we inquire whether scheduling changes to the noise colour over the course of training can outperform white noise for continuous control robotics. We perform a systematic study of 49 linear schedules between white and red noise, together with generalised state-dependent exploration (gSDE), on four sparse-reward manipulation tasks with a Fetch arm and a Shadow Dexterous Hand. We find that the noise colour at the end of training does not affect learning efficiency or end performance, whereas fading towards red noise halves the jerk of exploration in every task. Balancing learning efficiency, end performance and smoothness, red noise emerges as the preferred strategy: it learns as efficiently as pink noise with almost 2-fold less movement jerk. We therefore recommend exploration schedulers that fade towards red noise as the default for off-policy robot learning.
comment: 10 pages, 4 figures, 1 table. Accepted at the ACRA 2026 conference. Cite as: "López, F. M., and Cruz, F. (2026). Fade to... Red? Scheduling Coloured Noise Exploration Supports Efficient Robot Learning with Reduced Movement Jerk. In Proceedings of the Australasian Conference on Robotics and Automation (ACRA). Australian Robotics and Automation Association (ARAA)."
★ DeltaForge: A Post-hoc Mixture-of-Experts VLA for Multi-Domain Autonomous Driving
Fleets of driving vision-language-action (VLA) models are developed one domain at a time: each operator finetunes a shared base model on private data, producing specialized experts that are costly to consolidate into a single model. Weight merging loses accuracy as heterogeneous domains accumulate, and dense mixture-of-experts routing is prohibitively expensive to combine them. We present DeltaForge, to our knowledge the first post-hoc mixture-of-experts VLA for driving, which consolidates N independently finetuned per-domain models into a single multi-domain model from their weights and a small proxy set alone. When different operators finetune the same base model, their weight updates end up sharing a lot of structure. We exploit this by decomposing all the updates together using singular value decomposition (SVD), splitting each one into a shared part (the same across all domains) and a tiny domain-specific part. The router then only needs to work with those tiny domain-specific pieces, not the full expert weights, which is what keeps the approach efficient. Then two cooperating clip-level blind routers, one over the VLM reasoning experts and one over the flow-matching dynamics experts, are trained from soft labels on the proxy dataset. Across six diverse domains spanning urban, off-road, and highway driving on cars, trucks, and all-terrain vehicles (ATVs), DeltaForge approaches the accuracy of the per-domain experts and surpasses every merging, routing, centralized, and federated baseline at close to single-model parameter and compute cost. Adding a new domain requires only projecting its weight update onto the shared basis with no end-to-end retraining needed. The resulting model also generalizes to domain combinations it was never explicitly trained on.
★ Decoupled Asynchronous System for Humanoids (DASH): Latency-Agnostic Scaling of Vision-Language-Action Models for Humanoid Manipulation
Vision-language-action (VLA) models pair a vision-language backbone with an action head, but in closed-loop control the backbone sits on the control-critical path: while it computes, the robot keeps moving, so each executed action is conditioned on an increasingly stale observation. A larger backbone therefore directly blocks reactivity, especially in humanoid dexterous manipulation. We present DASH, a Decoupled Asynchronous System for Humanoids that removes this coupling. DASH splits the policy into a slow vision-language model (System 2) and a fast flow-matching action head (System 1) that run as concurrent processes over the LCM communication protocol, so the control rate is set by the action head alone, independent of backbone size. The action head fuses stale semantics from a frozen backbone with fresh vision from its own ResNet encoder, desynchronized training exposes it to the staleness it meets at deployment, and an adaptive temporal aggregator aligns overlapping action chunks under variable inference latency. On an asynchronous RoboTwin benchmark that charges inference latency in simulated time, DASH holds 20 Hz control rate from 0.8B to 8B backbones while the synchronous rate falls from 9.7 to 4.1 Hz, with higher success at every size. On a real humanoid with dexterous hands, DASH achieves clearly higher pick-and-place success than the synchronous pipeline.
★ A Benchmark and Layered Evaluation Protocol for Evidence-Grounded Shared Belief Updating in Connected Robotics
Connected physical systems fuse observations from robots, IoT sensors, and edge-AI modules into a shared account of the world, yet most evaluations score only the final answer or action and cannot tell whether a method attributed the evidence, updated its belief, handled contradictions, and justified its decision. We argue that shared reasoning in Connected Robotics should be evaluated as an evidence-to-belief trajectory, and introduce the Connected Robotics Evidence Graph (CREG) Benchmark to make that trajectory measurable. CREG comprises 350 deterministically generated scenarios across seven regimes (justified revision, belief preservation, relation association, contradiction onset and resolution, stale/delayed evidence, and source reliability), each exposing a normative posterior reference at every evidence prefix, validated by an independent log-odds checker and screened by a blinded expert audit. Nine representation conditions separate relation content from serialization, ordering, and metadata. Evaluating four open-weight instruction models and six transparent fusion baselines yields three findings. First, entity grounding accounts for a substantial part of the weak-model deficit. Second, deleting semantic relations costs 15.3 points of strict belief-state accuracy on relation-dependent scenarios (Holm-corrected p<0.001), but an information-matched structured list outperforms the full graph and shuffling the graph does not hurt: relation content matters where required, while graph serialization confers no consistent advantage. Third, belief accuracy is strongly non-monotonic along the evidence sequence, with mid-trajectory failures that endpoint-only evaluation cannot detect. All scenarios, references, prompts, metrics, and analysis code are released for offline, hardware-free reproduction.
comment: 15 pages (10 main + 5 supplementary), 6 figures, 4 tables; supplement adds 1 figure and 7 tables. Project page: https://creg-bench.github.io
★ ROS2MR: Connecting ROS 2 Motion Planning to Mixed-Reality Trajectory Visualization
Mixed reality (MR) allows users to inspect planned robot motion within their physical surroundings, but displaying trajectories from robotics software requires converting their joint configurations and timing into a format the MR application can play back. This paper presents ROS2MR, an extension of the MR-Robotics LAB platform that converts YAML-serialized ROS 2 joint trajectories, including MoveIt motion plans, into the platform's existing JSON playback format. The converter validates the input, maps arm joints by name, and resamples the trajectory at a nominal rate of 30 Hz, using cubic Hermite interpolation when joint velocities are supplied and linear interpolation otherwise, while binary gripper commands are sampled separately using a zero-order hold. A Python export helper combines sequences of motion plans and gripper states into a single trajectory and checks positional agreement at segment boundaries. The MR application was also adapted for Apple Vision Pro in addition to Meta Quest 3. A technical evaluation showed that the converter preserved the normalized duration and endpoint configurations for dense, sparse, and irregularly timed inputs. With supplied velocities, the position RMSE was approximately 2.91x10^-5 degrees for all sampling patterns; without them, sparse input increased it to 0.85 degrees, and close positional agreement did not guarantee accurate velocity profiles. A pick-and-place sequence retained its open--closed--open gripper order, although off-grid transitions were shifted to the output sampling grid. The resulting file-based pathway enables inspection of ROS-generated trajectories in MR without a live ROS connection.
comment: Pre-print submitted in Elsevier
★ Generalizable Output Feedback Optimal Control via Transformers Without Explicit State Estimation
Classical optimal control is used to design a separate controller for each plant. This paper asks a different question. Can one learned policy serve a whole family of partially observed, noisy linear systems? We show that one transformer can. The policy is trained to imitate optimal LQG control on a heterogeneous family of Linear Time-Invariant (LTI) systems with Multiple Inputs and Multiple Outputs (MIMO), whose dimensions, cost matrices, and noise levels all vary. A shared representation with system-wise standardization, zero-padding, masking, and an encoding of the cost and noise covariances lets one set of weights control every member of the family. At run time, a short window of noisy measurements is all the policy needs to produce a control action, with no plant matrices and no state estimator. We characterize the relation between the optimal LQG policy (represented as a dynamic state-space model) and a static transformer acting on a finite window of measurements. We then express the closed-loop excess cost of any causal policy as the energy of its deviation from the optimal action. Finally, we evaluate the approach on 28 simulated systems. The trained policy reaches the optimal cost to within a median gap of $0.4\%$ for seen systems. For unseen systems, a light fine-tuning step brings the median gap to $1.0\%$.
comment: 7 pages
★ Budget-aware Correspondence-driven View Selection for Real-time 3D Perception of Mobile Robots
3D visual perception and scene reconstruction are critical for autonomous robots and intelligent transportation, yet real-world deployments often generate long, redundant image sequences that hinder downstream processing. This report investigates BCVS (budget-aware correspondence-driven view sampling), a pose-free view selection algorithm that offers 22+ FPS on a CPU, enabling online filtering for efficient 3D perception by mobile robots. Unlike existing approaches that rely on full structure-from-motion (SfM) pipelines or camera pose estimation, BCVS employs a purely correspondence-based scoring strategy that prioritizes view novelty while maintaining sufficient overlap for geometric stability. Given a user-specified budget, BCVS iteratively selects the most distinct frames and suppresses near-duplicates, producing compact subsets that preserve connectivity and minimize reconstruction loss. Extensive experiments on benchmark show that BCVS outperforms CPU-friendly baselines at tight budgets and remains competitive as budgets scale. Field trials with large aerial and marine robotics data further validate its practicality and scalability: BCVS reduces redundancy up to 16% while sustaining robust SfM registration and high-quality 3D reconstructions.
comment: 12 pages
★ PAIR: Proprioception-Aligned Image-Ray Representations for Viewpoint-Robust Manipulation
While imitation learning has made substantial progress toward generalizable robot manipulation through scaling, viewpoint robustness remains a major bottleneck. Visuomotor policies rely heavily on visual inputs, yet images provide an inherently ambiguous signal under viewpoint change: similar scenes may appear markedly different across viewpoints, while distinct states may share certain visual overlaps. A policy must therefore infer not only the task-relevant state, but also how it is visually projected under a particular camera configuration. This entanglement of viewpoint-dependent appearance and task-relevant state can make learning inefficient and limit generalization to unseen viewpoints. Recent work has begun to address this challenge through camera-ray conditioning and representation learning using geometric cues to induce invariance. However, providing camera information as additional context does not by itself ground visual representations in how the task scene is visually projected, while geometric alignment without camera context can introduce spurious correspondences by ignoring viewpoint-dependent projection. We propose PAIR, a contrastive learning framework that aligns joint image-ray representations using robot proprioception. PAIR encourages observations with similar robot states to remain close in representation space across viewpoints, providing a structured grounding signal for learning viewpoint-robust features. PAIR is a highly adaptable framework: it can be used to train Diffusion Policy (DP) or post-train vision-language-action (VLA) models. In simulated and real-world manipulation benchmarks with camera variation, PAIR consistently outperforms the behavior cloning baseline without ray conditioning by 12.7% and 26.7% points, respectively, demonstrating stronger generalization to unseen viewpoints. Project website: PAIRrobot.github.io.
★ HolidayFactory 1.0: Simulating, Generating, and Evaluating Industrial Robot Tasks
Robots are typically evaluated in episodes that begin from a reset, while industrial work consists of repeated jobs whose outcomes affect subsequent operations. Evaluating such work at scale requires simulation of task-critical physical interactions, construction of industrial environments, and measurements of performance as scene state persists. We present HOLIDAYFACTORY 1.0, a framework for simulating, generating, and evaluating industrial robot work. Given a human-specified workplace and task, the framework defines a world pairing a scenario of scene variants and process logic with an evaluation defining a job, metrics, and a rubric. HOLIDAYFACTORY 1.0 comprises three components. HOLIDAY-SIM extends Newton's SDF-based hydroelastic contact with additional constitutive laws and a semi-implicit primal-dual solver, couples stateful actuator dynamics to articulated motion, and derives tactile and joint observations from simulated states. Agentic world construction generates simulation-ready assets and scene variants using geometric and interaction checks, and proposes task-conditioned evaluations for human confirmation and executable checks. 24/7-BENCH introduces seven industrial tasks in which jobs are evaluated without resetting the scene, measuring completed, failed, and unfinished jobs, throughput, latency, and motor energy. Controlled studies characterize the proposed contact, actuator, and tactile models and examine the effectiveness of verification-guided world construction. Experiments on scene and evaluation generation further reveal the limitations of model critics and the difficulty of ensuring that generated evaluations faithfully capture task requirements. We examine job-level evaluations with scripted trajectories (e.g., code-as-policy) but have not yet compared policies over persistent runs.
★ Planners Exploit Deterministic Error: Dithered Low-Bit World Models
Sampling-based planners such as MPPI and CEM query a learned world model thousands of times per control step, so low-bit world models are an obvious way to make model-based control cheaper on a robot. With round-to-nearest activation quantization and no calibration, we find that planning can collapse, by up to 100 points of success, while the world model's latent one-step prediction error stays no larger than that of a stochastic rounding scheme that plans well. At the same bit width, what decides the outcome is not the size of the rounding error but whether it persists across planner iterations and is shared across candidates. Round-to-nearest leaves an error that persists, and an iterating planner drifts toward the candidates that this error favors; in TD-MPC2, holding the same dither fixed within a step or an episode brings the drift back. Dither shared by all candidates and redrawn at every iteration removes most of the damage at the same bit width and without training: at least half of it in all 19 affected Meta-World cells (task $\times$ bit width) of TD-MPC2 (median 95%), in 4 of 5 DINO-WM cells, and in all 8 affected tasks of a 317M-parameter multi-task TD-MPC2. Captured as CUDA graphs on an Ampere GPU, a W8A4 integer TD-MPC2 world model runs its planning rollouts 1.9-3.3$\times$ and the whole planning step 1.3-1.5$\times$ faster than FP32, though not faster than BF16 with pre-converted weights; shared dither adds at most 2.3% latency. In TD-MPC2, block-scaled NVFP4 removes most of the damage and MXFP4 part of it, which ties the failure to coarse grids with a shared scale. Hypotheses, thresholds and analysis code were fixed before the data that test them, and the hypotheses that failed are reported.
comment: 7 pages, 3 figures, 1 table
★ Graph-Sparse Distributed Predictive Control for Multi-Robot Motion Planning vianEdge-Consensus ADMM
This paper presents a distributed model predictive control (MPC) architecture for collision-avoiding motion planning of planar robots with double-integrator dynamics. To reduce coordination demands, a time-varying predicted conflict graph selects relevant pairwise interactions. The associated collision-avoidance constraints are convexified once per MPC update using warm-start trajectories, yielding a fixed convex quadratic program (QP). The resulting QP is then decomposed using an edge-consensus formulation of the alternating direction method of multipliers (ADMM), enabling parallel robot-local optimization and independent closed-form projections onto separating half-spaces. Execution is selectively supervised by a centralized high-order control barrier function (HOCBF) QP when the distributed candidate does not satisfy the direct-execution conditions. Evaluated in C++ simulations involving 8, 16, and 32 robots, the proposed architecture achieves collision-free execution and substantially improves computational scalability while maintaining task-level performance comparable to a centralized complete-graph baseline.
★ Particle Filtering with Generative Trajectory Models for State Estimation and Control
Generative models can support both state estimation and control in robotics. For control, a common approach is to use the model as a policy that directly samples actions conditioned on observation histories. However, models that jointly represent states, actions, and observations can also predict how the robot state and its environment evolve. We investigate whether these predictions can improve state estimation and control through a change in inference procedure, without modifying the pretrained model. To this end, we use particle filtering to maintain competing hypotheses about state histories. A single pretrained diffusion model generates these hypotheses along with their predicted observations and associated action proposals. As new measurements arrive, particles are reweighted according to how well their predicted observations match the measurements. While particle filtering conventionally uses these weights only for state estimation, we also use them to guide action selection, favoring proposals associated with hypotheses that better explain the observations. Simulation experiments in partially observed navigation and vision-based manipulation demonstrate improvements in both state estimation and task performance compared to action generation without filtering, using the same pretrained model and no additional test-time information.
comment: 8 pages, 4 figures, 5 tables
★ Accelerated Reinforcement Learning for Soft Robotics: Hours Instead of Years
Reinforcement learning (RL) requires large amounts of interaction data or a reliable simulator, both hard to obtain for nonlinear soft robotic hardware. Recent work shows that a time-scale separation inherent to soft robot dynamics justifies their reduction to low-dimensional invariant manifolds, known as adiabatic spectral submanifolds (aSSMs). We demonstrate that an aSSM-reduced model, a simple polynomial ordinary differential equation learned purely from pre-collected data, can serve as an RL training environment. On a pneumatically actuated soft arm, we train a trajectory-tracking policy in 3 hours on a six-dimensional aSSM-reduced model and deploy it without fine-tuning, whereas reproducing this policy training extrapolates to 3 years of hardware interaction. Our policy tracks dynamic targets with 40% lower error than otherwise identical policies trained on a 4.8-million-parameter recurrent neural network (RNN) or a 1.7-million-parameter transformer, which take 13 and 100 hours to identify, against 5 minutes for our mathematically justified 6,924-parameter model.
comment: 25 pages, 6 figures, 3 extended data figures
★ Data-Driven Precision Control of Soft Helicoid Robots
A purely data-driven method grounded in the theory of adiabatic spectral submanifolds (aSSMs) was recently used to model and control nonlinear soft robotic simulators. Here, we extend the methodology to learn aSSM-reduced models from data collected from hardware soft robotic platforms. Specifically, we learn a five dimensional (5D) aSSM-reduced nonlinear ordinary differential equation (ODE) that accurately models the dynamics of a soft-helicoid trunk robot. We then use this ODE in a model predictive control scheme to demonstrate real-time precision control across various challenging closed-loop trajectory-tracking tasks, ranging from 2D to 3D trajectories with static and dynamic constraints. We find that our methodology enhances accuracy and precision across all control tasks compared to available data-driven linear methods and model-free reinforcement learning. While we focus here on a soft-helicoid trunk robot, our methodology relies solely on hardware data and is therefore applicable to a broader class of soft robotic systems.
comment: Submitted to npj Robotics. 44 pages, supplementary information included
★ Discrete Model-Based Diffusion for Combinatorial Optimization in Multi-Robot Task Allocation
Decision making in multi-agent and multi-robot systems often involves discrete decision variables, leading to NP-hard combinatorial optimization problems that must be solved within seconds for near real-time implementation. Denoising diffusion has emerged as a powerful tool for such problems, but existing approaches face a trade-off: Model-Based Diffusion (MBD) incorporates diffusion in sampling-based optimization (SBO) without training but only over continuous spaces, while existing diffusion-based combinatorial solvers operate over discrete spaces but require offline training. To this end, we propose Discrete Model-Based Diffusion (Discrete MBD), a training-free SBO method that directly optimizes discrete variables by approximating categorical reverse transition distributions through Monte Carlo sampling. We establish a finite-step lower bound on the probability of finding a global minimizer and show that this bound converges to one as the number of denoising steps increases. Extensive experiments on generic integer programs and heterogeneous multi-robot coalition formation and scheduling demonstrate that Discrete MBD outperforms SBO baselines in nearly all settings while achieving solution quality competitive with that of mixed-integer programming and reinforcement learning methods at a fraction of their computational cost.
comment: 11 pages, 3 figures
★ Fastener-centric acquisition and AND/OR graph representation of state-dependent disassembly knowledge from CAD-derived virtual assemblies
Design-stage disassemblability assessment requires knowledge of product structure and the operations available in each assembly state, but CAD assemblies do not directly specify state-dependent disassembly operations. This paper proposes a fastener-centric virtual-disassembly framework that acquires such knowledge from CAD-derived virtual assemblies. Fasteners jointly releasing the same constraint are grouped into operation-level sets, avoiding enumeration of arbitrary individual-fastener orders. Their removability is evaluated through state-dependent visibility and tool-access checks and torque-driven virtual-removal trials. After fastener removal, graph-based reasoning identifies candidate moving units, whose separation is evaluated by initial-motion screening, path generation, and physics-based validation. The acquired operations, states, dependencies, validated paths, and simulation-derived attributes are organized into a Transformed AND/OR Graph for subsequent planning and design evaluation. Exhaustive verification with a controlled toy assembly found no branching violations for introduced tool-accessibility and precedence conditions. Evaluation with a 54-component OpenManipulator Chain gripper demonstrated state-dependent fastener-removal and part- or subassembly-separation operations. Visibility and tool-accessibility evaluations pruned 51.4% of fastener-set candidate evaluations, while removal of obstructing components enabled previously inaccessible fastener operations. Operation-level grouping and screening thereby limited detailed physical validation to mechanically plausible candidates. The results demonstrate the feasibility of acquiring product-specific disassembly knowledge from CAD-derived virtual assemblies before manufacture, supporting subsequent disassembly reasoning and design-for-disassembly evaluation.
★ AHEADance: Anticipatory Humanoid Dance from Streaming Audio
A humanoid that dances to streamed music must commit each motion chunk before it has heard the music that chunk accompanies, and must continue from motion it has already committed. We present AHEADance, a robot-native streaming dance generator executed on a humanoid robot by a whole-body tracker. ListenAhead conditions each chunk on heard music and a frozen music model's forecasts of the coming music, aligned to their target times and weighted by reliability. The forecasts carry the timing of upcoming beats almost as well as the actual future; with them, the generator is better synchronized with the music and concentrates more motion energy at its song's beat frequency. Commit Forcing trains continuation from the model's own committed chunks, updating both the latent history and the decoded boundary of the next chunk; it reduces low activity from 83.5% to 10.3%, the level of real dance. Three music evaluators, each validated on real G1 dance, test synchronization, song and style. On a physical Unitree G1, it dances while the music plays and comes to rest when it stops, without a fall. Project page: https://ihcr.github.io/AHEADance
★ PneuGen: Task-Driven Agentic Co-Design of Soft Pneumatic Robots
Task-driven design of soft robots requires the consideration of factors such as geometry, morphology, material properties, and actuation, resulting in a large and complex design space that is challenging to explore. Computational optimisation methods leverage signals from simulation to iteratively improve designs to achieve task objectives. However, these methods typically demand high computational load for both optimisation and simulation. We present PneuGen, an agentic co-design framework for soft robots that leverages Large Language Model (LLM) agents to search over the soft robot design space efficiently. A designer agent proposes design programs from a parametric component library, and a separate critic agent analyses simulation diagnostics and recommends targeted revisions. Fitted module response models provide reusable physical priors relating geometry and pressure to deformation. Design programs are automatically validated and converted into finite-element simulations, enabling iterative evaluation. Across grasping tasks on primitive shapes and challenging objects, as well as shape-matching tasks that require multi-directional planar and 3D deformation, the proposed framework significantly outperforms prior computational optimisation baselines. Ablation experiments also highlight the utility of the critic agent and the module response model in facilitating efficient design. We additionally demonstrate the agent-generated soft robot designs in real-world shape-matching and grasping tasks.
★ One Trajectory, Two Requirements: Accuracy and Local Stability in Euler Sampling for Flow Matching Policies
Flow matching robot policies commonly use a fixed Euler step count selected from offline accuracy sweeps. At inference time, however, each policy input (visual observation, robot state, and language instruction) together with a source-noise draw defines a different ODE trajectory and sequence of local Jacobians. We ask whether a step count that accurately follows the nominal noise-to-action trajectory can still be too coarse for strongly contracting directions around it. An exact two-coordinate system and a controlled augmentation of six trained flow models show that trajectory accuracy and local Euler stability can impose different step requirements even when the original learned transport is unchanged. We then study a frozen $π_{0.5}$-LIBERO checkpoint without modifying its weights or action dynamics. Across 30 policy inputs from 10 LIBERO-10 tasks, the Jacobian exhibits strong contracting directions near the action endpoint, yet no sampled mode violates the Euler non-expansion condition. On five of those inputs, accuracy sweeps require 4--9 Euler steps at 5% terminal error and 87--112 at 1%. These five paired cases are therefore consistent with accuracy-limited rather than stability-limited sampling under this local criterion, while the controlled experiments show why endpoint accuracy alone cannot rule out a separate stability requirement.
★ Does Haptic Connection to a Healthy Partner Improve Motor Learning in Chronic Stroke? A Pilot Study of Robot-Based Dyadic Training
Purpose: Robot-based haptic dyads, where two users exchange forces through robots, have been proposed for robot-assisted neurorehabilitation. This intervention could increase motor learning by promoting motor variability, providing an implicit communication channel, and facilitating social interaction. Despite the potential benefits, few studies have investigated robot-based haptic dyads for neurological populations, and none have included a comprehensive clinical assessment. Design/Methodology/Approach: We present a pilot study investigation of individuals with chronic stroke learning a complex tracking task in solo and dyad conditions. Prior to training, participants completed a battery of cognitive, motor, and sensory assessments. We assessed changes in key kinematic outcomes including tracking error, distance traveled, mean absolute power, and sample entropy. We also performed a correlation analysis of clinical scores with baseline tracking errors. Findings: Baseline tracking error was strongly correlated (r > 0.6) with lower cognitive scores and higher motor scores. Only the solo group demonstrated lower tracking error at assessment (Δ = -5.33, 95%CI [-10.25, -0.40], p = 0.027), but this result was tempered by the removal of an outlier participant. Neither group demonstrated clear retention. The dyad group showed significantly elevated performance, effort, and movement complexity during training. Research Limitations/Implications: The selected task was too difficult to reveal learning for either group, which may be related to its high cognitive demand. Sample size and stroke heterogeneity were also limitation. Originality/Value: Dyadic training increased movement complexity and effort, which are associated with better motor learning. Clinical characteristics such as lesion location, cognition, sensation, and chronicity may influence potential benefits of dyadic training.
★ Large Language Model Explainability for Planning Lunar Rover Trajectories under Uncertainty
Planning planetary rover trajectories under probabilistic uncertainty is a multi-objective optimization problem that science teams currently iteratively solve and verify by hand. This process becomes impractical when timelines and constraints frequently shift throughout the mission planning phase. Sampling-based planners such as Monte Carlo tree search can speed up this process, but their outputs do not preserve traceability to the science objectives that justify each plan, which the science team needs to verify. We propose an interactive assistant based on a large language model (LLM) that can explain planner trajectories using domain vocabulary and re-run the planner under user-specified conditions to compare alternatives. Previous work has used LLMs inside planner decision-making loops, but we explore using them for post-planning explanation, where they are more likely to be trusted since they do not directly influence planner decisions. Our framework pairs a simulator inspired by the NASA Volatiles Investigating Polar Exploration Rover (VIPER) mission with a ReAct queryable interface that calls curated action tools and retrieval functions. We evaluate the assistant on a query bank consisting of plan comparisons, feasibility, and lookup queries. Experiments show that 90.9% of the assistant's responses are faithful to the planner's outputs, a nearly 65% improvement over zero-shot prompting of the same LLM without tool access, demonstrating the potential for trustworthy explainability of uncertainty-aware traverses with an LLM-based system.
★ RaMOD: Moving-Object Detection from Raw mmWave Radar Cubes under Ego Motion
Millimeter-wave (mmWave) radar provides weather-robust sensing and direct Doppler measurements, yet automotive perception pipelines often reduce its output to sparse point clouds or two-dimensional projections, leaving raw range-azimuth-Doppler (RAD) cubes underutilized. Ego motion induces Doppler responses from world-static structures, while Doppler wrapping aliases radial velocities, complicating moving-object detection and velocity estimation. We present RaMOD (Radar-based Moving-Object Detection), a real-time framework for moving-object detection and velocity estimation from raw RAD cubes acquired by a vehicle-mounted single-chip radar. RaMOD explicitly estimates frame-wise ego motion and uses the estimated radar translational velocity to predict the Doppler region of world-static structures and suppress returns within it. It temporally encodes the suppressed cubes into multi-scale features to predict RAD-space boxes with class labels. For each detected object, it predicts radial and forward velocities under raw-Doppler and Doppler-unwrapped hypotheses, together with Doppler-wrapping probabilities. On our BRIDGE-Liv dataset, RaMOD outperforms all compared raw-radar methods across RAD-cube, RA, and RD detection settings, achieving 80.8% cube mAP at IoU 0.30, 15.7 percentage points above the strongest RAD-cube baseline. Velocity MAE is 1.128 m/s, with 99.87% Doppler-wrapping accuracy.
★ Force-Closure Condition for Cooperative Multi-robot Pushing of Rectangular Objects
In cooperative object manipulation, the placement of pushing contacts determines whether force closure can be achieved and agents can generate arbitrary forces and torques on the object correspondingly. This study derives a necessary and sufficient closed-form condition for the planar force closure of a rectangular object, a common object shape in everyday settings, under four frictionless pushing contacts. For a rectangle of length $b$ and width $a$, force closure holds if and only if $(l_1+l_3-b)(l_2+l_4-a)<0$, where $l_1,l_2,l_3,l_4$ are the four edge contact locations. This condition is independent of the object's center of mass and is used to verify contact configurations before contact force allocation. When the condition is not satisfied, the signs of the two factors indicate which opposing contact pair should be adjusted. The condition and adjustment rule are then integrated into a full-pose trajectory-tracking framework as a geometric feasibility layer to demonstrate their use in control. Static wrench feasibility tests validate the proposed condition, while narrow-passage floating-body simulations show that maintaining a force-closure configuration reduces the RMS tracking errors in $x$, $y$, and unwrapped yaw from $12.05$ m, $18.11$ m, and $535.96^\circ$ to $2.24$ m, $2.66$ m, and $2.69^\circ$, respectively.
comment: 7 pages, 7 figures, 2 tables. Published in CAOS 2026
★ Curie's Jarvis: Autonomous Robotic Chemistry Experiments from Printed Protocols
A person running a bench experiment works from a printed instruction sheet, tells identical containers apart by the labels written on them, and looks at the bench again before each step. Lab automation usually works the other way around: the procedure is written as a script, and containers stand at fixed positions or carry fiducial markers. We present Curie's Jarvis, a system that lets a robot arm carry out a chemistry experiment on a bench set up for people. It reads the steps from a photo of the printed sheet, finds containers on an ordinary bench with no fixed positions or markers, tells identical-looking containers apart by reading their labels in four camera views, locates its target again before every step, and sets how each step is carried out, including its speed, from what is being handled. On a Franka Panda arm, Curie's Jarvis completed 5 of 5 runs of one protocol from a children's chemistry kit and 4 of 5 runs of another, with no human intervention, and in 160 locates on printed and handwritten labels it identified every container correctly. This brings robots a step closer to doing bench chemistry the way people do.
comment: 8 pages, 3 figures, 4 tables. Rumi Loghmani and Bozhang Song contributed equally
★ Incorporating World Models in Decision Making for Next Best View Selection in Active Sensing
Efficient robotic 3D reconstruction requires selecting views that contribute new surface measurements with few physical captures. Pretrained world models can guide this choice, but a predicted surface may not produce valid depth measurements under the operating conditions of the deployed sensor. We present a deployment adaptation methodology for next-best-view selection that learns a sensor-return adapter from limited local calibration data while keeping the pretrained predictor fixed. The adapter estimates measurement dropout over each predicted surface, allowing the selection method to favor views likely to fill gaps in the measured reconstruction. In a real-robot experiment with a structured-light sensor, the adapted method achieves higher measured surface coverage with 60% fewer captures than nominal selection. The adapted method recovers a surface region that the nominal method repeatedly targets but fails to measure. These results suggest that robots can reuse pretrained predictive models while adapting their decisions to local operating conditions in data-scarce environments.
comment: Accepted to CoRL 2026 Workshop "Do Robots Need World Models?"
★ Latent Barrier Steering: Hierarchical Safety for Generative Planning
We introduce Latent Barrier Steering (LBS), a hierarchical semantic-to-path framework for safe flow-matching-based generative planning. Existing control-barrier-function (CBF) safety filters for diffusion and flow-matching planners act locally on generated samples, either during sampling or after prediction. Such repair is effective for small violations, but can struggle when a new constraint blocks the whole selected behavior mode rather than merely perturbing it locally. LBS decouples behavior selection from path certification through two coordinated interventions: it first steers the generator's behavior latent toward rollouts with larger safety margin, then decodes the steered latent and applies a path-space CBF quadratic program (QP) only for residual violations. We show that latent steering increases the margin available to the path-space corrector, yielding finite-time recovery guarantees and reduced correction burden. Experiments across navigation and robot end-effector planning tasks show that LBS preserves safety while improving goal reaching and reducing correction burden compared with path-level safety filters.
★ Ground Reaction Force-Aware Model Predictive Control for Underactuated Upright Physical Assistance
Many rehabilitation robots execute prescribed motions using reactive tracking controllers that do not explicitly incorporate the user's instantaneous human-ground interaction. This paper presents Ground Reaction Force-Aware Model Predictive Control (GRF-MPC), a framework for upright physical assistance. By integrating live GRF and center of pressure (CoP) measurements directly into a human centroidal dynamics model, GRF-MPC formulates a real-time quadratic program (QP) that optimizes assistive forces and moments over a receding horizon. We deployed GRF-MPC on the Robotic Upright Stand Trainer (RobUST) in an underactuated 4-cable configuration and evaluated it against a fully actuated 8-cable Cartesian impedance baseline in nearly 300 trials across six healthy participants. Across tasks spanning impulsive perturbations, lunge trajectories, and voluntary limits of stability, GRF-MPC reduced peak planar center-of-mass (CoM) error by 19.6% during perturbations, lowered trajectory tracking error by 24.1%, required 28.7% less net mechanical work, and permitted a 45.9% greater normalized voluntary excursion area. The underlying QP solves in under 1 ms, comfortably enabling 100 Hz replanning and demonstrating that incorporating live human-ground interaction into predictive control yields stronger, more transparent physical assistance with half the actuators.
comment: Accepted for publication in IEEE Robotics and Automation Letters (RA-L), October 2026. 8 pages, 5 figures
★ How Much of Your VLA Is kNN
Vision-language-action (VLA) models now exceed 95% success on the LIBERO simulation benchmark, yet it remains unclear how much of this performance requires generalization beyond the training demonstrations. In this work, we show that retrieving nearby demonstrations and reusing their action chunks achieves a success rate of 91.7 % on LIBERO without policy training. Under LIBERO-Plus perturbations, retrieval shows the smallest drop of all evaluated policies. Across nine publicly available VLAs, fine-tuning mostly makes the vision-language backbone a better retrieval key, and the actions of the VLAs remain close to the demonstrations. In addition, we introduce Local Action Consistency (LAC), an offline metric that measures whether nearby observations in representation space are also associated with similar future actions. We show that LAC tracks the success rate of retrieval and can guide the training of a retrieval key. Our evaluation and source code are available at https://github.com/sereact/lerobot_knn.
comment: 21 pages, 10 figures, accepted to the CoRL 2026 G2A/TAG workshop
★ DOBB: Dynamic Oriented Bounding Boxes for Multi-Agent Collision Avoidance under State Uncertainty and Heterogeneous Dynamics
Learning-based decentralized collision avoidance must remain reliable under state-estimation uncertainty and heterogeneous agent dynamics. Existing methods typically address these factors through training-domain variation or by exposing uncertainty and capability estimates to the policy at runtime. However, both leave the policy to infer how these quantities should alter the physical separation required for safe interaction, which can couple collision-avoidance behavior to relationships learned from the training distribution. We introduce the Dynamic Oriented Bounding Box (DOBB), an online geometric representation that maps the current state estimate, uncertainty, and agent-specific response characteristics into a dynamic, direction-dependent safety margin. We derive geometric features that jointly characterize the relative configuration and severity of interactions between agents' DOBBs, along with a reward function that preserves the physical interpretation of those interactions. Across four evaluation environments, DOBB achieves the lowest mean total failure rate of 2.6%. This performance persists under out-of-distribution shifts in actuation dynamics and state-estimation uncertainty, with collision-based failure remaining at or below 2.3% in every tested condition. We attribute this robustness to the geometric abstraction of the safety margin, which is less coupled to the specific uncertainty and actuation regimes encountered during training.
★ ASPIRE: An Agentic System-Identification Platform for Iterative Reasoning and Exploration
System identification requires expert decisions on the identification method, incorporation of prior physical knowledge, and model validation. We propose ASPIRE, a framework for nonlinear system identification in which a large language model agent selects and executes the identification method. ASPIRE constructs closed-form, nonlinear static or dynamic models from measured data, optionally supplemented by a system description, the intended use of the model, and physical requirements expressed formally or in natural language. Each model includes its parameters, an estimation record, an interpretation of its terms, and an error evaluated on test data inaccessible to the agent. Formal physical requirements are verified independently. ASPIRE operates autonomously or interactively, allowing the agent to ask questions and the user to request further analyses and identification rounds. We evaluate ASPIRE on three measured systems: a mobile robot, a rotational system with dry friction, and a laboratory fermenter. Each system is tested with and without a description, in five runs per task. All 30 runs delivered valid models, at a cost of 0.96 to 3.38 US dollars per run. An assessment by the authors found that, for the 30 models recommended by the agent, one per run, the interpretation of the model and the documentation of its structure selection and parameter estimation fully satisfied 132 of 150 assessed items and partly satisfied 18. The most frequent discrepancy, found in nine models, concerned inaccurate or unsupported claims about model structure selection.
★ Robo-MLT: Dual-System Robotic Manipulation for Long-Horizon Medical Laboratory Tasks
Reliable medical laboratory automation requires robots to complete entire workflows in protocol order while handling changes during execution. A robot may execute each motion correctly yet fail the workflow by confusing visually similar stages or stalling after human intervention. We present Robo-MLT, a dual-system framework that couples explicit workflow monitoring with learned action generation. A vision--language planner composes an ordered atomic-skill sequence from the protocol and the initial workspace observation. A multiframe execution monitor detects when the active skill has completed or become infeasible, then advances the workflow or triggers replanning. A skill-conditioned vision--language--action policy generates the manipulation trajectories, and an asynchronous runtime coordinates it with the monitor. We evaluate Robo-MLT against four policy baselines and a VLM-based monitor baseline on blood gas analysis and blood sample tube sorting, with 50 nominal configurations per method and 20 sample-removal configurations. Robo-MLT achieves task success rates of 82\% and 70\%, exceeding $π_{0.5}$ by 30 and 22 percentage points, respectively. After an operator removes the sample being handled, the system revises the remaining plan and continues processing the other samples. These results indicate that, under the evaluated conditions, explicit workflow monitoring makes long-horizon laboratory manipulation more reliable and recoverable.
comment: 12 pages,7 figures,5 tables,1 algorithm
★ RoboACE: Adversarial Co-Evolution for Robot Self-Improvement
Robot agents can improve reusable skills, planning procedures, and recovery strategies from execution feedback, yet a fixed practice source leaves the search for new weaknesses unchanged. Useful practice depends on the agent's current capabilities, so the procedure that chooses practice must also be revisable. We present RoboACE (Robot Adversarial Co-Evolution), a framework that jointly evolves curriculum and execution harnesses around frozen foundation models. The curriculum harness uses accumulated capability evidence to propose informative challenges, while the execution harness responds with reusable revisions that expand execution capabilities. Alternating updates hold one side fixed as the other changes, and admitted challenges form fixed snapshots for comparing execution revisions. Starting from a compact set of LIBERO practice conditions, the resulting frozen deployment outperforms the evaluated controllers and agent systems across four LIBERO-family panels. On LIBERO-PRO, it reaches 74.0% success, exceeding the initial deployment by 15.0 percentage points and direct execution by its VLA controller by 13.5 points. Case studies document changes to evidence-guided practice selection, drawer-skill grounding, and controller intervention.
comment: 13 pages, 5 figures
♻ ★ Integrated hybrid edge-cloud architecture for multi-robot and human-robot collaboration using ROS 2-MQTT-FIWARE in Mediterranean Greenhouses
The imperative to develop more sustainable agriculture demands a transition from isolated automation toward the deployment of multi-robot systems (MRS) in agrifood environments. However, Mediterranean greenhouse settings-characterized by narrow corridors, dense biomass, and structural metallic interference-pose significant challenges for robust and scalable communication between agents. Traditional robotic frameworks, such as ROS 2, frequently encounter node discovery issues and latency spikes due to dynamic obstacles, dense foliage, and other characteristic greenhouse elements, creating a critical bottleneck for real-time coordination. This paper proposes an innovative cloud-based hybrid architecture that establishes a two-way communication bridge between ROS 2, acting as an edge computing platform, and iVeg as a Decision Support System (DSS), using MQTT and the European FIWARE platform. The proposed framework enables seamless interoperability between fleets of multiple robots in environments with communication constraints, facilitating the synchronised exchange of high-level telemetry, point cloud data and farmer identification for collaboration, amongst other critical data. The architecture was validated in a high-fidelity simulation environment and subsequently tested in a real-world greenhouse scenario, demonstrating its ability to maintain persistent connectivity and data integrity under adverse network conditions. The results indicate that the integration of MQTT effectively eliminates information silos, providing a scalable and decentralised solution for managing complex robotic missions, which are executed locally via Edge Computing. This work sets a new methodological precedent for the concept of Greenhouse Models as a Service (GMaaS), bridging the gap between low-level robotic control and high-level, cloud-based IoT decision-making.
♻ ★ An offline approach to fNIRS-guided reinforcement learning for robot behavior IROS 2026
Human-in-the-loop Reinforcement Learning has become a popular approach for training, finetuning, and aligning robot behavior with user preferences. Our paper explores the feasibility of using brain signals via functional near-infrared spectroscopy (fNIRS) to modulate robot learning in simulation. We compare agents trained on passive (observational) versus active (demonstrative) interaction tasks, and test multiple methods for enhancing the RL algorithm with the neural signal, focusing on parameter augmentation in contrast to replacement. We further examine how model granularity and noise affect agent learning. Our results show that this framework is effective. The neural signal improves learning when augmenting trajectory priorities and state-action q-targets. Additionally, the framework learns successfully from offline data, offering a practical alternative for settings where real-time BCI setups are impractical or only limited data is available.
comment: Accepted at the 4th Workshop on Nonverbal Cues for Human-Robot Cooperative Intelligence (IROS 2026). 5 pages, 3 figures
♻ ★ Toward Controlling Biology with Language: Offline Learning of Prompt-Conditioned Interventions for Cells, Organoids, and Biobots
Natural language has become an interface to code, images and molecules, but not to living matter: a stimulus applied to cells has no fixed linguistic meaning, and every paired example of an instruction and its outcome costs a wet-lab experiment. We ask how far an existing archive of experiments can take this interface, using xenobots, motile multicellular constructs made of frog cells, stimulated by a single electrical pulse of varying duration. We re-track 155 archived recordings with two independent methods, measure what the pulse does from each bot's own behaviour before and after it, and separate behaviours the pulse controls from behaviours it does not. Longer pulses stop bots and short pulses leave them moving, sometimes faster; turning direction and starting from rest do not depend on duration, and restarting at most weakly. We then replace hand-made behaviour labels with a blind language critic: fresh sessions of a multimodal language model read plots of each bot and score how well its behaviour matches each of 176 instructions. The critic agrees with the measured labels whenever its plots contain the information the labels are defined on, and dose curves learned from its scores match those learned from the measurements. A prompt-to-intervention model trained only on critic scores maps unseen instructions, including negations, to the right end of the dose range on unseen bots, as well as the same model trained on measured labels. The archive therefore supports a calibrated, refusing language interface for the three behaviours a pulse can set, and identifies, rather than hides, the behaviours it cannot.
comment: 18 pages, 9 figures. v2: full reanalysis. Archive re-tracked with two independent trackers; measured outcomes for eight behaviours; blind language critic validated against measurements; prompt-to-intervention model outputs a dose-response curve and is evaluated on unseen instructions and unseen bots
♻ ★ Learn the Task, Not the Schedule: Reducing Spurious Cross-Arm Coupling in Bimanual Imitation Learning
A bimanual demonstration specifies both what the robot should do and one particular schedule for when each arm should act. Standard imitation learning does not distinguish task-required cross-arm constraints from incidental scheduling choices, and may therefore exploit opposite-arm progress as a shortcut cue, creating spurious cross-arm coupling even where relative timing is irrelevant for the current task phase. We introduce Constraint-preserving Temporal Recombination (CTR), a training-time data intervention that preserves recorded arm-local sensorimotor content while retiming the corresponding segments to realize admissible schedules that preserve task-required cross-arm constraints. We use CTR to train Diffusion Policy, Action Chunking with Transformers, and $π_{0.5}$, and evaluate them on four simulated and four real-robot tasks. Across six real-robot bimanual progress-state initializations, CTR raises worst-state success from 56% for the strongest non-CTR schedule mixture to 76%. With $π_{0.5}$ on an Independent task, a 4 s pause transfers 2.56 s of delay to the opposite arm under the Concurrent schedule distribution, but only 0.25 s with CTR. On real robots, CTR violates task constraints in only 0--4% of trials, versus 36--98% for narrow-schedule baselines. Code will be released at https://github.com/MINT-SJTU/CTR.
♻ ★ Program-Level Adversarial Co-Adaptive Self-Play for Robotic Security
Coding agents can adapt robotic software by revising executable programs, but successful responses to an evolving opponent may not remain effective. We present a framework for program-level adversarial co-adaptive self-play in which coding agents generate attacks and defenses from tested opponent code and mission feedback, without model fine-tuning. Environment-specific interfaces support external programs for autonomous driving and SDK-level defenses for humanoid locomotion. Two eight-generation case studies reveal concrete software responses: a driving defender introduces direct-actuation recovery, while a humanoid defender revises incoming-state validation. These responses provide local task recovery but do not establish sustained robustness. The framework makes such adaptation and limitations observable through executable artifacts and task-level evidence.
comment: 8 pages, 5 figures
♻ ★ ALICE-LRI: A General Method for Lossless Range Image Generation for Spinning LiDAR Sensors without Calibration Metadata
3D LiDAR sensors are widely used in autonomous navigation, environmental monitoring, and precision mapping. Their point clouds are often projected into 2D range images for efficient processing, but conventional projection methods can lose points because of geometric inconsistencies. We present ALICE-LRI (Automatic LiDAR Intrinsic Calibration Estimation for Lossless Range Images), a metadata-free method for lossless range-image generation from calibrated spinning multi-beam LiDAR point clouds. It requires neither factory calibration files, packet-level data, nor beam identifiers, and targets operating regimes where scanlines contain enough returns for reliable parameter estimation. The method estimates the intrinsic calibration parameters relevant to projection, including beam configuration, angular distributions, and per-beam geometric corrections. Evaluation on the complete KITTI and DurLAR datasets yielded zero lost points across all point clouds. Reconstruction deviations remain below manufacturer-reported range-performance specifications, while the method runs in real time. A compression case study also shows improved reconstruction quality and a better quality-compression trade-off for a downstream application. These results enable range-image-based LiDAR processing in applications that require high measurement fidelity and preservation of acquired geometric information.
comment: 31 pages, 13 figures. Author accepted manuscript; published in Measurement 294 (2027), article 123662
♻ ★ Autonomous Precision Milling of Biological Structures via Generic Anatomical Priors and Active Boundary Perception
Autonomous precision milling of biological structures is challenged by incomplete knowledge of target geometry, local material thickness, and critical internal boundaries. Subject-specific preoperative models can address geometric and thickness variations, but static models cannot determine boundary status encountered during execution, while repeated target-specific imaging limits scalability. This article presents a state-adaptive autonomous milling framework that assigns complementary roles to generic anatomical priors and active boundary perception. A generic anatomical prior provides conservative global guidance and is transformed through semantic-guided registration and hybrid vision-force calibration into robot-executable guidance for individual targets. As milling approaches uncertain boundaries, the robot actively probes the remaining structure and uses relative stiffness changes to estimate boundary status and structural detachability. A state-adaptive controller governs transitions between active perception and spatially selective incremental refinement, repeating this cycle until the termination criterion is satisfied. Hierarchical experiments on biological surrogates and in vivo mouse cranial window creation demonstrate accurate anatomical prior transfer, reliable boundary adaptation, and autonomous precision milling of biological structures.
comment: ver 2. 18 pages, 12 figures. Submitted to IEEE Transactions on Automation Science and Engineering (T-ASE) for possible publication
♻ ★ EffiNav: Fusing Depth and Vision-Language for Efficient Object Goal Navigation
To locate a target object while exploring the unknown environment is a fundamental capability for autonomous agents, with applications ranging from search-and-rescue to field robots. A simplified version of such task is Object Goal Navigation (ObjNav). In ObjNav, successful arrival at the target object provides a basic measure of performance; however, the efficiency of the navigation trajectory is equally important, as it indicates how intelligently the agent explores and how much time remains for subsequent tasks. In unknown environments, the key to efficient navigation lies in deciding where to explore next. While many prior works aim to address this core challenge and achieved promising performance in certain settings, recent training-based models and non-training frameworks still suffer from generalization and efficiency issues respectively, which in the worst cases can lead to excessive exploration of already-visited areas or redundant back-and-forth motion. We evaluate EffiNav on two widely used simulation benchmarks Habitat Matterport 3D (HM3D) and Open-Vocabulary Object goal Navigation (OVON), and further validate its effectiveness on physical robots in real-world settings. We conduct failure analysis on massive simulation episodes. With minimal modification, we also extend EffiNav to a memory-augmented ObjNav task on the GOAT-BENCH dataset, demonstrating its adaptability beyond standard ObjNav settings. Across two standard metrics--Success Rate (SR) and Success weighted by Path Length (SPL), EffiNav matches or outperforms recent baselines, reflecting its efficiency, robustness, and practical applicability. Recognizing the different emphases of the two datasets, the performances reveals this framework is more balanced and generalizable for efficient ObjNav.
comment: I wrongly issued the institute of this paper, and I will resubmit this after several month with right institute name.
♻ ★ Towards Multi-Source Domain Generalization for Sleep Staging with Noisy Labels SP 2026
Automatic sleep staging is a multimodal learning problem involving heterogeneous physiological signals such as EEG and EOG, which often suffer from domain shifts across institutions, devices, and populations. In practice, these data are also affected by noisy annotations, yet label-noise-robust multi-source domain generalization remains underexplored. We present the first benchmark for Noisy Labels in Multi-Source Domain-Generalized Sleep Staging (NL-DGSS) and show that existing noisy-label learning methods degrade substantially when domain shifts and label noise coexist. To address this challenge, we propose FF-TRUST, a domain-invariant multimodal sleep staging framework with Joint Time-Frequency Early Learning Regularization (JTF-ELR). By jointly exploiting temporal and spectral consistency together with confidence-diversity regularization, FF-TRUST improves robustness under noisy supervision. Experiments on five public datasets demonstrate consistent state-of-the-art performance under diverse symmetric and asymmetric noise settings.
comment: Accepted to the MLESP 2026 workshop (The 7th international workshop on Machine Learning for EEG Signal Processing) in conjunction with IEEE BIBM 2026
♻ ★ Swim2Real: Vision-Language Models for Simulator Calibration of a Soft Robotic Fish
Calibrating a soft aquatic robot simulator requires matching coupled deformation and propulsion from limited pool recordings. Swim2Real uses a vision-language model (VLM) to compare real and simulated swimming videos and propose updates to 16 simulator parameters. A backtracking line search tests each direction and accepts updates that reduce marker error. To test whether the proposals carry useful directions, we compare them with random and signed-permuted directions under the same line search and simulator budget. In an independent, preregistered fifty-seed replication, Swim2Real reduces the mean best-so-far error over 40 simulator evaluations by 63.4 mm against random directions and 53.2 mm against permuted directions. Both paired comparisons pass Holm correction ($p<10^{-5}$), conditional on run completion. A 24-seed experiment with a second VLM supports the same direction effect. Backtracking averages 16.1 VLM proposals per run versus 39 for Vid2Sid, which attains lower error at the same simulator budget. Ten hardware trials demonstrate open-loop execution of simulator-generated motor commands. Project page and videos: https://swim2real.github.io.
comment: 8 pages, 8 figures, 5 tables. Accepted for publication in IEEE Robotics and Automation Letters. v2: accepted version, substantially revised after peer review (new title, new experiments). Project page: https://swim2real.github.io
♻ ★ MemTransfer: Benchmarking Memory Beyond Matched Experience in Embodied Decision-Making
Memory lets an embodied agent reuse past experience, yet retaining useful information does not ensure that the agent can apply it when conditions change. We present MemTransfer, a benchmark comparing six memory representations, a working-memory baseline and five representations of past experience, under a shared frozen vision-language-model policy. It comprises 100 navigation cases across ten task types in a simulated warehouse, with expert demonstrations supplying the history. Three comparisons vary the starting pose, route availability, and amount and task relevance of history. With one demonstration per task, Full-context and Episodic memory reach 95.3% and 100.0% success at the original demonstration start, but lose 48-49 percentage points at a new test start. Summary changes little between these two test starts, yet with four demonstrations per task it retains a smaller fraction of its unchanged-route success after blocking (39.3%) than Working memory (44.8%) or the two trajectory memories (56-58%). At the new test start, increasing from one to four relevant demonstrations raises Episodic success by 14.3 percentage points, while the other evaluated representations gain no more than 1.3 percentage points. Replacing half of the relevant histories with other-task experience lowers success for both trajectory memories. These results show that robustness to one kind of mismatch does not imply robustness to another, motivating evaluation of both stored information and its use at decision time.
♻ ★ Teaching Robots to Say 'I Don't Know' : SENTINEL for Uncertainty-Aware SLAM ICRA 2026
Low-cost 2D LiDARs lack the intensity channel that higher-end sensors use to diagnose measurement failures, yet they are widely used on educational and budget robotics platforms. We present SENTINEL, a training - free, label - free reliability estimation framework that gives range - only LiDAR an effective diagnostic signal. SENTINEL combines geometry-based scan statistics with cross - modal depth consistency between LiDAR and an RGB - D camera to compute a per - scan reliability score between 0 and 1. When the score falls below a threshold, corrupted scans are rejected and the robot falls back to calibrated wheel odometry, preventing silent SLAM corruption. We evaluate SENTINEL on a GEFIER R1 four - wheel skid-steer robot equipped with an RPLidar A2M12 and an Intel RealSense D435i in a 185 cm by 245 cm arena containing controlled transparent and reflective failure elements on a central obstacle. Spatial reliability maps across five surface conditions, including glass, mirror, shiny paper, and a mixed mirror and shiny-paper condition, show clear separation between clean and failure cases, allowing affected regions to be identified as reject or noise. Because these failure modes are absent in simulation, validation is performed entirely on real hardware.
comment: 6 pages, 10 figures, 3 tables, Accepted at the Uncertainty in Open-World Robotics Workshop, held in conjunction with the 2026 IEEE International Conference on Robotics and Automation (ICRA 2026)
♻ ★ CADENCE: Predicting Realized MAPF Execution Time Beyond Sum of Costs ICRA 2026
Multi-Agent Path Finding (MAPF) algorithms are increasingly used to plan motion for robot teams in industrial warehouses and robotic shared workspaces, but standard MAPF algorithm evaluation metrics, such as Sum of Costs (SoC), makespan, and planner runtime, can obscure how planner choices translate into realistic execution performance. We present CADENCE (Coordination and Action-Driven Estimation for Networked Continuous Execution), a hardware study of this evaluation gap on a fixed 7 by 7 workcell with seven differential drive robots, asking which features available before execution can best predict final wall-clock completion time. We compare SoC, total planned travel cost, primitive motion burden (how much basic motion the plan requires, such as makespan, turns, consecutive moves, and start-stop transitions), and interaction aware coordination structure (how much inter-robot coordination the plan induces, such as dependency links, interacting robot pairs, dependency depth, and crowding exposure). To test this, we generate 120 plans across 15 scenarios -- 5 Empty, 5 Medium Random, and 5 Bottleneck and execute each plan four times, yielding a 480 trial hardware corpus. Using both a scenario-held -- out ridge model and a trial-level mixed-effects model, we find that SoC alone is informative but incomplete, while primitive motion burden gives the strongest improvement, reducing held out error by about 48.6%-59.8% in MAE and 44.2%-61.4% in RMSE relative to SoC-only models. Interaction-aware coordination features add smaller, less uniform gains, most clearly in the mixed-effects analysis. Across both models and uncertainty checks, primitive motion burden is the most reliable additional signal beyond SoC, suggesting that much of the execution time gap is already visible in the offline plan before any robot starts moving.
comment: 7 pages, 4 figures, 3 tables, and This paper was accepted for a poster presentation at Multi-Agent Robotic Systems: Real-World Collaboration and Interaction, a workshop held in conjunction with the 2026 IEEE International Conference on Robotics and Automation (ICRA 2026)
♻ ★ MineXplore: An Open-Source Reinforcement Learning Exploration Benchmark for GNSS-Denied Underground Environment ICRA 2026
Underground mines present extreme conditions for autonomous robot navigation: GPS is denied, lighting is degraded, and tunnel topology is loop-rich and non-convex. Simulation benchmarks grounded in real production-mine geometry and compatible with GPU-accelerated learning pipelines do not yet exist in the open-source ecosystem. We present MineXplore, an open-source MuJoCo-based navigation benchmark derived from the Leung et al. 2017 Chilean underground copper mine dataset. The environment reconstructs a 104,423 sq.m tunnel network through an six-stage contour-to-MJCF pipeline incorporating octagonal wall cross-sections, LiDAR-sourced jagged wall geometry, three terrain friction zones, a global 5 degree incline, and periodic spot lighting. Geometric fidelity is validated at an Intersection over Union (IoU) of 0.9538 against the source survey map, and surface texture similarity scores 79.4% across six structural dimensions. A single-agent PPO baseline trained via RLlib across five independent random seeds achieves a best rolling coverage of 88.89% (3 of 5 seeds reaching the 90% coverage target), confirming that MineXplore supports stable and reproducible policy learning under realistic underground sensing and topology.
comment: 7 pages,11 figures, This paper was accepted at Xplore: Cross-Disciplinary Aspects of Exploration in Robotics, Reinforcement Learning, and Search, a workshop held in conjunction with the 2026 IEEE International Conference on Robotics and Automation (ICRA 2026)
♻ ★ Guardian: Detecting Robotic Manipulation Failures with Vision-Language Models WACV2027
Robust robotic manipulation requires reliable failure detection and recovery. Although recent Vision-Language Models (VLMs) show promise in robot failure detection, their generalization is severely limited by the scarcity and narrow coverage of failure data. To address this bottleneck, we propose an automatic framework for generating diverse robotic planning and execution failures across both simulated and real-world environments. Our approach perturbs successful manipulation trajectories to synthesize failures that reflect realistic failure distributions, and leverages VLMs to produce structured step-by-step reasoning traces. This yields GuardianFail-36k, a large-scale failure reasoning dataset built upon the RLBench simulator and the BridgeDataV2 real-robot dataset. Using GuardianFail-36k, we train Guardian, a multi-view reasoning VLM for unified planning and execution verification. Guardian achieves state-of-the-art performance on three unseen real-world benchmarks: RoboFail, RoboVQA, and our newly introduced UR5-Fail. When integrated with a state-of-the-art LLM-based manipulation policy, it consistently boosts task success rates in both simulation and real-world deployment. These results demonstrate that scaling high-quality failure reasoning data is critical for improving generalization in robotic failure detection. Code, Data, and Models are available at https://www.di.ens.fr/willow/research/guardian/.
comment: Code, Data, and Models available at https://www.di.ens.fr/willow/research/guardian/. The paper contains 8 pages, 7 figures, 7 tables Accepted at WACV2027
♻ ★ Flexible Piezoelectric Tactile Sensing System for Intelligent Object Recognition
Tactile sensing provides information about object properties that can improve robotic grasping and manipulation. However, accurate object recognition during the early stages of grasping remains challenging, particularly under strict real-time constraints. This study presents a flexible, distributed piezoelectric tactile sensing system for early-stage object recognition. The system was experimentally evaluated using six everyday objects and three recognition approaches: feature-based classification using support vector machines (SVMs), raw-signal classification using 1D convolutional neural networks (1D-CNNs) and convolutional recurrent neural networks (C-RNNs), and spatiotemporal heatmap classification using a 2D-CNN. The effects of sensor count and temporal window size on recognition accuracy and real-time feasibility were systematically investigated. Increasing the number of sensing elements improved recognition performance across all approaches. The best-performing method, based on spatiotemporal heatmaps and a 2D-CNN, achieved F1-scores of 0.9796, 0.9846, and 0.9903 using 16, 24, and 32 sensors, respectively. The optimal configuration, comprising 32 sensors and a 100-ms temporal window (50 samples), was deployed on an embedded edge platform, achieving an end-to-end recognition latency of approximately 101 ms from contact onset and an energy consumption of 63.8 microjoules per inference. These results demonstrate the potential of distributed piezoelectric tactile sensing for accurate, energy-efficient, and real-time object recognition during robotic grasping.
♻ ★ FUSE-PnPL: Flexible, Unified, Statistically Consistent, and Asymptotically Efficient Camera Pose Estimation from Points and Lines
Camera pose estimation from 3D-2D correspondences is fundamental to robotics. Three practical considerations motivate this work. First, point and line features provide complementary geometric information, motivating a unified estimator. Second, the number and composition of available features vary across scenarios, requiring support for minimal and arbitrary non-minimal configurations. Third, modern visual front ends extract abundant features from individual images, enabling highly accurate pose estimation. We propose FUSE-PnPL, a flexible and unified camera pose estimator that is statistically consistent and asymptotically efficient. Specifically, we represent each line using two distinct points and adopt a unified image-space noise model for point and line measurements. By employing Plücker coordinates for line parameterization, we explicitly characterize the propagation of image endpoint noise into line residuals and formulate a joint maximum likelihood (ML) estimation problem for combined point and line measurements. To efficiently solve this problem, FUSE-PnPL employs a two-stage estimation scheme. First, noise variance estimation and bias elimination yield a consistent initial pose estimate. A single Gauss-Newton iteration then refines this estimate, achieving asymptotic efficiency as its mean squared error asymptotically attains the Cramér-Rao lower bound. Moreover, the proposed estimator has linear computational complexity with respect to the number of correspondences, making it suitable for high-precision and real-time robot pose estimation. Extensive experiments on synthetic data and real-world benchmarks validate the theoretical properties of FUSE-PnPL and demonstrate its advantages in both static global localization and dynamic SLAM systems.
♻ ★ HandMimicX: Learning Human-Like Dexterous Manipulation via an Object-Centric Policy from Egocentric Videos
Learning dexterous manipulation from human videos remains an open challenge, particularly in the context of transferring a single demonstration into a spatially generalizable robotic skill and maintaining physically plausible human-like interactions. Unlike prior works that emphasize hand-motion imitation, we introduce an object-centric framework for acquiring dexterous skills from egocentric human RGB-D videos. Within our framework, we formulate the reinforcement learning (RL) objective in terms of object motion to decouple the underlying skill from the specific hand trajectory and impose constraints on hand-object interaction states to promote more anatomically natural hand configurations. Extensive experiments in simulation and on real robots demonstrate that our method enables efficient learning from human videos and generalizable skill transfer across novel spatial configurations.
♻ ★ Scene-Adaptive Readout and Decoding in Diffusion Driving VLAs
Diffusion driving vision-language-action models repeatedly execute the backbone at every denoising stage, incurring substantial inference cost, while it remains unclear whether all scenes require the same decoding computation. To distinguish when a trajectory becomes readable from how much further refinement it requires, we analyze trajectory readout across network depth and decoding progress. We identify a depth-decoding dissociation in which late-layer representations already support useful trajectory readout after the first forward sweep, whereas early-layer representations benefit more from continued decoding. We further find that both the preferred readout layer and decoding depth vary across scenes. Based on these observations, we propose a scene-adaptive readout and decoding framework. A scene-conditioned readout MoE adaptively weights and combines trajectory predictions from layer-specific late-layer representations after a single backbone sweep, while an adaptive decoding policy applies additional native refinement only when required by the scene. We evaluate our framework on two diffusion driving VLAs across the Waymo E2E and nuScenes benchmarks. Our method improves validation RFS by up to 0.56 while also improving test performance, and achieves an end-to-end speedup of up to $6.0\times$ on Waymo E2E, while consistently reducing trajectory error on nuScenes. These results demonstrate that scene-level readout and decoding allocation improves planning quality while avoiding unnecessary decoding. Our code is available at https://github.com/sjyu001/SARD
comment: The manuscript requires substantial revisions to its methodology, experimental analysis, and presentation of the findings. The authors believe that the current version may not accurately reflect the intended research contributions and could potentially mislead or confuse readers. Therefore, we request withdrawal of this version to avoid further dissemination of potentially misleading information.
♻ ★ RoboWits: Unexpected Challenges for Robotic Creative Problem Solving
The ability to reason, adapt, and creatively solve problems under unexpected challenges is essential for robots operating in real-world environments. However, current robotic benchmarks primarily emphasize skill-level execution and provide limited insight into such cognitive reasoning capabilities. We introduce RoboWits, a bi-manual robotic benchmark designed to systematically evaluate cognitive reasoning, creative tool use, and robustness to unexpected conditions. To enable scalable construction of high-quality reasoning-centric unexpected scenarios, we propose an automated task generation pipeline formulated as a multi-agent cooperative framework, comprising agents for seed task generation and verification, metric generation, scene generation, and task mutation. Using the pipeline, we curated 30 diverse seed tasks and 200 tasks with mutations and graded difficulty across geometry, material, and assembly-based reasoning. We benchmark popular robot policies, pre-trained VLAs, a zero-shot policy built on a frontier VLM, and oracle-state planners. Our results reveal a significant performance gap: while pre-trained VLAs exhibit preliminary success on seed tasks after fine-tuning, they struggle to perform on mutated tasks, implying their brittleness in manipulation tasks requiring reasoning, strategy adaptation, and robustness to deceptive or constrained environments. A zero-shot policy built on GPT-6 Astra outperforms all fine-tuned VLAs, yet still loses about a fifth of its success on mutated tasks, showing that stronger foundation models alone do not close the robustness gap. See our project page https://umass-embodied-agi.github.io/RoboWits/ for videos.
comment: The first two authors contributed equally
♻ ★ Bias-Eliminated PnP for Stereo Visual Odometry: Provably Consistent and Large-Scale Localization
In this paper, we first present a bias-eliminated weighted (Bias-Eli-W) perspective-n-point (PnP) estimator for stereo visual odometry (VO) with provable consistency. Specifically, leveraging statistical theory, we develop an asymptotically unbiased and $\sqrt {n}$-consistent PnP estimator that accounts for varying 3D triangulation uncertainties, ensuring that the relative pose estimate converges to the ground truth as the number of features increases. Next, on the stereo VO pipeline side, we propose a framework that continuously triangulates contemporary features for tracking new frames, effectively decoupling temporal dependencies between pose and 3D point errors. We integrate the Bias-Eli-W PnP estimator into the proposed stereo VO pipeline, creating a synergistic effect that enhances the suppression of pose estimation errors. We validate the performance of our method on the KITTI and Oxford RobotCar datasets. Experimental results demonstrate that our method: 1) achieves significant improvements in both relative pose error and absolute trajectory error in large-scale environments; 2) provides reliable localization under erratic and unpredictable robot motions. The successful implementation of the Bias-Eli-W PnP in stereo VO indicates the importance of information screening in robotic estimation tasks with high-uncertainty measurements, shedding light on diverse applications where PnP is a key ingredient.
comment: 10 pages, 7 figures
♻ ★ Beyond Retargeting: Low-Latency and Robust Humanoid Whole-Body Teleoperation with Learned Atomic Motion Primitives
Humanoid whole-body teleoperation translates human motion into stable robot behavior in real time. Existing systems typically rely on online motion retargeting to bridge human--robot morphological differences, but this process adds latency and can produce physically infeasible targets. Meanwhile, diverse, noisy, and partial human-motion observations often fall outside the training distribution, potentially causing unstable robot behavior. We propose a retargeting-free policy that maps raw human motion directly to robot joint commands in a single forward pass, eliminating online kinematic adaptation. To improve robustness, we learn a codebook of full-body motion primitives that projects out-of-distribution observations onto plausible motion prototypes and recovers full-body motion from partial inputs. Experiments on a Unitree~G1 in simulation and on hardware, using virtual reality, optical mocap, text-to-motion generation, and monocular video inputs, show that our method outperforms baselines in latency and robustness.
comment: This paper is withdrawn pending acquisition of the required data-use license and completion of the company’s internal open-source compliance review.
♻ ★ Frequency-Aware Flow Matching for Continuous and Consistent Robotic Action Generation NeurIPS 2026
Flow matching has emerged as a standard paradigm for robotic manipulation owing to its strong expressive power for modelling complex, multimodal action distributions, alongside similar approaches like diffusion policy. However, existing methods rely on discretized action chunks, making them brittle to demonstrations collected at heterogeneous control frequencies and prone to temporally inconsistent actions that degrade control stability. In this paper, we propose Frequency-Aware Flow Matching (FAFM), which outputs continuous, temporally consistent actions. To handle heterogeneous frequency input, we transform discrete action sequences into the frequency domain with the discrete cosine transform (DCT), perform flow matching over the resulting coefficients, and reconstruct continuous actions via cosine basis expansion. To generate temporally consistent actions, we regularize the first-order temporal derivative to promote smooth actions. This corresponds to a Sobolev-type constraint that suppresses high-frequency errors and discourages abrupt action changes. Our FAFM is simple, introduces no additional network parameters and applies to standalone flow-matching policies and vision-language action models. Across synthetic toy benchmark, obstacle avoidance, LapGym, and LIBERO, FAFM improves success rates, multimodal expressivity, motion smoothness, convergence speed, robustness to mechanical bias and mixed-frequency input. These gains are consistent when deployed on a real-world Franka robot. Code available at https://github.com/gakakulicc/FAFM.
comment: Published as a conference paper at NeurIPS 2026
♻ ★ Decoupling Ego-Motion from Target Dynamics via Dual-Interval Motion Cues for UAV Detection
Object detection from Unmanned Aerial Vehicles (UAVs) is challenged by severe ego-motion, camera jitter, and large scale variations. While modern detectors perform well on static images, their direct application to UAV video often fails, particularly for small objects in dynamic scenes. Existing motion-based methods often rely on computationally expensive optical flow. We propose a vision-only motion-guided detection framework that decouples target motion from camera-induced disturbances. Homography-based Global Motion Compensation (GMC) first aligns adjacent frames. We then introduce a Dual-Interval Motion Extraction strategy that captures both short-term and long-term motion cues. To integrate these cues, a lightweight Dual-Interval Motion Injection (DIMI) module enhances feature representations within a Feature Pyramid Network. Experiments on the VisDrone-VID dataset demonstrate consistent improvements over the YOLOv8n baseline under severe ego-motion. Ablation studies further confirm the effectiveness of the dual-interval design and DIMI.
♻ ★ Bird-Inspired Spatial Flapping Wing Mechanism via Coupled Linkages with Single Actuator
Spatial single-loop mechanisms such as Bennett linkages offer a unique combination of one-degree-of-freedom actuation and nontrivial spatial trajectories, making them attractive for lightweight bio-inspired robotic design. However, existing synthesis methods rely on dual quaternion algebra and rational motion interpolation and factorization, placing task-based linkage design outside the reach of most engineers, a gap addressed by the Rational Linkages library recently developed by the authors. This paper presents a bird-inspired flapping-wing mechanism built from two coupled spatial four-bars, driven by a single motor. One linkage is actuated to generate the desired spatial sweeping stroke, while the serially coupled linkage remains unactuated and passively switches between extended and folded wing configurations over the stroke cycle. Additionally, we introduce a simplified kinematic methodology for constructing Bennett linkages from quadrilaterals that contain a desired surface area and further leverage mechanically induced passive state switching. This architecture realizes a coordinated sweep-and-fold wing motion with a single actuation input, reducing weight and control complexity. A 3D-printed prototype is assembled and tested, demonstrating the intended spatial stroke and passive folding behavior.
♻ ★ DreamTrue: Action-Faithful Robot World Model with Counterfactual Post-Training
We present DreamTrue, a multi-view, cross-embodiment robot world model for action-faithful and physically plausible video prediction. Training such a model on existing robot datasets faces two obstacles: imprecise calibration can impair action following, while limited coverage of unsuccessful interactions can bias predictions toward successful outcomes. To improve action following across embodiments, we render action trajectories into image-space conditions and introduce offline geometric calibration to align these conditions with the target videos. To broaden interaction coverage, we introduce counterfactual post-training, modifying recorded action trajectories and generating future videos under a wider range of actions and contact configurations. To provide feedback on these predictions without paired ground-truth futures, we construct a human-annotated video dataset covering robot, object, and interaction defects and use it to train an embodied video reward model. Its scores guide reinforcement-learning post-training toward more physically plausible interaction outcomes. On AgiBot, DreamTrue attains state-of-the-art action following, while reducing the human-assessed interaction defect rate from from 48.12% to 6.25%. Notably, our model ranks first in the world model track of the AgiBot World Challenge 2026. The project page can be found at https://brave-eai.github.io/DreamTrue.
comment: project page: https://brave-eai.github.io/DreamTrue; code: https://github.com/brave-eai/DreamTrue
♻ ★ Beyond Topology: A Morphological Symmetry Graph Representation for Locomotion Policy Learning
Reinforcement learning has enabled impressive locomotion skills on articulated robots, but common policy representations remain only weakly aligned with robot physics. Generic networks ignore kinematic structure, while graph-based policies encode connectivity without specifying how physical quantities transform across symmetric body parts. We introduce a morphological symmetry graph representation for locomotion policy learning, instantiated as MS-PPO with PPO on Go2 and MS-SAC with FastSAC on G1. Starting from the robot's topological graph, our representation explicitly incorporates the permutations and sign changes induced by morphological symmetry on observations and actions, yielding a graph actor with an equivariant action mean and an invariant critic. We evaluate the representation on Unitree Go2 command tracking and asymmetric joint-failure tasks, and on Unitree G1 29-DoF whole-body locomotion, including zero-shot sim-to-real deployment. Our approach achieves the lowest tracking errors in both simulation and hardware, and the highest mean simulation returns across all three joint-failure types on Go2 among the evaluated baselines. On Go2, MS-PPO uses only 34.6% of the parameters of the PPO baseline. These results support the effectiveness of jointly encoding topology and morphological symmetry across robot embodiments and learning frameworks.
comment: Project page: https://lunarlab-gatech.github.io/MS-PPO/
♻ ★ cuRoboV2-Fast: Fast Torque-Limited Collision-Free Motion Planning
We present cuRoboV2-Fast, a GPU-accelerated planner that reduces both planning latency and execution time for collision-free, payload-aware, torque-limited motions. B-spline trajectory optimization reduces the required iterations, while accelerated robot computations and differentiable inverse dynamics reduce refinement cost and accommodate changing payloads. TimeOpt optimizes traversal speed along the fixed path under sampled velocity, acceleration, jerk, and torque constraints, exploiting local spline support to solve banded Newton systems efficiently on the GPU. With a 3,kg payload, cuRoboV2-Fast achieves 4.4 ms mean planning time on RTX 5090, 14.7x faster than cuRoboV2. Our method on Jetson AGX Orin plans faster than cuRoboV1 + TOPPRA on RTX 5090. A stereo-camera-based deployment on YAM achieves 5.93 s mean completed pick-and-drop cycles, 34.6% shorter than cuRoboV2. Code (Apache 2.0): https://github.com/NVlabs/curobo , Project Page: https://research.nvidia.com/labs/lpr/curobov2-fast .
♻ ★ SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model
Tracking large human-motion datasets has become the dominant route to general, expressive humanoid whole-body control, but scaling across many motions is expensive. Current pipelines use model-free reinforcement learning, either merging per-motion specialists through lossy distillation, or brute-forcing a single policy with cluster-scale compute. SuperMimic amortizes learning across all motions with one differentiable world model of their shared dynamics, fit from logged transitions of a standard rigid-body simulator. The world model and the tracking policy train together in one loop, and backpropagation-through-time over short imagined rollouts gives the policy analytic gradients. Fast imagined rollouts and analytic gradients make policy learning efficient. In 3 hours on a single GPU, SuperMimic surpasses the return of a state-of-the-art motion tracker baseline trained for 7 days on 128 GPUs. It reduces GPU-hours by more than $1000\times$ and fails on only 25 of 129,514 benchmark motions, versus 179 failures for the baseline. After 9.8 hours on two GPUs, SuperMimic achieves zero failures. From 1 to 264 hours of motion, the world model's prediction error stays stable, and GPU-hours to a target return grow slowly. Our policy transfers to a physical Unitree G1 across diverse and acrobatic motions. SuperMimic removes compute as the bottleneck to scaling humanoid whole-body control, making it practical to measure motion-tracking scaling laws.
comment: Accepted at CoRL 2026 (Oral). Project website: https://supermimic.github.io
♻ ★ Seeing What Execution Needs: Benchmarking Task-Conditioned Actionable Spatial Belief for Manipulation
When should a robot look at the cup, and when at the portafilter? A view that is informative for one task can be useless for another, even at the same workstation. We introduce Seeing What Execution Needs, a benchmark for task-conditioned actionable spatial belief in simulated tabletop manipulation. Its aligned three-domain track contains 1,080 frozen cases (360 each for coffee workstation, drawer/cabinet, and microwave/appliance). The primary coffee protocols -- closed-loop actionability and same-object functional out-of-distribution generalization -- produce over 115,000 evaluator episodes across evaluated methods, budgets, and inference repeats. Each case pairs candidate RGB views with a manipulation command, shared scripted skills, and hidden postconditions, directly measuring whether selected observations support execution. A metadata-assisted diagnostic reference pipeline maintains spatial-functional beliefs and ranks views by missing evidence. At primary budget settings, it reaches 98.8% closed-loop and 98.9% functional-OOD success; removing functional fields or command binding reduces functional-OOD success to 0.0% and 19.7%, respectively. Under a separate full-preview contract, seven vision-language models reach 26-43%. Replacing coverage metadata with frozen RGB-derived scores retains 98.5%/98.9% success; shuffling these scores reduces success to 15%/11%, supporting dependence on image content. Separately, the aligned three-domain evaluation at H=3 supports portability of the benchmark interface under frozen domain-specific schemas. These results suggest that spatial belief for manipulation should be evaluated by whether acquired evidence meets the current command's execution requirements.
comment: 22 pages (11 main + 11 supplement). Extended cross-domain evaluation and physical robot validation in progress
♻ ★ Learning Beyond Rigid Manipulation with Hybrid Soft-Rigid Robots
Rigid-soft robot hybrids promise to unite the precision and repeatability of rigid manipulators with the compliance and adaptability of soft structures. Yet, full-fledged rigid-soft manipulators remain largely unexplored, in part because the intelligence required to exploit this combination is still missing. Here, we introduce a hybrid robotic system obtained by the series interconnection of a rigid arm and a soft manipulator, and systematically examine its ability to perform both established and previously inaccessible manipulation tasks. We show that a tailored imitation learning algorithm -- generally applicable to this class of hybrid systems -- leverages the complementary roles of rigid precision and soft embodiment to acquire manipulation skills and generalize them in a single shot across task variations. As a result, the system matches or exceeds the performance of rigid robots in classical tasks such as stacking, while enabling qualitatively new behaviors, including grasping objects through narrow openings.
comment: Revised version with new title, abstract, and experiments
♻ ★ Evaluating Long-Horizon Robot Planning under Uncertainty
Household robots must successfully navigate im-perfect circumstances, both in their internal models and the external world. We introduce a benchmark of seven long-horizon household tasks in Habitat that vary initial robot memory and environment configurations. We evaluate adaptations of ReAct, VLM-TAMP, and Tru-POMDP within a shared execution framework across 90 tasks-environment-memory combinations, totaling 1,350 episodes. Our evaluation measures task success, partial completion, and action efficiency, along with rule-based analysis of execution traces. ReAct achieves the highest overall task success (45.3%) and completion (71.7%), while having accurate initial memory does not consistently yield the highest performance for every planner. We identify recurring placement errors, precondition errors, and redundant actions through trace analysis. These findings highlight differences in how the evaluated planners use initial knowledge and respond to uncertainty.
♻ ★ An Integrated Open-Source Stack for Vision-Based Autonomous Navigation of Wheeled Rovers
Vision-based rover autonomy requires integrating perception, localization, planning and platform-specific control under limited sensing and computing resources. We present an open-source ROS 2 stack for following directional arrows, reaching a terminal cone and returning autonomously to the start. It connects YOLOv8n detection and geometric direction, range and bearing estimation with RGB-D SLAM, wheel-inertial state estimation, kinematics-aware planning, behavior-tree mission management and STM32 actuation. Standard interfaces separate reusable autonomy from the hardware layer; a proportional path-following plugin supports low-speed rover motion. Public resources include software, configuration, a labeled dataset, trained weights and evaluation logs. On 478 held-out images, the detector achieves mAP@0.5 of 0.720 under the Ultralytics evaluator and outperforms a classical OpenCV baseline under a shared evaluation protocol. The reported perception latency is 25 ms on a Jetson Orin NX. Three missions, one indoor and two outdoor, demonstrate marker-guided navigation and autonomous return without operator motion commands or reported collisions. Field results document integration failures and corrective choices; repeated instrumented trials remain necessary to quantify physical navigation errors and reliability.
comment: Accepted at RiTA 2026
♻ ★ Show, Don't Tell: Detecting Novel Objects by Watching Human Videos
How can a robot quickly identify and recognize new objects shown to it during a human demonstration? Existing closed-set object detectors frequently fail at this because the objects are out-of-distribution. While open-set detectors (e.g., VLMs) sometimes succeed, they often require expensive and tedious human-in-the-loop prompt engineering to uniquely recognize novel object instances. In this paper, we present a self-supervised system that eliminates the need for tedious language descriptions and expensive prompt engineering by training a bespoke object detector on an automatically created dataset, supervised by the human demonstration itself. In our approach, "Show, Don't Tell," we show the detector the specific objects of interest during the demonstration, rather than telling the detector about these objects via complex language descriptions. By bypassing language altogether, this paradigm enables us to quickly train bespoke detectors tailored to the relevant objects observed in human task demonstrations. We develop an integrated on-robot system to deploy our "Show, Don't Tell" paradigm of automatic dataset creation and novel object-detection on a real-world robot. Empirical results demonstrate that our pipeline significantly outperforms state-of-the-art detection and recognition methods for manipulated objects, leading to improved task completion for the robot.
Multiagent Systems
★ Multi-Agent Screening: What Agreement and Acceptance Cannot Tell You
When AI agents screen items for a costly human assessor, as in agentic search, e-discovery and AI reviewing of papers, only what they pass gets judged. We show what each kind of evidence can establish about their errors, even with unlimited data. Agreement shows that agents concur, not that they are right, however many there are. Acceptance by the assessor gives precision but only a lower bound on recall. Knowing which agents found which relevant items gives an upper bound on recall if agents' finds are not negatively correlated, and caps what one more agent can add if agents are interchangeable. Without assumptions about the agents, recall can be pinned down only by judging items they did not pass: a random sample of these estimates how many relevant items were missed, to any chosen accuracy, and the sample needed does not grow with the number of agents.
★ Teamknitt: Configuring, Observing and Intervening in LLM-Agent Teams through the Lens of Dynamical-Systems and Human-Team Theory
Most frameworks for teams of large language model (LLM) agents fix the team's shape inside the software: who talks to whom, how much is passed along, and what a human can do once a run has started. Human-team research has long asked those questions: where team phenomena live (in members or in their interaction), how team-level properties arise, and what separates what a team does from how a team is. I propose using that theory as the organizing lens for configuring, observing and intervening in LLM-agent teams, without assuming its findings transfer. I introduce teamknitt, a framework and open reference implementation in which a user configures a team of 1-16 AI agents for task completion, solution generation, achieving a goal or a research mission; selects one of fourteen deterministic routing topologies or a custom directed graph; chooses a sharing profile that limits what may travel along permitted routes; and binds sources, representations, work dependencies, handoffs and an expertise directory. The recorded process can be observed through non-system views (member reports and behavior) and system views (distributed problem-solving, differentiated transactive memory, interactive coordination, team dynamics). Interventions are explicit and human-led: reviewed reconfiguration between runs, planned disturbances, and private feedback, pause, resume or stop applied at dependency-wave boundaries, with "queued", "applied", "assembled", "understood" and "improved" kept as separate claims. Emergent states such as trust and shared understanding are declared unmeasured. This is an idea and system-description paper with falsifiable hypotheses on topology, information flow and intervention timing, including whether the fractal scaling of communication output between members, and between members on different models, relates to task and team performance. No comparative result is reported.
comment: 17 pages, 3 figures, 6 tables. Reference implementation v0.9.0: https://github.com/JeranRolevs/teamknitt
★ Unprompted coordination through shared memory in LLM agents: a locked two-implementation replication and extension
Coordination between large language-model agents has become a central question for AI safety and software engineering, as such agents are deployed in groups that share infrastructure. Existing research tells the agents that partners exist or asks them to coordinate. Whether coordination arises through shared state alone, with nothing disclosing that anyone else is present, is the key question for deployments that share caches and memory between agents, yet it has rarely been examined, and never with independent replication; this study does both. Four agents each received a short arithmetic task and a key-value store presented to each as its own scratch space; two held complementary halves of one computation, and no prompt mentioned other agents, sharing or collaboration. For reliability, the environment was rebuilt independently from a written protocol, and both implementations ran under hashed locks on design, data and analysis. The results bear novelty value in three respects. Joint success occurred only through the shared store. Whether a model coordinated depended not on its vendor tier but on whether it wrote its intermediate result and looked at the store again, and one ordinary sentence, "check your workspace again before submitting", switched that second look on and off. The findings held under replication. For practice, state shared between agents is a communication channel and should be separated, namespaced and logged. For research, designs should be locked before the data, and the parse rule, endpoint and parameters reported. Future work should test open-weight models and separate using another agent's output from intending to communicate.
comment: 13 pages, 7 tables, 2 figures. Replication package: https://doi.org/10.5281/zenodo.23965306
★ A Modular Perspective for Modeling MARL Systems
The growing complexity of Multi-Agent Reinforcement Learning (MARL) has fragmented the literature, obscuring underlying architectural choices behind diverse implementations. This fragmentation makes it difficult to evaluate critical modeling dimensions in isolation, identify how components affect the resulting system dynamics, and compare, reuse, or combine contributions across the literature. This paper proposes a modular perspective for modeling, analyzing, and implementing MARL systems. We decompose them into eight functional modules: agents, environments, learning, reward modeling, training and execution modes, communication, modeling of other agents, and evaluation criteria. For each module, we clarify its role and review representative approaches. We instantiate this decomposition in MARLKIT, a framework whose modules are explicit and interchangeable. MARLKIT demonstrates how the proposed perspective supports transparent experimental assumptions and the controlled variation and combination of MARL mechanisms. Experiments illustrate its ability to prototype, compare, reuse and combine modules while making differences between configurations explicit. Overall, the paper provides a structured perspective on MARL modeling and implementation, offering a common language for making modeling choices explicit, supporting clearer descriptions of MARL approaches, and supporting more controlled experimentation through the systematic isolation and combination of MARL mechanisms.
comment: 71 pages. 18 figures. Associated repository: https://github.com/tmaisonhaute/marlkit
★ Modelling socio-economic effects of the pandemic in a metropolitan area
One of the most notorious measures taken by authorities in every country trying to prevent the uncontrollable spread of the Covid 19 pandemic was to impose severe measures to diminish the mobility of people. In recent publications we proposed a mathematical model to address the consequences of these measures and to predict the infections under different scenarios, for various different countries. In this paper we extend the model, called the BTH--SEIRS-model, to deal with the reduction of mobility in an urban area. The chosen region was the densely-populated metropolitan area around Helsinki, the capital of Finland. This choice obeyed to the fact that there are precise data related to mobility from a mobile telephone company, and that this region was completely isolated from the rest of the country for a short period during the pandemic. We also used public transport data from railways and the accommodation occupation rates in the area of interest. We show that the BTH--SEIRS model is capable of simulating the real infection data as well as the mobility-related economic activity, exhibiting the main trends present in the three indicators used for comparison. Our calculations allow us to highlight the difference of the behaviour of the pandemic in a totally different spatial scale, and to discover the hidden factors that govern the behaviour of people who are forced to commute every day from home to work.
comment: 22 pages, 9 figures
★ Agent4Crypto: A Dual-Reflective and Market-Feedback-Aligned LLM-Based Multi-Agent System for Cryptocurrency Trading
Cryptocurrency markets provide a natural testbed for autonomous AI agents because they are digital-native, high-frequency, and machine-speed environments, yet existing methods often struggle with non-stationary market regimes, opaque decisions, and fixed aggregation rules. In this paper, we propose Agent4Crypto, a dual-reflective and market-feedback-aligned multi-agent system. The framework coordinates specialized agents for data collection, analysis, risk assessment, reflection, and trading, synthesizing heterogeneous signals from prices, indicators, news, and charts in an interpretable decision pipeline. Our central design is a trader-centric decision mechanism: three style-specific trader agents, Aggressive, Neutral, and Conservative, generate candidate actions and natural-language rationales from the same market context, while their influence is learned rather than manually fixed. A PPO-based gating policy dynamically allocates trader weights through a dual-reflective feedback process. One reflection stream evaluates post-trade decision quality from realized portfolio outcomes and distills reusable lessons into a Financial ReasoningBank for retrieval in subsequent trader prompts, while the other estimates trader-level counterfactual contribution, indicating which style has recently aligned better with market movements. The PPO state combines Counterfactual Contribution Memory with market features, previous trader weights, and current trader actions, so trader influence adapts across bull, sideways, and bear regimes. Experiments on BTC, ETH, and XRP show that Agent4Crypto improves total return and Sharpe ratio over traditional strategies, machine learning and reinforcement learning baselines, and existing LLM-based trading systems, demonstrating the value of interpretable trader competition and market-feedback-driven adaptation in volatile cryptocurrency markets.
comment: Guangyan Gan and Jiahao Li contributed equally to this work
★ Competing Influence in LLM Populations: Budget Timing and Baseline Drift
When competing campaigns target a population of large language model (LLM) agents, observed opinion changes may reflect both campaign influence and changes that would occur without intervention. We study how the timing of campaign expenditure and the population's own evolution shape outcomes in a network of persona-conditioned LLM agents that exchange and revise opinions in natural language. A proponent and a challenger are tasked with generating opposing messages under equal delivery budgets, while a separate stance probe measures opinions without influencing the simulation. Across eight claims, we compare single-campaign and competitive conditions against matched no-campaign runs, and evaluate all nine combinations of early, sustained, and late spending on a reference topic. Baseline stance drift varies substantially across claims and, on some, accounts for a sizeable fraction of apparent campaign effects. After accounting for this drift, the direction of observed campaign-associated changes aligns more closely with the claim's epistemic standing than with the campaigner's assigned role. Uninfluenced populations also consistently lose stance diversity, demonstrating that directional opinion change and convergence are distinct phenomena. In networks with two opinion-segregated communities, campaigns primarily reduce differences between communities rather than diversity within them. LLM campaign agents do not always argue for their assigned positions, particularly when these conflict with established evidence. Finally, campaign outcomes depend on the opponent's expenditure schedule: unlike a matched numerical bounded-confidence population receiving interventions at the same times and locations, the LLM population exhibits a payoff structure with no pure-strategy equilibrium.
★ A Benchmark and Layered Evaluation Protocol for Evidence-Grounded Shared Belief Updating in Connected Robotics
Connected physical systems fuse observations from robots, IoT sensors, and edge-AI modules into a shared account of the world, yet most evaluations score only the final answer or action and cannot tell whether a method attributed the evidence, updated its belief, handled contradictions, and justified its decision. We argue that shared reasoning in Connected Robotics should be evaluated as an evidence-to-belief trajectory, and introduce the Connected Robotics Evidence Graph (CREG) Benchmark to make that trajectory measurable. CREG comprises 350 deterministically generated scenarios across seven regimes (justified revision, belief preservation, relation association, contradiction onset and resolution, stale/delayed evidence, and source reliability), each exposing a normative posterior reference at every evidence prefix, validated by an independent log-odds checker and screened by a blinded expert audit. Nine representation conditions separate relation content from serialization, ordering, and metadata. Evaluating four open-weight instruction models and six transparent fusion baselines yields three findings. First, entity grounding accounts for a substantial part of the weak-model deficit. Second, deleting semantic relations costs 15.3 points of strict belief-state accuracy on relation-dependent scenarios (Holm-corrected p<0.001), but an information-matched structured list outperforms the full graph and shuffling the graph does not hurt: relation content matters where required, while graph serialization confers no consistent advantage. Third, belief accuracy is strongly non-monotonic along the evidence sequence, with mid-trajectory failures that endpoint-only evaluation cannot detect. All scenarios, references, prompts, metrics, and analysis code are released for offline, hardware-free reproduction.
comment: 15 pages (10 main + 5 supplementary), 6 figures, 4 tables; supplement adds 1 figure and 7 tables. Project page: https://creg-bench.github.io
★ ShadowBench-MCP: Measuring Shadow Invocations in MCP-Style Tool Routing USENIX Security
LLM agents increasingly act as routers for tools, APIs, and software-supply-chain workflows, making an execution path - rather than generated text alone - a security-relevant model output. We study shadow invocation: a task-preserving request variant that causes an LLM router to select a riskier execution path, potentially skipping validation, provenance, staging, redaction, or audit controls even though the request does not ask for a malicious action. We introduce ShadowBench-MCP, a benchmark and metric suite that separates task preservation (TPSS), operational pressure (OPSS), and route-geometric drift (RGAS), and combines them in the pre-execution SODS score. In a focused policy-ablation study using a Groq-hosted qwen/qwen3-32b router, 120 routing decisions yield 96 original-versus-variant comparisons across six security-sensitive workflows. Efficiency-first routing produces a 29.2% shadow-invocation rate and 26.7% safety-tool skip rate. Policy-aware routing reduces these to 4.2% and 1.7%, respectively, corresponding to an 85.7% relative reduction in shadow invocations. Pre-execution signals distinguish shadow routes with ROC AUC up to 0.94. These results are early, single-router evidence of a policy-dependent routing-layer security effect, not a claim of universal vulnerability.
comment: Presented at the 35th USENIX Security Symposium (USENIX Security 2026)
★ Steering Emergent Behavior in Multi-Agent LLM Systems via Information Networks
Large language model (LLM) agents increasingly operate in groups, each pursuing an individual objective while jointly determining system performance. If an operator controls which agent observes which other agent, but not their payoffs or messages, how should observations be structured to raise collective welfare? We make the information network (who observes whom) a design object, decoupled from the fixed interaction network over which agents act. Ten identical LLM agents play repeated group-decision games; by default each observes only an anonymous summary of the group's previous-round outcome, and a manager with a fixed budget adds directed observation links that reveal peers' past actions. Across 1,080 episodes and six games spanning coordination, anti-coordination, social dilemmas, and discovery, we ask how much information to share and, at a fixed budget, who should observe whom. Disclosure creates behavioral coupling: observation acts as a contagion channel whose welfare effect depends on the alignment between individually attractive behavior and the collective objective. First, the value of information is alignment-contingent: more peer information tends to help or be neutral when individual behavior is aligned with the collective goal, but under free-riding or congestion externalities it amplifies collectively harmful behavior; in a public-goods game the group does best with no information at all. We read this sign separation as a cross-game regularity within one controlled LLM population, not a model-invariant law. Second, at a fixed budget, who observes whom matters primarily when coupling is harmful: a modular "barbell" that lets sub-populations entrench harmful behavior is reliably the worst wiring, reversing the network-reciprocity intuition from human evolutionary game theory. These results argue against a default of full transparency for agent collectives.
comment: 20 pages, 8 figures, 7 tables; supplementary information included as an appendix
★ Do AI Agents need Preventive Care? An Epidemiologic Cohort Design for Agentic AI
As AI agents increasingly perform tasks autonomously across sectors, including healthcare and biomedical research, understanding how their behaviors develop and influence outcomes over time becomes increasingly important. Existing evaluations largely focus on model capabilities and performance on predefined tasks. Here, we propose an epidemiologic cohort design for studying agentic AI, drawing on principles established in human population studies. The framework treats individual agents as members of a cohort, their underlying model as genetic makeup, accumulated work as their time scale, and naturally occurring behaviors as exposures that may influence subsequent outcomes. By following agents longitudinally, this approach enables investigation of behavioral determinants of adverse events, estimation of their potential preventable burden under causal assumptions, and characterization of behavioral changes over an agent's work life. We illustrate the design through two prospective observational cohorts examining curiosity and probing as a safety question, and data checking and analytical accuracy as a reliability question. Together, the framework and illustrative studies provide a generalizable blueprint for designing epidemiologic cohorts of AI agents.
★ From Process Models to Composable Agentic Workflows: Combining BPMN, Agents and Enterprise Knowledge
LLM-based agents can reason, use tools and collaborate. However, coordinating increasingly complex agentic applications becomes difficult to understand, monitor and improve. Decisions distributed across prompts, tool calls and application logic make workflows implicit. We introduce **Recursive Capability Composition**, enabling agents to execute business processes and expose them as reusable Agentic Tools. This paper presents a BPMN-based approach to Agentic Workflows, combining explicit orchestration with autonomous reasoning. SemTalk BPMN models are transformed into executable workflows using a LangGraph/Pregel runtime. Agentic Tasks introduce LLM reasoning at selected activities, while conventional BPMN controls process flows, deterministic decisions, system interactions and human approvals. Agentic Participants provide reusable interfaces for agent behavior or other executable processes. BPMN orchestration makes multi-agent systems more understandable and manageable. Explicit process models provide developers and operators with a shared view of workflow structures and execution progress, facilitating debugging, monitoring and optimization. Enterprise agents also need organization-specific context. Process Models and Knowledge Graphs describe activities, participants, information objects, decisions and relationships between processes. They provide structured knowledge of how an organization operates, supporting informed agent decisions and actions. Our contributions are threefold: 1. **Model-Driven Execution:** Transforming BPMN models into executable Agentic Workflows. 2. **Recursive Capability Composition:** Exposing agents and executable business processes as reusable tools for other agents. 3. **Enterprise Context:** Using Process Models and Ontologies to provide organization-specific knowledge for agent reasoning.
comment: 14 pages, 3 figures
♻ ★ Bench-MFG: A Benchmark Suite for Learning in Stationary Mean Field Games
The rise of Mean Field Games (MFGs) has fostered a growing family of algorithms designed to solve large-scale multi-agent systems. However, the field currently lacks a standardized evaluation protocol, forcing researchers to rely on bespoke, isolated, and often simplistic environments. This fragmentation makes it difficult to assess the robustness, generalization, and failure modes of emerging methods. To address this gap, we propose a comprehensive benchmark suite for MFGs, focusing on the discrete-time, discrete-space, stationary setting. We introduce a taxonomy of problem classes, ranging from no-interaction and monotone games to potential and dynamics-coupled games, and provide prototypical environments for each. Furthermore, we present MF-Garnets, a method for generating random MFG instances to facilitate rigorous statistical testing. We benchmark a variety of learning algorithms across these environments, including a novel population-based approach (MF-PSO). To our knowledge, this is the first benchmark for MFGs.
♻ ★ Modelling Opinion Dynamics at Scale with Deep MARL NeurIPS 2026
Modelling opinion dynamics typically relies on hand-crafted local interaction rules to study emergent macroscopic phenomena such as consensus and polarisation. In contrast, multi-agent reinforcement learning (MARL) enables agents to learn such behaviours directly by optimising simple rewards. To explore the potential of MARL for opinion dynamics, we introduce a GPU-accelerated consensus and truth-finding game that scales to populations of up to 1000 agents, comparable to many real-world social sub-networks. To prevent unrealistic conventions, we extend other-play to general-sum social interactions. We next validate our model on a subset of the Bluesky network by recovering agent importance structures from graph topology alone via a learned attention layer, finding that highly conforming populations most closely match human data. In large social media networks, such high levels of conformity significantly reduce collective accuracy and promote dishonest agents that lie to fit in. By contrast, small hunter-gatherer networks are less affected; here, conformity can even improve collective agreement. This suggests a mismatch between evolved human conformity heuristics and modern social media environments as a potential contributor to misinformation.
comment: 39 pages, 31 figures. Accepted at the 40th Conference on Neural Information Processing Systems (NeurIPS 2026)
♻ ★ Rate-Aware Quantum-Inspired Trajectory Learning for Interference-Limited Multi-UAV Networks
Placing unmanned aerial vehicles (UAVs) as aerial base stations is non-convex and combinatorial, so practical methods first discretize the continuous airspace into a finite set of candidate positions---a uniform grid or clustering centroids---and optimize over that set. Built from Euclidean distance alone, the candidate set is channel-blind: it ignores line-of-sight probability and the co-channel interference that limits achievable rate under full frequency reuse. No optimizer can select a position the candidate set omits, so good locations the geometry missed are lost before the search begins. We propose Rate-Aware Quantum-Assisted Graph Condensation (RA-QAGC), a two-stage framework that makes the candidate set itself the object of optimization. The first stage constructs it by annealing a cost that embeds tier-weighted achievable rate, mean co-channel interference, and an explicit minimum-rate penalty, yielding a compact, mobility-feasible waypoint graph. The second stage learns trajectories over that graph with decentralized independent Q-learning agents, replacing a centralized joint-action table with one compact table per UAV. RA-QAGC improves priority-tier throughput by 21--54\% over seven baselines, including recent graph-neural and deep Q-network schemes, at no significant cost in aggregate throughput, and the margin widens as the fleet densifies. We additionally examine the quantum-inspired acceptance rule that recent aerial-networking work advocates, and report that its tunnelling floor confers no measurable advantage over classical simulated annealing at any fleet size tested, while annealing itself clearly outperforms greedy and random placement.
♻ ★ SocialGym: Evaluating LLM Social Intelligence through Verifiable Multi-Agent Games
Large Language Model (LLM) agents are increasingly deployed in multi-agent social settings where they must cooperate, negotiate, and adapt to other agents. Measuring these social skills is hard because most social interactions offer no objective ground truth. Evaluations often fall back on LLM judges, which are costly, subjective, and noisy. We introduce SocialGym, an environment of 22 multi-agent social games, from matrix games to hidden-role deduction, with verifiable outcomes determined by game rules. We benchmark ten LLMs across all games and roles using an Elo tournament. Our results show that no model is best in every game, and rankings shift across game families and sides. Secret keeping correlates more strongly with theory-of-mind and privacy benchmarks than with general knowledge benchmarks. Half of the models give away their side's secret in Chameleon and Spyfall, and the smallest models are not reliably better than chance at voting for hidden opponents in most hidden-role games. Finally, we examine whether models can improve their play by reflecting on self-play and writing strategy playbooks, finding mixed results across models and games. SocialGym provides verifiable measures for diagnosing LLM social capabilities across settings and roles.
comment: Code and data: https://github.com/Keyu-He/Social_Gym_Spartan
♻ ★ Physics-Informed Multi-Agent Coordination for Hospital Patient Flow Optimization
Efficient patient flow coordination across autonomous hospital departments is critical for mitigating overcrowding and balancing resource utilization. While classical queueing theory, specifically open Baskett--Chandy--Muntz--Palacios (BCMP) networks, provides an interpretable mathematical topology for healthcare operations, analytical models rely on stationary assumptions and fixed routing matrices that degrade under state-dependent real-world dynamics. Conversely, centralized reinforcement learning approaches struggle to accommodate the decentralized structure of hospital governance, where individual clinical departments function with localized observations, heterogeneous resources, and divergent operational objectives. In this paper, we present a Multi-Agent Systems (MAS) framework titled \emph{Physics-Informed Multi-Agent Coordination}, which embeds empirically calibrated BCMP queueing topologies as physical priors within a decentralized multi-agent reinforcement learning architecture. Formulated as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP) under coupled resource constraints, our method enables autonomous departmental agents to cooperatively negotiate patient routing and dynamic service scaling. To mitigate environmental non-stationarity without inducing excessive communication overhead, agents exchange localized action fingerprints along network edges and optimize a spatially decomposed reward structure. Empirical evaluations driven by real-world MIMIC-IV patient trajectories indicate that this cooperative multi-agent approach substantially reduces cumulative system delay compared to static Markovian approximations, heuristic dispatching, and independent multi-agent baselines, while maintaining clinical safety constraints.
comment: Accepted to PRIMA 2026 (Full paper). 16 pages, 2 figures, 3 tables
♻ ★ A Channel-Boosted Multi-Agent System with Iterative Consultation for Document Sensitivity Classification
Organizations in critical national infrastructure sectors must assess heterogeneous documents for sensitivity before routing or storage. Manual assessment is slow, inconsistent, and unscalable. Extending our prior leakage-controlled benchmark, BERT established the top single-encoder baseline (89.14% accuracy, 89.33% F1-score under 5-fold cross-validation on the Strategic 16K corpus). However, transformer baselines suffer from a structural limitation: fixed input length truncation discards evidence beyond the retained window-precisely where sensitive cables tend to be longest. We present Channel-Boosted MAS (CB-MAS) and instantiate it as IC-MAS (Iterative Consultation Multi-Agent System) to solve this without long-context computational costs. A Channel Critic Agent learns document-adaptive trust weights governing Gated Channel Boosting between two first-window encoders, while paired Consultation Agents iteratively exchange belief states to reconcile evidence from the beginning and end of long documents. IC-MAS holds computation constant regardless of document length by reconciling fixed windows in a compact representation space. Ablation studies show critic-controlled Channel Boosting provides the bulk of accuracy gains, while consultation recovers recall without precision collapse. Critic-Controlled Gated Channel Boosting with Max-Pool fusion and Blackboard Adaptive Consultation achieves 90.72% accuracy, 91.23% F1-score, 92.01% sensitive recall, and 90.46% sensitive precision, using about 54% less average computation than a fixed-round baseline. Gains over the single-encoder baseline are statistically significant (McNemar's test, p less than 0.000001; paired t-test). We include LIME/SHAP explainability, multi-agent evaluation, and an honest accounting of limitations.
comment: 40 pages , 14 figures
♻ ★ When Disclosure Is Compulsory: Evaluating Ride-Hailing Rebalancing Policies with Self-Interested LLM Driver Agents
The EU Platform Work Directive, to be transposed by December 2026, requires ride-hailing platforms to state in plain language the main parameters of their automated decisions and how a worker's behaviour affects them. Such rules are exactly what a large language model (LLM) can read, so drivers will be able to let one decide for them. We compare two main families of rebalancing policy once their rules are public: a reactive policy (R) that sends idle drivers to cover requests just lost, and an anticipatory policy (A) that plans moves on a demand forecast. Each driver is an LLM agent maximizing its own net income, given no rules (N) or the rules at the directive's level of detail (L). We test two LLMs, Claude Haiku 5.5 and Jev, which understand the rules equally well, in a three-zone model and in Manhattan. With the rules disclosed, drivers follow the platform more closely: in Manhattan, the share of relocation offers they decline falls from about four in five to one in four under both policies. Yet the effects differ by policy. For the platform, disclosure costs R 3.7% of its profit in Manhattan, as R now pays for moves drivers used to make for free, while A's profit is unchanged; the two policies, which tie on profit under full compliance, end 6.1% apart. For drivers, disclosure raises income in the small model under both policies and LLMs; for users, it raises the service rate there by 10 to 18 percentage points. In Manhattan, self-interested drivers also make A a win-win-win against full compliance, raising driver income, platform profit and service rate by 10 to 12% each. Rebalancing policies that perform equally well under full compliance can thus show opposite results under mandatory disclosure. The study offers a framework to evaluate such policies under the new requirement and is, to our knowledge, the first to do so with LLM agents.
♻ ★ AIM: Adaptive Interaction in Multi-Agent Debate for Multimodal LLM Inference NeurIPS 2026
Multi-Agent Debate (MAD) improves reasoning on multimodal tasks, but its accuracy and token efficiency depend on how agents interact during debate. A largely overlooked design choice is which interaction modalities agents use to exchange evidence. Existing methods rely on fixed text- or graph-based interaction, which fails to convey fine-grained multimodal evidence and thus degrades accuracy while inflating token cost due to verbose descriptions. To address this, we propose Adaptive Interaction in Multi-Agent Debate (AIM), an adaptive routing framework that dynamically selects the optimal modality combination for inter-agent interaction in multimodal MAD. Beyond text and graph, AIM introduces task-dependent modality interaction, where agents exchange targeted regions of interest from the task-specific multimodal input, such as image regions, audio segments, or spatio-temporal video clips. To determine the best combination for each instance, AIM first generates a structured baseline response and extracts interpretable routing features that capture which evidence the instance requires, how confident the baseline agent is, and how much new evidence each modality would add beyond the baseline response. A lightweight router then uses these routing features to select a modality combination that avoids both under-fusion (i.e., agents miss decisive evidence) and over-fusion (i.e., redundant modalities inflate token cost) risks. Across eight benchmarks spanning three multimodal task families, AIM achieves the highest accuracy. Combining modalities improves accuracy by up to 6.4% over the best single-modality baseline, and adapting the combination per instance adds up to 4.2% over the most accurate fixed combination of each benchmark with up to 53.4% fewer tokens.
comment: Accepted in The Fortieth Annual Conference on Neural Information Processing Systems (NeurIPS 2026)
♻ ★ Policy-Controlled Offline-Verifiable Accountability for Cross-Organization Agent Messaging: A Preserved Evidence-Bundle Approach
Cross-organization agent workflows require preserved evidence that remains independently verifiable during later audit or dispute review. Such workflows may involve multiple organizations, delegated actions, policy-relevant events, and disputed accountability claims. This is difficult when live systems are unavailable, controlled by one party, or not trusted by all participants. Existing mechanisms provide useful pieces, including authenticated logging, delegation semantics, signed checkpoints, and consistency checks. What remains missing is a verifier-centered event-level bundle for checking evidence sufficiency offline under an explicit policy. We propose a preserved evidence-bundle model and a policy-controlled offline verifier for agent-to-agent workflow events. Each bundle preserves policy-required evidence, including sender authentication, authenticated log commitment, witness-backed checkpoint evidence, append-only continuity, delegation-aware authorization evidence, and, when issued and required by policy, explicit receiver-signed receipt evidence. The verifier accepts a bundle only when all mandatory and policy-required checks succeed, giving a later dispute reviewer an offline basis for assessing evidence sufficiency. It does not infer delivery or receipt from transport behavior or log inclusion alone. In a prototype evaluation, the event-type study used 300 synthetic four-event healthcare workflow cases and 1200 valid preserved bundles. The policy-profile comparison observed its largest median latency reduction relative to the fully enabled profile when checkpoint-context validation was disabled. All 17 tested cases in the original negative-evidence diagnostic suite were rejected under the fully enabled profile. Additional functional studies examined context and signed-reference binding. These results support prototype feasibility for the evaluated workloads.
comment: v2: Revised title and abstract, clarified policy-controlled verification, and expanded the evaluation. 23 pages, 5 figures, 16 tables
♻ ★ Graph Attention-based Decentralized Actor-Critic for Dual-Objective Control of Multi-UAV Networks
This research focuses on optimizing multi-UAV systems with dual objectives: maximizing service coverage as the primary goal while extending battery lifetime as the secondary objective. We propose a Graph Attention-based Decentralized Actor-Critic (GADC) to optimize the dual objectives. The proposed approach leverages a graph attention network to process UAVs' limited local observation and reduce the dimension of the environment states. Subsequently, an actor-double-critic network is developed to manage dual policies for joint objective optimization. The proposed GADC uses a Kullback-Leibler (KL) divergence factor to balance the tradeoff between coverage performance and battery lifetime in the multi-UAV system. We assess the scalability and efficiency of GADC through comprehensive benchmarking against state-of-the-art methods, considering both theory and experimental aspects. Extensive testing in both ideal settings and NVIDIA Sionna's realistic ray tracing environment demonstrates GADC's superior performance.
♻ ★ AGiR: Mitigating Gift Over-Reliance in Mixed-Motive Games NeurIPS 2026
In this paper, we consider mixed-motive games (MMGs) which model various real-world multi-agent scenarios where self-interested agents must both cooperate and compete. We identify an issue with existing gifting methods, where agents become overly dependent on incoming gifts and fail to learn reward-yielding behaviors, a phenomenon we call gift over-reliance. To address this, we propose a refusal-augmented gifting framework that balances cooperation and competition in MMGs. Our approach introduces an adaptive refusal mechanism that allows agents to selectively reject a portion of received gifts from their policy-update signals, thereby mitigating gift over-reliance and preventing the agents from being exploited by other agents. This mechanism is gifting-method-agnostic, serving as a plug-in for existing schemes while enabling decentralized learning. We demonstrate that across diverse MMG environments and gifting schemes, our mechanism consistently reduces gift over-reliance and prevents lock-in to sacrificing policies, improving $α$-fairness as a result, without necessitating major changes to existing training pipelines.
comment: Accepted at NeurIPS 2026
Systems and Control (EESS)
★ Distributed Global Pose Synchronization for Heterogeneous Multiple Rigid Body Systems on SE(3)
This paper addresses the problem of global pose synchronization for heterogeneous multiple rigid-body systems evolving on the matrix Lie group $\SE(3)$. We first construct a new potential function on $\SE(3) \times \mathbb{R}$ with a generalized non-diagonal weighting matrix, together with a set of switching variables governed by continuous-discrete hybrid dynamics. Based on this new potential function and these switching variables, we propose a distributed hybrid feedback control scheme that uses relative pose on $\SE(3)$ and group velocity measurements, endowed with global asymptotic stability guarantees. We then further propose a distributed velocity-free hybrid feedback control scheme that uses relative and absolute pose measurements, without requiring group velocity measurements, while preserving global asymptotic stability. Numerical simulations are presented to demonstrate the performance of the proposed distributed hybrid control schemes.
comment: 17pages, 9 figures
★ Safe Performance-aware Online Input Design
In this paper, we propose an input design framework for the safe online excitation and identification of uncertain linear systems without requiring informative initial data. The objective is to actively excite the system and collect sufficiently informative input-output data while maintaining safe operation and satisfactory control performance. Achieving this objective is challenging due to the potential conflict between the excitation required for learning and the safety and performance requirements, particularly when limited prior knowledge of the system is available. Inspired by the dual control principle, the proposed scheme balances exploitation for control performance with exploration for model improvement. A Bayesian learning approach is employed to update the model uncertainty online, starting from a conservative initial uncertainty set. As informative data are collected, the uncertainty set contracts, enabling progressively less conservative operation. To reduce the online computational burden, the resulting program is reformulated into a computationally tractable form. Furthermore, an online ellipsoidal terminal set with non-increasing conservatism is designed to guarantee the recursive feasibility of the proposed scheme. Numerical experiments are conducted to demonstrate the effectiveness of the proposed framework, particularly in terms of uncertainty reduction and control performance.
★ Certified Zonotopic Reachability Analysis for Trajectory Tracking on Lie Groups
Autonomous vehicles track planned trajectories close to obstacles. Unknown disturbances make them deviate from these trajectories. With a bound on the tracking error that disturbances can cause, a planned trajectory can be certified as safe before execution. Existing set-based methods bound the tracking error by one invariant ellipsoid for the whole trajectory or by intervals, which are both conservative. For aerial and space vehicles, the tracking error evolves on a matrix Lie group with nonlinear dynamics. However, for group-affine systems, the error dynamics are linear in Lie algebra coordinates when the vehicle receives the inputs of the reference. The control input and the disturbance enter through the inverse right Jacobian, a matrix that depends on the error. We propagate the linear part of the error dynamics with zonotopes and enclose the effect of the inverse right Jacobian on the disturbance in closed form. The resulting tubes are tight and provably contain all reachable tracking errors in continuous time. When some error components have no input, the inverse right Jacobian leaves unmatched nonlinear terms in the closed loop. None of the tested methods certifies such a closed loop. Hybrid coordinates make these terms matched, so that our controller cancels every nonlinear term that does not contain the disturbance. Our tubes give tighter bounds than the ellipsoid method in simulations of a planar aircraft and a fully actuated multirotor and in real flights of a quadrotor, where the tube contains every measured attitude error. Under larger disturbances and the same feedback gain, the ellipsoid method yields no valid ellipsoid, whereas our tubes remain valid.
★ Succinct Infinite-Horizon Lyapunov and LQR Value Problems Are BQP-Complete
Fundamental tasks in linear systems analysis and control require evaluating infinite-horizon output energies for autonomous systems and optimal value functions for controlled systems. These quantities are characterized by Lyapunov and algebraic Riccati equations, respectively. In this work, we study promise problems for their evaluation when the system matrices are sparse and represented succinctly by Boolean circuits specifying their nonzero entries, with input length polynomial in the logarithm of the system dimension. For dissipative autonomous systems with coordinate-vector initial states, we define the succinct Lyapunov value problem (SLV). Its extension to linear-quadratic optimal control is the succinct LQR value problem (SLQRV). We prove that SLV is $\BQP$-hard: every problem in bounded-error quantum polynomial time ($\BQP$) reduces to it in polynomial time. We then establish that the more general SLQRV problem admits a quantum algorithm running in time polynomial in the succinct input length. Since SLV is a special case of SLQRV, both problems are $\BQP$-complete. Consequently, unless bounded-error probabilistic polynomial time coincides with $\BQP$, no classical randomized algorithm can solve either problem on all promised instances in time polynomial in the succinct input length.
★ Sliding into Balance: Benefits of a Sliding Payload for Load Distribution and Disturbance Rejection in Dual-UAV Transportation
This paper analyzes the cooperative transportation of a payload suspended from two drones in which, instead of being attached to two independent ropes, the payload is allowed to slide along a single rope connecting both UAVs. First, a planar nondimensional dynamical model of the sliding motion with viscous friction is presented. Then, considering stationary drones, this configuration is compared with a fixed payload with the same total rope length, both at equilibrium and under disturbances. The analytical results show that the sliding payload equalizes the tension in both rope segments for any relative position of the drones, so that no rope loses tension at equilibrium; it doubles the region reachable by the payload and, except for low friction combined with a large drone separation, it enlarges the set of admissible disturbances. Finally, a dynamic simulation also shows that under realistic gusty wind profiles, the average tension imbalance between the drones is reduced with respect to the fixed-payload solution.
comment: 6 pages, 6 figures
★ KL-Regularized Policy Iteration for Adaptive LQR: Gain Updates and Probing from One Policy Improvement
Policy-gradient methods for adaptive linear quadratic control improve the feedback gain of an unknown plant online by gradient-type steps computed from closed-loop data. A recent framework of such methods, policy-gradient adaptive control (PGAC), takes one gradient-type step per sample, which its indirect form computes from an estimated model and its direct form directly from the data. Because data generated by a fixed linear feedback cannot identify the plant, PGAC adds a probing perturbation to the input and leaves its covariance as a design parameter. We study the adaptive LQR problem through a KL-regularized formulation, in which the probing is the randomness of a Gaussian policy kept close to a Gaussian reference policy, so that the cost of the probing is part of the objective. One policy improvement of this formulation gives a gain update together with a probing covariance. We show that, for suitable references centred at the current gain, this gain update is exactly PGAC's Gauss-Newton or natural-gradient update with any step size below a bound. We repeat this policy improvement, each time against a tempered copy of the current policy, and refer to this scheme as KL-regularized policy iteration (KLPI). On a known plant and with tempering, the cost of KLPI's gains converges linearly to the optimal cost, and the probing covariance keeps a floor proportional to the regularization weight. On an estimated plant, under conditions like PGAC's (small updates and sufficiently informative data), the closed loop stays stable, the cost of the gains converges linearly up to a bias, and the probing covariance again keeps this floor. In simulations at equal probing cost, the shape of KLPI's probing covariance, which PGAC can adopt, lowers the extra cost caused by suboptimal gains on nearly all tested plants, whereas KLPI's use of past iterations in its gain update changes this cost little.
★ Hierarchical Nonlinear Control for Swarm Encirclement Around a Moving Target UAV
This paper presents a hierarchical inner-outer loop nonlinear control strategy using a geometric Lyapunov-based framework for the cooperative encirclement of a target by multi-ple unmanned aerial vehicles (UAVs) in three-dimensional space. The encirclement problem is formulated on a two-sphere with arbitrary radius and center, denoted Gamma-2, by projecting agent positions from R3 onto a spherical manifold centered at the target, being the unit two-sphere at the origin S2 used for intermediate calculations. Desired encirclement configurations are generated via a point separation method on Gamma-2, while stable convergence from R3 to these points is guaranteed using a Lyapunov-based reduced-attitude control law inspired by geometric control on the two-sphere. Formal stability results are derived using Lyapunov analysis and the framework is validated in simulation using a nonlinear quadrotor model and controller. Results demonstrate stable encirclement and highlight the suitability of the proposed approach for three-dimensional multi-agent aerial systems.
comment: Pre-print. Accepted at the 17th APCA International Conference on Automatic Control and Soft Computing (CONTROLO 2026); 6 pages, 6 figures
★ Offset-Free Data-Driven Predictive Control: Implicit and Explicit Disturbance Formulations
Offset-free tracking is a key objective of any control algorithm. For predictive controllers with linear models, it is well understood that rejecting asymptotically-constant disturbances necessitates augmenting the prediction model with $p$ integrating states, where $p$ is the number of measured outputs. This can be achieved in model predictive control (MPC) by using a velocity form or a combination of disturbance model and observer. In data-driven predictive control (DPC), implicit incorporation of integrating states using the velocity form has been successfully adopted to obtain offset-free tracking; however, formulations based on an explicit disturbance model and observer have not yet been explored. In this paper, we first study how the integrating states are incorporated into the data-based representation used in DPC, translating the requirements of $p$ integrating states into the required modifications of the DPC formulation. We then propose a novel offset-free DPC formulation that incorporates an explicit constant output-disturbance model and an output disturbance observer. Numerical examples show that this formulation has similar disturbance-rejection capabilities to the velocity-form DPC, but using a smaller optimisation problem.
comment: 6 pages, 3 figures. Submitted to ECC 2027
★ Reshaping Checkpoint-Induced Power Variations Through Inference Workload Scheduling
Synchronous checkpoints in large language model training cause rapid power variations that challenge frequency regulation in islanded data-center microgrids. This paper proposes a checkpoint-aware two-layer framework that schedules flexible inference workloads to compensate for checkpoint-induced disturbances. Using measured GPU power profiles and anticipated checkpoint power deficits, the upper layer coordinates heterogeneous task selection and launch times, while the lower layer adjusts the batch sizes of newly launched tasks online to correct power-tracking errors. Early launching (EL) and staged retirement (SR) further smooth power transitions at checkpoint onset and recovery, respectively. PSCAD/EMTDC simulations with reference checkpoint drops of 1-3 MW demonstrate reductions of 41.6-61.6% in maximum absolute frequency deviation and 68.5-89.6% in mean absolute frequency deviation relative to no scheduling. Battery energy storage system (BESS) peak charging power and charging energy decrease by 65.1-100% and 89.7-100%, respectively. These benefits exceed those of greedy scheduling when GPU resources are relatively abundant.
★ Converse Lyapunov theorem for input-to-state stability of infinite-dimensional systems
We prove converse Lyapunov theorems for input-to-state stability (ISS) of a broad class of infinite-dimensional systems applicable to evolution equations in Banach spaces, boundary control systems, and delay systems. To this end, we employ a novel method that does not require constructing auxiliary closed-loop systems. We start by showing that ISS of a control system implies uniform global asymptotic stability of the same system with respect to trajectory-dominated inputs. Then, under an additional regularity requirement, we prove that an ISS Lyapunov function can be constructed. Based on our general result, we prove that for linear systems with unbounded $\infty$-admissible operators, ISS is equivalent to the existence of a Lipschitz continuous coercive ISS Lyapunov function.
★ Covert System Identification Under Timing Constraints: A Static Linear Model and Its Detectability Implications
This paper studies covert system identification in cyber-physical systems, where an attacker infers plant dynamics while attempting to remain stealthy. We introduce a lightweight static linear model in which the time to execute a setpoint change is proportional to the commanded speed difference. On a Siemens PROFINET testbed this gain captures the drive's ramp response accurately but not its terminal overshoot. That omitted transient becomes the attacker's main exposure: on real traces, the residual left by the linear masking procedure is detected by two anomaly monitors, Process-Aware Stealthy-Attack Detection (PASAD) and the CUSUM change detector. Across the cascade identification and staircase attack strategies defined in the paper, the stealth boundary relates to the plant's own settling transients, a structural limit for this class of models under the adopted detector-calibration regime. Comparison against dynamic models obtained by N4SID, trained at multiple excitation amplitudes, shows that the evaluated N4SID models do not improve stealth over the structure-informed ramp model. Since per-cycle computation is negligible for both procedures, the binding trade-off is not inference cost but whether the attacker can acquire and refresh a model without injecting detectable perturbations.
comment: Presented at the 27th International Symposium on Measurement and Control in Robotics (ISMCR 2026), Rio de Janeiro, Brazil. Proceedings forthcoming in Springer's Mechanisms and Machine Science series
★ Degradation-Aware Participation of Battery Storage in Frequency Regulation Markets
As data centers make the load on the power grid fluctuate faster, frequency regulation grows more valuable. Battery energy storage systems (BESSs) are well suited to provide frequency regulation, but providing this service speeds up aging, especially during operation at extreme state of charge (SoC). In the PJM interconnection, the grid operator supplies a regulation signal that a participating BESS is expected to follow, within a capacity that it must bid before the signal is known. Accordingly, we decompose the degradation-aware frequency regulation problem into a bidding layer and an operations layer. For bidding, we propose an adaptive strategy, assuming perfect tracking in which the bid depends on the SoC expected at the start of the bid period by means of choosing basepoints. For operation, the BESS further protects its battery by deliberately giving up a perfect performance score, to keep its SoC away from the extremes. On the PJM regulation signal and prices of 2023, adaptive bids lengthen the battery life by $31\%$ against bidding full capacity, for $2.2\%$ of the revenue. Giving up a perfect score in operation lengthens the battery life of constant bids by $10\%$ to $27\%$, at a change in revenue of at most $0.2\%$.
comment: Extended version of IEEE PESGM submission
★ Towards Passivity-informed Data-driven Predictive Control
We propose a passivity-informed data-driven predictive control framework based on passivity-preserving sampling of unknown continuous-time linear systems. We incorporate the passive supply rate into the objective functional alongside an output-regulation cost, thereby explicitly accounting for energy extraction. Despite being practically motivated, the resulting objective is not always convex; its convexity depends on the collected input-output training data. We exploit passivity-induced positive semidefiniteness to derive an equivalent reformulation that remains a convex quadratic program independently of the data. Furthermore, we establish recursive feasibility and asymptotic output convergence using passivity and a terminal equality constraint. Numerical simulations illustrate the closed-loop performance and computational benefits of the proposed approach.
★ KPG193-ext: A Synthetic Korean Power Grid Test System for Stability Analysis
Growing shares of inverter-based generation and constrained transmission expansion increase the need to assess stability across operating conditions. Production-cost models provide chronological commitment and dispatch, but stability analysis requires corresponding electrical detail and device dynamics. This paper introduces KPG193-ext, an extension of the synthetic Korean power-grid test system KPG193 that links chronological operating points with detailed electrical and dynamic models. The extension adds synchronous-machine, controller and renewable-converter models, together with explicit voltage levels, generation terminals, inter-voltage and step-up transformers, and reactive compensation. The resulting 731-bus network retains the original 193 regional locations. Daily unit commitment followed by hourly AC optimal power flow establishes 8,760 operating points for 2025, each distributed as paired steady-state and dynamic-model files. Selected-hour studies examine generation-loss frequency response, steady-state flow-voltage characteristics, and rotor-angle response with special protection system actions. The results reveal differences in delivered frequency support, internal voltage stress along regional transfer paths, and the angle-frequency tradeoff of corrective generator tripping. The database provides a common basis for examining the stability implications of commitment, dispatch and corrective action across daily and seasonal operating conditions.
★ Scalable Practical Synchronization of Heterogeneous Linear Systems
We study practical state synchronization of heterogeneous linear multi-agent systems over directed graphs using static diffusive state feedback. Persistent mismatches between the agent dynamics generally preclude exact synchronization and are treated as bounded disturbances within an Input-to-State Stability (ISS) framework. We derive a scalable synthesis procedure based on fixed-dimension linear matrix inequalities (LMIs), whose dimension is independent of the number of agents. Local robust dissipation conditions are combined through a small-gain argument to obtain global ISS guarantees and explicit bounds on the synchronization error. The framework is formulated in a unified manner for continuous- and discrete-time systems, while highlighting a fundamental difference in disturbance attenuation between the two domains. In particular, exact ZOH discretization of a continuous-time controller need not preserve closed-loop stability, motivating a native discrete-time synthesis. Numerical experiments on heterogeneous harmonic oscillators illustrate the proposed approach and the continuous/discrete-time distinction.
★ Sensor placement for real-life water distribution networks, ensuring observability under uncertainties
Leakage detection in water distribution networks (WDNs) is crucial for limiting water losses. However, effective detection strongly depends on sensor availability and placement. Most existing sensor placement methods rely on extensive simulations or precise knowledge of hydraulic parameters, which are often uncertain in practice. In this paper, we propose using a scalable sensor placement framework based on structural observability that depends only on the network topology and the underlying pressure-flow interconnection structure. The approach guarantees structural observability independently of hydraulic parameters, and is therefore robust to parametric uncertainties, assuming that the network topology is correctly specified. The method is computationally efficient and its performance is demonstrated on several real-life WDNs provided by Brabant-Water. While the framework ensures full network observability, the required number of sensors can be high in large and cyclic networks, which may limit applicability in real-life and motivates future work on relaxed observability guarantees for sensor placement.
comment: Accepted for WDSA/CCWI 2026
★ Observer-based parameter estimation in a Darcy model towards dam internal seepage monitoring with validation on real data
This paper addresses internal seepage monitoring in hydraulic infrastructures through the estimation of physical parameters in a distributed model. A physics-based approach relying on Darcy's law is adopted to describe the spatiotemporal evolution of hydraulic head within the structure. To enable real-time applicability, a Proper Orthogonal Decomposition (POD) is employed to derive a reduced-order model of the underlying partial differential equation. An Extended Kalman Filter (EKF) is then designed to perform online estimation of the hydraulic diffusivity, a key parameter related to the structural state. The proposed method is validated directly on real measurement data through a twofold approach. First, the estimated parameter is compared with the value obtained via offline batch optimization performed on the same dataset, showing consistent results. Second, the ability of the resulting model, parametrized by the estimated diffusivity, to reconstruct the observed measurements is assessed, to evaluate its suitability for seepage monitoring.
★ Safety-Constrained Intelligent Control of a 2-DOF Robotic Manipulator Using Control Barrier Functions
Control barrier function (CBF) filters depend on dynamic models that can be inaccurate when their parameters and external forces are uncertain. This paper investigates a radial basis function (RBF) neural network compensator whose output is shared by a Cartesian tracking controller and a joint-space CBF filter. An auxiliary state driven by the difference between requested and applied torques corrects the learning signal, reducing the direct influence of filter intervention on adaptation. A two-link manipulator is evaluated through constrained regulation, ablations, matched uncertainty and disturbance trials, and operating-range and timing studies. The results demonstrate that the proposed controller improves tracking while maintaining joint, velocity, and torque limits under parameter uncertainty, and that both the shared compensation and the auxiliary correction contribute to this improvement.
comment: Paper submitted to the European Control Conference 2027
★ Tensor Network Kernel Machines for Nonlinear System Identification
Over the past ten years Tensor Network Kernel Machines have enabled efficient nonlinear system identification through the use of low-rank tensor networks. The goal of this article is to provide the reader with a basic understanding of this model class in terms of its two defining properties. Furthermore, both the optimization and regularization of such models are discussed, together with recent developments and starting points for further exploration.
comment: This work has been submitted to IFAC for possible publication
★ A Scale-Space Theory of Memory
Scale-space theory has emerged as a central attempt to restrict the {\it general} theory of signal processing with geometric constraints inspired from the animal world. In the mathematical language of operator theory, scale-space theory restricts the Toeplitz operator. The present paper investigates scale-space theory for memory systems. We investigate how scale space theory restricts a memory system encoding past information, that is, restricts the Hankel operator. Our main result is to establish a close correspondence between scale-space theory and the long-studied {\it relaxation} property.
★ Prospective Action-Resource Graph Multi-Agent Reinforcement Learning for Electric HGV Fleets under Shared Charging Infrastructure
Electric heavy goods vehicle (eHGV) fleet coordination requires each vehicle to choose driving, charging and resting actions whose consequences depend on shared infrastructure. Existing multi-agent reinforcement learning (MARL) methods mainly represent current states, leaving action-specific futures and resource interactions to be learned indirectly. We develop Horizon-Preview and Graph-Coordinated Proximal Policy Optimisation (HGC-PPO), which integrates horizon preview and candidate-resource graph coordination into the PPO decision architecture. Horizon preview represents each legal action through its short-horizon operational consequences. Graph message passing then updates these candidates through station, charger-type and time-slot resources shared across the fleet. On the 794km Folkestone-Glasgow task, HGC-PPO achieved the lowest mean charging cost among five RL methods at £504.32 per vehicle, with a fleet completion time of 24.920h. It also completed the task 1.763h faster and cost £36.51 less per vehicle than a replay-verified finite-budget MILP benchmark. Across six unseen routes and fleet sizes from 5 to 20 vehicles, HGC-PPO delivered the lowest macro-average completion time, charging cost and charging frequency among the RL methods. A 2 by 2 ablation showed that the full preview-graph architecture performed best across all four operational metrics. These results show that prospective action information and fleet-wide resource coordination improve eHGV decisions under charging and driver-hours constraints.
★ A Game-Theoretic MPC Planner for Cooperative and Adversarial Drone Coordination
We present and evaluate a game-theoretic model predictive control planner for cooperative goal reaching and adversarial pursuit-evasion with drones. The planner combines 3D prediction, individual objectives, optional shared separation constraints and trajectory-tracking interfaces. A comparison of cooperative game solvers assesses their ability to find feasible trajectories that resolve conflicts. A computational study quantifies how number of drones, prediction horizon and move blocking affect computation time and numerical acceptance. Move blocking holds control inputs constant over groups of prediction steps, reducing the number of independent control variables and lowering computation time. Timing measurements identify the practical limits of real-time planning in the tested configurations. PyBullet simulations demonstrate cooperative goal reaching and pursuit-evasion. A laboratory flight complements the simulations by validating the trajectory-tracking interfaces on physical drones. The planner can serve as a basis for further evaluation and controller development.
comment: 7 pages, 4 figures, submitted to European Control Conference (ECC27)
★ Environmental Assessment of GaN Current-Source and Voltage-Source AC-AC Variable Speed Drives
Three-phase ac-ac converters for variable-speed drives can be realized conventionally as voltage-source converters (VSCs) or, leveraging emerging monolithic bidirectional GaN power transistors (M-BDSs), as current-source converters (CSCs). This paper presents a comparative life-cycle assessment of a realized VSC and a CSC demonstrators including grid-side and motor-side EMI filters, and with identical specifications (1.4 kW rated power, 200 V line-to-line rms grid and motor voltage). The embodied carbon footprints are similar (44.8 kgCO2eq for the VSC and 47.4 kgCO2eq for the CSC) and dominated by auxiliary and control electronics, and the printed circuit boards. Over 10 years of continuous operation under the NEMA Power Index mission profile for variable-load centrifugal systems (pumps, fans), the CSC's higher part-load efficiency translates into a clearly lower life-cycle carbon footprint of around 900 kgCO2eq compared to the VSC's 1130 kgCO2eq when operating with a carbon-intense electricity mix (440 gCO2eq/kWh). However, this difference almost vanishes when considering a future renewable electricity mix (25 gCO2eq/kWh).
comment: Submitted to an IEEE journal
★ DC Voltage Control Dominated Oscillations in Electromechanical Frequency Range - Insights from Synchronous Coefficient Reduction Effect of IBR Plants and Network Interactions
Large-scale integration of inverter-based resources (IBRs) is widely known to cause subsynchronous oscillations. This paper investigates a distinct class of oscillations that occur in the electromechanical frequency range (EFR), referred to as EFOs, and are dominated by the DC voltage control (DVC) loops of grid-following IBRs rather than synchronous generator-like controls or phase-locked loops (PLLs). Although open-loop oscillation modes of DVC loops are subsynchronous, the corresponding closed-loop dominant oscillation mode related to DVC can shift downward into the EFR without retuning the control parameters. This frequency shift is caused by network-mediated interactions among the IBRs, which introduce a negative synchronous coefficient into the DVC oscillation loop. This synchronous coefficient reduction effect (SCRE) becomes stronger as the IBR plant scale increases and/or as the minimum eigenvalue of the network admittance matrix decreases. Furthermore, a complex vector-based stability analysis method is proposed, and a stability criterion for DVC-EFOs is derived. Finally, the analytical conclusions are validated using a practical power system configuration implemented in Simulink. The SCRE differs from traditional explanations, in which the open-loop oscillation modes are already located in the EFR, and provides new insight into DVC-dominated oscillations in the EFR for integration of IBR plants.
★ On necessary conditions for feedback stabilizability when the control space is a fiber bundle
Within control theory, a question of great interest is to understand when a stabilizing feedback exists. Building upon Krasnosel'skiĭ and co-workers, necessary conditions for the existence of a continuous stabilizing feedback have been pioneered by Brockett and Coron, providing so-called adversarial and homological conditions, respectively. Kvalheim recently generalized their conditions from equilibria to compact attractors and---in line with earlier work by Coron---showed that when a control system can be identified with a vector bundle, the homology condition is stronger, that is, the homology condition implies the adversary condition. Kvalheim also showed that this comparison breaks when extending the case of vector bundles to fiber bundles. In this work, we propose and characterize an appropriate adversarial condition for control systems represented by fiber bundles. The homology condition, on the other hand, is already well-defined, showcasing its general applicability. With the generalized conditions at hand, we show that by appropriately relaxing the regularity of the feedback under consideration, that is, by allowing for mild discontinuities, the homology condition implies the adversarial condition for fiber bundles. Thereby, in this work, we extend these earlier comparative results.
comment: 18 pages, 1 figure. Submitted
★ Compact and Wide-Force-Range Magnetorheological Hand Exoskeleton for Kinesthetic Feedback in Teleoperation
Hand kinesthetic feedback is crucial for restoring and enhancing human perception of force and displacement in robotic hand teleoperation. Wide perception range of force feedback is vital in rendering objects of different stiffness, especially for high-stiffness rendering. However, existing approaches struggle to achieve wide range of force feedback while maintaining compact and wearable design. This paper presents a hand exoskeleton system employing magnetorheological (MR) brakes to provide a wide range kinesthetic feedback during teleoperation. The MR brake provides a braking torque ranging from 0.01 to 0.918 Nm, while maintaining a compact structure with a total mass of only 39.8 g. The proposed exoskeleton can render fingertip forces up to 21 N while maintaining back-driving force of 1.01 N. Experimental results show that the system allows users to discriminate object stiffness using kinesthetic feedback alone. In a tele-grasping task, when both conditions achieved a 100% success rate, kinesthetic feedback led to a 28.2% reduction in object contact force compared to visual feedback only, enhancing object safety and operator immersion. Questionnaire results further indicate that kinesthetic feedback improves reliability, realism, and user confidence during teleoperation.
★ One number per hour is not enough: directional carbon intensity of data-center ramps on the PJM fleet
A data-center campus can change its load by hundreds of megawatts within an hour, yet every carbon signal in use gives one value per hour for a load change, whatever its size or direction. We compute the directional carbon intensity, the emission caused or avoided per megawatt-hour of a finite upward or downward ramp, for every hour of 2025 on the PJM Interconnection's fossil fleet, and compare it with the locational marginal emission rate (LME) and the regional average. The LME misstates a 1 GW ramp by more than 10% in 60% to 83% of hours in eleven months. Upward and downward ramps differ in a quarter to three quarters of hours but agree on their monthly mean. Location-based reporting assigns 47% to 59% of a ramp's emission to other consumers. A campus should schedule on the signal for its ramp's size and direction and report the emission it causes.
comment: 15 pages main text, 6 pages supplemental information. Code and derived results: https://doi.org/10.5281/zenodo.24074188
★ Online Digital Twins: Data Assimilation and Control
The online assimilation of data and the application of controls are essential ingredients of the digital twin paradigm. In this paper, we formalize the online interaction between the physical process and its virtual digital representation in the setting of continuously arriving data from the physical process and a receding horizon virtual planning process, which provides the control for the physical process. Our mathematical formulation relies on a mean field representation of the data assimilation process as well as a mean field Pontryagin minimum principle for the optimal control problem. Such mean field formulations are computationally attractive because they allow for simultaneous implementation in terms of Hamiltonian interacting particle systems. A more detailed discussion is included for the ensemble Kalman-Bucy filter in combination with affine approximations of the receding horizon control laws. This paper sets out the mathematical framework, while a computational study is left for future research.
★ Generalizable Output Feedback Optimal Control via Transformers Without Explicit State Estimation
Classical optimal control is used to design a separate controller for each plant. This paper asks a different question. Can one learned policy serve a whole family of partially observed, noisy linear systems? We show that one transformer can. The policy is trained to imitate optimal LQG control on a heterogeneous family of Linear Time-Invariant (LTI) systems with Multiple Inputs and Multiple Outputs (MIMO), whose dimensions, cost matrices, and noise levels all vary. A shared representation with system-wise standardization, zero-padding, masking, and an encoding of the cost and noise covariances lets one set of weights control every member of the family. At run time, a short window of noisy measurements is all the policy needs to produce a control action, with no plant matrices and no state estimator. We characterize the relation between the optimal LQG policy (represented as a dynamic state-space model) and a static transformer acting on a finite window of measurements. We then express the closed-loop excess cost of any causal policy as the energy of its deviation from the optimal action. Finally, we evaluate the approach on 28 simulated systems. The trained policy reaches the optimal cost to within a median gap of $0.4\%$ for seen systems. For unseen systems, a light fine-tuning step brings the median gap to $1.0\%$.
comment: 7 pages
★ MAC+: Mobility-Aware Co-Expansion Planning for Distribution Grid and Electric Vehicle Charging Infrastructure
We study the least-cost co-expansion of power distribution networks and public charging ports to accommodate the growth of electric vehicles (EVs). Our approach, MAC+, exploits the flexibility of charging along each EV's itinerary and coordinates it across the fleet to reduce the peak violations of each feeder's hosting capacity and port capacity. This coordination is an extremely large, spatio-temporally coupled optimization problem with binary plug-in variables. We solve it by iterated price response (IPR), which repeatedly posts a two-tier tariff on feeder load and port occupancy, lets millions of EVs best-respond in parallel, and certifies the optimality gap at every iteration. We develop a custom exact dynamic-programming algorithm that solves each EV's best-response problem, a mixed-integer quadratic program, about 20 times faster than off-the-shelf solvers. The case study covers the San Francisco Bay Area at 30% EV adoption, with 1.8 million EVs on week-long itineraries and 1,303 feeder service regions, each with a utility-published hosting capacity and a port capacity set by its existing public ports. On this instance, IPR reaches a certified gap below 10% in a few hours. MAC+ saves about $15 billion of the upgrade investment that unmanaged charging would require. It also needs over 90% less investment than charging that is fully coordinated but flexible only within each session, and over 80% less than charging that is flexible along the itinerary but uncoordinated.
comment: 20 pages, 6 figures, 7 tables
★ Graph-Sparse Distributed Predictive Control for Multi-Robot Motion Planning vianEdge-Consensus ADMM
This paper presents a distributed model predictive control (MPC) architecture for collision-avoiding motion planning of planar robots with double-integrator dynamics. To reduce coordination demands, a time-varying predicted conflict graph selects relevant pairwise interactions. The associated collision-avoidance constraints are convexified once per MPC update using warm-start trajectories, yielding a fixed convex quadratic program (QP). The resulting QP is then decomposed using an edge-consensus formulation of the alternating direction method of multipliers (ADMM), enabling parallel robot-local optimization and independent closed-form projections onto separating half-spaces. Execution is selectively supervised by a centralized high-order control barrier function (HOCBF) QP when the distributed candidate does not satisfy the direct-execution conditions. Evaluated in C++ simulations involving 8, 16, and 32 robots, the proposed architecture achieves collision-free execution and substantially improves computational scalability while maintaining task-level performance comparable to a centralized complete-graph baseline.
★ Silinx: An Open, Browser-Based HDL Design Platform for Teaching Digital Systems on FPGA Boards
Many digital systems courses teach on FPGA boards whose vendor tool is no longer maintained: the Digilent Basys2 and Nexys2 (Xilinx Spartan-3E) and Nexys3 and Atlys (Spartan-6) need Xilinx ISE 14.7, discontinued since 2013 and hard to install on the computers and tablets students use today. The author presents Silinx, a free (AGPL-3.0) HDL design platform for teaching that keeps ISE's Project Navigator workflow but runs in a web browser with nothing to install. A module can be written in VHDL or Verilog, in C or C++ through Silinx HLS, which is compatible with AMD's Vitis HLS and passes all 73 of its introductory examples, or drawn as a schematic, an ASM chart, a state diagram, a Boolean function with its Karnaugh map, or a Petri net, from which its HDL is generated. Designs are checked with explanations written for beginners, simulated, run on an emulated board, built, and programmed onto a real board over WebUSB. The implementation flow is open and runs in the browser: Yosys and nextpnr compiled to WebAssembly, with Silinx's own device data, timing model and bitstream writer for the Spartan-3E and Spartan-6, built by black-box observation of ISE's documented tools; Lattice iCE40 boards use the existing open iCE40 tools, and further families are planned. ISE is the oracle: Silinx's bitstreams are byte-identical to those ISE writes for the same placed and routed design, and pass ISE's design rule check. Against ISE at the same optimization goal, the open flow uses 15% fewer slices with 14% shorter critical paths when optimizing for area, and 21% fewer slices with 8% shorter critical paths when optimizing for speed (geometric means over 45 open-source designs built by both flows). The author also describes how the platform was developed with AI coding agents under test-first project rules.
★ A Bifurcation-Theoretic Framework for the Controllability of Emergent Intelligent Systems
Real-world intelligent systems must reconcile two demands: the adaptability conferred by emergence and the steerability required for accountable deployment. We show that these demands are governed by two distinct bifurcations on the same object and that, consequently, increased capability does not imply increased controllability. Emergence is a bifurcation of the capability variable $S$, with criterion $Γ:=\mathcal F'(0)>1$ equivalent to the existence of a unique positive stable fixed point of a self-consistency equation, following solely from the concavity of $\mathcal F$. Controllability is a bifurcation of the permission variable $c$, whose dynamics include an erosion term and exhibit a saddle-node fold with hysteresis as the gate criterion $ρ^\ast$ crosses unity. Three quantitative results follow: the critical exponent distinguishes families rather than classifying the transition; the gating-cost bound $K(M)\propto\log(1+M)$ grows with scope, not capability; and the fold threshold is $γ_c\approx4μκ$. Calibrated in both directions, the criteria reproduce the two-dimensional Ising transition, admit two true-feedback order parameters, and reject nine candidate quantities, supporting the conclusion that the order parameter is geometric rather than capability-related. Since permission cannot be endogenized and no deployed system has been measured to satisfy $Γ>1$ and $ρ^\ast>1$ simultaneously, increased capability does not imply controllability.
comment: 21 pages, 1 figure, 5 tables
★ Scarcity Pricing for State-of-Charge Management in Real-Time Markets Under Intra-Day Uncertainty
The growth in renewable resources will increase net load variability and forecast error. Energy storage resources can provide flexibility when ramp capability is limited and contribute to load leveling over the day. The challenge for system operators is how to balance these two functions in real time, when updated net load forecasts are available for a limited lookahead horizon. U.S. electricity markets are seeking ways to manage state-of-charge (SoC) within their energy markets to more efficiently utilize these resources. We propose a method to manage energy storage SoC in a real-time market by extending Hogan's (2013) scarcity pricing framework. A day-ahead SoC schedule is modified using existing multi-interval real-time unit commitment engines as net load forecasts change. To address 30-minute-ahead forecast errors, an updated 5-minute SoC target and a dynamically updated shortage price for violating the constraint are provided to each economic dispatch. We simulate PJM's energy markets for 365 days and demonstrate reliability improvements relative to a strategy that enforces the day-ahead schedule. We show cost reductions and improved price formation relative to a strategy that updates target quantities but uses an administratively chosen shortage penalty.
★ ShadowBench-MCP: Measuring Shadow Invocations in MCP-Style Tool Routing USENIX Security
LLM agents increasingly act as routers for tools, APIs, and software-supply-chain workflows, making an execution path - rather than generated text alone - a security-relevant model output. We study shadow invocation: a task-preserving request variant that causes an LLM router to select a riskier execution path, potentially skipping validation, provenance, staging, redaction, or audit controls even though the request does not ask for a malicious action. We introduce ShadowBench-MCP, a benchmark and metric suite that separates task preservation (TPSS), operational pressure (OPSS), and route-geometric drift (RGAS), and combines them in the pre-execution SODS score. In a focused policy-ablation study using a Groq-hosted qwen/qwen3-32b router, 120 routing decisions yield 96 original-versus-variant comparisons across six security-sensitive workflows. Efficiency-first routing produces a 29.2% shadow-invocation rate and 26.7% safety-tool skip rate. Policy-aware routing reduces these to 4.2% and 1.7%, respectively, corresponding to an 85.7% relative reduction in shadow invocations. Pre-execution signals distinguish shadow routes with ROC AUC up to 0.94. These results are early, single-router evidence of a policy-dependent routing-layer security effect, not a claim of universal vulnerability.
comment: Presented at the 35th USENIX Security Symposium (USENIX Security 2026)
★ Rapid Heat Removal from Chip Cores: Near-Junction Active Cooling of Power Transistors via Electron Entropy Transport and Spatial Decoupling
Chip performance is limited by a shared bottleneck: the core produces heat faster than it can be moved out. We study the hardest case, RF power GaN HEMTs, whose hotspots exceed 5 kW/cm^2, and propose a horizontal near-junction active cooling topology built on electron entropy transport. Classical on-chip thermoelectric cooling faces a contest: Peltier absorption grows linearly with current, Joule heating with its square. In a vertical pillar, its own conduction resistance locks about half the Joule heat back toward the cold end, capping net cooling. From the Onsager framework we show the origin of this asymmetry: Peltier absorption localizes at the interface where the Seebeck coefficient jumps, whereas Joule heating fills the volume. The topology exploits this: a thermoelectric thin film lies flat on a diamond substrate beside the device drain (the electron exit), so bulk Joule heat leaves vertically into the substrate. A lateral fin model gives a closed form for the recirculation factor gamma: the vertical pillar yields gamma = 1, while the horizontal film yields gamma_max = sqrt(2) L_z/L_x, about 0.03 for L_z = 10 um, L_x = 500 um. The decoupling scales the net cooling ceiling and the critical runaway current by roughly 1/gamma. At equal resistance, the same film lifts maximum cooling from 13 mW to 500 mW, and the critical current from 0.65 A to 24 A. A time-constant and linear stability analysis with temperature-sensitive parameters defines the working window and safety margins for microsecond pulse cooling. The architecture leaves the device structure untouched; the principle, absorb at an interface and extract bulk heat vertically, applies to any chip with a hot core and offers a reproducible design path for near-junction thermal management.
comment: 10pages
★ AHEADance: Anticipatory Humanoid Dance from Streaming Audio
A humanoid that dances to streamed music must commit each motion chunk before it has heard the music that chunk accompanies, and must continue from motion it has already committed. We present AHEADance, a robot-native streaming dance generator executed on a humanoid robot by a whole-body tracker. ListenAhead conditions each chunk on heard music and a frozen music model's forecasts of the coming music, aligned to their target times and weighted by reliability. The forecasts carry the timing of upcoming beats almost as well as the actual future; with them, the generator is better synchronized with the music and concentrates more motion energy at its song's beat frequency. Commit Forcing trains continuation from the model's own committed chunks, updating both the latent history and the decoded boundary of the next chunk; it reduces low activity from 83.5% to 10.3%, the level of real dance. Three music evaluators, each validated on real G1 dance, test synchronization, song and style. On a physical Unitree G1, it dances while the music plays and comes to rest when it stops, without a fall. Project page: https://ihcr.github.io/AHEADance
★ Modal-Augmented Chebyshev Physics-Informed Kolmogorov-Arnold Networks for Power Systems
Physics-informed neural networks (PINNs) offer a promising alternative to numerical integration for power system dynamic simulation; however, their conventional architectures can struggle to capture electromechanical oscillations. Although physics-informed Kolmogorov-Arnold networks (PIKANs) offer an alternative, their B-spline implementations incur substantial training costs. To address these limitations, this paper investigates Chebyshev PIKANs (c-PIKANs) and proposes a modal-augmented c-PIKAN (mac-PIKAN) based on a trial solution that incorporates the closed-form modal response of the linearized swing equation. First, the modal response is used to capture the dominant oscillatory dynamics. Then, a c-PIKAN learns a nonlinear correction weighted by a quadratic time-dependent factor, ensuring that the initial rotor angle and speed are satisfied exactly. Evaluations on the single-machine infinite-bus benchmark demonstrate the computational efficiency of c-PIKAN and show that mac-PIKAN reduces PINN rotor-angle and frequency errors from 1.59% to 0.29% and from 5.26% to 0.99%, respectively, while maintaining higher accuracy across training-data sizes and at lower damping levels.
comment: 5 pages, 6 figures. Submitted to the 2027 IEEE PES General Meeting. Code and data: https://github.com/komeilnosrati/mac-PIKAN
★ Implicit DeepReach Without Implicit Differentiation
We consider the problem of approximating backward reachable and reach-avoid tubes for high-dimensional nonlinear systems whose Hamiltonians require numerical optimization over constrained controls and disturbances. Existing neural Hamilton-Jacobi (HJ) reachability methods alleviate the dimensionality bottleneck of grid-based solvers, but evaluating Hamiltonians without closed-form expressions introduces an additional computational challenge. We show that, under suitable assumptions, the gradients required for value-function training can be computed without differentiating through the inner optimization problem. Specifically, we establish an envelope-type gradient identity for the saddle point Hamiltonian, under which the optimal control and disturbance enter backpropagation as constants without gradient tracking. This enables end-to-end neural HJ reachability with numerically evaluated Hamiltonians while avoiding both solver unrolling and implicit differentiation. We demonstrate the accuracy and computational efficiency of the proposed approach on nonlinear reachability and reach-avoid problems of different scales.
★ Pulsatile Endocrine Regulation with a Releasable Hormone Pool: Impulsive Feedback Model
An impulsive model of endocrine regulation equipped with frequency- and amplitude-modulated pulsatile feedback is extended to study the dynamical effects pertaining to limitations due to the releasable hormone pool. The model describes a three-hormone regulatory loop and is applicable to endocrine systems such as the reproductive, stress, and growth hormone axes. Conditions guaranteeing the existence and uniqueness of a periodic solution with a single feedback-triggered impulse per period (a 1-cycle) are derived, together with monotonicity properties of its characteristics. An explicit Jacobian formula is obtained to assess local orbital stability. Model analysis demonstrates that a reduced hormone-pool replenishment rate produces a lower-amplitude, higher-frequency 1-cycle. This behavior is consistent with experimentally observed alterations in testosterone regulation in aging human males.
★ Force-Closure Condition for Cooperative Multi-robot Pushing of Rectangular Objects
In cooperative object manipulation, the placement of pushing contacts determines whether force closure can be achieved and agents can generate arbitrary forces and torques on the object correspondingly. This study derives a necessary and sufficient closed-form condition for the planar force closure of a rectangular object, a common object shape in everyday settings, under four frictionless pushing contacts. For a rectangle of length $b$ and width $a$, force closure holds if and only if $(l_1+l_3-b)(l_2+l_4-a)<0$, where $l_1,l_2,l_3,l_4$ are the four edge contact locations. This condition is independent of the object's center of mass and is used to verify contact configurations before contact force allocation. When the condition is not satisfied, the signs of the two factors indicate which opposing contact pair should be adjusted. The condition and adjustment rule are then integrated into a full-pose trajectory-tracking framework as a geometric feasibility layer to demonstrate their use in control. Static wrench feasibility tests validate the proposed condition, while narrow-passage floating-body simulations show that maintaining a force-closure configuration reduces the RMS tracking errors in $x$, $y$, and unwrapped yaw from $12.05$ m, $18.11$ m, and $535.96^\circ$ to $2.24$ m, $2.66$ m, and $2.69^\circ$, respectively.
comment: 7 pages, 7 figures, 2 tables. Published in CAOS 2026
★ Image-Driven Control with Application to Thermoforming
This paper proposes Image-Driven Control (IDC), an approach to vision-based feedback that operates directly in the image space. IDC offers a direct end-to-end solution in the behavioral framework. Targeting a limited subset of vision-based feedback problems, including applications in manufacturing, the IDC design builds on a computationally advantageous subspace predictive control formulation and offset-free tracking to enable reference tracking in the image space under input and pixel-level constraints. IDC provides a control-theoretic alternative to vision-based feedback systems, enabling improved transparency for analysis and tuning of end-to-end control. It offers straightforward experiment design, an advantage in applications where data collection is expensive. The effectiveness of IDC is shown in a simulation example using a nonlinear high-fidelity model of the heating phase of the thermoforming process.
comment: 6 pages, 2 figures. Presented at the 23rd IFAC World Congress (IFAC 2026), Busan, Republic of Korea, August 2026
★ Ground Reaction Force-Aware Model Predictive Control for Underactuated Upright Physical Assistance
Many rehabilitation robots execute prescribed motions using reactive tracking controllers that do not explicitly incorporate the user's instantaneous human-ground interaction. This paper presents Ground Reaction Force-Aware Model Predictive Control (GRF-MPC), a framework for upright physical assistance. By integrating live GRF and center of pressure (CoP) measurements directly into a human centroidal dynamics model, GRF-MPC formulates a real-time quadratic program (QP) that optimizes assistive forces and moments over a receding horizon. We deployed GRF-MPC on the Robotic Upright Stand Trainer (RobUST) in an underactuated 4-cable configuration and evaluated it against a fully actuated 8-cable Cartesian impedance baseline in nearly 300 trials across six healthy participants. Across tasks spanning impulsive perturbations, lunge trajectories, and voluntary limits of stability, GRF-MPC reduced peak planar center-of-mass (CoM) error by 19.6% during perturbations, lowered trajectory tracking error by 24.1%, required 28.7% less net mechanical work, and permitted a 45.9% greater normalized voluntary excursion area. The underlying QP solves in under 1 ms, comfortably enabling 100 Hz replanning and demonstrating that incorporating live human-ground interaction into predictive control yields stronger, more transparent physical assistance with half the actuators.
comment: Accepted for publication in IEEE Robotics and Automation Letters (RA-L), October 2026. 8 pages, 5 figures
★ Robust State-Parameter Estimation in Second-Order Thévenin Batteries
We propose a scheme for estimating the full states and parameters of a second-order Thévenin battery model, using distinct features of the measured current. An identification step, run during suitable rest intervals, identifies the resistances and capacitances of the two RC branches from the voltage relaxation by linear least squares, without knowledge of the open-circuit voltage (OCV) map. An online estimator then runs continuously: the polarization voltages are estimated by open-loop observers; the series resistance is updated at current jumps using least squares; the state of charge (SOC) is propagated by Coulomb counting using the current capacity estimate, and both estimates are updated periodically by a sliding-window regression. We show that the identification is exact under a mild rank condition, that the series resistance is recovered at the first current jump, and that the online state and capacity estimates converge exponentially under verifiable excitation conditions. We also establish robustness of the online estimator under bounded process disturbances and measurement noise.
comment: Under review for European Control Conference (ECC)
★ Risk-Aware Dynamic Operating Envelopes for Battery Arbitrage in Unbalanced Distribution Networks
Dynamic operating envelopes (DOEs) provide network-aware limits for distributed energy resource scheduling while respecting network operating constraints. This paper develops a risk-aware DOE framework for battery arbitrage in unbalanced three-phase networks using a semidefinite program (SDP). We derive sufficient conditions for exactness of the SDP relaxation, ensuring that the recovered operating points satisfy all constraints of the original AC network model. The method computes hourly active-power DOEs under uncertain load conditions and incorporates them directly into a 24-hour battery arbitrage problem.On a modified IEEE 13-bus feeder, empirical comparisons with unconstrained arbitrage and local Volt--VAR/Volt--Watt control show that DOE-constrained scheduling eliminates the observed out-of-sample voltage violations while retaining 94.6% of the unconstrained arbitrage profit.
comment: 8 pages, 2 figures, 1 table. Extended version of a manuscript prepared for the 2027 IEEE PES General Meeting
★ Fast Sampling or High Gain? iP versus PI Control
Intelligent proportional (iP) controllers, introduced in the framework of model-free control, are often presented as an alternative to proportional-integral (PI) controllers that requires no model of the plant. This paper studies this property for a first-order linear plant under sampled-data implementation with a zero-order hold. When the time derivative of the output is estimated by a backward difference, the discrete-time iP and PI controllers are known to obey the same control law under a one-to-one correspondence between their parameters. Building on this correspondence, the paper derives necessary and sufficient conditions for exponential stability of the iP closed loop, in closed form, in terms of the controller parameters KP and alpha, the sampling period h, and the plant parameters. It then shows that, when the iP parameters are held fixed and the sampling period tends to zero, the closed-loop decay rate tends to the iP gain KP, provided that KP is positive and that the ratio rho between the input gain of the plant and alpha satisfies 0 < rho < 2. One closed-loop pole then behaves as 1 - h KP, while the other tends to 1 - rho. With fixed PI gains, by contrast, the decay rate tends to that of the continuous-time PI loop, which depends on the plant parameters. The two parametrizations therefore suggest different tuning procedures, although they achieve the same closed loops at every fixed sampling period. High gain for the PI is, in some sense, replaced by fast sampling of the iP. The results also make explicit which plant information the iP controller requires: the sign of the input gain and a bound on its magnitude in the limit of fast sampling, and bounds on both plant parameters at a given sampling period. Numerical experiments illustrate these results.
comment: Submitted to IEEE ECC 2027
★ ASPIRE: An Agentic System-Identification Platform for Iterative Reasoning and Exploration
System identification requires expert decisions on the identification method, incorporation of prior physical knowledge, and model validation. We propose ASPIRE, a framework for nonlinear system identification in which a large language model agent selects and executes the identification method. ASPIRE constructs closed-form, nonlinear static or dynamic models from measured data, optionally supplemented by a system description, the intended use of the model, and physical requirements expressed formally or in natural language. Each model includes its parameters, an estimation record, an interpretation of its terms, and an error evaluated on test data inaccessible to the agent. Formal physical requirements are verified independently. ASPIRE operates autonomously or interactively, allowing the agent to ask questions and the user to request further analyses and identification rounds. We evaluate ASPIRE on three measured systems: a mobile robot, a rotational system with dry friction, and a laboratory fermenter. Each system is tested with and without a description, in five runs per task. All 30 runs delivered valid models, at a cost of 0.96 to 3.38 US dollars per run. An assessment by the authors found that, for the 30 models recommended by the agent, one per run, the interpretation of the model and the documentation of its structure selection and parameter estimation fully satisfied 132 of 150 assessed items and partly satisfied 18. The most frequent discrepancy, found in nine models, concerned inaccurate or unsupported claims about model structure selection.
★ Control Barrier Functions for Counting Tasks in Multi-Agent Systems
Multi-agent systems increasingly operate in dynamic environments and execute complex tasks that require coordination, posing significant computational challenges. Counting tasks, which require a minimum number of agents to satisfy a given condition, arise naturally in such systems. Examples include surveillance tasks, where at least m drones must continuously monitor a critical area, and emergency-response tasks, where at least m robots must converge to an incident location. The identities of the agents assigned to these tasks may change dynamically depending on their states, input bounds, and individual objectives. In this work, we propose a control barrier function (CBF)-based framework for enforcing counting invariance and counting convergence tasks, without explicitly assigning agents or enumerating all possible subteams. We develop a recursion-based method for constructing differentiable CBFs for counting tasks. The resulting CBF constraints are incorporated into quadratic programs, enabling online controller synthesis. Numerical experiments demonstrate scalability to large multi-agent systems, while a multi-drone emergency-response case study illustrates the satisfaction of counting tasks.
comment: 8 pages, 6 figures; This paper has been submitted to the European Control Conference 2027
★ Opacity of Discrete Event Systems under Sensor Attacks
Many information control problems in discrete event systems can be addressed using strong opacity, weak opacity, and non-opacity. These opacities depend on observations. If cyber attackers can attack sensors (or observation channels), then opacities will change. An opaque system may become not opaque, or a not opaque system may become opaque. Therefore, it is important to investigate opacities under sensor attacks. In this paper, we consider opacities under sensor attacks for two settings. The first one is the user has prior knowledge of sensor attacks in which the opacities are called as W-CA-opacities. The second one is the user has no prior knowledge of sensor attacks, and opacities are called as N-CA-opacities. For the former, we construct an augmented automaton that integrates the information of sensor attacks and then converts W-CA-opacities into classical opacities to solve the verification problem. For the latter, we construct the observers for the given system without sensor attacks and with sensor attacks, respectively. We then do the parallel operation on these two observers to obtain a method of verifying N-CA opacities. Furthermore, by adopting the detector technique, we find polynomial algorithms for verifying weak opacity and non opacity for both settings.
★ Proportional Neuro-Impulsive Control of SISO LTI Systems
This paper explores a controller that is at the crossroads of neuromorphic, i.e., spike-based, and impulsive control systems. The controller consists of a single integrate-and-fire-like neuron that generates an input proportional to the system output when the neuron variable reaches a threshold. We prove that any static output-feedback stabilizable single-input single-output (SISO) linear time-invariant (LTI) plant can be made semi-globally asymptotically stable (sGAS) in closed loop with the proposed neuro-impulsive controller. This improves on existing fixed-amplitude neuromorphic controllers, which only achieve practical stability, but at the price of using graded spiking inputs. Moreover, we provide an algorithm for choosing appropriate controller parameters given a set of admissible initial conditions. A numerical example of an inverted pendulum linearized around its upright equilibrium illustrates the method.
comment: 8 pages, 1 figure, 1 algorithm
★ An Open Implementation Flow for the Xilinx Spartan-3E and Spartan-6 that Runs in the Browser, Checked Byte for Byte Against ISE
The author describes an open implementation flow, from HDL to a programmable bitstream, for the Xilinx Spartan-3E (XC3S100E to XC3S1200E) and Spartan-6 (XC6SLX16, XC6SLX45), devices that only the discontinued ISE 14.7 supports. The flow runs unchanged in a web browser or in Node.js: Yosys and nextpnr compiled to WebAssembly, with nextpnr's Himbaechel framework given Spartan-3E and Spartan-6 architectures generated from Silinx's own device descriptions, bit database and timing model, and Silinx's own bitstream writer. The bit database was built by black-box observation of ISE's documented XDL and bitgen tools, using codeword-batched experiments; the router is given only switches whose bits were measured, and the writer refuses a feature it has not measured. ISE is the oracle: for the same placed and routed design, the bitstream is byte-identical to the one bitgen writes and ISE's DRC is clean. Placement and routing were first done by an in-house annealer and PathFinder router, which nextpnr replaced once it matched them. On 67 open-source designs for the XC3S1200E, compared with ISE at the same optimisation goal (45 designs built by both), the flow uses 15% fewer slices and has 14% shorter critical paths at goal Area, 21% fewer slices and 8% shorter paths at Speed, and 20% fewer slices but 12% longer paths at Fast (geometric means). XST cannot read 15 of the designs. The flow is published as the npm package silinx-fpga (Apache-2.0).
♻ ★ Control policies for a two-stage queueing system with parallel and single server options
We study a two-stage queueing system attended by two flexible servers. Once a server begins to interact with a job, it remains attached to its job throughout both stages of service. After completing the first-stage service, the server chooses between two second-stage options. Option 1 sends the server-job pair to a parallel-service station where both servers may operate simultaneously; therefore, the server immediately begins the second-stage service on its job. Option 2 routes the pair to a single-service station that admits at most one server-job pair into service at a time. While we consider the complete range of parameters, the usual reason for choosing Option 2 is that jobs may receive higher-quality (faster and/or lower expected cost) service. If the single-service station is occupied, the pair must wait together until the station becomes available. The decision-making scenario is modeled via a Markov decision process formulation of a clearing system with holding costs at each station. Focusing on cases with a large initial queue, we fully characterize the structural properties of an optimal control policy based on the relationship between the service rates at the downstream stations. A numerical study highlights the significance of understanding the structure of the optimal policy by comparing its performance against several natural heuristic policies.
♻ ★ Feedback Stabilization for Sampled Linear Systems with Control-linear Noise
In this paper, we consider linear stochastic systems with control-linear noise and periodically sampled measurements. We address the problem of feedback stabilization in the mean-square sense. The main contribution of the paper is to provide a necessary and sufficient condition for feedback stabilization. In particular, we relate feedback stabilizability of the stochastic system to the existence of a fixed point of a Riccati-type algebraic equation and, further, to the existence of a solution to an infinite-horizon optimal control problem.
♻ ★ Measure the Deployment, Not Just the Model: Rethinking Hardware-Aware Model Selection for Microcontrollers
Selecting an energy-efficient model for periodic sensing requires accounting for time between inferences. Hardware-aware model search typically measures inference alone, even when the application cares about energy per release. We present CREST (Cross-platform Runtime Evaluation and Search Tool), an open-source hardware-in-the-loop experimentation system that evaluates model and hardware configurations under a specified release schedule. Researchers can vary the objective, schedule, target and candidate proposer while reusing evaluation and replay. Records preserve comparison conditions and separate valid measurements from deployable candidates. Three case studies demonstrate the system. First, searches and controlled replay on STM32 N657 show how waiting changes model and clock selection. For one architecture, continuous inference favors 800 MHz over 300 MHz at 8.0 versus 17.1 mJ per inference, but additional waiting offsets 97% of this saving in a fitted decomposition. With STOP between releases, 300 MHz uses about 0.3-0.4% less board energy per release at 100 and 500 ms. At a fixed clock, the preferred architecture changes between release periods, which a model with shared waiting parameters cannot produce. Latency predicts continuous energy within a median 1.7% on held-out observations, yet most release-energy error remains in scheduled terms. In paired replay, one of six randomly selected near-latency pairs reverses its continuous-energy order by 5.8 mJ per release. Second, replay on three Arduino targets shows proxy-selected inertial-odometry candidates can fail to fit or use more energy than measured selections, with only 9 of 16 FLOPs-selected candidates fitting the most memory-constrained board. Third, an audio classification study reuses the evaluator with LLM-based candidate proposers, and cross-target replay moves a selected configuration from third to last in energy rank.
comment: 23 pages, 11 figures, 14 tables, including appendices
♻ ★ Belief-Driven Search in Connectivity-Constrained Mobile Sensor Networks with Convergence Guarantees
Multi-agent search and coverage is critical to autonomous exploration, search-and-rescue, and defense applications. This paper investigates settings where a mobile sensor network is tasked with coordinating to discover the position of a stationary target within a bounded region. We develop a Bayesian update scheme where agents iteratively combine local uncertainties derived from their sensor measurements through inverse variance weighting to get a better global understanding of where the target may be located given some prior belief. Network agents' trajectories are governed by a primal-dual gradient update rule that simultaneously minimizes the expected sensing uncertainty and enforces network connectivity among the agents. We prove that the belief converges to the target under mild planner conditions, with an explicit exponential rate of false-hypothesis rejection and a bound on expected search time. Simulation results show that we can successfully find the target and collapse belief to the ground truth target location while ensuring connectivity of our sensor network. They also empirically confirm our theoretical bounds and planner conditions. Finally, we compare against a naive lawnmower sweeping strategy, showing that our method locates the target more quickly when the prior is informative.
comment: 8 pages, 6 figures, 2 tables. Extended version with full proofs
♻ ★ Uniform Exponential Stability Analysis of Impulsive Linear Time-Invariant Systems on Banach and Hilbert Spaces: Non-Coercive and Coercive Stability Conditions
We consider the uniform exponential stability analysis of infinite-dimensional impulsive systems defined on a Banach or Hilbert space, whose flow is governed by a fixed $C_0$-semigroup generator and whose jumps occur at a prescribed time sequence. While the flow and jump maps are themselves time-invariant, the time-triggered impulses render the propagator a genuinely time-varying evolution family, which is the source of the analysis difficulty addressed here. We combine ideas from hybrid systems theory and infinite-dimensional systems to produce operator-based stability conditions, which can be analytically or numerically checked via convex programming. Necessary and sufficient conditions for the uniform exponential stability of impulsive systems on Banach spaces are obtained in the context of a fixed impulse-times sequence but also of arbitrary, constant, minimum, and range dwell-times using both non-coercive and coercive Lyapunov functionals. Some of those results are then adapted to systems on a Hilbert space and quadratic Lyapunov functionals. As an application, linear switched systems are shown to be an exact special case: reformulated as impulsive systems with unit-norm selector jumps, they inherit non-coercive and clock-dependent dwell-time stability conditions on both Banach and Hilbert spaces. Theoretical and numerical examples are given for illustration, notably on the sampled-data control of time-delay systems.
comment: 90 pages
♻ ★ State space modeling of RLC ladder circuits
Large-scale electrical circuits span a wide range of research topics from highly integrated circuits in microelectronics up to power grids for cities and countries. As the complexity increases significantly by each additional component, we find a need to describe large-scale circuits in a manner easy to understand. In this article, we study cascades of simple circuits consisting of resistors, capacitors and inductors, which we call ladder circuits. Such electrical ladders are common in electronics to design filters and they are also used in other disciplines, for example to describe diffusion or wave phenomena. In particular, we derive linear time-invariant state space models for three simple ladder types and we discuss the specific matrix structures of the large-scale second-order differential equations. Furthermore, we exemplify our findings with simulations to unveil the intrinsic dynamical behavior of each ladder type.
comment: 18 pages, 7 figures, references added, section 4 added, typos corrected
♻ ★ Certified Set Convergence for Piecewise Affine Systems
When all individuals of a system receive the same input, one open-loop input sequence has to steer every possible initial state to a target. We study this problem for piecewise affine systems with bounded disturbances, where every state of an initial box must reach a robustly invariant target. When the reachable set crosses a switching surface, it splits into fragments that follow different dynamics, and existing set-based controllers then give each fragment its own state feedback, which a common input does not allow. Instead, we propose a planner-independent and sound verification framework for common-input control of disturbed piecewise affine systems. Any planner can propose candidate sequences, and the verifier certifies a sequence by following every fragment under the dynamics of its own region. To certify all possible initial boxes in a given range, we partition the range into finitely many cells and certify each cell. On three evaluation systems, every possible initial box in the considered ranges is certified.
♻ ★ Industrial Load Modeling and Optimization for Market-Based Interaction with Power Systems
The transition toward a renewable-dominated power system increases the flexibility needed to balance net-load variations. Industrial loads account for over 40% of the world's electricity consumption and can shift large amounts of demand through automated production scheduling. However, making industrial processes a dependable flexibility resource for power systems remains challenging, because industrial process constraints are non-convex, equipment parameters are often private, and device-level models of thousands of facilities are too detailed for aggregator and power-system optimization. This dissertation builds computationally efficient methods for modeling and optimizing industrial flexibility for power-system interaction. Specifically, this dissertation uses smart meters as the external measurement interface and maps industrial process constraints into a concise flexibility space for electricity-market operation. The resulting chain of mechanism modeling, parameter identification, constraint dimension reduction, and optimal decision-making makes industrial-load participation computationally tractable while retaining precise process models for final execution.
comment: PhD thesis, Tsinghua University, June 2026, 175 pages
♻ ★ Autonomous Precision Milling of Biological Structures via Generic Anatomical Priors and Active Boundary Perception
Autonomous precision milling of biological structures is challenged by incomplete knowledge of target geometry, local material thickness, and critical internal boundaries. Subject-specific preoperative models can address geometric and thickness variations, but static models cannot determine boundary status encountered during execution, while repeated target-specific imaging limits scalability. This article presents a state-adaptive autonomous milling framework that assigns complementary roles to generic anatomical priors and active boundary perception. A generic anatomical prior provides conservative global guidance and is transformed through semantic-guided registration and hybrid vision-force calibration into robot-executable guidance for individual targets. As milling approaches uncertain boundaries, the robot actively probes the remaining structure and uses relative stiffness changes to estimate boundary status and structural detachability. A state-adaptive controller governs transitions between active perception and spatially selective incremental refinement, repeating this cycle until the termination criterion is satisfied. Hierarchical experiments on biological surrogates and in vivo mouse cranial window creation demonstrate accurate anatomical prior transfer, reliable boundary adaptation, and autonomous precision milling of biological structures.
comment: ver 2. 18 pages, 12 figures. Submitted to IEEE Transactions on Automation Science and Engineering (T-ASE) for possible publication
♻ ★ Characterization of the Frequency-Current Curve of First-Order High-Pass Neurons
A spiking model, called the High-Pass (HP) model, has recently been proposed as a minimal excitable primitive. This model is composed of a static activation function, usually a sigmoid, in positive feedback with a high-pass filter. The HP model enters a tonic spiking regime whenever its input lies in the high-gain zone of the sigmoid, where the sigmoid's slope exceeds 1. In neuroscience, the frequency-current (FI) curve is the standard way of characterizing the tonic spiking regime of a neuron. So far, the FI curve of the HP model has been characterized analytically only for specific sigmoids, and has otherwise been computed through simulation. These simulations are computationally heavy, since simulating neurons requires solving a stiff system of differential equations. This paper bridges this gap by analytically characterizing the FI curve of the HP model with a first-order high-pass filter, for a broad class of sigmoid functions. The FI curve of the first-order HP model can be computed by solving two input-independent equations and evaluating two input-dependent integrals, with no need to simulate the model itself. A spiking primitive is said to be Type I (resp. Type II) if tonic spiking starts at a zero (resp. non-zero) frequency. This paper also shows that, for the HP model, the type depends solely on the local shape of the sigmoid at the outer edges of its high-gain zone. A sharp corner, where the derivative approaches a value strictly smaller than 1 from the outside, yields a Type I primitive. A locally twice differentiable corner, with a bounded second derivative and a derivative approaching 1 at the edge, yields a Type II primitive.
comment: 8 pages, 4 figures
♻ ★ Towards Optimal Passive Feedback Control of LTI Systems under LQR Performance
We study state-feedback design for continuous-time LTI systems with a control input and an external input-output pair. Our objective is to determine feedback gains that render the closed-loop system (strictly) passive with respect to the external port while minimizing the standard LQR cost in the disturbance-free case. The resulting constrained optimization problem is intractable due to bilinear matrix inequalities. We analyze the set of passivating gains, showing it is unbounded, possibly nonconvex, path-connected, and contractible. We propose an indirect approach, in which the set of passivating feedback gains is inner-approximated by a compact, convex polytope. A projected gradient flow is employed to compute a gain within this polytope that minimizes the LQR cost. Numerical examples illustrate the effectiveness of the method.
♻ ★ Design and Formulation of a Hydromechanical Fin for AUV Operations and Wave Parameter Estimation
Ocean dynamics play a crucial role in global climate, ecosystems, and human activities, necessitating accurate and efficient methods to characterize ocean currents and waves. This paper presents the development of a novel bio-inspired hydrofoil system for detecting and characterizing ocean currents and waves based on a dolphin's flipper foil design. The prototype's performance was assessed through controlled experiments, demonstrating the system's ability to quickly and accurately orientate itself in the direction of flow. Wave mechanics and a geometric proof were applied to estimate wave parameters such as wave height and current from the hydrofoil's positioning. The proposed hydrofoil system shows potential for use in various marine applications, including oceanographic research, environmental monitoring, and navigation. The bio-inspired hydrofoil system offers a promising approach to ocean current and wave characterization, with the potential to significantly impact our understanding and monitoring of ocean dynamics without greatly impacting vehicle performance or increasing power utilization in operation.
comment: Is not representative of my best work; submitted too early.
♻ ★ A Tutorial on Learning-Based Radio Map Construction: Data, Paradigms, and Physics-Awareness
Radio maps (RMs) provide the digital representation of the wireless propagation environment, mapping complex geographical and topological boundary conditions to critical spatial-spectral metrics that range from received signal strength to full channel state information matrices. The integration of artificial intelligence into next generation wireless networks further necessitates the accurate construction of RMs as a foundational prerequisite for electromagnetic digital twins. This paper presents a comprehensive survey of learning-based RM construction, systematically addressing three intertwined dimensions: data, paradigms, and physics-awareness. From the data perspective, we review physical measurement campaigns, ray tracing simulation engines, and publicly available benchmark datasets, identifying their respective strengths and fundamental limitations. From the paradigm perspective, we establish a core taxonomy that categorizes RM construction into source-aware forward prediction and \added{source-agnostic} inverse reconstruction, and examine five principal neural architecture families spanning convolutional neural networks, vision transformers, graph neural networks, generative adversarial networks, and diffusion models. We further survey optics-inspired methods adapted from neural radiance fields and 3D Gaussian splatting for continuous wireless radiation field modeling. From the physics-awareness perspective, we introduce a three-level integration framework encompassing data-level feature engineering, loss-level partial differential equation regularization, and \added{architecture-level} structural isomorphism. Open challenges including foundation model development, physical hallucination detection, and amortized inference for real-time deployment are discussed to outline future research directions.
♻ ★ Price-Optimized EV and Heat-Pump Scheduling Can Exceed Uncoordinated Feeder Peaks
Electric vehicles and heat pumps add large, time-flexible loads to low-voltage feeders. When each household independently minimizes its own charging cost against the same electricity price, this flexible demand piles into the cheapest hours, and the resulting synchronization can stress the feeder more than uncoordinated operation does. This work quantifies that effect on a rural low-voltage feeder with photovoltaics (97 buses, 250 kVA transformer) in 913 simulation runs that compare seven strategies across four seasons, device penetrations of up to 90 %, 208 price weeks from 2022-2025, and heterogeneous heat pumps. At 30 % penetration of each device, this decentralized cost minimization drives the transformer to 253 % of rated capacity, more than three times the peak of non-optimized Fixed-random charging with the same mobility and charged energy, and 478 % with electric vehicles alone at 90 % penetration, with undervoltage at the far end of the longest feeder arm. It exceeds Fixed-random and Fixed-times charging in every winter, spring, and autumn price week, and each 30 pp of heat-pump penetration raises its peak by about 110 pp, twelve times as much as without optimization. Centralized peak minimization removes all demand-driven violations. In the heat-pump-free summer case at 60 % photovoltaics, midday reverse power flow sets a transformer floor of 128.9 % that no strategy undercuts, since no commuting electric vehicle is at home at the export peak. With a time-variable grid fee added to the spot price, the weekly peak of decentralized cost minimization falls into the reduced-fee window in 99 of the 104 fee-active price weeks, compared with 53 without the fee, while keeping at least 89.8 % of its height, so the fee relocates rather than removes the synchronization. The simulation and optimization code is available at https://github.com/lukas-wagner/SynchronizationEffectsAnalysis.
comment: Accepted at 2026 IEEE IES ONCON
♻ ★ First-order plants are globally asymptotically stable under antithetic feedback
We show that first-order linear plants are globally asymptotically stable under antithetic feedback. This is shown by constructing an explicit smooth, convex Lyapunov function and by appealing to a Krasovskii-LaSalle invariance principle.
comment: 3 pages, 1 figure. Submitted
♻ ★ Repeated Binary Direct Collinear Impacts Under Incremental Contact Laws With Residual Indentation: A Hybrid Systems Formulation
Incremental contact laws specify the normal contact force through a differential relation, possibly carrying an internal state, driven by the indentation and its rate. The force may vanish at a nonzero indentation, through plasticity, an elastic aftereffect, or, under some laws with a viscous component, a load drawing the bodies apart rapidly, leaving a residual deformation at separation. Such a deformation sits uneasily within rigid body dynamics, which admits none. The tension is tolerable when the indentation is small relative to the bodies, so that it may be carried constitutively rather than geometrically. Even then, the law alone does not determine the interaction: because force and indentation no longer vanish together, conditions for the commencement and termination of contact must be supplied separately, as must the fate of the deformation and internal state at separation, both absent from the equations of motion. This article formulates the repeated direct collinear impact of two convex bodies under external forces as a hybrid dynamical system. The contact interface is modeled as a massless element carrying the contact law and its state, coupled to the bodies through relative velocity and an interaction force dictated by the contact state. Consequently, all switching and resets are confined to the interface, leaving the geometry and equations of motion of the bodies unaltered. The principal analytical properties of the formulations are established, among them passivity and completeness; the branching of solutions at the onset and termination of contact is examined. The framework is demonstrated through numerical simulations.
comment: 34 pages, 6 figures; v1-v7: a variety of amendments and corrections; v6-v7: further analytical properties
♻ ★ Anytime-Valid Runtime Monitoring of Learned Barrier Certificates
Learned barrier certificates provide a principled mechanism for establishing safety of stochastic systems, but their validity may deteriorate during deployment when the underlying dynamics differ from those used for training. We propose an anytime-valid runtime monitor that audits a learned barrier certificate using observed closed-loop trajectories. The monitor is intended to detect loss of statistical support for the deployed barrier condition; it does not by itself guarantee detection before unsafe entry or safety of a subsequent intervention. Specifically, the conditional barrier-drift condition is formulated as an episode-wise sequential hypothesis test, and observed barrier increments are transformed into a nonnegative test supermartingale, yielding finite-sample false-alarm control under arbitrary stopping times. Inspired by finite-horizon safety verification, we further introduce a deterministic temporal shaping of the monitoring boundary, making the monitor more conservative early in a prescribed deployment horizon while preserving anytime-validity. The monitor is designed to detect statistical evidence that the safety certificate is no longer consistent with the observed dynamics, rather than to predict unsafe entry. To assess the monitor empirically, we evaluate empirical alarm behavior under valid and invalid certificates. Code and data are available at https://github.com/FormalCert-AI/runtime-barrier-monitor.
comment: Update-v2: add stepwise falsification guarantee theorem, VLA experiments, and acknowledgment
♻ ★ The Smart Buildings Control Suite: A Diverse Open Source Benchmark to Evaluate and Scale HVAC Control Policies for Sustainability
Commercial buildings account for 17\% of U.S. carbon emissions, with roughly half of that from Heating, Ventilation, and Air Conditioning (HVAC). HVAC devices form a complex thermodynamic system, and while model predictive control and reinforcement learning have been used to optimize control policies, scaling to thousands of buildings remains a significant unsolved challenge. Most current approaches are over-optimized for specific buildings and rely on proprietary data or hard-to-configure simulators. We present the Smart Buildings Control Suite, the first open source interactive HVAC control benchmark with a focus on solutions that generalize across building. It has 3 components: real-world data from 11 buildings over 6 years, a lightweight data-driven simulator, and a modular Physically Informed Neural Network (PINN) building model. The buildings span multiple climates, management systems, and sizes, and both the simulator and PINN easily transfer to new buildings, ensuring solutions using this benchmark are robust to these factors and only reliant on fully scalable building models. This represents a major step towards scaling HVAC optimization from the lab to buildings everywhere. To facilitate use, our benchmark is compatible with the Gym standard, and our data is available at our webpage. Code at https://github.com/google/sbsim
comment: Accepted to Neurips 2026
♻ ★ Passive Identification of Unequal Phase Grid Impedances by Phase-Domain Source Cancellation
Passive grid impedance estimators that return a single resistance--inductance pair cannot represent a grid whose phase impedances are unequal. This paper identifies a separate pair for each phase from recorded point-of-common-coupling (PCC) voltage and current without injected perturbations. In each phase, the unknown grid-source voltage is cancelled by a periodic difference (PD) or by a second-order-generalized-integrator notch difference (SOGID), and moving-window integration (MWI) replaces the current derivative by an integral and an endpoint feature. The resulting two-parameter regression requires a period or notch-frequency estimate but no phase-locked-loop angle. A voltage-balance analysis gives the source residual left by frequency mismatch and bounds the parameter error by the ratio of this residual to the conditioning of the two features. In the unequal-phase interval (8.3--9.8~s) of a 10-s EMTP grid-forming record, SOGID-MWI attains phase-averaged R/L errors of 5.09/0.33\%; a shared value has a minimum error of 41.67\% for the phase ratio 1:0.5:2. On a 50-s constant-impedance grid-following record, the errors are 0.56/0.06\% over the 20--30-s synchronization disturbance. In a controlled 40-dB noise study, a noise-referenced update rule reduces the white-noise GFL error from 55.9/65.0\% to 12.3/11.5\%.
♻ ★ Informed Posterior Sampling: More Efficient Online Learning with Few Offline Demonstrations in Average-Reward MDPs NeurIPS 2026
We study efficient online reinforcement learning in the infinite horizon setting when there is an offline dataset to start with. We assume that the offline dataset is generated by an expert but with an unknown level of competence, i.e., it is not perfect and not necessarily using the optimal policy. We show that if the learning agent models the behavioral policy (parameterized by a competence parameter) used by the expert, it can do substantially better in terms of minimizing cumulative regret than if it does not do that. We establish an upper bound on regret of the exact informed PSRL (iPSRL) algorithm that scales as $\widetilde{\mathcal{O}}(\sqrt{T})$. This requires a novel prior-dependent regret analysis of Bayesian online learning algorithms for the infinite horizon setting. We then propose the informed RLSVI (iRLSVI) algorithm to efficiently approximate the iPSRL algorithm. Empirical results show that small offline expert datasets are sufficient for iRLSVI to outperform online-only algorithms on hard exploration domains.
comment: 47 pages, 16 figures. NeurIPS 2026
♻ ★ Spurious-Free Lithium Niobate Bulk Acoustic Wave Resonator with Grounded-Ring Electrode
High-performance piezoelectric resonators are promising energy-storage elements for piezoelectric power conversion because of their compact footprint and low loss at frequencies where conventional magnetic components become bulky and inefficient. However, their practical use is often limited by the trade-off between a high electromechanical coupling coefficient ($k^2$) for wide-band operation and the emergence of spurious acoustic modes that limit the resonators' inductive bandwidth. This work reports a spurious-free thickness-extensional (TE)-mode bulk acoustic wave (BAW) resonator in single-crystal lithium niobate (LN) based on a grounded-ring electrode architecture. The proposed structure is analyzed through simulation and experimentally validated using electrical characterization and laser Doppler vibrometry (LDV). The results show that the grounded ring modifies the effective boundary conditions of the acoustic device, enabling a piston-like modal response that suppresses lateral spurious modes across the inductive band. The demonstrated device operates at 10.14 MHz and achieves an electromechanical coupling coefficient of 29.6%, a maximum in-band Bode quality factor ($Q_\textit{Bode}$) of 5230, and a figure of merit (FoM, $Q\cdot k^2$) of 1548. These results establish the grounded-ring TE-mode LN BAW resonator as a practical platform for piezoelectric power conversion and a broader design approach for realizing high-performance spurious-free acoustic resonators.
comment: 15 pages, 17 figures
♻ ★ A Hybrid Framework for Explainable Aviation Risk: Causal Chain Extraction from Safety Reports with Physics-Based Failure Quantification and Interventional Analysis
Aviation authorities hold decades of accident reports whose narratives record how investigators reconstructed each accident. Computational work on these archives has mostly sorted reports into fixed taxonomies. In this work, we propose a hybrid framework in which a text-based corpus identifies the leading factors of a failure mode. Physics-based models then quantify how likely that failure is at a stated operating point. Ordered causal chains extracted from 56,202 American accident reports with a local large language model aggregate into a causation knowledge graph. On an evaluation sample of 2,000 reports, the chains recover 90.8% of investigator-flagged causes. Each physical factor whose drivers the record carries receives an occurrence model calibrated to the accident population. A leaky noisy-OR fitted to the corpus combines the factors into a failure mode probability that supports intervention. For engine power loss, the carburetor icing model separates 1,205 icing accidents from other accidents using temperature and dewpoint alone, reaching 0.61 in area under the receiver operating characteristic curve against a density ratio ceiling of 0.64. Routine weather hours give no valid control, because flying concentrates in fair weather. On later years the physics adds 0.03 to that area for occurrence and nothing for severity, so its value lies in interpretability. On four named accidents, abduction gives a probability of necessity of 0.29 to 0.50 for the heat omission, under monotonicity. The Fast Causal Inference algorithm does not license the largest backdoor adjustment on the learned networks.
comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible
♻ ★ A Quantitative Approach to Reach-Avoid Reachability Analysis for Discrete-Time Nonlinear Control Systems
A reach-avoid specification requires a dynamical system to reach a target set of states while avoiding a failure set. Reachability analysis partitions the state space into parts starting from which the specification can be satisfied by following some control signal and those from which it cannot despite the best control effort. In this paper, we present a quantitative approach for reachability analysis to analyze the satisfaction of avoid and reach-avoid specifications for discrete-time nonlinear control systems. In such an approach, a value function assigns a value to each state that determines how robustly the system satisfies or violates the specification starting from it and following the optimal control signal. However, computing such a value function is difficult. Instead, our approach constructs discrete abstractions on which it performs value iteration to compute sound bounds on the desired value function. The resulting lower bound enables a sound under-approximation of the reach-avoid set (RAS) and, in the avoid case, a sound over-approximation of the backward reachable set (BRS). When the control set is finite, the upper bound additionally enables a sound over-approximation of the RAS and, in the avoid case, a sound under-approximation of the BRS. Moreover, the obtained bounds provide quantitative guarantees on safety margins and, in discounted reach-avoid problems, on safe target-reaching times. When the control set is finite, they also bound failure-reaching times. Our approach further refines the abstraction by splitting the grid cells that could not be classified as within or outside the BRS (or RAS) to improve the bounds. We evaluate our approach on five systems with up to five-dimensional state spaces.
♻ ★ Computing GR(1) Assume-Guarantee Contracts
Assume-guarantee contracts provide a set of algebraic operations that enable the efficient compositional design and analysis of complex engineering systems. The explicit computation of these operations at component interfaces is currently supported only for some logics lacking a notion of time. This paper develops theory and algorithms to compute these operations when the contracts are expressed in the GR(1) modulo theory. The key challenge is the computation of two quantifier elimination problems for temporal formulas. We develop context-aided variable elimination procedures on GR(1)-modulo formulas, establish their soundness, characterize subproblem-wise optimal solutions within an admissible class of formulas, and use the procedures to compute contract operations, including composition and quotient. We implement the resulting framework in Pacti, automating compositional reasoning over GR(1) contracts. Finally, we demonstrate the framework on an automated emergency-braking case study, where we compute system-level contracts and expose compatibility conditions arising from the interaction of component specifications. Experiments on synthetic families show that the cost is governed by the number of eliminated variables and liveness conjuncts, and a comparison with exact automata-based elimination shows complementary scaling.
comment: 55 pages, 10 figures
♻ ★ Holonic Graceful Transitions Across Centralized, Decentralized, Distributed, and Local Control in DER-rich Cyber-Power Distribution System
The increasing penetration of distributed energy resources (DERs) in distribution system necessitates adaptive coordination frameworks. These frameworks must remain optimal during normal operation and resilient under cyber physical disturbances. Existing coordinated control approaches are typically deployed as static architectures with limited ability to adapt when communication degrades, local instability emerges, and operating conditions become spatially heterogeneous. This work addresses the gap by proposing a DER service-independent edge autonomous holonic adaptive coordination framework. In this framework, each DER controller executes its own control actions and transitions among centralized, distributed, decentralized, and local autonomous coordination modes without always relying on static coordination commands from the grid operator. The framework preserves coordination continuity across modes by retaining local controller states while reconfiguring only the coordination topology, information exchange pattern, and fallback action associated with the active DER service. Graceful transitions are enabled through dwell timers, rate limiting, and safety overrides to prevent dynamic instability during mode changes. Volt VAR control is used as a representative distribution automation application to validate the proposed architecture in a cyber-physical Hardware-in-the-loop (HIL) testbed. The proposed approach is evaluated under diverse cyber-physical event-based scenarios, showing region-confined adaptation under localized disturbance, reduced coordination traffic during coordination switching, and node-confined mitigation under cyber attack through edge anomaly detection and neighbor-corroborated impact estimation.
comment: Need full revision of the paper due to strong comments from reviewers
♻ ★ Neural Network-Enhanced Frequency-Constrained Optimal Power Flow With Worst RoCoF and Frequency Deviation
Frequency-constrained optimal power flow (FCOPF) should keep the rate of change of frequency (RoCoF) and the frequency deviation (FD) within their limits after a specified generator out-age. The linearized frequency constraints commonly used in FCOPF are derived from aggregated system frequency response (SFR) models, which describe the center of inertia (COI) fre-quency with simplified governor dynamics and can underesti-mate the frequency excursions experienced by individual genera-tors. To address this issue, this paper develops a neural network-based frequency predictor (NN-FP) trained on electromagnetic transient (EMT) simulations in PSCAD/EMTDC. The NN-FP predicts the worst RoCoF and the worst FD over the monitored generators, namely the generators at all buses other than the contingency bus. The trained NN-FP is reformulated and embed-ded into FCOPF with tightened frequency limits obtained from the test set prediction errors of NN-FP, leading to the proposed NN-FCOPF. The proposed model is compared with a traditional optimal power flow and a linearized FCOPF. EMT validation of 24-hour dispatches on the IEEE 9-bus and 39-bus systems shows that NN-FCOPF satisfies both limits in all hourly cases for the specified contingency, whereas the linearized FCOPF violates the limits in some hours because it underestimates the worst genera-tor RoCoF and FD.
♻ ★ Computationally Constrained First-Order Prediction-Correction Approach for Time-Varying Convex Optimization
Time-varying optimization methods often predict the locations of time-varying optima, then sample the objective function being minimized, and then compute corrections to those predictions. However, existing methods typically do not account for settings in which the number of computations is limited. Therefore, in this work we formulate a prediction-correction algorithm that is designed to obey a budget constraint on the total number of iterations it can execute over a given time horizon. We first identify a threshold on the average number of computations between samples of the objective, above which a bound from prior work certifies contraction of iterates toward a minimizer. Below this threshold, we introduce a finite-horizon contraction certificate, which we use to give budget conditions under which the tracking error admits an explicit bound that decays at a prescribed rate. We also present an algorithm to determine how many prediction and correction steps should be executed for each sample of the objective, while obeying a given budget on the allowable number of computations. Empirically, we show in a simulation that our approach reduces cumulative tracking error by 45.5 % relative to a comparable baseline approach.
comment: 8 pages, 2 figures, Submitted to ECC
Robotics
★ Compute Regeneration through Re-Productization: Reorganizing Retired Edge AI Silicon into a Delivered Multilingual Service Robot
Useful edge hardware can lose product support while retaining substantial computation. We treat this as systems design: compute regeneration here means re-productization, retaining the compute module while redesigning the surrounding product organization. Our case is GEdgeBot, a multilingual service robot built around a retired 32 GB Jetson AGX Xavier. A source-grounded inventory shows the box alone supplies inference, while the combined system exposes five on-device AI services, a seven-language speech chain, three sensing domains, and one deterministic seven-byte control boundary. AI and orchestration stay on the Xavier; finite motor-pulse generation on an STM32F407. Audits cover the six-mode control state machine, readiness barriers and sound orientation. Historical single-prototype measurements report 1,782 MB GPU memory with YOLO, Whisper-small, and NLLB-200 resident, a direct-control p99 deviation rising from 0.24 to 49.93 ms under concurrent AI load, and six of six trigger-to-follow demonstrations at 4.35 s mean latency, with conversational voice unavailable. A November campaign restores an isolated inference stack, re-measures AI workloads, and repeats control-frame runs. Driving two execution organisations with the same 6,000 measured 50 ms frame intervals shows the finite-pulse boundary removes host jitter from the pulse train but not the command start phase; its stopping bound is host-enforced, while the MCU 16-bit pulse field is unclamped. A six-check static audit exposes a host--MCU queue mismatch, and a 14-scenario replay executes the audited main() dispatcher across six modes with zero physical device opens and 54 contract-checked frames. Results support an existence claim and a reproducible architectural hypothesis, not fleet-level reliability. The 208 direct-control ratio remains unreplicated; MCU waveform capture and on-site fault injection remain open.
★ SurgTAP-3D: A Large-Scale Cross-Surgery Video Dataset with Contact-Aware 3D Motion Tracking
Understanding 3D scene dynamics in minimally invasive surgery (MIS) is a fundamental step toward physically intelligent and autonomous surgical systems. However, existing public datasets offer limited scalability and diversity in real-world clinical scenarios. Moreover, annotations typically focus on either tissue or instruments alone, overlooking tool-tissue interactions. This gap limits applications such as augmented reality guidance and autonomous surgery driven by vision-language-action (VLA) models or world models, which demand large-scale, high-quality 3D supervision. In this work, we construct SurgTAP-3D, a large-scale in-vivo pseudo-3D surgical motion tracking dataset and benchmark, comprising 520K frames, 4,549 action clips, and 1.4M 2D/3D point trajectories from diverse robotic and manual procedures under unified action categories. To make annotation scalable, we design a unified framework for surgical 3D point tracking that converts uncalibrated monocular surgical videos into structured 4D annotations through contact-region prompting, active-tool segmentation, dynamic bundle adjustment, and global 4D refinement. This pipeline adapts foundation priors to surgical videos and produces temporally consistent trajectories, pseudo-scale geometry, camera poses, and instrument masks. The dataset captures realistic challenges inherent to surgical environments, including occlusions, smoke, bleeding, and rapid instrument motion. We establish a comprehensive benchmark on SurgPose, SurgT, and STIR, covering both instrument and tissue tracking. We demonstrate instrument displacement estimation, deformable surface reconstruction, and foundation-model fine-tuning. Extensive experiments and analysis demonstrate the effectiveness of our dataset and annotation framework for 3D surgical point tracking, reconstruction, and surgical action understanding. Homepage: https://wrld.github.io/surgtap-3d/
comment: Accepted to IEEE Transactions on Robotics (T-RO)
★ A Physical Testbed for Self-Driving Laboratories: Comparing Bayesian and Agentic Optimization in a Minimal Closed-Loop Color Formulation Task
Self-driving laboratories (SDLs) integrate laboratory robotics and automation with AI-driven decision-making to close the design-build-test-learn (DBTL) cycle. However, physical testbeds that combine off-the-shelf hardware and AI into a functioning closed-loop workflow remain in early stages of development, and their integration with commercial laboratory automation is comparatively less developed. In this work, we develop a physical testbed that assembles and integrates commercially available laboratory automation instruments and demonstrate continuous, closed-loop, autonomous operation over 24 cumulative hours of unattended experimentation. We validate this platform on a minimal closed-loop discovery task in which the system autonomously formulates aqueous dye mixtures to match a target absorbance spectrum. The system completed six independent campaigns, each consisting of 48 closed-loop iterations over approximately four hours. In every campaign, the system autonomously selected, dispensed, and measured each mixture, with an AI-driven decision-making algorithm determining subsequent formulations without human intervention. Using this testbed, we compared the performance of two AI-driven decision-making algorithms, traditional Bayesian optimization (BO) and an agentic approach, in selecting mixtures under a fixed total-volume constraint. The agentic optimizer matched target absorbance spectra using fewer iterations than BO, with reductions up to 52%, and converged on a solution for one target that BO failed to match within 48 iterations. These results demonstrate a reference physical platform for SDLs that can evaluate both established and emerging AI strategies, including large language model (LLM)-directed optimization, before their deployment in real-world applications where experiments may be costly and time-consuming to prepare.
★ Streaming Failure Forecasting: VLA Failure Monitors That Keep Learning After Deployment
Failure monitors for vision-language-action (VLA) policies are trained offline and then frozen. After deployment, lighting, objects and cameras drift, and the policy itself is updated or moved to new tasks. On recorded real-robot runs, after a switch of task or policy version, a frozen monitor fit on the old task ranks failures little better than chance. We present Streaming Failure Forecasting (SFF), a monitor that keeps learning from the one label every episode provides: whether it succeeded. SFF keeps the offline monitor's network fixed and refits only its last linear layer after each episode, at 2.18 ms per policy call on a Jetson CPU core. Its alarm threshold moves only after successes, which in simulation kept false alarms near target except when two drifts came at once. With every setting locked before testing, SFF ranks failures better than the same monitor kept frozen on 4 of 5 simulated test suites under unannounced drift, and above resetting on detected drift on every suite. Carried with its locked learner settings to 434 recorded runs of four VLAs on real SO-101 arms, replayed across unannounced switches of task or policy version, it ranks failures better than the frozen monitor for every VLA and, in an exploratory comparison, above a full reset at the true switch. A deployed failure monitor need not stay frozen: refitting its readout from outcomes is cheap and helps after the changes we tested.
comment: 25 pages, 13 figures, 9 tables. A short version was accepted at the Sim2Real2Field and EEAI workshops at CoRL 2026. Project page: https://sff-research.org
★ Brain-Aligned Latent Actions: Imposing Premotor Cortex Geometry on a Robot's Action Vocabulary
Robot policies increasingly act through a small vocabulary of discrete latent actions learned from unlabeled video, yet training leaves open which motion receives which code. We find this choice highly unstable: two independently trained runs of Moto's latent motion tokenizer assign the same robot clips to nearly unrelated codes (adjusted Rand index 0.001), and evaluating one tokenizer in bf16 instead of fp32 reassigns 74% of its tokens. We use this freedom to impose the representational geometry of macaque ventral premotor cortex (PMv), recorded while a monkey drew shapes, on the tokenizer's code assignment, with a straight-through similarity loss trained alongside its reconstruction objective on robot video. In one training run per target, on five shape categories the loss never saw, the PMv-trained tokenizer reaches a representational similarity of 0.52 with PMv, against 0.20-0.26 for released and no-prior tokenizers and 0.42 for the 95th percentile of random kinematic encoders. Kinematic, language, and random targets do not transfer (-0.17 to -0.03). Two results limit the claim: a hand-picked kinematic descriptor, the stroke's initial direction, matches PMv better (0.65), and a label-shuffled PMv control scores 0.41. We specify the downstream manipulation test, on a CALVIN pipeline that reproduces Moto's published results, before running it.
comment: 16 pages, 3 figures, 6 tables. Robot manipulation results (CALVIN) will be added in a later version
★ HINT-VLM: Humanoid Interactive Navigation over Rough Terrain with Embodiment-Grounded Vision-Language Model
Humanoid robots have achieved significant progress in locomotion and manipulation, enabling traversal over rough terrain and richer interaction with the environment, such as moving obstacles. However, most traditional navigation methods assume non-interactive environments and focus only on traversal. While Navigation Among Movable Obstacles (NAMO) approaches relax this assumption, prior efforts have largely focused on flat environments and reactive interaction, rather than predicting object movability and planning ahead. These limitations become critical in outdoor rough-terrain scenarios, where robots must jointly reason about terrain traversability and object movability. In this work, we address NAMO for humanoid robots in cluttered outdoor rough terrain by proposing a hybrid planning framework that combines semantic priors from a Vision-Language Model (VLM) with embodied robot interaction data through a Gaussian process (GP) residual model. The GP corrects the VLM-derived movability prior to produce embodiment-grounded probabilistic movability estimates, which are then incorporated into hierarchical uncertainty-aware planning. The proposed approach allows humanoids to jointly leverage locomotion and manipulation for efficient and safe navigation, representing the first solution to NAMO in rough terrain through predictive movability reasoning and probabilistic VLM integration.
★ Enhancing Balance and Posture in Material Handling Tasks with a Wearable Robotic Tail
Frequent heavy lifting is linked to musculoskeletal strain, with lower back pain affecting up to 40% of manual labourers. Inspired by animals using tails for balance, we developed a wearable robotic tail as a dynamic counterbalance. Ten participants performed an imitated shelf loading/unloading task across three conditions: active tail, passive tail, and no tail. Kinematics, centre of pressure, and muscle activation (gastrocnemius, erector spinae, tibialis anterior, biceps brachii) were measured. Compared with no tail, the active tail achieved >20 mm reduction in centre of pressure peak displacement, 4.3 degree reduction in trunk motion, and >9% and >20% reductions in erector spinae and gastrocnemius activity, respectively. The passive tail produced smaller reductions, suggesting passive counterbalancing helps while active movement augments it further. Biceps brachii activation increased with the active versus passive tail. The ankle kinematics and tibialis anterior activation were unchanged. These findings support wearable robotic tails for improving balance during material handling.
comment: 16 Pages, 5 Figures
★ FlowEval: How Much Adaptation Does an Embodied Foundation Model Need at Deployment?
Large-scale pretraining is reducing the amount of adaptation required to deploy embodied foundation models, but it is not guaranteed that a pretrained model already provide the level of performance required in every deployment. Practitioners therefore need to understand how well a pretrained model matches a target deployment and how much adaptation data is left to be collected. We study whether a small set of successful target-domain demonstrations can reveal whether and how much target-domain data will be required before any adaptation. We introduce FlowEval, a set of training-free metrics that use the flow-matching objective to probe an embodied foundation model with a set of demonstrations. FlowEval quantifies predictive disagreement on successful target demonstrations to assess how well a pretrained model fits a target deployment. Across four embodied foundation model families: Pi-0.5, MolmoAct2, GR00T, and DreamZero, FlowEval distinguishes seen from unseen deployments across real-world and simulated manipulation. Further, by finetuning Pi-0.5 and GR00T on 17 RoboCasa365 manipulation tasks, we showed that FlowEval correlates with how success rate improves as additional adaptation data is provided. Together, these results suggest that FlowEval can serve as a lightweight, training-free diagnostic of how much model--deployment adaptation may be required.
★ PA-SaR: Perception-Aware Learning, Control, and Coordination for Multi-Robot Search and Rescue
Autonomous SaR requires robots to explore unknown or partially known environments, avoid hazards, coordinate team activities, and guide discovered victims to safety. Improving individual capabilities of robots does not necessarily improve mission outcomes: safer motion can restrict the exploration behavior, faster discovery can increase the rescue demand, and effective guidance depends on victim responses. We introduce PA-SaR, a mathematical and architectural framework for coordinating heterogeneous local control, learning-based exploration, supervisory task allocation, and adaptive approach-lead victim guidance around mission-level objectives. PA- SaR distinguishes environmental ground-truth, robot-specific perceptions, and simulated victim behavior. This provides a common basis for integrating complementary mechanisms and analyzing their interactions across mission stages. We demonstrate the framework through single- and multi-robot simulation studies incorporating model-predictive and learning-based exploration, perception-based safety shielding, event-driven task allocation, and adaptive victim guidance. The results show trade-offs between collision avoidance and search completion, indicate that supervisory allocation improves search completion beyond information sharing alone, and demonstrate that increased exploration does not necessarily yield more rescues. Adaptive guidance increases rescues relative to continuous leading, without a detected improvement in mean or final victim trust or a reduction in mortality. These findings support evaluating local control, team coordination, and evacuation as related but distinct aspects of autonomous SaR within the considered simulation environments.
★ ColNav: Uncertainty-Aware Spacetime Collision Checking with Bounding Volume Hierarchies
Safe navigation for mobile robots in dynamic and uncertain environments requires planning paths that avoid collisions with other moving targets. Typically this involves forward propagating the robot motion and checking for collisions at sampled points in space and time. Such methods can miss collisions between sampled points and are not robust to uncertainty in the state estimates. We propose ColNav, a robust collision check formulation with geometric volumes. The key aspect of our approach is encoding the uncertainty in the targets' and the ego robot's positions and velocities into spacetime tubes, in two variants that consume bounded-set and Gaussian uncertainty descriptions, respectively. The tubes are organised in bounding volume hierarchies, so that a single tree traversal covers the entire horizon, rather than testing sampled time instants, and returns a robust geometric certificate together with an estimate of the time to collision. Static and dynamic obstacles share the same representation. Our approach reaches the goal with a 100% success rate under prediction model mismatch, where the best baseline reaches 65%, and maintains up to three times the clearance in dense traffic. We also validate real-time operation on a physical robot.
★ ScanDreamer: A Map-Grounded World Model for Quadrotor Navigation
Quadrotors inspecting buildings and tunnels must navigate unknown clutter with a forward-facing depth camera whose limited field of view constrains scene understanding and collision-avoiding navigation. Recent world-model agents learn navigation through imagined experience (e.g., AirDreamer), but remain data-intensive and often lack explicit geometric memory and awareness of what the camera can observe. We propose ScanDreamer, a map-grounded world model that constructs an online 2D scan-memory map from depth images and vehicle pose. Our world model, actor, and critic are conditioned on this map, which also defines a geodesic progress reward. Moreover, a forward-cone penalty discourages horizontal motion outside the camera's viewing direction. During imagination, ScanDreamer propagates the quadrotor pose over a frozen map and derives progress and wall-contact signals from the mapped geometry. We train ScanDreamer for 0.8 million environment steps (5.5 h on one GPU) and evaluate it in NVIDIA Isaac Sim over 1560 episodes spanning unseen layouts and parameter ranges. Across three seeds (mean $\pm$ SD), ScanDreamer achieves 96.3 $\pm$ 0.3% success with 3.6 $\pm$ 0.3% collision, compared with 59.6 $\pm$ 3.0% success and 28.8 $\pm$ 2.8% collision for AirDreamer at the same online-interaction budget. Removing the forward-cone penalty increases collisions to 9.9 $\pm$ 3.1%. These results show that combining explicit map memory, map-grounded imagination, and perception-aware motion substantially improves navigation performance and data efficiency. The source code is available at https://github.com/WANGSHUAIzzzz/ScanDreamer.
comment: 8 pages, 6 figures, 3 tables. Submitted to AICAP 2027; decision pending
★ Kinora: A Versioned Workflow for Text-to-Motion Generation, Robot Retargeting, and Interactive Validation
Motion generation tools produce candidate trajectories, but interactive robot authoring also requires recoverable execution, traceable edits, explicit geometry, and model-specific validation. We present Kinora, a system that connects text-to-motion generation, a motion library, local agent tools, robot retargeting, and trajectory refinement through versioned artifacts. The current release uses an independently specified 52-joint authoring rig and procedural skeleton previews, separating the distributed human geometry from externally installed generation models and robot assets. Human-motion edits preserve frames outside a selected interval, while robot edits create immutable candidates whose reports govern acceptance and export. We evaluate server 0.5.1 and Gadget 0.15.2 with 156 core regression test executions, 49 Gadget executions, four CUDA generation/editing cases, a 30-frame G1 retargeting check, and a 3,600-frame synthetic G1 fixture paired with a suspended negative control. All four CUDA cases passed their numerical gates; the turning splice nevertheless retained a 14.3 cm support-foot drift proxy warning. The G1 positive fixture completed bounded simulation replay, whereas the suspended control was quarantined before replay. These results support selected integration and preservation contracts. They do not establish semantic-quality improvements, population-level feasibility, or hardware safety.
comment: 10 pages, 1 figure, 3 tables. Technical report on Kinora server 0.5.1 and Gadget 0.15.2. Code and release artifacts: https://github.com/PoirotJin/kinora
★ Qubit-to-Motion: Traceable Action Provenance in Hybrid QRL for Robotic Bin-Packing Sequencing
Hybrid quantum reinforcement learning has been explored for robotic decision-making, but evaluations often emphasize task success, return, or convergence, leaving unresolved whether quantum circuit outputs influence constrained actions and remain traceable through robot execution. We present \texttt{Qubit-to-Motion}, an evidence-centered framework for support-constrained sequencing and simulated robotic placement. Four Pauli-$Z$ readouts provide the sole learned input to the actor and critic, while a support mask enforces admissible object selection. Each decision is recorded and linked via step indices and SHA-256 fingerprints to a sealed route, deterministic Cartesian and inverse-kinematics waypoints, prescribed joint motion, and MuJoCo contact replay. A frozen checkpoint is evaluated across five deterministic 30-object scene variants. All 150 placements satisfy support, containment, and settling criteria. A fixed-state zero-readout intervention changes $90.0\pm0.0\%$ of decisions, demonstrating dependence on the joint circuit representation. Mean macro travel is $34.537\pm0.206$~m, inference requires $8.60\pm1.27$~ms per route, and replay achieves $8.764\pm0.220~μ$m mean final position error with 0.356~mm maximum interpenetration. These results establish reproducible decision-to-contact traceability. The implementation is available at \href{https://anonymous.4open.science/r/Qubit-to-Motion-Quantum-Reinforcement-Learning-F7DC}{\textcolor{linkpink}{the anonymous repository}}.
★ Concurrent Learning-Based Inertial Parameter Estimation and Control of Quadrotors
In this letter, we consider trajectory tracking control and inertial parameter estimation of a quadrotor UAV whose center of mass (CoM) does not coincide with its geometric center. Onboard payloads like an IMU, a camera, or a battery are not necessarily placed at its geometric center. Moreover, during the flight these sensors might move because of external disturbances and cause a CoM offset. Thus, these events and equipment introduce an offset and cause the inertia matrix to be non-diagonal. We develop an integral concurrent learning (ICL)-based geometric controller that estimates the mass, the CoM offset, and the full inertia matrix online, without an accelerometer or numerical derivatives, and tracks a given trajectory corrected by the offset estimated online. We prove that the estimated parameters converge to their true values under a finite excitation condition. We validate the effectiveness of the proposed algorithm using a simulation example.
comment: 6 pages, 4 figures. To be submitted to IEEE Control Systems Letters (L-CSS) with the CDC 2027 option
★ Kimodo-Policy: Learning Kinematic Action Models for Humanoid Loco-Manipulation
Humanoid loco-manipulation requires continuous coordination of high-dimensional whole-body motion to realize task objectives. While vision-language-action models and world action models transfer semantic and visual-dynamics priors to robot control, these priors do not necessarily capture the structure of robot motion. We introduce Kinematic Action Models (KAMs), a paradigm that treats whole-body motion as a direct pretraining target and uses a pretrained motion generator as the primary action-generation backbone. KAMs ground learned temporal and cross-body coordination priors through task conditioning, visual observations, and execution feedback. We present Kimodo-Policy as an initial instantiation, combining a frozen Kimodo text-to-motion backbone with lightweight ControlNet-based visual conditioning. A root-body-hand hierarchy coordinates locomotion and manipulation, while motion-history conditioning and repeated action-chunk prediction bridge motion synthesis and closed-loop control. Following large-scale loco-manipulation pretraining, the policy achieves the highest overall success among compared policies on multi-task HumanoidArena (72.14%), competitive performance on SIMPLE, and 60% mean success across eight real-world tasks, with 678M parameters (92M trainable). Ablations support the contribution of the pretrained motion backbone. Together, these results provide initial evidence for motion generation as a complementary foundation for humanoid action learning.
comment: 23 pages, 14 figures, 4 tables. Project page: https://kimodo-policy.github.io/
★ K-PWM: Control-Oriented Structured Bilinear World Models under Partial Observation NeurIPS 2026
World models learn internal simulators of environment dynamics for prediction, planning, and control, yet learning them from limited, noisy, and partial observations remains challenging. While much progress has focused on pixel-based observations, many physical systems instead provide vector-valued measurements that reveal only part of the underlying state. We introduce K-PWM, a probabilistic world model for prediction and control under partial observation. K-PWM couples Koopman-bilinear latent dynamics with a nonlinear observation decoder and performs probabilistic latent-state inference through the learned model via extended Kalman filtering and smoothing, without a separately trained inference network. Its bilinear structure enables exact propagation of the maintained Gaussian belief through the latent transition and efficient rollouts for uncertainty-aware model predictive path integral (MPPI) control. For training K-PWM, we present a generalized expectation conditional maximization (GECM) procedure with closed-form updates for the generators, noise covariances, and latent prior, complemented by gradient-based decoder learning. A Koopman-consistent initialization selects the latent dimension from data and initializes the action-dependent dynamics. With limited training data, K-PWM achieves lower long-horizon prediction error than the evaluated baselines on partially observed Gymnasium MuJoCo systems, while substantially reducing training time relative to the strongest prediction baseline. On high-dimensional MuJoCo Menagerie robots, it achieves competitive tracking accuracy with low per-step computation times, demonstrating potential for real-time deployment. These results support structured probabilistic dynamics as a data- and compute-efficient approach to world modeling and control under partial observation.
comment: Accepted at NeurIPS 2026
★ Breaking survivorship bias in driving world models
World models allow autonomous vehicles to imagine the consequences of candidate actions before execution, but their reliability depends on the futures represented in their training data. We identify a survivorship bias in driving world models: because the futures recorded in prevailing driving datasets almost always end safely, existing models often imagine benign outcomes even when a collision is imminent. We introduce a neutral world model (NWM) that learns both routine and safety-critical futures from a mixture of routine driving and real accident videos with a bounding-box-guided objective. The NWM more faithfully anticipates imminent collisions, while maintaining a low false collision rate and competitive generation quality in routine driving. We further use the model as an imagination engine for decision-making. Given multiple ego trajectories, the NWM predicts the future associated with each candidate, and a reward function evaluates these imagined outcomes before selecting an action. On challenging NAVSIM scenarios, this procedure selects the better trajectory more reliably than the baseline world model. Zero-shot experiments with real-world pedestrian dummy crossing data collected using an autonomous research vehicle further demonstrate transfer to unseen sensor observations. These results show that correcting survivorship bias enables world models to more faithfully reason about futures and support safety-critical decision-making in autonomous driving.
comment: Project Website: https://zihaosheng.github.io/NWM/
★ Review the Episodes, Refine the Lessons: Simple Memory Evolution for Embodied Agents
Recent studies of embodied agents based on frontier multimodal large language models (MLLMs) develop harness engineering around the MLLM, such as harnesses with post-trained policies and recursive self-improvement loops. Yet whether these additions exceed the native capability of the MLLM remains unexplored. We introduce SimplEvo to explore how far an MLLM can improve under direct control when it retains its execution experience in self-written text memories. It requires no weight updates, post-trained policies, or privileged information. The MLLM writes a coordinate-free task memory once from a single demonstration. Under a strict train-test protocol, it then creates a bounded working memory on training episodes only and refines it after each episode with lessons on procedure, completion, and recovery. Both memories stay fixed at test time. On EBench, SimplEvo lifts the success rate (SR) of GPT-6-Astra under direct control from 46.7% to 81.7%, and tops the RoboDojo leaderboard with 69.63% SR and a score of 75.32. Compared with memories that also store past visual frames, SimplEvo reaches a similar SR with text memory alone while using fewer tokens. With its memories fixed, SimplEvo also generalizes to unseen layouts and randomized scene configurations. Moreover, the working memory transfers across models: written by GPT-6-Astra, it can be reused as is by weaker MLLMs and improve their performance without any adaptation of their own.
★ ForeAct: Geometry-Guided Action Learning from World Representations
Robotic manipulation requires representations that capture both scene geometry and its evolution during interaction. Existing approaches have largely treated these two aspects separately. World action models capture such dynamics by coupling action prediction with future video generation, but their reliance on iterative diffusion or rectified-flow inference leads to high computational cost, while their representations lack explicit spatial structure. Explicitly modeling scene geometry typically requires auxiliary modules that increase both parameter count and computational cost. Conversely, geometry-centric policies primarily recover observed spatial structure and often lack explicit supervision for anticipating future geometric changes during interaction. To address these limitations, we introduce ForeAct, which repurposes a pretrained video world model as a single-pass representation backbone for low-latency control. ForeAct directly processes the clean latent of the current observation in one forward pass, eliminating future-frame synthesis and iterative denoising while retaining the backbone's pretrained spatiotemporal priors. Rather than introducing a separate geometry backbone, ForeAct uses learnable tokens supervised by next-frame relative depth to shape the shared video representation through bidirectional self-attention. This compact interface turns the spatial knowledge already encoded in the backbone into a future-aware representation for action generation, with little parameter overhead. The lightweight geometry readout is used only during training and removed at deployment, enabling low-latency, geometry-aware control without depth input or an external geometry backbone. Without additional large-scale pretraining, ForeAct achieves 96.6% average success on LIBERO, 63.5% on SimplerEnv-Bridge, and a 74.4% success rate across 90 real-world rollouts.
★ Beyond Memory Limits: A Hierarchical Spatio-Temporal Map Representation for Large-Scale 3D Autonomous Exploration
Autonomous UAV exploration is increasingly used for large-scale surveying and inspection in inaccessible or disaster-affected environments, where onboard memory and computational resources are often severely constrained. Most existing UAV exploration systems maintain globally queryable geometric map representations for motion planning and exploration decision-making, causing memory consumption and spatial-query costs to increase with the explored area. Robot-centric local mapping bounds dense-map memory and computation but discards global task context needed for long-horizon exploration.We present FLAVOR, a hierarchical spatio-temporal framework that confines dense metric geometry to a robot-centric local map while retaining compact global observation and task context. At the observation level, lightweight LiDAR-keyframe visibility records preserve ray-wise evidence for frontier reasoning. At the task level, unresolved frontiers beyond the local-map extent are retained as compact pending tasks associated with odometry anchors. A sparse topological graph connects these anchors through retained routes for future revisits.Upon returning to a task region, FLAVOR reconstructs the frontier and exploration viewpoint from observations, while the topological route is instantiated and validated against the current local map before execution. Dense-map capacity remains fixed as exploration expands, with exploration-dependent memory growth confined to compact observation and task context.FLAVOR achieves 31.98--41.88% shorter exploration time and a 94--99% smaller map memory footprint than the best baseline that completes all three simulated scenes. In real-world deployments, it reduces this footprint by 92.67% in the forest and 62.42% at the construction site relative to an explicit point-cloud ikd-tree while preserving long-term task continuity under constrained onboard resources.
comment: 16 pages, 12 figures, preprint submitted to Elsevier
★ Off-Policy RL with Online Inference Tuning: My Solution to the CoRL 2026 RoboEval Challenge
The CoRL 2026 RoboEval challenge has eight bimanual tasks, and it ranks a policy not only on whether it succeeds but on how: every stage of the task, done safely, efficiently and with both arms in coordination, scored against human demonstrations. My solution has two parts. The generic part, carried over from my LeHome challenge solution, is off-policy RL on a $π_{0.5}$-based policy that is also its own value function. It is trained with AWR and RECAP on advantages of the exact official score, in a loop where every rollout is at the same time a Thompson-sampling trial of about 20 inference parameters. Out of the box, it took the policy to 0.85 RDS on held-out seeds in two days (and it was still very far from convergence). The competition-specific part adds data that local exploration cannot find: generated demonstrations that reach the full reward, and recovery episodes that start from the policy's own failures. The submitted policy reaches 0.990 RDS on held-out seeds and 0.987 on the official seeds.
comment: Technical report for the RoboEval Challenge at CoRL 2026. Code: https://github.com/IliaLarchenko/RoboEvalSolution ; checkpoint: https://huggingface.co/IliaLarchenko/RoboEvalSolution ; project page: https://ilialarchenko.com/projects/roboeval2026/
★ When Coding Agents Get Arms: Safety Risks Across the Digital-Physical Boundary
Advances in spatial understanding have enabled frontier models to power coding agents that control a real robot arm zero-shot and end-to-end, without task-specific demonstrations or retraining. This brings two new safety challenges. First, the extensive knowledge of frontier models lets these agents perform tasks that earlier VLA and WAM paradigms could not, opening the door to new forms of capability misuse. Second, coding agents can access digital resources such as web pages and emails. Threats in these resources, such as indirect prompt injection, can carry over into the physical world when the agent controls a robot. We therefore introduce ArmedAgent, the first systematic benchmark to evaluate both risks for coding agents performing zero-shot robot control. It covers robustness, malicious use, and indirect injection across 36 tasks in 5 scenarios, evaluated in simulation and validated on a physical robot arm. A distinctive portion of the tasks target harmful behaviors that earlier VLA paradigms could not perform. Testing several frontier models, including GPT-6-Astra, under widely used agent frameworks, we find substantial safety risks in current systems.
comment: Technical report, work in progress
★ Single-Radar Bilateral 3D Near-Field Imaging for Mobile Robots
Mobile robots operating in homes need reliable 3D perception of nearby furniture and surrounding structures, yet recovering this information on both sides of the robot's path with a single forward-mounted millimeter-wave radar remains challenging because of near-field wavefront curvature, viewpoint-dependent target visibility, and limited elevation diversity. This paper presents a motion-assisted framework for reconstructing structures on both sides of a robot's trajectory. Our near-field model relates scene reflectivity to stepped-frequency multiple-input multiple-output (MIMO) observations across frequencies and robot positions using spherical paths from transmitter (Tx) to scatterer to receiver (Rx). Bilateral Tx combining provides wideband directional weighting, while sliding-aperture back-projection coherently focuses the measurements using near-field propagation distances. Top-K power fusion combines local images with different visibility. The reconstruction provides lateral-forward support for multi-layer height extraction, followed by guarded adaptive height refinement using the original MIMO channels. Experiments in controlled, office/study, and corridor environments produce mean lateral-forward localization errors of 4.65, 5.63, and 5.98 cm, respectively, with peak-to-background ratios (PBRs) exceeding 13 dB. The framework recovers bilateral 3D scattering structure from one forward pass using a single radar, without relying on learned representations of the scene.
★ Cable construction effects on the lateral forces and wakes of helically stranded cables under oblique inflow
Helically stranded cables in aerial towing systems experience oblique inflow, but are often modeled as smooth cylinders, neglecting three-dimensional cross-stream coupling and surface texture. We perform large-eddy simulations and spectral proper orthogonal decomposition for a smooth cylinder and four stranded cables at four inclinations from $22.5^\circ$ to $90^\circ$, all stationary, with $Re \approx 1.0 \times 10^4$. Aerodynamic differences are governed mainly by cross-sectional symmetry and strand-scale groove depth. Force-source decomposition shows that surface pressure contributes 94.9%--98.3% of the mean normal force and dominates the mean lateral force. High-symmetry, quasi-circular envelopes maintain balanced circumferential pressure, whereas deep-grooved, low-symmetry tri-lobed geometries produce asymmetric separation and a persistent lateral pressure imbalance. In the wake, high-symmetry cables sustain spanwise-synchronized Karman shedding at moderate-to-high inclinations, concentrating fluctuation energy in a dominant low-rank mode. Deep grooves disrupt shedding coherence, deflect the recirculation zone laterally, and spread energy across broadband scales. Mean lateral force and lateral-force fluctuations are therefore decoupled: the quasi-circular $(1 \times 19)$ cable has minimal mean lateral force but strong periodic fluctuations, while the deep-grooved $(3 \times 7)$ cable has weak fluctuations but appreciable mean lateral force. These results support construction-dependent cable selection and aerodynamic load modeling.
★ DeepSoMo: Deformable Object Motion Estimation under Proprioceptive Partial Observability via Predictive Latent Dynamics
Estimating the motion of deformable bodies in soft robotics from proprioceptive sensing is challenging due to partial observability, as internal observations provide only indirect access to global motion, contact, and shape evolution. In such settings, instantaneous observations are insufficient, and accurate estimation requires temporally consistent latent representations. This paper studies deformable motion estimation under proprioceptive partial observability from a latent dynamics perspective. We argue that a key requirement for long-horizon estimation is latent temporal consistency, where latent states remain predictive of future observations. To this end, we present DeepSoMo, a learning-based framework that enforces predictive structure in latent dynamics through a self-supervised latent prediction constraint. We evaluate the approach in a canonical deformable dynamics setting using high-fidelity finite-element simulation as a controlled source of ground truth. Results demonstrate consistent improvements in trajectory estimation, orientation recovery, shape reconstruction, and contact inference over architecture-matched recurrent baselines.
★ DexForesight: Grounding Dexterous Policies with Implicit Spatial Foresight
Dexterous manipulation requires rich spatial foresight over both the surrounding scene and the robot hand. Existing world-action models often acquire such predictive knowledge through explicit future prediction. We introduce DexForesight, a framework that learns spatial foresight implicitly by injecting dynamics-aware world representations into the latent space of a dexterous policy. During training, DexForesight learns an action-conditioned spatial world model from offline scene depth, kinematic hand geometry, and demonstration trajectories. The world model predicts 3D hand-point motion and visual feature changes, capturing both articulated hand dynamics and scene changes under demonstrated actions, including the motion of fingers occluded from the cameras. We then freeze the world model and align policy queries with its predictive world queries, transferring the learned spatial dynamics into policy latents and using them to condition action generation. At deployment, only the policy is retained, operating on RGB observations, language, and robot state without depth input or explicit future prediction. On 11 DexJoCo simulation tasks, DexForesight improves the overall success rate of the base policy from 49.4% to 60.5%. On three real-robot dexterous manipulation tasks, DexForesight achieves 58.3% success, compared with 50.0% for the strongest baseline.Project page https://dexforesight.github.io/
comment: 13 pages, 4 figures
★ Tutorial: Vision-Based Formation Control with Differential-Drive Robots
This tutorial presents a step-by-step introduction to multi-robot formation control using differential- drive robots, an overhead camera, and Python. ArUco markers are used to estimate robot positions and orientations, while desired pairwise displacement vectors define a circular formation. By comparing measured and desired relative positions, the controller computes movement vectors that are converted into differential-drive wheel commands, with basic proximity-sensor-based obstacle avoidance. The tutorial guides readers through hardware and software setup, robot pose estimation, formation geometry, feedback control, system integration, and formation-error evaluation. Motivated by the need to bridge the gap between theoretical understanding and practical implementation in robotics education, we emphasize how mathematical concepts are translated into coordinated behaviors on physical robots. Although the implementation uses a specific robotic platform, the underlying perception and formation-control principles can be adapted to other differential-drive robots and simulation environments. The accompanying open-source GitHub repository provides the Python implementation, setup instructions, and demonstrations to support hands-on learning and further exploration of multi-robot systems.
★ MOSU2: Long-range Navigation System with Multi-modal Scene Understanding
Robots that travel long distances through public outdoor spaces must do more than avoid obstacles: they must remain on terrain suited to their mobility and decide when to proceed, yield, or detour around pedestrians, cyclists, vehicles, traffic signals, and temporary restrictions. These decisions are difficult because their correctness often depends on how a scene evolves, such as whether an approaching car is slowing or a cyclist will cross the robot's path. Vision-language models (VLMs) can interpret such cues, but they reason over images that lack metric geometry and motion. Supplying this evidence at every step is too slow for real-time navigation, and hand-crafted triggers cannot anticipate the open-ended variety of outdoor situations. We present MOSU2, a long-range navigation system with two-stage trajectory selection, in which first-stage contextual reasoning determines when additional evidence is needed and what to retrieve. Only when an interaction remains unresolved does the first stage trigger the second stage to retrieve geometric and dynamic information and align it with scene context to make navigation decisions. Building on MOSU, MOSU2 follows routes from GPS-based route planning and filters candidate trajectories through multi-modal traversability analysis before selecting a trajectory to move the robot. In real-world experiments spanning stop gestures, construction sites, crosswalks, and roads shared with vehicles, cyclists, and pedestrians, MOSU2 raises mean navigation success from 61.8% to 94.2% and condition-detection accuracy from 72.2% to 91.4% over its single-image predecessor, at about 2.2 times its mean decision latency. Project website: https://jinggm.github.io/mosu2/
★ CrossLiDAR: Controllable Multi-Sensor LiDAR Sequence Generation WACV 2027
High-fidelity LiDAR simulation remains a bottleneck for scalable development and validation of autonomous systems. Existing generators can synthesize realistic scans, but they are typically constrained to a single sensor configuration, offer limited control over structure and geometry, and drift in long-horizon 4D generation. We present CrossLiDAR, a unified transformer-based framework capable of generating controllable and spatio-temporally consistent multi-sensor LiDAR sequences. Our approach leverages a rectified flow objective with a Multi-Modal Diffusion Transformer (MM-DiT) backbone, within a compact latent space of range images. To support multiple LiDAR sensors, an Inter-sensor Module enables interaction across sensors, while a ControlNet-style branch provides structural conditioning (e.g., BEV priors or 3D bounding boxes). For modeling the temporal consistency, CrossLiDAR generates short $N$-frame sequences and extends them autoregressively to improve long-horizon coherence. Our model significantly outperforms state-of-the-art baselines, achieving an average improvement of 29.56\% in JSD and 15.36\% in MMD evaluated on nuScenes and the Waymo Dataset. CrossLiDAR bridges the gap to modern autonomous-driving simulation by enabling calibration-consistent multi-sensor LiDAR rollouts over long horizons for controllable, scenario-based testing.
comment: WACV 2027
★ From Festival Video to the Stage: A Sim-to-Real Ladder for Long-Horizon Expressive Whole-Body Dance on a Commercial Humanoid
We present a pipeline for transferring Awa Odori, a traditional Japanese festival dance, from monocular video to a commercial Unitree G1 EDU humanoid. The pipeline combines human mesh recovery, kinematic retargeting, reinforcement learning for motion tracking in simulation, and onboard deployment via ONNX. Training ran on a single RTX 5070 GPU. An eleven-version deployment ladder evaluated on hardware examines three interventions. We detect and repair silent corruption of deployment artifacts, model state-estimator drift during training, and edit the reference motion to support stable execution and a controlled ending. The final policies achieved three consecutive rope-free completions of a 43-second routine, a 46-second performance with automatic handoff, and a continuous 9.7-minute rope-free performance composed from the captured motion. In a live handover, a single remote request let the robot start from and return to the manufacturer's walking controller on stage, with a measured torque-free window of 45-140 ms. We report the training dynamics and hardware outcomes and discuss failure boundaries involving compliant flooring and occlusion in monocular capture.
comment: 19 pages, 12 figures. Four supplementary videos are linked in the manuscript
★ HARP-Bench: Benchmarking Robot Manipulation Around Active Humans
Recent robot manipulation benchmarks have enabled systematic evaluation of general-purpose robot policies across diverse objects, scenes, instructions, and embodiments. However, most existing benchmarks still assume that the robot is the only active agent and that task semantics are determined primarily by the instruction and object state. This assumption overlooks a common source of real-world variation: humans actively sharing the workspace. In such settings, humans may reveal an implicit goal, complete part of the task, temporarily change object availability, or constrain safe robot motion while working nearby. We introduce HARP-Bench, a benchmark for evaluating robot manipulation policies in workspaces shared with active humans. HARP-Bench treats human activity as part of the manipulation problem, rather than as background scene variation. It organizes human-present manipulation into four conditions: Cueing, Contribution, Interruption, and Co-presence. These conditions are instantiated through everyday manipulation tasks, including paired human-present variants for controlled comparison with standard manipulation. HARP-Bench also provides a demonstration-generation pipeline for dynamic human-present settings. We evaluate VLA models, imitation-learning methods, and heuristic policies on HARP-Bench. Human-present variations challenge current policies, and higher task success does not consistently improve safety. Quantitative failure analysis identifies manipulation and recovery as the dominant remaining challenges. Training with HARP-Bench demonstrations improves human-present performance in simulation. A real-world study on one task family shows improved performance under human contribution, but not interruption, providing initial evidence of the data's utility and identifying limits of transfer. Code is available at https://github.com/UMass-Embodied-AGI/harp.
comment: Accepted at CoRL 2026
♻ ★ IMPULSE: Push First, Then Predict in V-JEPA 2 Feature Space
Latent world models such as V-JEPA 2 forecast in the feature space of a video encoder, but one look at an object does not reveal how hard it is to rotate, how much it bounces or how much it slides. People find out by trying: a few pokes reveal how an object responds. IMPULSE, named after the impulse response it infers, gives a predictor in frozen V-JEPA 2 feature space the same ability. A Bayesian contact memory summarizes three short calibration pushes, a predicted object pose carries it into the predictor, and a lightweight random-Fourier pose head lets the predictor use the pose. On 512 fresh objects, in a pre-specified held-out confirmation with models sealed before the data existed, the memory lowers the 1-second feature-forecast error by 11.6-12.8% relative to the same predictor without experience, for all 8 training seeds, with a 95% seed-and-object interval excluding zero. The gain also holds for 128 objects with out-of-range physics (11.7-14.0%), and the full extension beats a plain predictor by 15.5-16.6%. An information-ladder argument, exact for ideal predictors, shows that physics reaches the features only through motion, so we measure how much the pathway lets pose help. On fresh objects, the Fourier head roughly triples the gain of a perfect memory over plain pose tokens (2.8-3.2 times). Raw push measurements fed through the same pathway give a similar gain, so the Bayesian summary is not required; it condenses the trials into eight interpretable numbers. The gain is absent at contact onset and grows afterwards.
comment: 8 pages, 6 figures, 3 tables. Adds a larger-predictor check and a push-count analysis, corrects several statements, and adds a co-author
♻ ★ MolmoB0T: Large-Scale Simulation Enables Zero-Shot Manipulation
A prevailing view in robot learning is that simulation alone is not enough; effective sim-to-real transfer is widely believed to require at least some real-world data collection or task-specific fine-tuning to bridge the gap between simulated and physical environments. We challenge that assumption. With sufficiently large-scale and diverse simulated synthetic training data, we show that zero-shot transfer to the real world is not only possible, but effective for both static and mobile manipulation. We introduce MolmoBot-Engine, a fully open-source pipeline for procedural data generation across robots, tasks, and diverse simulated environments in MolmoSpaces. With it, we release MolmoBot-Data, a dataset of 1.7 million expert trajectories for articulated object manipulation and pick-and-place tasks. We train three policy classes: MolmoBot, a Molmo2-based multi-frame vision-language model with a flow-matching action head; MolmoBot-Pi0, which replicates the $π_0$ architecture to enable direct comparison; and MolmoBot-SPOC, a lightweight policy suitable for edge deployment and amenable to RL fine-tuning. We evaluate on two robotic platforms: the Franka FR3 for tabletop manipulation tasks and the Rainbow Robotics RB-Y1 mobile manipulator for door opening, drawer manipulation, cabinet interaction, and mobile pick-and-place. Without any real-world fine-tuning, our policies achieve zero-shot transfer to unseen objects and environments. On tabletop pick-and-place, MolmoBot achieves a success rate of 79.2% in real world evaluations across 4 settings, outperforming $π_{0.5}$ at 39.2%. Our results demonstrate that procedural environment generation combined with diverse articulated assets can produce robust manipulation policies that generalize broadly to the real world. Technical website: https://allenai.github.io/MolmoBot
♻ ★ Human-Machine Interfaces for Subsea Telerobotics: From Soda-straw to Natural Language Interactions
This review explores the evolution of human-machine interfaces (HMIs) in subsea telerobotics, charting the progression from traditional first-person "soda-straw" consoles-- characterized by narrow field-of-view camera feeds-- to contemporary interfaces leveraging gesture recognition, virtual reality, and natural language processing. The review makes three key contributions. First, we introduce a structured framework that analyzes subsea HMIs through three interrelated aspects: operator experience, robotic autonomy, and bidirectional human-machine communication. Second, using these aspects, we compare visual, haptic, extended-reality, gesture, and natural-language interfaces; examine the progression from direct teleoperation to shared autonomy; and assess the distinct roles of simulators and digital twins in training, prototyping, and mission support. Third, we synthesize the reported evidence to distinguish established industrial tools, selectively deployed technologies, and research-stage capabilities, while identifying limitations in interoperability, communication reliability, immersive feedback, and field validation. From this synthesis, we outline standardization needs and research directions for hydrodynamic feedback, latency-aware shared autonomy, grounded language interaction, and real-to-simulation transfer. The report concludes by identifying open research questions and proposing a roadmap toward more intelligent and intuitive subsea HMI systems.
comment: 28 pages. First pre-print released in 2024; revised as technical report in November 2026
♻ ★ Latent World Models with Monotone Planning Costs for Image-Goal Navigation
Image-goal navigation with latent world models requires not only accurate future prediction, but also a planning cost that reliably ranks candidate action sequences. We define the cost as the cosine distance between the predicted future embedding and the goal embedding, and show that poor cost ordering can mislead sampling-based planners such as Cross-Entropy Method (CEM). To address this, we propose a latent world model built on a frozen DINO-family encoder and train it with two complementary objectives. An autoregressive rollout loss reduces the gap between training and multi-step planning rollouts, while a Monotone Cost Ranking (MCR) loss directly encourages increasingly perturbed action sequences to receive higher planning costs. We also study InfoNCE-based action-contrastive training and find that temporal permutation negatives distort the latent geometry and degrade planning performance. On the GNM navigation dataset, our method outperforms Navigation World Models (NWM), DINO-WM, OmniVLA, and NoMaD, achieving state-of-the-art image-goal navigation performance while reducing orientation error by $2.7\times$ over the same-encoder DINO WM baseline. We also deploy the model zero-shot on a physical robot, where it follows goal-directed paths in unseen indoor and outdoor environments.
♻ ★ Manufacturing Complex Airtight Soft Pneumatic Actuators for Soft Robotics: Process Evaluation and Optimization
Manufacturing complex soft pneumatic actuators remains challenging because geometric fidelity, compliance, structural integrity, and airtightness must be achieved simultaneously. This study presents a manufacturing-focused evaluation of several fabrication routes for complex pneumatic structures, including heat-shrink forming, silicone casting, powder- and liquid-based additive manufacturing, and fused deposition modeling (FDM). The processes were assessed through process screening, baseline fabrication, failure analysis, and process improvement to distinguish inherent process limitations from correctable manufacturing defects. Heat-shrink forming was limited by geometric conformity, casting by mold accessibility and bonded interfaces, powder-based methods by residual material trapped within enclosed passages, and digital light processing by the material properties and post-processing requirements of the investigated system. FDM provided the most adaptable route because its dominant defects could be progressively reduced through process optimization. The results further showed that airtightness depends not only on nominal wall thickness but also on extrusion-path architecture, while support-free geometry is important when access for internal post-processing is limited. These findings establish a practical design-for-manufacturing approach in which process selection is guided by the compatibility between actuator architecture and manufacturing constraints. The proposed approach provides practical guidance for developing complex, flexible, and airtight soft pneumatic actuators for soft robotic applications
♻ ★ A complete analytical characterization of optimal collision-free motion for two robots
Collision-free motion planning for multi-agent robotic systems remains an active research area because of the complexity of coordinating robot trajectories, with applications ranging from coordinated drone shows to autonomous operations on land, in the air, and on water. Importantly, the analytical difficulty of nonconvex collision-avoidance constraints has so far driven the development of exclusively numerical methods. We derive a complete analytical characterization of the optimal trajectories for two robots with zero initial and final velocities, under double-integrator dynamics and a squared-acceleration objective. We prove that these trajectories belong to eight distinct contact families and determine which family applies for each configuration of initial and final positions. We analyze the geometrical properties of the optimal trajectories and provide a numerical algorithm for computing them. These insights have already been adopted to improve the performance of numerical methods for multi-agent motion planning by several orders of magnitude.
♻ ★ Tactile MNIST: Benchmarking Active Tactile Perception NeurIPS 2026
Tactile perception has the potential to significantly enhance dexterous robotic manipulation by providing rich local information that can complement or substitute for other sensory modalities such as vision. However, tactile sensing is an inherently local sensor modality, providing information only at the points of contact. This strict locality necessitates the use of active perception techniques to acquire useful, task-relevant information from manipulated objects or the environment in general. Hence, the agent must actively guide sensors toward regions with more informative or significant features and integrate this information over time to understand a scene or complete a task. While active perception and tactile sensing methods have received significant attention recently, both fields lack standardized benchmarks. To bridge this gap, we introduce the Tactile MNIST Benchmark Suite, an open-source, Gymnasium-compatible benchmark specifically designed for active tactile perception tasks, including classification, localization, pose estimation, and volume estimation. Our benchmark suite offers diverse simulation scenarios, from simple toy environments to complex tactile perception tasks using vision-based tactile sensors. Furthermore, we offer a comprehensive dataset comprising 13,580 synthetic 3D MNIST digit models and 153,600 real-world tactile samples collected from 600 3D-printed digits. Using this dataset, we train a CycleGAN for realistic tactile simulation rendering. By providing standardized protocols and reproducible evaluation frameworks, our benchmark suite facilitates systematic progress in tactile sensing and active perception more broadly.
comment: Advances in Neural Information Processing Systems (NeurIPS 2026), Evaluations & Datasets Track
♻ ★ LFX: Fusion-Form-Flexible Light Field Modeling for Dense Semantic Segmentation and Salient Object Detection
Two distinct light field (LF) observation forms, namely sub-aperture images (S.Aperture images) and focal stack (FS), correspond to two fusion settings: the central image with S.Aperture images (CS) and the central image with FS (CF). Existing LF perception methods are typically form-specific, while a modeling design that can be separately instantiated under either setting remains underexplored. To address this gap, we present LFX, a form-flexible LF perception framework. Here, ``X'' represents the LF observation form, indicating that the one modeling design can be separately instantiated under two fusion settings (CS and CF). Specifically, we propose Field-of-Parallax Angular Subspace Modeling (FoP-ASM), which maps the feature difference between each auxiliary observation and the central image to an observation-specific angular marker. This avoids pairwise modeling among auxiliary observations and form-specific geometric assumptions. Meanwhile, a shared manifold subspace and the associated regularization losses constrain the angular residuals within a common feature space, promoting semantic consistency across auxiliary observations. Extensive evaluations across three LF datasets show that LFX achieves state-of-the-art results across distinct LF forms. Under the S.Aperture images and FS settings, respectively, LFX reduces MAE by 12.12%/20.59% on DUTLF-V2 and by 6.15%/7.69% on HFUT, while achieving the best performance on UrbanLF-real. The source code will be made publicly available at https://github.com/FeiT-FeiTeng/LFX.
comment: Accepted to IEEE Transactions on Circuits and Systems for Video Technology (TCSVT). The source code will be made publicly available at https://github.com/FeiT-FeiTeng/LFX
♻ ★ RCT-AD: Reliability-Aware Context and Temporal BEV Reasoning for End-to-End Autonomous Driving
Reliable temporal representation learning is important for end-to-end autonomous driving, particularly when visual observations are degraded by occlusion, motion blur, illumination variation, and sensor noise. Existing Bird's-Eye-View (BEV) approaches provide a unified representation for perception and planning; however, temporal features are commonly aggregated across frames without explicitly accounting for the reliability of individual observations. Consequently, low-quality or inconsistent features may propagate through temporal representations and degrade downstream perception, prediction, and planning. To address this problem, we propose RCT-AD, a Reliability-Aware Context and Temporal BEV Reasoning framework for end-to-end autonomous driving that explicitly models feature reliability and temporal consistency within a shared BEV representation. RCT-AD introduces a Reliable Context Awareness (RCA) module that estimates frame-level reliability using spatial, semantic, temporal, and visual-quality cues, and selectively preserves trustworthy information through a reliability-guided temporal memory mechanism. When the current observation is unreliable, historical high-reliability features are aligned and integrated to construct a more stable temporal BEV representation. A Temporal Trajectory Planner models long-term temporal dependencies and multi-agent interactions, while semantic segmentation and motion-guided detection provide complementary supervision for refining the shared BEV representation. Experiments on the nuScenes benchmark demonstrate consistent improvements across perception, motion prediction, and trajectory planning. RCT-AD achieves 61.5 NDS, 52.9 mAP, and 52.3 mIoU while maintaining competitive computational efficiency. These results demonstrate the effectiveness of reliability-aware temporal BEV reasoning for unified perception, prediction, and planning.
comment: 15 pages, 6 figures. Substantially revised version with an updated title, revised methodology, expanded technical descriptions, updated experimental analysis, and reorganized manuscript structure. The central research objective remains unchanged
♻ ★ Multi-quadrotor systems interconnected via structurally unknown forces and moments
This work addresses the challenge of controlling multi-quadrotor systems subject to tight uncertain couplings. We consider interconnection forces and moments among the quadrotors to be state-dependent functions of the coordination errors, without assuming knowledge of their functional/parametric structures, e.g., no linear-in-parameters assumptions are made. We only rely on general state-dependent quadratic bounds valid for several Euler-Lagrange mechanical systems. We propose a coordination strategy embedding adaptive laws dedicated to compensating for the lack of structural knowledge. Unlike conventional decoupled position-attitude designs, we present a coupled stability analysis accounting for the position-attitude couplings induced by the interconnection forces and moments. The analysis, along with real experiments involving cooperative aerial transportation of rope-interconnected payloads, demonstrates the effectiveness of the proposed solution in compensating for these complex interconnections. Comparative experiments show that, when a payload drop induces a disturbance propagating through the ropes across the whole formation, the proposed strategy reduces significantly post-disturbance transients.
comment: 14 pages, 6 figures, 3 tables. Video available at: https://tinyurl.com/4mzpb9rf
♻ ★ LARC: Lazy Adaptive Reachability Certification of Robot Manipulator Trajectories
Discrete trajectory checks can miss collisions between sampled robot states. Reachability-based certification bounds motion between states, but uniform time partitions waste computation where clearance is large. We present lazy adaptive reachability certification (LARC), which checks a planned trajectory by bisecting only intervals with an inconclusive clearance test. For piecewise-cubic Hermite joint trajectories, the method bounds link occupancy using midpoint capsules inflated by exact componentwise speed maxima. Certified intervals covering the trajectory provide continuous-time external-obstacle clearance, subject to geometric containment, static obstacles, and a prescribed margin. On 160 AgileX PIPER trajectories from 80 start-goal pairs, LARC matched all decisions of the fixed-fine baseline at depth nine. It used 20328 interval evaluations (24.8% of baseline work), with a median paired speedup of 10.28x. A separate MoveIt/FCL audit checked 158051 states and detected collisions in 21 direct-interpolation controls, none of which LARC certified. The method reduced computation under a shared certificate model, but 27 of 139 sampled-clear trajectories remained uncertified. The sampled audit cannot independently prove continuous-time clearance.
♻ ★ Lights, Camera, Malfunction: When Illumination Robustness Leaves VLA Models Blind to Color
Vision-Language-Action (VLA) models have emerged as a powerful paradigm for general-purpose robot manipulation; however, their transition to real-world environments reveals vulnerabilities to minor environmental perturbations. We propose FLARE, an optimization framework that exploits these vulnerabilities via targeted illuminations, causing critical disruption of baseline tasks. While adversarial training is the standard countermeasure, we identify a critical and previously underestimated defensive pitfall: naive data augmentations condition VLA models to largely discard color as noise, biasing their visual perception toward shape cues. We expose this degradation through a diagnostic grayscale evaluation, in which a naively defended model maintains high success rates on grayscale inputs, while its success rate on benign, color-dependent real-world tasks drops to at most 48.8%, compared with 78.8% for the undefended SmolVLA baseline. To address this, we propose ChromaGuard, a chroma-preserving adversarial training method. On a physical 6-DoF robotic platform using SmolVLA, we demonstrate that ChromaGuard achieves 95.0% and 88.8% success rates in benign and attacked color-dependent tasks, respectively. Demo videos are available at https://keio-csg.github.io/ChromaGuard/.
♻ ★ SpectraTac: A Compact Camera-Free Optical Tactile Sensor with Distributed Color Sensing
Tactile sensing is essential for physical interaction in robotics and human--machine systems. However, combining rich tactile information with compact hardware, low cost, and low computational overhead remains challenging. This work presents SpectraTac, a compact, camera-free optical tactile sensor that combines active red--green--blue (RGB) illumination with spatially distributed color sensing. Contact deforms a compliant transparent elastomer and modulates its internal light field, producing spatially differentiated changes in color and intensity. Three distributed color sensors capture these responses as low-dimensional spatio-spectral features, avoiding cameras, imaging optics, and high-dimensional image processing. The device measures 19.2 mm in diameter and 4 mm in height, with a material cost below USD~5. A data-driven decoding framework simultaneously estimates three-dimensional (3D) force and the contact region from the optical measurements. For 3D force prediction, the sensor achieved mean absolute errors (MAEs) of 0.161, 0.164, and 0.429 N along the x-, y-, and z-axes, respectively. The nine-region contact-classification accuracy was 99.9%. We further evaluated real-time 3D force tracking and contact-region-based human--machine interaction through an interactive control task. These results indicate that distributed color-resolved optical sensing offers a compact, low-cost alternative to camera-based tactile sensing for robotics, wearable sensing, and interactive systems.
♻ ★ Robotic Spatiotemporal Editing: A Unified Interface for Controllable Action Generation via Masked Action Reconstruction
Generative policies can synthesize robot behaviors, yet controlling and editing them through diverse spatiotemporal references remains challenging. In the real world, human intent requires robot behaviors that satisfy constraints on where, when, and how actions are executed. Our key insight is that spatiotemporal control references, such as keypoints, flows, orientations, and keyposes, can be viewed as partial specifications of the same temporal Cartesian action space. We introduce the MAP (Masked Action Space) interface to unify these action references and formulate controllable manipulation as an action inpainting problem. Our Inpainting Policy (IP) reconstructs missing action dimensions of the end-effector, combining progress-guided alignment with multi-mask training supporting unseen action-constraint combinations. We further adopt a training-free RePaint variant that iteratively inpaints the action during denoising to reduce control error. On ManiSkill and LIBERO-10, our policy achieves average task success rates of 92.67% and 64.67%, respectively.Real-world experiments demonstrate spatiotemporal control with average positional and rotational errors of 44.3 mm and 7°, respectively, showing that our policy can use diverse references to guide executable robot behaviors.
♻ ★ FlowPilot: Real-Time World-Action Modeling for Agile UAV Navigation
We present FlowPilot, the first world-action model for real-time onboard UAV navigation from depth. Unlike map-then-optimize pipelines that require local reconstruction or end-to-end policies that lack explicit scene prediction, FlowPilot jointly denoises future depth observations and executable trajectories with flow matching. A dual-stream mixture-of-transformers couples video and action experts through shared attention, allowing future-scene prediction and trajectory generation to inform each other. At deployment, the model runs action-centrically and outputs only a trajectory. To generate smooth, state-consistent references, actions are parameterized as degree-7 Bernstein polynomials: the current state constrains the initial control points, and the network predicts five free control points, yielding C^2-continuous references with closed-form velocity, acceleration and jerk. FlowPilot is trained on a three-level depth pyramid spanning high-throughput simulation, photorealistic simulation, and real onboard data. In closed-loop simulation, it outperforms learning- and optimization-based baselines under increasing clutter and commanded speeds up to 10 m/s. On a physical quadrotor, FlowPilot achieves a mean inference latency of 10.2 ms on a Jetson Orin NX and reaches a peak speed of 10 m/s during outdoor flight using only onboard sensing and computation.
comment: 8 pages, 8 figures, 4 tables. Accepted for publication in IEEE Robotics and Automation Letters (RA-L)
♻ ★ You Don't Need To Stay in The Loop: An Agentic Robotics Loop for Robot-Policy Improvement
Coding agents close the software loop for training a robot policy without having a human in the loop: a main agent manages the training, subagents analyze and execute, treat robot policies and hardware as tools and do the work. We port this architecture to robot-policy improvement, where one difference dominates the design: robotic tools such as trained policies, training pipelines, and data collection fail routinely, so a tool's quality must be measured, recorded at every call, and expired when the artifact behind it changes. AgenticRobotics is a backend-independent control plane in which an LLM controller drives disposable workers through durable train--evaluate--improve transactions: an immutable objective, controller-owned measurement, commit-keyed crash recovery, an evidence-graded skill library, and a tool registry with a standardized, recorded call surface. The title is an operational claim, not a selection claim: the operator can leave because promotion is evidence-gated, state is recoverable, and capability quality is derived from records, not because the loop picks better checkpoints than a human. On the one lineage we measured, it does not. The gates provide measured false-promotion control (0.001 false promotions per run with the hardened margin, against 0.005--0.021 with the shipped rule), anytime-valid decisions under optional stopping, zero lost or duplicate effects under kill injection, and a signed verifier that caught six of six artifact-tampering classes.
comment: Agentic Robotics Preview Version
♻ ★ Dual-Part Multi-Lateral Branched Network for Multi-Class Segmentation in Cardiovascular Catheterization Angiograms
Catheterisation image processing requires segmentation models that are fast, accurate and explainable. While most of the existing studies usually focus on binary segmentation, there is a recent demand for simultaneous segmentation of multiple structures found in catheterization scenes. In this study, a dual-part MLBNet architecture is designed with multi-lateral encoder blocks and multi-head decoder branches for class-aware segmentation in cardiovascular catheterization scenes. Lateral branches in the encoder enable repeated feature extraction to learn diverse shared representations, while multiple decoder heads are used to introduce class-skewed branches that specialize in different structural properties in catheterization scenes. To analyze the performances of the dual-part MLBNet architecture, several multi-class segmentation angiogram data obtained during cardiovascular catheterization in phantom models, synthetic human-simulated aorta, and animal model are used for model training and evaluation. Results obtained showed the dual-part models could effectively separate guidewire, catheter, vessels and background pixels to their classes of memberships with high probability. The results demonstrate that all models were able to distinguish the dominant background class from foreground structures with high overall accuracy.
♻ ★ Sample-Efficient Reinforcement Learning for Policy Improvement via Flow-Reversed Action Anchors
Generalist robot policies offer strong behavioral priors, but achieving deployment-level reliability and throughput remains difficult. Reinforcement learning (RL) can improve these policies, yet its effectiveness is constrained by the cost of online interaction. Existing methods adapt frozen flow-matching policies by treating noise space as a continuous action space. We show that this behaviorally unstructured space makes exploration inefficient, often preventing these methods from extracting meaningful learning signals on challenging tasks. We introduce AnchorQ, which enables coarse-to-fine policy improvement through a structured, refinable latent action representation. At each decision step, AnchorQ uses flow reversal to map reference behaviors to state-conditioned latent anchors, samples local particles around those anchors, and applies value-guided sampling to explore promising refinements. Our analysis shows that anchors reduce identification complexity at the cost of support bias; particle refinement trades this bias against action-class complexity; and critic accuracy determines the reliability of KL-regularized, value-guided selection. Across simulated and real-world manipulation tasks, AnchorQ outperforms state-of-the-art post-training baselines in sample efficiency, delivering a 37% improvement in real-world success rate and 97% higher simulated task throughput under the same interaction budget. Additional visualizations are available on our website https://anchorq-rl.com/.
♻ ★ Controllable Road Marking Generation
Lane and road markings provide critical guidance for vehicle navigation and multi-agent coordination, yet authoring them at scale remains a manual workflow that limits quantitative analysis and scenario testing. We introduce Controllable Road Marking Generation, which synthesizes a missing center-region marking layout from a drivable-area mask, optional outer-ring markings, and a textual description. Our benchmark uses deterministic, metadata-derived prompts and three output channels: lane dividers, road dividers, and pedestrian crossings. We develop a conditional bird's-eye-view (BEV) pipeline that combines (i) a text-conditioned latent rectified-flow DiT trained with a topology-aware auxiliary loss, (ii) Gaussian-blurred training targets that provide smoother supervision for thin, sparse markings, and (iii) Structured Gaussian Render (SGR), a training-free post-process that recovers crisp divider geometry by extracting polylines, fitting cubic Bézier curves, and re-rendering them as anisotropic super-Gaussian primitives. On 4,597 Argoverse~2 test tiles, our system achieves Buffered F1 of 76.8 and clDice of 50.2, compared with 38.8 and 24.6 for an adapted mask-refinement baseline. When trained independently on the Waymo Open Dataset, it yields 88.0 Buffered F1 and 66.2 clDice. Component ablations show complementary connectivity gains from topology-aware supervision and SGR. Text-editing experiments reveal that stronger guidance improves edit success but also increases changes to non-target structures. We see this framework as a step toward simulation-ready road-marking variation, automated map completion, and early-stage infrastructure design exploration.
Multiagent Systems
★ PA-SaR: Perception-Aware Learning, Control, and Coordination for Multi-Robot Search and Rescue
Autonomous SaR requires robots to explore unknown or partially known environments, avoid hazards, coordinate team activities, and guide discovered victims to safety. Improving individual capabilities of robots does not necessarily improve mission outcomes: safer motion can restrict the exploration behavior, faster discovery can increase the rescue demand, and effective guidance depends on victim responses. We introduce PA-SaR, a mathematical and architectural framework for coordinating heterogeneous local control, learning-based exploration, supervisory task allocation, and adaptive approach-lead victim guidance around mission-level objectives. PA- SaR distinguishes environmental ground-truth, robot-specific perceptions, and simulated victim behavior. This provides a common basis for integrating complementary mechanisms and analyzing their interactions across mission stages. We demonstrate the framework through single- and multi-robot simulation studies incorporating model-predictive and learning-based exploration, perception-based safety shielding, event-driven task allocation, and adaptive victim guidance. The results show trade-offs between collision avoidance and search completion, indicate that supervisory allocation improves search completion beyond information sharing alone, and demonstrate that increased exploration does not necessarily yield more rescues. Adaptive guidance increases rescues relative to continuous leading, without a detected improvement in mean or final victim trust or a reduction in mortality. These findings support evaluating local control, team coordination, and evacuation as related but distinct aspects of autonomous SaR within the considered simulation environments.
★ ODSA: Orchestrated Discovery and Self-Aggregation
Test-time scaling improves the accuracy of a language model's answers by spending more computation at inference time. Established methods do so either sequentially, revising a single solution over several passes, or in parallel, generating several independent solutions and selecting among them. We present ODSA (Orchestrated Discovery and Self-Aggregation), an LLM-directed scaling framework in which an orchestrator LLM is given existing test-time scaling methods as actions --- discovery of a new independent solution, comparison of existing solutions to build a better one, and acceptance of a proposed answer, which triggers its verification --- and directs stateless workers to carry them out. The orchestrator does not solve the problem itself: it allocates computation, feeds the results of earlier actions into later ones, and takes every action from a single context, so that each decision is informed by everything tried before. On Humanity's Last Exam, ODSA improves accuracy by 4.7 to 18.0 points across three models over a single direct call to the model with no test-time scaling. On LiveCodeBench, where a single direct call already solves most problems, it improves accuracy by 0.6 to 14.0 points.
★ ErosionBench: Probing Alignment Decay in Enterprise Multi-Agent Systems
As enterprises delegate consequential work to multi-agent LLM systems, benign-looking requests can escalate into unsafe actions as they propagate across sessions through poisoned shared state, long-context documents, and trusted peer messages: no single message is malicious, yet the assembled tool action is. Prior benchmarks isolate a single agent under prompt injection or score multi-agent attacks by their final output, leaving cross-session erosion of shared state out of scope. We introduce ErosionBench, an environment-grounded benchmark that executes such attacks and verifies alignment decay at the process level, scoring the realized tool call against authorized intent and provenance with deterministic contracts, not output text. Each scenario is authored as a deterministic state graph before any prose is rendered, so every state change traces to a known cause. It spans four enterprise domains and four decay mechanisms across 1,728 scenarios with matched benign controls, labeled against the OWASP Top-10 for LLM Applications, plus 198 image-based attacks, each paired with a text equivalent, so a defense must survive the same attack in either channel. Perimeter defenses suppress attacks only by over-refusing 54-92% of legitimate work, so a low attack-success rate can conceal severe over-refusal. As a reference defense, we propose a calibrated Meta-Auditor pairing synchronous state checks with asynchronous trajectory monitoring; on a family-disjoint 906-scenario held-out split it reaches 98.5% recall at a 2.9% false-positive rate. Since it monitors trajectory, not surface content, its detection is channel-invariant: the strongest auditor flags image-delivered attacks about as reliably as their text equivalents (recall gap $\leq$2 points per model). Across GPT, Gemini, and Claude auditors, calibration keeps false positives low while recall tracks model capability, strongest for Claude.
comment: 10 pages plus references and appendices
★ LoMASA: A Local Multi-Agent Sales Assistant with Confidence-Gated Delegation for B2B Quoting
Frontline sales staff spend much of their day re-coordinating quotes after customers change quantities, ask for discounts or new payment terms, or costs move. We present LoMASA (Local Multi-Agent Sales Assistant), a training-free multi-agent system that runs entirely on one desktop computer. A 35B mixture-of-experts model (System-1) makes each decision in a single forward pass and reads its confidence from the probabilities of the answer options. A 27B dense model (System-2) writes customer replies and escalation briefs and is consulted when System-1 is unsure. Code computes every price, margin and option, and hard pricing rules cannot be overridden. On a test split of 500 simulated episodes with 1512 decisions built from public supply-chain data, the salesperson has to decide only 19.5% of decisions: 38.0% are settled automatically, with a median latency of 2.67 seconds, and 42.5% are escalated to a manager by the hard rules. On the automatically settled decisions LoMASA is as accurate as a rule baseline (0.894 vs.\ 0.894) and makes fewer commitments whose margin later falls below the floor (2.61% vs 4.53%); the price is additional escalations to the manager. Option probabilities are well calibrated without any training, unlike confidence stated in words by the same model. We also report where the design does not help: the 27B model rarely changes a decision.
★ Deterministic Narrative Gravity: Bridging Theory of Mind and Historical Simulation through ABM-LLM Architectures
Historical social simulation faces a fundamental reproducibility challenge: the stochastic nature of large language models (LLMs) introduces a digital butterfly effect that prevents exact replication across multi-turn runs. Furthermore, the inability to control low-level server parameters in commercial LLM hosting environments severely limits reproducibility. To overcome this opacity, we present a deterministic multi-agent framework coupling LLMs to an Agent-Based Modelling (ABM) pipeline grounded in Theory of Mind (ToM) and Cognitive Archaeology, and validate exact artifact identity within local, pinned execution environments using SHA-256 hashing. Run locally at temperature zero with clean per-call model reloads, the system injects differentiated cognitive states per turn through dissonance-driven behavioural modes and gated internal voices. Evaluating seven model configurations spanning 4B to 31B parameters against an ablated baseline reveals clear operational boundaries. Small models ($\leq$8B) suffer from context capacity overload, producing repetitive, specular dialogue loops. However, at $\geq$12B parameters, the ABM scaffolding breaks the monotonous response patterns seen in ablation. Crucially, these scaffolded models exhibit spontaneous diegetic expansion: they coherently expand scenario lore through tactically motivated, anachronism-free details rather than escalating into aggression. By resisting the structural pull toward premature scene closure, scaffolded sessions sustain an open terminal Narrative Vector ($V_N = \mathrm{Open}$ in 12/21 overall; 9/9 in official $\geq$12B configurations) whereas unscaffolded ablation never does (0/21). This framework provides a transparent, auditable benchmark architecture for virtual archaeology, with the full pipeline released as open-source software to enable independent replication.
comment: 28 pages, 2 figures. To appear in the Journal of Computer Applications in Archaeology. Code: https://github.com/alixiacf/cognitive-abm
★ GARP: Real-Time Generative Agents on Local Hardware in a Game Engine
Generative agents are game characters whose plans, conversations, and memories are written on the fly by a large language model (LLM). The original Generative Agents demonstration used cloud inference in a custom simulation. We present GARP, a platform for real-time generative agents inside Godot, and describe the Character Behavior System used in GARP 2. The complete loop runs on local hardware using a 3B-parameter open-weight model and no cloud service. Alongside the original planning, memory, and reflection, GARP gives each agent an emotion system driven by LLM appraisal, four graded personality axes drawn from the Big Five that set simulation parameters as well as prompt text, six needs whose urgency grows over time, and pairwise relationships updated after every conversation. The system narrows 50+ candidate actions to typically 3-8, constrains model responses with JSON schemas, and surfaces important memories in decision prompts. A layered optimization strategy (an adjustable game clock, decisions taken ahead of need, multi-tier caching, prompt compression, and fully asynchronous execution) keeps the game responsive while the model thinks. In a nine-scenario evaluation, 25 of 27 runs passed log-based checks. Both failures occurred in the birthday party scenario, where one guest omitted the event from their daily plan.
comment: 13 pages, 13 figures
★ Advancements in Cloud-Native Systems: A Systematic Mapping Study of Reliability and Scalability across Cloud-Native, Edge-Native and AI-Enabled Systems
Cloud-native systems now extend to the network edge and host artificial-intelligence (AI) workloads, including AI agents, but evidence on their reliability and scalability is scattered and uneven. We report a PRISMA 2020 systematic mapping study with an evidence-graded synthesis. Five logged OpenAlex searches on 11 October 2026 covered cloud-native systems, edge AI, agentic AI for operations, reliability of agentic applications and physical AI; 189 records from an earlier exploratory search were added separately. After 301 duplicate removals and an automated peer-review venue rule, 4,611 records were screened against written criteria and a five-level relevance scale. The review includes 1,794 primary studies from 2017-2026 and 95 secondary studies, with every decision released in Dataset S1. Each primary study was assigned to a 13-category taxonomy and graded as simulation, prototype or testbed, production data, production deployment, or not determinable from the abstract. Of 1,003 studies with a determinable level, 66.2% used prototypes or testbeds, 18.0% simulations and 15.8% production data or deployments. A 93-study synthesis core, appraised from indexed abstracts, shows production deployments of cascading-failure prevention, trace- and graph-based root-cause analysis and predictive autoscaling, and production-trace evaluations of cold-start mitigation. Edge reliability and physical AI rely mainly on simulations and testbeds. AI-agent reliability uses tool-grounded diagnosis, explicit safety properties, trace-driven repair and workload-aware scheduling, while empirical studies show that agent frameworks remain fragile. Conclusions are limited to the documented corpus. The principal gaps are production evidence for edge-native systems, shared resilience benchmarks and dependable agentic operations.
♻ ★ Implicit Credit Learning for Sparse-Reward Multi-Agent Reinforcement Learning
When a cooperative team receives feedback only at the end of an episode, a single outcome must guide many agents' earlier actions. Learning a dense team reward helps, but its decomposition does not automatically give each agent a useful policy update. We introduce IMAP, an online method that fits a shared Q/V reward model by comparing completed episodes according to their observed terminal outcomes. Local Q/V residuals provide agent-specific rewards; the same normalized weights combine them for a centralized critic and scale local advantages for decentralized PPO actors. We characterize when these local updates follow the learned team objective. With full returns and frozen reward heads, a sharp bound relates their gradient error to variation in future Q/V compatibility; an exact identity identifies additional continuation terms for general GAE. In experiments with terminal-only feedback, IMAP's local-actor configuration attains the highest reported mean among the compared methods on all nine SMACv2 scenarios and four MAMuJoCo tasks, and on two of three SMACLite scenarios. Controlled relay interventions confirm the predicted changes in actor credit while holding the learned team reward fixed. The gradient result concerns the fitted objective; benchmark returns measure performance on the original task outcomes.
comment: 41 pages; working paper
♻ ★ Autonomous Analysis of Thermoelectric Materials Big Data by Multi-Agentic AI and Construction of a Knowledge Base and Specialized AI
Large databases of paper-derived experimental materials data hide essential information on materials. Their analysis is demanding, large-scale and complex, making manual knowledge extraction very difficult. To address this challenge, we developed Multi-Agentic AI (MAA) for automated, autonomous analysis of materials experimental big data and a specialized AI built on the obtained knowledge. MAA's nine role-specific AI agents, under predefined instructions and quality checks, autonomously perform data extraction, compositional feature generation, machine-learning modeling, feature-importance analysis and knowledge-base recording, iterating while inheriting previous knowledge. Delegating numerical processing to code makes large-scale data tractable; recording every choice and result enables later tracing to sources. In four thermoelectric-data experiments varying theme and cycle counts, the knowledge graph grew: knowledge clustered by theme and vocabulary was shared across themes. The specialized AI, searching and reading these records, answered materials questions citing sources. Retrospective verification also revealed errors, traced through the records to their sources.
comment: 78 pages (31 pages main text with 5 figures; 47 pages Supplementary Information with 7 Supplementary Notes, 10 Supplementary Tables and 3 Supplementary Figures). Analysis records, Supplementary Data 1-5 and code are available from the corresponding authors upon request. v2: abstract and text synchronized with the journal submission
♻ ★ Distributed Learning of $α$-Potential Functions in Strategic Multi-agent Systems
We study the problem of computing the tightest \(α\)-potential approximation of a general-sum game over continuous action spaces, within a prescribed class of potential functions and when each player has access only to its own utility function. The difficulty is twofold: the approximation error involves a worst-case search over an infinite set of unilateral deviations, and the required utility information is distributed across players. For a linear-in-parameters potential class, we use an exact finite-tuple reformulation that separates the problem into a global outer search over deviation tuples and distributed convex inner problems. We develop a primal--dual inner oracle tailored to this structure and establish a uniform one-sided accuracy guarantee. This oracle can be combined with global outer search to obtain an end-to-end guarantee on the outer optimization error. We also develop a projected zeroth-order outer method as a computationally lighter alternative for higher-dimensional problems. Numerical experiments illustrate the accuracy--computation tradeoff between the two outer-search methods and show that the proposed optimization framework can improve upon analytical \(α\)-potential constructions.
comment: 15 pages, 4 figures
Systems and Control (EESS)
★ PA-SaR: Perception-Aware Learning, Control, and Coordination for Multi-Robot Search and Rescue
Autonomous SaR requires robots to explore unknown or partially known environments, avoid hazards, coordinate team activities, and guide discovered victims to safety. Improving individual capabilities of robots does not necessarily improve mission outcomes: safer motion can restrict the exploration behavior, faster discovery can increase the rescue demand, and effective guidance depends on victim responses. We introduce PA-SaR, a mathematical and architectural framework for coordinating heterogeneous local control, learning-based exploration, supervisory task allocation, and adaptive approach-lead victim guidance around mission-level objectives. PA- SaR distinguishes environmental ground-truth, robot-specific perceptions, and simulated victim behavior. This provides a common basis for integrating complementary mechanisms and analyzing their interactions across mission stages. We demonstrate the framework through single- and multi-robot simulation studies incorporating model-predictive and learning-based exploration, perception-based safety shielding, event-driven task allocation, and adaptive victim guidance. The results show trade-offs between collision avoidance and search completion, indicate that supervisory allocation improves search completion beyond information sharing alone, and demonstrate that increased exploration does not necessarily yield more rescues. Adaptive guidance increases rescues relative to continuous leading, without a detected improvement in mean or final victim trust or a reduction in mortality. These findings support evaluating local control, team coordination, and evacuation as related but distinct aspects of autonomous SaR within the considered simulation environments.
★ Predictive Neuromorphic Control of a Scanning Mirror System: An Event-Centric Framework
Organizing control around events can reduce sensing and actuation demands while directly linking control decisions to the occurrence and timing of desired system behaviors. This paper proposes a novel event-centric model predictive control (MPC) framework that unifies event sensing, event prediction, and event regulation. Events define the feedback information, prediction horizon, and event-timing objectives, placing them at the center of predictive control design. The framework is developed for a scanning mirror system equipped with a baseline neuromorphic controller composed of spiking neurons that supports sparse actuation. The control objective is to achieve prescribed zero-crossing times and a desired scanning range while minimizing the mechanical energy transferred between the actuator and the mirror. A sampling-based algorithm is presented to obtain suboptimal solutions of the event-centric MPC optimization problem without requiring an explicit event-to-event prediction model. By grounding predictive optimization in events, the proposed framework opens a new direction for MPC design and its integration with neuromorphic control.
comment: 8 pages, submitted to ECC27, this work is the first, foundational work that proposes Event-Centric Model Predictive Control (EC-MPC), a framework that combines model predictive control and neuromorphic control. Philosophy: predicting events but not trajectories; regulation without calibration
★ Concurrent Learning-Based Inertial Parameter Estimation and Control of Quadrotors
In this letter, we consider trajectory tracking control and inertial parameter estimation of a quadrotor UAV whose center of mass (CoM) does not coincide with its geometric center. Onboard payloads like an IMU, a camera, or a battery are not necessarily placed at its geometric center. Moreover, during the flight these sensors might move because of external disturbances and cause a CoM offset. Thus, these events and equipment introduce an offset and cause the inertia matrix to be non-diagonal. We develop an integral concurrent learning (ICL)-based geometric controller that estimates the mass, the CoM offset, and the full inertia matrix online, without an accelerometer or numerical derivatives, and tracks a given trajectory corrected by the offset estimated online. We prove that the estimated parameters converge to their true values under a finite excitation condition. We validate the effectiveness of the proposed algorithm using a simulation example.
comment: 6 pages, 4 figures. To be submitted to IEEE Control Systems Letters (L-CSS) with the CDC 2027 option
★ Ultrawideband Polarization-Rotating Antenna for Enhanced Ground Penetrating Radar Detection
Ground penetrating radar (GPR) represents a powerful and widely adopted tool for nondestructive subsurface detection and imaging. To overcome the impaired detection performance caused by polarization mismatch in conventional single-channel GPR systems, this paper presents an innovative ultrawideband polarization-rotating (UPR) antenna that achieves rotating linear polarizations (LPs) by varying the source frequency for enhanced detection. The developed UPR antenna achieves a continuous polarization rotation from $0^\circ$ to $170^\circ$ across 1.2 to 3.2 GHz, as verified by both full-wave simulations and experimental measurements. The antenna is subsequently integrated into a single-channel stepped-frequency GPR system and evaluated against targets of varying geometries in a sandy environment. Compared with a conventional fixed-LP GPR system, the UPR antenna-enabled GPR system delivers markedly more consistent detection performance across all target types, with particularly pronounced enhancements for polarization-sensitive targets. By enabling ultrawideband frequency-dependent polarization agility, the proposed GPR system enhances subsurface target detection while maintaining the simplicity and low cost of a single-channel measurement architecture.
★ K-PWM: Control-Oriented Structured Bilinear World Models under Partial Observation NeurIPS 2026
World models learn internal simulators of environment dynamics for prediction, planning, and control, yet learning them from limited, noisy, and partial observations remains challenging. While much progress has focused on pixel-based observations, many physical systems instead provide vector-valued measurements that reveal only part of the underlying state. We introduce K-PWM, a probabilistic world model for prediction and control under partial observation. K-PWM couples Koopman-bilinear latent dynamics with a nonlinear observation decoder and performs probabilistic latent-state inference through the learned model via extended Kalman filtering and smoothing, without a separately trained inference network. Its bilinear structure enables exact propagation of the maintained Gaussian belief through the latent transition and efficient rollouts for uncertainty-aware model predictive path integral (MPPI) control. For training K-PWM, we present a generalized expectation conditional maximization (GECM) procedure with closed-form updates for the generators, noise covariances, and latent prior, complemented by gradient-based decoder learning. A Koopman-consistent initialization selects the latent dimension from data and initializes the action-dependent dynamics. With limited training data, K-PWM achieves lower long-horizon prediction error than the evaluated baselines on partially observed Gymnasium MuJoCo systems, while substantially reducing training time relative to the strongest prediction baseline. On high-dimensional MuJoCo Menagerie robots, it achieves competitive tracking accuracy with low per-step computation times, demonstrating potential for real-time deployment. These results support structured probabilistic dynamics as a data- and compute-efficient approach to world modeling and control under partial observation.
comment: Accepted at NeurIPS 2026
★ Local Certificate Repair for Asynchronous Controller Updates in Recommender Systems
We study how to adapt an existing distributed recommendation controller when local dynamics or objective weights change. Starting from a quadratic stability and cost certificate, we construct positive semidefinite corrections with finite spatial support and an explicit residual. A joint admission test allocates the available dissipation margin to these residuals and to interactions among changes and corrections, including pending updates. The test covers complete declared parameter ranges, allowing different operators to progress independently. For a fixed compatible batch, the corrections define a common storage that guarantees exponential stability and bounds the cost accumulated throughout deployment. When the admitted collection evolves, a reservation protocol preserves the certificate and a rate bound on storage increases ensures exponential convergence. Regional linear-quadratic regulator (LQR) design proposes new gain rows for the operators' own users, and a separate residual-based calculation can certify that the final gain lowers the expected cost. A recommendation-network example illustrates the tradeoff between repair depth, data access, and the benefits of deploying an update.
★ A Distributed Observer with Random Agent Activations over Random Communication Networks
This paper develops a distributed observer for the forward-Euler discretization of a continuous-time linear time-invariant system under random local operations and random communications. Under uniform stochastic strong connectivity, we establish almost sure exponential convergence of all local state estimates for sufficiently small discretization step sizes. We further examine how the underlying parameters and components affect the admissible range of the discretization step size and the guaranteed convergence rate bound. Unlike related approaches, the proposed method does not require each local unobservable subspace to admit an $A$-invariant complement.
★ Scenario-Based Assessment of Transport Electrification for Low-Carbon Urban Planning: A Case Study of Phu Quoc, Vietnam
Urban net-zero strategies require a defensible emissions boundary and quantitative estimates of the contribution achievable through individual mitigation measures. This study assesses road-transport electrification within a planned residential development in Phu Quoc, Vietnam, using the assumptions and scenario data of an urban-planning study originally prepared in 2022. A vehicle-class inventory combines population-dependent fleet projections, assumed travel activity, gasoline and diesel consumption, fuel-specific carbon dioxide (CO_2) factors, and electricity-related emissions from battery-electric vehicles. The reference case projects annual road-transport emissions of 59.41, 73.86, and 87.23 tCO_2 for 2022, 2025, and 2030, respectively. Three year-specific substitution scenarios, corresponding to nominal electric-vehicle shares of 25%, 50%, and 100%, yield source-reported avoided emissions of 9.43, 13.64, and 55.41 tCO_2/year, or 15.9%, 18.5%, and 63.5% of their respective baselines. However, a cross-scenario audit identifies an inconsistency in the 2025 mixed-fleet results under time-invariant vehicle-class emission-intensity ratios: under that additional assumption, 2025 avoided emissions would be 23.45 rather than 13.64 tCO_2/year. The original detailed vehicle-energy input tables are not included in the available source document, preventing definitive reconciliation. The study therefore provides a transparent, preliminary planning assessment rather than evidence of citywide net-zero attainment. Its principal value is a reproducible accounting structure and an explicit identification of the data, electricity-system, and infrastructure constraints that must be resolved before policy or investment decisions.
★ GFM Control Automatic Compliance Toolbox
The increasing penetration of converter-interfaced generation reduces system inertia, short-circuit strength, and voltage support. European grid codes, including the ENTSO-E Network Code on Requirements for Generators and the ACER Demand Connection Code, now define disturbance-based validation procedures for grid-forming (GFM) generation and converter-based demand. Commercial RMS tools, however, lack an integrated workflow aligned with these recent requirements. This paper presents an automated Python-PowerFactory compliance toolbox and an RMS test bench for a co-located system comprising a 1 MW GFM photovoltaic (PV) plant, a 0.3 MW GFM battery energy storage system (BESS), and a 1 MW converter-interfaced dynamic load at a 20 kV point of common coupling fed from a 110 kV grid. Extended virtual synchronous machine models and a DCC-oriented load model are tested in batch, with results automatically extracted and plotted for assessment. Results show compliant RMS performance for the slower, electromechanical tests, while the paper identifies which tests require complementary electromagnetic transient (EMT) verification.
★ Grid-forming Control of Offshore Wind through a Diode-Rectifier HVDC System: a 2 GW Case Study
Diode-rectifier-unit (DRU)-based high-voltage direct current (HVDC) transmission considerably reduces the volume and weight of the offshore converter platform for remote wind farms and, hence, its cost, but shifts the task of forming and regulating the offshore AC network to the wind-turbine converters. This paper examines which grid-forming control strategy is the most suitable for DRU power transfer. The DRU commutation relationship shows that the delivered active power depends on the offshore voltage magnitude (but not on the converter voltage angle), so the angle-based active-power loop of virtual synchronous machine control is not closed through the rectifier. A $P-V/Q-ω$ grid-forming architecture, which regulates active power via the voltage magnitude and derives frequency from the reactive power error, is therefore adopted. Electromagnetic transient simulations of a 1GWaggregated grid-forming turbine on a 66 kV collector network with a $\pm$525 kV bipolar DRU link show that the commanded power is delivered over 800 to 1000MW with the offshore voltage held near nominal. The paper also examines how grid-forming operation changes the allocation of control duties within the Type-4 turbine: once the grid-side converter forms the network, DC link regulation must move to the machine-side converter, which limits the achievable bandwidth of the DC-link control. Extending the system to 2 GW with two 500 MW grid-following units achieves full power transfer in simulation, but with the grid-forming converter loaded beyond its continuous rating and a steady-state offshore frequency deviation of approximately 1.8 Hz.
★ Spot Markets for AI Inference: Dynamic Capacity Allocation and Pricing on a Shared Accelerator Fleet
Inference providers size their accelerator fleets for the interactive peak and sell spare capacity through fixed discounts, such as batch tiers at half price. However, inference differs from conventional cloud capacity in four respects: capacity depends on the latency tier through batching, moving weights between models is slow and costly, spot demand can wait until its deadline, and power can bind. In this paper, we develop a framework for dynamic capacity allocation and pricing of AI inference on a shared accelerator fleet. We consider a provider that serves several models under a power cap and sells each model in an interactive tier with a speed promise and in a spot tier for preemptible work with a deadline. In each period, the provider chooses how many replicas of each model to keep loaded and what spot price to quote. We show that one interactive token displaces a fixed number $ρ_m>1$ of spot tokens, and that the optimal spot price lies midway between the interactive price and the effective cost of capacity in the deadline window, which is close to the prevailing half price only when the fleet is slack. We further show that a spot tier raises the optimal reliability of the interactive tier, and that placement should follow persistent demand shifts while prices absorb transients. Building on these results, we design a model predictive controller based on a linear program with explicit approximation bounds, which solves each period in milliseconds. Numerical experiments on a simulated week with a measured daily demand profile, a flash crowd, and a grid curtailment show that the controller raises profit by 7.4% over fixed half-price practice, captures 93% of the gain available with perfect forecasts, keeps the interactive promise through both events, and leaves spot buyers better off.
comment: 39 pages, 12 figures, 4 tables
★ A Multistage Extension of Witsenhausen's Counterexample with Linear Signaling
We study a multistage scalar extension of Witsenhausen's counterexample in which controller 1 observes the initial state exactly, whereas every subsequent controller observes only a local noise-corrupted state and does not inherit its predecessors' observations, so that every nonterminal action both regulates the state and shapes the signal available to the next controller. Restricting the signaling controllers to linear policies while leaving the terminal controller unrestricted, we compute the exact value within this class and reduce the optimization to a deterministic dynamic program in a single propagated variance. The optimal intermediate gains are shown to have a shrinkage form, and the value functions of that program are concave. For two stages, the benchmark admits a closed form on a matching locus, where for small effort weights it has two distinct minimizers, and at the standard parameter instance on that locus a two-point nonlinear policy strictly improves upon it. For three stages, numerical continuation at a reference instance reveals three distinct local minima, and two values of the initial variance at which the lowest of the three changes abruptly. Under the terminal-cost normalization we further prove, for every finite horizon, that the optimal linear value becomes asymptotically equal to the signaling-idle benchmark as the initial variance tends to infinity.
comment: 8 pages, 3 figures
★ Numerical Stability Issues and a Mitigation Method in Switching Events Calculation Considering State Variable Discontinuities for EMT Simulation
Some existing methods for switching events calculation considering state variable discontinuities for EMT simulation can encounter numerical stability issues, either over-stabilizing or destabilizing the studied power systems. This paper documents these issues in detail with a concrete example. The underlying mechanism of the issues are revealed via systematic analysis. A mitigation method for the issues is proposed. The convergence behavior of the method is analyzed. Numerical examples demonstrate the effectiveness of the method. It is observed that the method also helps to improve the accuracy of the simulation results.
comment: Copyright may be transferred without notice, after which this version may no longer be accessible
★ Learning the NMPC with Stability Guarantees
Although nonlinear model predictive control (NMPC) is widely used, its online optimization is computationally expensive. Neural network approximations of NMPC can effectively reduce the online computational burden but generally lack stability guarantees. This paper proposes an approximate NMPC framework with guarantees of constraint satisfaction and closed-loop stability. The approximation error is addressed through constraint tightening, and the resulting tightened NMPC is used to generate the training dataset of the approximate NMPC. The maximum allowable approximation error is prescribed a priori for the design of tightened NMPC and satisfied in a probabilistic sense through an iterative trial-and-error procedure alternating between network training and a posteriori verification. Finally, the effectiveness of the proposed framework is demonstrated through a numerical example of a constrained nonlinear system.
comment: 13 pages, 7 figures, this paper was not presented at any IFAC conference
★ Analysis increments for model bias diagnosis and correction: Theoretical foundations
A common method for diagnosing and correcting model bias in weather and climate prediction is based on analysis increments: the difference between the analysis and the forecast at each data assimilation cycle. Their time average is used to estimate and correct the bias, and machine learning is increasingly used to map the forecast state to the increments, giving a state-dependent correction. Despite its widespread use, the method lacks theoretical foundations. We first analyse a Kalman filter whose forecast model has a constant bias. We show what the averaged increment measures, and prove that iterating the correction removes all of the bias seen through the observations; for linear dynamics, no method can identify the rest. We then extend the analysis to state-dependent biases. For linear dynamics, each increment records the model error over the filter's memory, as seen through the observations. We use this to determine what the increments measure when averaged over a long period, averaged over periods with similar slowly varying conditions, and used without averaging, as in machine-learned corrections. Numerical experiments confirm the theory for a linear model and suggest that its main conclusions carry over to the nonlinear Lorenz '96 model, including for a correction learned by a neural network.
comment: 15 pages, 5 figures
★ Scalable Neural Safety Certification via Monotonicity
Learning-based safety certification offers a promising route for verifying complex dynamical systems when explicit models are unavailable, but existing data-driven methods often scale poorly: they treat the system as a black box, rely on dense discretizations or Lipschitz bounds, and consequently suffer from sample complexity that grows exponentially with dimension. This paper shows that a common structural property of dynamical systems--monotonicity--can be used to break this curse of dimensionality. We develop a data-driven framework for robust safety verification and safe controller synthesis for unknown monotone systems using only simulator queries. Our main theoretical result proves that, for monotone systems with upper-closed unsafe sets, safety is completely characterized by the existence of monotone inductive barrier certificates. This structure reduces certificate verification over continuous state spaces to localized boundary checks over finitely many cells. To learn such certificates, we introduce Max-Monotone Networks, a monotone neural architecture with universal approximation guarantees for monotone functions. We then propose a training and refinement algorithm that, upon successful termination, returns formally valid neural barrier certificates and, under mild partition assumptions, requires only $O(n)$ simulator queries. Across high-dimensional benchmarks, including oscillator networks with up to $13{,}659$ states and traffic networks with $1{,}000$ states, our method certifies safety in regimes where prior approaches either do not converge or exceed memory limits. These results demonstrate that exploiting order structure can make neural safety certification both scalable and formally sound.
★ Feasibility-Aware Learning for Network-Constrained Unit Commitment
Learning-based approaches to network-constrained unit commitment (NCUC) usually still rely on an optimization solver to repair or complete their predictions, which limits the speedup they can deliver. This paper proposes FALCON-UC, a feasibility-aware graph learning framework that predicts commitment and dispatch jointly and produces final schedules without solving a mixed-integer program at inference time. It combines an LSTM--GraphSAGE predictor over a fully connected generator graph, a differentiable correction that embeds commitment-dependent dispatch bounds, power balance, and PTDF-based branch flows in training, and deterministic commitment, dispatch, and ramp-coupled reserve repairs at inference. On the 2383-bus system, FALCON-UC achieves $99.09\%$ commitment accuracy and a $0.4350\%$ mean absolute cost gap with all evaluated constraints within tolerance at $0.0562$ s per instance, a $2.27\times10^{3}$ speedup over the solver. MTLR R-SCUC-FL is slightly more accurate but still needs an optimization solver to complete each schedule, which takes $262.2$ s per instance, $4.67\times10^{3}$ times the FALCON-UC runtime. A label-only baseline without repair violates the balance and branch limits on every 2383-bus test instance, and FALCON-UC also meets all evaluated constraint tolerances on a 1354-bus system.
★ Floquet-Parameterized Koopman-MPC for Tracking Periodic Orbits
Designing real-time control for nonlinear systems to track periodic orbits can be challenging, especially for complex and coupled dynamical systems (e.g., tethered kites), where the true model parameters are hard to obtain. To address this, we propose a physics-informed Koopman learning approach that incorporates periodic-orbit stability directly into the learned linear predictor. The approach combines the underlying periodic structure of the dynamics along with periodic state-input trajectory data to learn physically consistent linear predictors (in lifted/observable space). We leverage Floquet theory, which studies the stability of periodic orbits in nonlinear systems, to obtain the periodic stability information of a given orbit via a monodromy matrix (i.e., the state transition matrix over an entire period). Subsequently, we impose open-loop monodromy constraints to learn Floquet-parameterized linear predictors using a least-squares Extended Dynamic Mode Decomposition (EDMD) formulation, thereby constraining the search space and aiding in Koopman learning. We then utilize these linear predictors to design a model predictive control (MPC) law for tracking periodic references. Finally, we evaluate the designed framework on a cartpole system under periodic forcing and compare it against a purely data-driven baseline.
comment: 6 pages, 2 figures
♻ ★ Conditions for Normality of Pontryagin's Maximum Principle in the Absence of State Constraints
Pontryagin's Maximum Principle is said to be "normal" for the problem at hand when it has a solution (maximizer u*) that explicitly involves the problem's cost function, making the maximum principle helpful (while in the opposite situation it is of no help). The aim of the present paper is, for Lagrange optimal control problems without state constraints, to identify classes of problems for which the Principle is normal. General conditions are introduced that guarantee normality, and classes of problems that satisfy these conditions are identified. Such classes include problems for which the constrained (by bounds on the control values) "augmented" reachable set, with cost along an upward vertical axis, has a convex lower-boundary function and the optimal terminal state lies in the relative interior of the reachable state set, with the terminal constraint within the affine hull of that set. Such classes include problems for which the constrained (by bounds on the values of the control signal) "augmented" (with the cost value along an upward "vertical axis") reachable set has a "convex" lower boundary for which the optimal terminal state lies in the relative interior of the set of reachable states. Special attention is given to Lagrange steering problems: sufficient conditions are obtained on a nested sequence of sub-classes of such problems, culminating with the class of steering problems with scalar control signal and linear time-invariant (LTI) state dynamics and objective integrand, for which normality is guaranteed whenever all eigenvalues of the state dynamics matrix A are real (e.g., if A is symmetric or triangular or the system is overdamped) or have small enough imaginary parts; simple 2 x 2 examples show that, in contrast with the situation in the finite-dimensional case, this property does not extend to arbitrary A matrices.
♻ ★ Digital Twin Networks for 6G Wireless Systems: Architecture, Enabling Technologies, AI-Driven Control, and Open Challenges
The transition to the Sixth Generation (6G) of mobile networks requires proactive and deterministic orchestration to satisfy the stringent key performance indicators of future services, including ultra-reliable low-latency communications, enhanced mobile broadband, and massive machine-type communications. Digital Twin Networks (DTNs) have emerged as a foundational technology to meet these demands, offering real-time and high-fidelity virtual replicas of the physical network. Although the literature explores DTNs conceptually, a gap exists in the coverage of technical classification and computational feasibility evaluations. This survey addresses this gap by formally categorizing state-of-the-art DTN architectures into passive monitoring twins and active control twins. We evaluate their enabling technologies, namely ray-tracing, reconfigurable intelligent surfaces, artificial intelligence, and multi-access edge computing. Importantly, we analyze the complexity of ten state-of-the-art frameworks, five reporting complexity expressions, four derived in this survey, and one not derivable, to assess hardware scalability and inference bottlenecks. These architectures are then linked to forthcoming 6G use cases, including smart cities, Industry 5.0, healthcare, and smart grids. Finally, we synthesize crucial unresolved challenges, highlighting graphics processing unit hardware limitations, cyber-physical actuation latency, and the need for a zero-trust security paradigm, offering research directions toward the unified Internet of Everything.
♻ ★ All-Pass Function OPF: A Solver-Friendly, Physics-Preserving Approximation of AC OPF
This paper presents a rational approximation of the AC optimal power flow (AC OPF) problem based on an all-pass approximation of the exponential power flow kernel. The classical AC OPF relies on trigonometric coupling between bus voltage phasors, which yields a nonconvex program with oscillatory derivatives that can slow, or in some cases destabilize, interior-point methods. We replace the trigonometric terms with an all-pass function (APF) approximation whose real and imaginary components act as smooth surrogates for the cosine and sine functions, and we introduce a pre-rotation to shift the argument of the approximation toward its most accurate region, ensuring that the reformulated power flow model preserves physical loss behavior, maintains the symmetry of the classical kernels, and improves the conditioning of the Jacobian and Hessian matrices. The proposed APF OPF formulation remains nonconvex, as in the classical model, but it eliminates trigonometric evaluations and empirically produces larger and more stable Newton steps under standard interior-point solvers. Numerical results on more than 25 IEEE and PGLib test systems ranging from 9 to 10{,}000 buses demonstrate that the APF OPF model achieves solutions with accuracy comparable to that of the classical formulation while reducing solver times, indicating a more solver-friendly nonconvex representation of AC OPF. All code, functions, verification scripts, and generated results are publicly available on \href{https://github.com/LSU-RAISE-LAB/APF-OPF}{GitHub}, along with a README describing how to run and reproduce the experiments.
♻ ★ Robust Adaptive Backup Control Barrier Functions
We propose a notion of robust adaptive backup control barrier functions for nonlinear control affine systems with parametric uncertainty in both the drift dynamics and actuation matrix. Backup control barrier functions guarantee safety by predicting the system's trajectory under a pre-certified safe controller. However, these predictions rely on the model and can be inaccurate when the system contains unknown parameters. To address this issue, we estimate the unknown parameters using element-wise certified adaptive estimators that provide a parameter adaptation law and component-wise estimation error bounds. We compute the backup flow using the estimated model and tighten the safety conditions using these certified bounds. The resulting safety conditions account for the sensitivity of the predicted flow to parameter estimation errors. Moreover, to handle uncertainty in the actuation matrix, we use a duality-based reformulation that enables the use of a computationally efficient quadratic-program-based safety filter. We prove that controllers satisfying the proposed robust adaptive backup control barrier function constraints guarantee safety under parametric uncertainty and input constraints.
comment: IEEE Control Systems Letters (L-CSS)
♻ ★ Pricing Short-Circuit Current via a Primal-Dual Formulation for Preserving Integrality Constraints
Synchronous Generators (SGs) currently provide important levels of Short-Circuit Current (SCC), a critical ancillary service that ensures line protections trip during short-circuit faults. Given the ongoing replacement of SGs by power-electronics-based generation, which has a hard limit on current injection, it has become relevant to optimize the procurement of SCC services provided by remaining SGs. Pricing this service, i.e., computing dual variables of SCC constraints, is however challenging due to the integrality constraints in Unit Commitment (UC). Existing methods, e.g., dispatchable pricing and restricted pricing, attempt to address this issue but exhibit limitations in handling non-convexities, resulting in SCC prices that either fail to cover the operating costs of units or lack interpretability. To overcome these pitfalls, we adopt a primal-dual formulation of the existing SCC-constrained dispatch that preserves the binary UC for effectively computing shadow prices of SCC services. Using a modified IEEE 30-bus system, the proposed method is compared with the previously developed pricing schemes. It is demonstrated that, under the proposed pricing method, revenue-adequate and explicit service prices can be assigned without the need for make-whole payments, an advantage that cannot be achieved by other pricing approaches.
♻ ★ Pre-Solve Horizon Sampling of Frozen Acceleration Residuals for Quadrotor NMPC under External Forces
We investigate how a frozen supervised acceleration-residual model should be sampled within nonlinear model predictive control (NMPC) of a quadrotor subject to external forces. Cyclewise deployment holds the current-state residual across the prediction horizon, whereas stagewise deployment evaluates the same network at shifted warm-start nodes before each optimization solve. The correction sequence is fixed during optimization and the network is not updated online. In 220 simulated pulse-load conditions, cyclewise and stagewise deployment reduce mean integral absolute position error (IAE) relative to nominal NMPC by 4.38% and 7.77%, respectively. However, both worsen all 55 hover conditions, and most of the aggregate improvement arises after pulse removal rather than during loading. In 96 prescribed random-force blocks, stagewise deployment reduces mean IAE by 11.99%; the evaluated finite-budget reinforcement-learning add-ons do not improve their matched supervised means in either force bank. A separate causal innovation compensator reduces mean IAE by 27.47% relative to stagewise deployment on 32 confirmation conditions, at the cost of a 10.19% increase in angular-rate root-mean-square error. In one synthetic computational-fluid-dynamics inflow field, stagewise deployment reduces four-task mean IAE by 2.01% against a same-graph zero-residual control. These simulation results quantify deployment-dependent tracking gains and trade-offs, without establishing general wind rejection or hard-real-time guarantees.
comment: 25 pages, including supplementary material. Revised title and abstract; clarified the contribution, residual sampling procedure, and discussion. Numerical results unchanged
♻ ★ A complete analytical characterization of optimal collision-free motion for two robots
Collision-free motion planning for multi-agent robotic systems remains an active research area because of the complexity of coordinating robot trajectories, with applications ranging from coordinated drone shows to autonomous operations on land, in the air, and on water. Importantly, the analytical difficulty of nonconvex collision-avoidance constraints has so far driven the development of exclusively numerical methods. We derive a complete analytical characterization of the optimal trajectories for two robots with zero initial and final velocities, under double-integrator dynamics and a squared-acceleration objective. We prove that these trajectories belong to eight distinct contact families and determine which family applies for each configuration of initial and final positions. We analyze the geometrical properties of the optimal trajectories and provide a numerical algorithm for computing them. These insights have already been adopted to improve the performance of numerical methods for multi-agent motion planning by several orders of magnitude.
♻ ★ Finite-Sample Probabilistic Safety Certification for AI-Based Grid-Edge Coordination
Coordinating a large population of flexible grid-edge devices can alleviate the need for time-consuming and capital-intensive network upgrades. Artificial intelligence (AI)-based control methods such as multi-agent reinforcement learning or imitation learning are promising to this coordination due to their real-time decision scalability. However, system operators still need an independent and rigorous way to decide whether a given AI system is safe enough for deployment. This paper develops a \emph{finite-sample} probabilistic safety certification framework for AI-based grid-edge coordination. The central idea is to reduce the complete input--AI--decision--evaluation workflow to a binary unsafe outcome under an operator-defined safety specification, and then use exact binomial inference to certify the corresponding unsafe operation probability. Given a set of held-out calibration scenarios, the framework returns the tightest one-sided upper certificate and an accept/reject deployment criterion that controls the probability of false safety certification. Because the certification is for the calibration distribution that may \emph{deviate} from the future operation, we further combine the nominal certificate with physically interpretable sample-space adversarial attacks, a concept widely used in AI models to investigate their fragility. Case studies on grid-edge flexibility coordination with 1{,}000-agent AI models (independent parameters) verify the safety guarantee and the value of integrating adversarial attacks into a rolling-window training-certification-deployment flow.
♻ ★ Over-the-Air Consensus-based Formation Control of Heterogeneous Agents: Communication-Rate and Geometry-Aware Convergence Guarantees
This paper investigates the formation control problem of heterogeneous, autonomous agents that communicate over a wireless multiple access channel. Instead of avoiding interference through orthogonal node-to-node transmissions, we exploit the superposition property of the wireless channel to compute, at each receiver, normalized convex combinations of simultaneously broadcast neighbor signals. At every communication instant, agents update their reference positions from these aggregates, and track the references in continuous time between updates. The only assumption on the agent dynamics is that each agent tracks constant reference positions exponentially, which accommodates a broad class of platforms. Under this assumption, we analyze the resulting jump-flow system under time-varying communication graphs and unknown channel coefficients. We derive a communication-rate based sufficient condition that guarantees convergence to a prescribed formation. We then provide a geometry-aware refinement showing how favorable tracking transients can relax the required condition. Simulations with unicycle agents illustrate the theoretical results and demonstrate a substantial reduction in the number of required orthogonal transmissions compared to interference-avoiding node-to-node communication protocols.
♻ ★ Scaling and Trade-offs in Multi-agent Autonomous Systems
Designing autonomous drone swarms is hampered by a vast design space spanning platform, algorithmic, and numerical-strength choices. We perform large-scale agent-based simulations in three canonical scenarios: swarm-on-swarm battle, cooperative area search with attrition, and pursuit of scattering targets. We demonstrate how dimensional-analysis and data-scaling can be leveraged to collapse performance data onto scaling functions that are mathematically simple, yet counterintuitive and therefore difficult to predict a priori. These scaling laws reveal success-failure boundaries, including sharp break points which we show can be framed as an ``effective swarm size.'' Additionally, we show how this technique can be used to quantify trade-offs between agent count and platform parameters such as velocity, sensing or weapon range, and attrition rate. Furthermore, we show the benefits of embedding an optimal path planning loop within this framework, which can qualitatively improve the scaling laws that govern the outcome. The methods we demonstrate are highly flexible and would enable rapid, budget-aware sizing and algorithm selection for large autonomous swarms.
comment: Published in Physical Review E. 16 pages, 12 figures
♻ ★ Equilibria and Consensus in Mixed Weighted-Average and Weighted-Median Opinion Dynamics: The Consensus-Promoting Role of Weighted-Average Individuals
How individuals respond to their neighbors' opinions shapes whether a networked group reaches consensus. This paper analyzes synchronous discrete-time opinion dynamics on a fixed influence network, where individuals use either weighted-average (WA) or weighted-median (WM) opinion-aggregation mechanisms with individual-specific inertia coefficients. We characterize equilibria through graph-theoretic conditions on opinion level sets and WA balance equations, proving that every solution converges to a Lyapunov-stable equilibrium. We define type-dependent cohesive sets, whose WA members assign all their influence weight to the set and WM members assign at least half. The absence of two disjoint such sets gives a necessary and sufficient graph-theoretic criterion for consensus from any initial condition. Otherwise, an open set of initial conditions leads to persistent dissensus. Converting WM individuals to WA cannot create type-dependent cohesive sets and preserves existing guarantees of consensus from any initial condition. If the conversion eliminates every disjoint pair of these sets, the converted system also guarantees consensus from any initial condition. Four numerical studies examine the locations and proportion of weighted-average individuals, as well as the influence weights, while comparing the consensus criterion with finite-horizon simulation outcomes. In the four network ensembles with 100 individuals, the observed frequency of reaching a prescribed consensus tolerance is nondecreasing with WA proportion at each tested simulation horizon. This trend supports the consensus-promoting role of WA individuals, although individual realizations need not follow it.
comment: 29 pages, 10 figures, including supplementary material
♻ ★ Low-Thrust Trajectory Optimization for Cubesat Lunar Mission: HORYU-VI
A three-stage low-thrust trajectory is investigated for the HORYU-VI CubeSat lunar mission: pre-flyby deceleration, lunar capture, and orbit circularization. Impulsive reference trajectories initialize finite-burn targeting in the General Mission Analysis Tool (GMAT). The archived case uses constant 12 kg spacecraft mass and 1.08 mN effective thrust. The first reported bound lunar state occurs at day 184.1. At the end of the 381.3-day record, the osculating lunar orbit has semi-major axis 3,862 km and eccentricity 0.0867. Integrating the reported thrust state yields an estimated velocity increment of 839.8 m/s over 108.00 days of thrust. At the configured 2800 s specific impulse, the ideal rocket-equation propellant equivalent is 0.361 kg; mass depletion was disabled in the propagation. The record demonstrates lunar capture and partial circularization. Completion of the intended 100 km science orbit, coupled mass-depleting propagation, and uncertainty analysis remain necessary to assess operational feasibility.
comment: 8 pages, 8 figures. Presented at the 71st International Astronautical Congress, 2020
♻ ★ Distributed Learning of $α$-Potential Functions in Strategic Multi-agent Systems
We study the problem of computing the tightest \(α\)-potential approximation of a general-sum game over continuous action spaces, within a prescribed class of potential functions and when each player has access only to its own utility function. The difficulty is twofold: the approximation error involves a worst-case search over an infinite set of unilateral deviations, and the required utility information is distributed across players. For a linear-in-parameters potential class, we use an exact finite-tuple reformulation that separates the problem into a global outer search over deviation tuples and distributed convex inner problems. We develop a primal--dual inner oracle tailored to this structure and establish a uniform one-sided accuracy guarantee. This oracle can be combined with global outer search to obtain an end-to-end guarantee on the outer optimization error. We also develop a projected zeroth-order outer method as a computationally lighter alternative for higher-dimensional problems. Numerical experiments illustrate the accuracy--computation tradeoff between the two outer-search methods and show that the proposed optimization framework can improve upon analytical \(α\)-potential constructions.
comment: 15 pages, 4 figures
Multiagent Systems
★ Network Contention State Aware Dynamic Optimizer for Accelerator Collective Communication Libraries
Collective Communication Libraries (CCL) are critical for large-scale distributed AI/ML training, yet their configurations are determined by static heuristics that do not account for real-time network conditions. Our empirical study shows that these heuristics perform near-optimally in contention-free environments, but degrade substantially under background traffic. To address this, we present a cooperative multi-agent run-time optimizer for accelerator collective communication. Our approach employs a two-agent architecture: (i) detector- a Long Short Term Memory(LSTM) recurring neural network fed classifier head with a sequential probability ratio test that distinguishes sustained contention from transient spikes; and (ii) selector- a configuration selector built on a Dueling Double DQN (D3QN) with Conservative Q-Learning that adapts configuration parameters in response. We implemented this dynamic tuner over NCCL, a widely used CCL developed by NVIDIA. When evaluated on Chameleon Cloud's isolated bare-metal cluster, this dynamic tuning approach reduces AllReduce latency by 36% over NCCL and 32% over a fully converged AutoCCL. We further evaluated it across six real AI workloads spanning convolutional networks, transformers, and recommendation architectures. Our multi-agent tuner approach delivers up to 65% speedups on Delta cluster with NVIDIA A100 GPUs and Slingshot-11 fabric internode connectivity.
★ MultiTurnRoute: A Platform for Building and Evaluating Multi-Turn LLM Routers NeurIPS 2026
In a multi-turn conversation, the choice of language model at one turn can affect which model should answer the next. The selected model's reply shapes the context for the next query, and its cached prompt prefix can reduce the cost of using that model again. We introduce MultiTurnRoute, a platform for building and evaluating multi-turn LLM routers. A shared interface gives built-in and user-defined routers the current query, dialogue history, and an estimate of each model's reusable prefix length. Our routing algorithm, Chain-of-Route (CoR), uses a learned gate over recent and recurrent dialogue history to predict response quality and estimates cost from each model's reusable prefix. MultiTurnRoute Arena ranks routers on recorded conversations and compares session quality and cost in interactive runs where a simulated user responds to each reply. On a benchmark drawn from WildChat and LMSYS-Chat-1M with per-turn quality annotations for five LLMs, CoR reduces the normalized cost required to match the strongest model's quality by up to 20.3% relative to the strongest routing baseline, while matching or improving peak response quality.
comment: Accepted by NeurIPS 2026
★ A Gamified Framework for Conflict-Driven Design: Discovering Strategies for Collaborative Resilience with Multi-Agent Learning
Conflicting stakeholder objectives make it difficult to coordinate investments in urban resilience. We model conflict-driven urban design as a non-zero-sum digital board game in which stakeholder agents allocate scarce resources and balance individual payoffs against shared urban development and resilience objectives. We use multi-agent reinforcement learning (MARL) to train policies and examine their behavior across three settings. On a 4 x 4 board, learned policies reach about 84% of the total return found by a simulated annealing benchmark. On an 8 x 8 board with spatial and resource constraints, trained agents favor projects that improve resilience in self-play. In a 12 x 12 simulation initialized from historical maps of Tatsumi, Tokyo, trained agents achieve a mean total score of 452.1, compared with 357.3 for a random-policy baseline across 10,000 simulation runs. Gains in adaptability, sustainability, flood resistance, and greenery come with reductions in vitality and density. A human-in-the-loop interface supports replay and lets participants take over stakeholder decisions, providing a testbed for exploring these trade-offs in participatory urban planning.
comment: 10 pages, 11 figures, 1 table. Author version of a paper published in the proceedings of ACADIA 2025: Computing for Resilience
★ Selective Cooperative Perception via Blind Spot Reasoning and Agentic Dialogue with Vision-Language Models
Real-time cooperative perception (CP) in multi-agent traffic environments is highly resource-intensive, which posing scalability challenges for large-scale deployment. Efficiency can be significantly improved by selectively performing CP only in safety-critical scenarios and for key occluding objects. Vision-Language Models (VLMs) are well-suited for complex reasoning in dynamic and uncertain environments, making them a promising tool for this selective CP task. This study proposes a VLM-driven agentic framework for blind spot reasoning and dialogue-based CP. A mixed-model architecture is designed that integrates conventional object detection for stable perception, vision-language reasoning for identifying impactful occlusions, and a physics-based geometric module to align reasoning with real-world visibility constraints. The framework also incorporates a Chain-of-Thought (CoT) prompting mechanism to enhance interpretability and intent-aware decision-making. We collect real-world data from connected vehicles operating in urban traffic and compare the driving-critical blind spot identification and dialogue performance of four state-of-the-art VLMs. Experimental results demonstrate that our method achieves over 80% accuracy in driving-critical blind spot detection and over 90% success in dialogue-driven perception sharing. This work provides a scalable, interpretable, and bandwidth-efficient alternative to conventional CP pipelines, and offers new insights into agentic communication design for future connected traffic systems.
★ What Do Collusion Probes Read? Three Pitfalls in Evaluating Multi-Agent Collusion Monitors
Activation probes look like near-perfect monitors for secret collusion among large language model (LLM) agents. We show that they mostly detect that an agent was told to collude, not that it decided to. Our benchmark (2,500 committee and 1,476 code-review episodes) adds agents who receive the collusion instruction and refuse it, separating instruction from decision. Published detectors barely tell colluders from these refusers, and our best probe separates them mostly by reading the instruction: without it, agent AUROC falls from 1.00 to 0.61. Two further pitfalls inflate reported results: scored after the vote, a rule that only reads the votes reaches AUROC 0.990 but finds nothing before it; and in 57% of attempts no colluder argues for the target in public. The decision is still there: a probe trained on agents' own choices reads it (AUROC 0.73 without the instruction), and probes find colluders whose goal was fine-tuned into the weights (AUROC 0.995; pre-registered). We turn the controls into a protocol for testing any collusion monitor, and adapt scan statistics into an online monitor for groups of any size.
comment: 30 pages, 5 figures. Code: https://github.com/mainul3195/collusion-probe-pitfalls ; data: https://huggingface.co/datasets/mainul3195/collusion-probe-pitfalls ; archive: doi:10.5281/zenodo.23636508
★ M3FT: Micro-Macro-Micro Reasoning for Multi-Agent Cryptocurrency Perpetual Futures Trading
Cryptocurrency perpetual futures form a large-scale online financial market in which automated agents manage interacting asset positions under changing conditions. Policies relying solely on micro-level numerical states may overlook cross-asset relationships and become vulnerable to regime shifts, while macro-level linguistic judgments alone cannot ensure precise measurement or optimize positions under trading costs and risk constraints. Connecting market interpretation to coordinated multi-asset decisions remains challenging. We propose M3FT, a micro-macro-micro framework for multi-agent perpetual futures trading. The first micro stage constructs time-aligned numerical states from exchange and account observations through deterministic processing. The macro stage transforms these measured states into explicit market interpretations without adding external market information. Four asset-specific teams, each comprising a Market Interpreter, an Asset Communicator, and a Risk Reviewer, refine these interpretations through experience retrieval, cross-asset communication, and review of joint-position risks. The final micro stage combines encoded linguistic representations with the original numerical states to learn continuous target positions. We implement this framework with a language-conditioned multi-agent policy: asset actors use global and private numerical states together with linguistic context, while centralized critics evaluate joint actions using portfolio-wide information and shared-account rewards. Experiments on real cryptocurrency perpetual futures data, including comparisons across language models and policy-learning algorithms and component ablations, show that the best-performing M3FT configuration achieves a higher mean test return than the evaluated direct LLM, numerical RL, and rule-based baselines.
comment: 9 pages, 3 figures, 2 tables
★ Who Remembers What? Multi-Granularity Memory Extraction for Multi-User, Multi-Agent Interaction
Self-evolving agents improve their own skills, memory, and harness code from experience without retraining the base LLM, and group-evolving agents go further by sharing these lessons across agents. Yet most such systems overlook that sharing across users and heterogeneous agents requires deciding who can benefit from each lesson and what information must remain private. We present MixUA, a memory system that distills independent user-agent trajectories into lessons and assigns each lesson to a user-agent pair, user, agent, or global scope according to whom it benefits and what it may disclose. For evaluation, we introduce CrossLearnBench, a mixed-user-agent benchmark that pairs established agent tasks with user preferences and privacy probes that test whether private facts leak into memory other users can read. With five users and seven agents, MixUA raises pass rates by up to 16 points over no memory and 5 over a single global memory, even on tasks previously tried only by other users on other agents. It retains up to 90% of previously stated preferences, and at most 0.02% of privacy probes find a planted private fact in a forbidden scope. We hope MixUA offers a scalable path toward a global system where users and agents co-evolve.
★ LSRP*: Scalable and Anytime Planning for Multi-Agent Path Finding with Asynchronous Actions
Multi-Agent Path Finding (MAPF) seeks collision-free paths for multiple agents from their respective starting locations to their respective goal locations while minimizing path costs. Although many MAPF algorithms were developed, most of them rely on a common assumption on synchronized actions, where the actions of all agents start at the same time and always take a time unit. This assumption may limit the use of MAPF planners in practice. To get rid of this assumption, recently, an algorithm called Loosely Synchronized Rule-based Planning (LSRP) was proposed, which can find sub-optimal solutions for many agents. However, LSRP often finds poor quality solutions due to its unbounded sub-optimality. This paper develops a new anytime planner called LSRP* that can keep improving solution quality after the initial solution is obtained until the runtime budget depletes. We analyze the properties of LSRP* and test it against several baselines with up to 1000 agents on various maps. LSRP* can handle up to 25% more agents than LSRP and can reduce up to 40% of the solution cost found by LSRP.
comment: 15 pages, 14 figures
♻ ★ Convergence of Byzantine-Resilient Gradient Tracking via Probabilistic Edge Dropout
We study distributed optimization over networks with Byzantine agents that may send arbitrary adversarial messages. We propose Gradient Tracking with Probabilistic Edge Dropout (GT-PD), which combines a universal self-centred projection that clips incoming messages to a radius-$τ$ ball around the receiver with a fully decentralized probabilistic dropout rule driven by a threshold-gated, dual-metric trust score in the decision and tracking channels. These identity-free defences bound adversarial perturbations while preserving doubly stochastic mixing. The projection preserves the honest network average exactly in every realization, even when honest messages are clipped. Under complete Byzantine isolation ($p_b=0$), GT-PD converges linearly to a neighbourhood of order $ασ^2/(μn_{\mathcal H})$ plus higher-order network terms, matching centralized stochastic gradient descent to first order in the step size. We also give a radius computable from problem data that ensures honest messages remain unclipped with high probability. Under partial isolation ($p_b>0$), a leaky variant, GT-PD-L, controls accumulated tracking perturbations and converges linearly to a bounded neighbourhood governed by stochastic variance and the clipping-to-leak ratio, under explicit conditions on the step size, leak, and aggregate Byzantine mixing weight. With two-tier dropout and $p_h=1$, Byzantine isolation adds no variance to honest consensus dynamics. MNIST experiments under Sign Flip, ALIE, and Inner Product Manipulation attacks show that GT-PD-L outperforms coordinate-wise trimmed mean by up to 4.3 percentage points under stealth attacks.
♻ ★ Your Agents Are Aging Too: Agent Lifespan Engineering for Deployed Systems NeurIPS 2026
Long-lived AI agents are increasingly deployed as persistent operational systems, yet they are still mostly evaluated like freshly initialized models. Day-one benchmarks miss a basic systems question: how does an agent's reliability change after deployment? Even when model weights are frozen, an agent's effective state keeps changing as it compresses interaction history, retrieves from a growing memory store, revises facts after updates, and undergoes routine maintenance. Reliability therefore becomes a lifespan property of the full agent harness, not only a snapshot property of the base model. We introduce AgingBench, a longitudinal reliability benchmark for agent lifespan engineering: measuring not only whether deployed agents degrade, but what form the degradation takes and where repair should target. AgingBench organizes agent aging into four mechanisms: compression aging, where writing and rewriting drops future-relevant details; interference aging, where accumulated similar memories crowd out the target fact; revision aging, where changed or derived state is not updated correctly; and maintenance aging, where lifecycle events such as model swapping trigger regressions. To diagnose these failures, AgingBench uses temporal dependency graphs and counterfactual probes that produce diagnostic profiles for the write, retrieval, and utilization stages of the memory pipeline. Across 6 scenarios, 10 memory systems and 6 models from two families, we find that agent aging is not one-dimensional over 400 runs spanning 50-200 sessions: a well-formed reply can hide a value the memory no longer supplies; derived state collapses as updates accumulate; and the same wrong answer can require different repairs depending on what the diagnostic profile points to.
comment: NeurIPS 2026
♻ ★ Sensitivity Analysis of Decision Rules in an Agent-Based Agro-Ecological Model
Agent-based models are well-suited for modelling agro-ecological systems for decision-making, yet turning their simulations into actionable insight requires identifying which decisions actually drive the behaviours. In such a context, sensitivity analysis stumbles upon a structural feature of many complex systems: the parameters are not a flat vector but a decision tree in which many parameters appear only when a higher-level choice activates them. Classical global sensitivity analysis presupposes independent inputs over a fixed Cartesian product and is ill-posed on such a branched space and biased by construction. We instead treat the structure of the decisions as a first-class modelling object. Using advanced surrogate modelling techniques, we encode the crop-management itinerary of the high-resolution agro-ecological modelling platform MAELIA as a hierarchical design space featuring both categorical and numerical variables. We propose a novel dedicated method to analyse this kind of hierarchical space, a random-forest-supervised HSIC-ANOVA, which ranks main effects and interactions natively on the tree while placing rarely and always active decisions on an equal footing. We show how, when applied to simulation outputs such as nitrogen leaching, soil organic carbon variation, and yield predictions, it can deliver an interpretable, decision-oriented reading.
comment: Published, proof-read version, PRIMA 2026
♻ ★ Multi-Agent Conditional Diffusion Planning for Decentralized Wireless Resource Allocation
In wireless communication systems, efficient and adaptive resource allocation is essential for enhancing overall Quality of Service (QoS). Compared with conventional Model-Free Reinforcement Learning (MFRL), Model-Based RL (MBRL) first learns a generative world model for subsequent planning, enabling more stable training through the reuse of historical experience. However, deploying MBRL in large-scale wireless networks remains challenging due to high-dimensional stochastic dynamics, strong inter-agent cooperation, and stringent communication constraints. To overcome these challenges, we propose a Multi-Agent Conditional Diffusion Model Planner (MA-CDMP) for decentralized communication resource management. Following the Decentralized Training with Decentralized Execution (DTDE) paradigm, MA-CDMP models each communication node as an autonomous agent and employs Diffusion Models (DMs) to capture environment dynamics. An inverse dynamics model is further incorporated to guide action generation, improving sample efficiency and policy scalability. Moreover, to efficiently approximate large-scale agent interactions, we introduce a Mean Field (MF) mechanism to assist the classifier in conditional DMs. This mitigates inter-agent non-stationarity and enhances cooperation with minimal communication overhead. We further establish an upper bound on the distributional approximation error induced by MF-based diffusion generation, guaranteeing convergence stability and reliable modeling of multi-agent stochastic dynamics. Extensive experiments demonstrate that MA-CDMP consistently outperforms existing MARL baselines in terms of average reward and QoS metrics, showcasing its scalability and practicality for real-world wireless network optimization.
♻ ★ Independent Learning of Nash Equilibria in Partially Observable Markov Potential Games with Decoupled Dynamics NeurIPS'26
We study Nash equilibrium learning in partially observable Markov games (POMGs), a multi-agent reinforcement learning framework in which agents cannot fully observe the underlying state. Prior work in this setting relies on centralization or information sharing, and suffers from sample and computational complexity that scales exponentially in the number of players. We focus on a subclass of POMGs with independent state transitions, where agents remain coupled through their rewards, and assume that the underlying fully observed Markov game is a Markov potential game. For this class, we present an independent learning algorithm in which players, observing only their own actions and observations and without communication, jointly converge to an approximate Nash equilibrium. Due to partial observability, optimal policies may in general depend on the full action-observation history. Under a filter stability assumption, we show that policies based on finite history windows provide sufficient approximation guarantees. This enables us to approximate the POMG by a surrogate Markov game that is near-potential, leading to quasi-polynomial sample and computational complexity for independent Nash equilibrium learning in the underlying POMG.
comment: NeurIPS'26
♻ ★ MAC-Bench: Benchmarking Multi-Agent Collaboration via Interconnected Tasks
Large language model (LLM)-based multi-agent systems (MAS) tackle challenging tasks by dividing work among agents and coordinating their contributions toward shared objectives. Evaluating these systems therefore requires benchmarks that assess not only task-solving performance but also the ability to handle interdependent work. However, existing benchmarks often emphasize task difficulty without substantial collaboration requirements or evaluate only a limited range of predefined collaboration patterns. In this paper, we introduce MAC-Bench, a comprehensive benchmark for evaluating multi-agent collaboration through interconnected tasks. Rather than imposing predefined agent roles or communication protocols, MAC-Bench embeds collaboration requirements directly into task dependencies and shared-resource constraints. Specifically, we mine reusable collaboration structures from selected benchmark tasks at two levels. Local collaboration patterns capture recurring interactions among subtasks, while global task structures characterize how these interactions contribute to complete workflows. Through coarse-to-fine composition, these structures are recombined into task graphs combining different collaboration requirements. Tasks are instantiated in executable environments with automated verification of partial progress and complete task resolution.
♻ ★ FlowMAS: Learning Multi-Agent Workflow Topology via Information-guided Generative Flow Network
Automated multi-agent systems offer clear advantages over manually designed ones in scalability and adaptability, but existing workflow topology methods still face important limitations. Search-based methods are often computationally expensive, textual-gradient-based methods rely on coarse-grained feedback, and existing generation-based methods are not well suited to discrete workflow topologies with complex dependencies. To address these limitations, we propose FlowMAS, a multi-agent workflow topology method based on Generative Flow Networks (GFlowNets). FlowMAS models workflow generation as reward-guided flow over the topology space and introduces three components: a GFlowNet-based topology generation backbone, a curiosity-driven module for structure-aware exploration, and an information-guided optimization module for evaluating intermediate topologies. Concretely, the curiosity-driven module encourages exploration of structurally novel workflows, while the information-guided module measures both the information contribution and the communication efficiency of different operators to favor more informative and effective collaboration patterns. Experiments on six benchmark datasets with three LLM backbones show that FlowMAS consistently outperforms multiple baselines.
♻ ★ LAMaS: A Modular Platform for Learning Low-Latency Multi-Agent Orchestration
In a multi-agent system (MAS), removing an expensive agent call can save tokens without making the answer arrive sooner: latency depends on the longest chain of dependent calls. We present Latency-Aware Multi-agent System (LAMaS), a modular platform for learning low-latency orchestration. A framework can send a query to LAMaS and receive an answer and execution trace, or keep its own graph builder and use the platform's latency-optimized orchestration layer. During training, the layer gives operators on the critical path the greatest latency penalty while targeting a floor on expected accuracy. During inference, a lightweight controller uses partial results to decide whether to skip work that has yet to start. Across four benchmarks, LAMaS reduces end-to-end latency by over 50% relative to the evaluated learning-based MAS baselines while maintaining competitive or better accuracy. GDesigner and AgentDropout retain their graph builders and train the controller on their own traces; both achieve lower latency across the same benchmarks.
comment: Preprint. Previously this version appeared as arXiv:2607.13359 which was submitted as a new work by accident
♻ ★ SkillOps: Managing LLM Agent Skill Libraries as Self-Maintaining Software Ecosystems NeurIPS 2026
Large language model agents increasingly rely on skill libraries for multi-step tasks, yet these libraries can accumulate persistent defects as skills are added, reused, patched, and linked to changing dependencies. We call this failure mode skill technical debt: library-level defects that may not break a single skill locally but can harm future retrieval, composition, and execution. Existing skill-based agents mainly focus on task-time retrieval, planning, and repair, while library-time maintenance remains underexplored. We propose SkillOps, a method-agnostic plug-in framework for maintaining skill libraries. SkillOps represents each skill as a typed Skill Contract (P, O, A, V, F), organizes skills with a Hierarchical Skill Ecosystem Graph, and diagnoses library health across utility, compatibility, risk, and validation dimensions. Given a raw skill library, SkillOps produces a maintained library that can be used by existing retrieval or planning agents without changing their internal code. On ALFWorld, SkillOps achieves 79.5 percent task success as a standalone agent, outperforming the strongest baseline by 8.8 percentage points with no additional task-time large language model calls. As a plug-in layer, it improves retrieval-heavy baselines by 0.68 to 2.90 percentage points. The current rule-based maintenance implementation uses nearly zero library-time large language model calls or tokens, showing that skill-library maintenance can be added as a low-overhead architectural layer.
comment: 23 pages, 9 figures. Submitted to NeurIPS 2026
♻ ★ Finite-Sample Convergence in Networked Average Reward MARL: Decentralization Pitfalls and Entropy Remedies
We study the problem of average reward Multi-Agent Reinforcement Learning (MARL) where agents interact within a network. Each agent in the network conducts local updates with information within its $k$-hop neighborhood and collaboratively maximizes the overall reward of the entire network. We first provide impossibility results in achieving convergence to the global optimal joint policies in our setting. These impossibility results highlight the challenges of decentralized learning with local information in multi-agent systems, namely the issues of multiagency, where each agent independently optimizes its policy, and partial observability, where agents have limited visibility into the global state. Given these challenges which indicate that decentralized policy optimization can generally be certified only up to stationarity, we provide finite-sample convergence guarantees for a decentralized actor-critic algorithm with linear function approximation that converges to an approximate stationary point with small gradient norm. To overcome such impossibility results, we further show that adding fixed-state entropy regularization reshapes the objective landscape which leads to even stronger global convergence guarantees for our proposed actor-critic algorithm.
♻ ★ Scale-Dependent Collective Adaptation in Self-Amending LLM Societies: A Cross-Family Study of Emergent Governance
We study group decision-making in artificial societies where the rules of play are themselves subject to collective amendment. Using the self-amending game Nomic, we compare multiple scales across two LLM families and find that collective adaptation does not improve monotonically with model size. Instead, both families exhibit a narrow mid-scale regime that supports sustained rule adoption, diverse amendments, and balanced consensus. Smaller models tend to remain rule-inert, whereas larger models often converge on restrictive voting patterns, and heterogeneous mixed-size groups collapse into veto-driven gridlock. These cross-scale contrasts persist under temperature perturbations and under a shift from unanimity to majority voting, although latent-state structure varies by family and scale. Hidden-state divergence alone does not explain collective performance: high representational divergence can coincide with poor behavioural outcomes. Linear probes reveal regime-selective coupling between latent vote-predictive signals and collective behaviour, but decodability is necessary rather than sufficient for adaptive play. Overall, the recurring regularity is non-monotonicity, not the particular scale at which the optimum appears. Self-amending games therefore provide a controlled testbed for studying collective adaptation in artificial societies beyond raw model scale.
♻ ★ Measuring What Tool-Using LLM Agents Actually Do: Structured Validation, Causal Error Propagation, and Fabricated Tool Use
Tool-using agents built on large language models (LLMs) are usually scored by comparing the final answer with a reference text, using a string heuristic or an LLM judge. This is unreliable when the references describe the task rather than its answer. We replace it with a deterministic validator that checks the tools called, the executed parameters, the resulting state, and whether the values in the answer are supported by the correct tool observations. We re-scored 33,583 traces from 13 agents on simulated calendar, weather, medical and retail tools. The simulator reproduced all 10,396 logged observations, and the gold observations supported the validator in 16 of 18 disagreements with a three-annotator majority (the annotators in none). The instrument changes conclusions: on the same 306 traces, Gemini-2.0-Flash scores 63.4% under the substring heuristic and 5.6% under the validator. In a paired design (same task and model, with and without one schema-valid wrong argument injected into the first tool call), the error rate rises from 52.4% to 74.5% (+22.0 percentage points, 95% CI 18.4-25.5; 1,085 pairs, 7 models), and by 44.7 points (38.4-51.0) where the corrupted call executed without errors. An LLM-judge ensemble estimates +2.5 points on the same pairs. Agents also report tool results without executing any call, in 0-42.5% of traces depending on the model (rule-based lower bounds), largely because the harness accepts an action and a final answer in one completion. A rule-based runtime monitor costs about 7 microseconds per trace but reduces wrong answers mainly by abstaining.
comment: 25 pages, 4 figures. v3: substantially revised. Correctness is now measured by a deterministic gold-grounded validator; the propagation estimate uses a paired causal design. Claims from v1-v2 that were withdrawn or corrected are listed in Section 8. Code and data: https://github.com/bhaskargurram-ai/agenthallu-bench
Systems and Control (EESS)
★ An Interline Multiport DC Circuit Breaker with Power Flow Controller
Currently, the high-voltage multi-terminal DC (MTDC) grids face two major challenges. First, fast rise rate and lack of zero-crossing point in DC-side fault currents render AC circuit breakers inoperable. Second, DC line currents are passively determined by terminal DC voltages and line resistances, which inherently limits the capability of changing individual line currents for essential grid services such as power flow management and overloading prevention. To address these two limitations, DC circuit breakers (DCCBs) and DC power flow controllers (DCPFCs) have been developed. The conventional DCCBs are two-port devices installed in series with DC lines at their terminals in an MTDC converter substation, so that any fault current flowing through the line can be rapidly cleared. Additionally, a DCPFC can be installed at line terminals to adjust terminal voltages and power exchanges among adjacent lines. As a result, for a substation that connects to multiple lines, it needs to host multiple DCCBs and a DCPFC, which significantly increases the cost and footprint of both the substation and the MTDC grid. To this end, this work proposes an integrated equipment combining multiple DCCBs and a DCPFC, called the interline multiport DCCB with DCPFC, which can reduce the number of components and enhance the economic feasibility of MTDC networks. This paper presents the topology, operation principles, and component selection procedures of the proposed equipment, and compares it with the state-of-the-art technologies. Electromagnetic transient (EMT) simulation results validate the fault interruption and power flow control performance of the interline multiport DCCB with PFC.
★ BRACE: Belief-guided Residual Adaptive Control for Safe Navigation under Unknown Disturbances
Disturbances encountered during autonomous navigation generally vary across space, often discontinuously, as obstacles redirect wind and changing terrain alters traction. Tracking controllers typically assume sufficient control authority to mitigate disturbances and maintain safety; the regime where this assumption fails remains underexplored. We present Belief-guided Residual Adaptive Control (BRACE), a hierarchical reinforcement learning (RL) framework for reach-avoid navigation under unknown, bounded, spatially varying disturbances. BRACE infers a disturbance belief from model prediction errors and uses that to condition a high-level policy that adapts the reference trajectory. The policy combines a nominal and a belief-guided residual term, trained jointly with a likelihood-ratio advantage reweighting heuristic. We evaluate BRACE in three simulated quadcopter environments with abruptly switching wind fields, and validate it on Crazyflie hardware with fan-generated disturbances. Across these settings, BRACE improves reach-avoid performance over model predictive control and RL baselines.
★ Bisect-Index-Choose: A Fast and Exact Algorithm for EV Charging under Minimum Charging Rates
Electric vehicle (EV) charging stations are increasingly operated with an aggregate power limit below the sum of their port ratings, and require scheduling algorithms to divide the available power among the connected EVs. Charging standards permit a current that is either zero or at least a minimum rate, typically 6 A, which makes this allocation problem non-convex. Existing methods either solve a mixed-integer program with a commercial solver, which can be computationally expensive, adds software dependencies, and incurs licensing costs, or apply heuristics without optimality guarantees. This paper proposes Bisect-Index-Choose (BIC), a fast and solver-free scheduling algorithm that computes a globally optimal allocation among EVs under mild assumptions on their utilities. For $n$ EVs, BIC runs in $O(n \log n)$ time when all EVs share one maximum rate and in $O(n^2)$ time for two, times a logarithmic factor in the numerical precision. In tests with up to 1000 EVs, BIC attains global optimality and is up to three orders of magnitude faster than the solver. On real-world charging data, it needs less capacity and delivers more energy than existing heuristics. We implement BIC on a low-cost microcontroller and show that it solves instances with 1000 EVs in less than 90 ms.
comment: 10 pages, 6 figures. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible
★ Computationally Efficient Data Generation for Learning Nonlinear MPC Policies
Learning MPC requires large datasets generated by repeatedly solving MPC problems. For Nonlinear MPC (NMPC), dataset generation is computationally expensive because feasible initial states are difficult to sample efficiently. The feasible set is often non-convex or even disconnected, and many randomly sampled states are infeasible, leading to wasted optimization solves and unnecessary computational effort. In this work, we propose a data-generation framework for learning NMPC policies that combines active-set exploration with projection-based sampling. First, a bilevel formulation samples activity weights to promote solutions associated with different active sets while preserving optimality for the original NMPC problem; second, a simpler slack-based formulation uses weighted slack variables to target different active sets at the single level. These reference solutions are then used to construct local linearized feasible sets, from which candidate initial states are sampled and projected through the nonlinear NMPC problem to recover feasible and locally optimal training samples. Our approach is up to $37\times$ faster than baseline uniform sampling with $99\%$ acceptance rate, while maintaining a representative coverage of the feasible set.
★ Sampling-based Model Predictive Control as Sequential Monte Carlo Inference
Sampling-based model predictive control (MPC) is treated as a family of distinct algorithms: predictive sampling, MPPI, the cross-entropy method, and their biased, colored-noise, and annealed variants all sample control sequences and reweight them by cost. We show a single Sequential Monte Carlo (SMC) inference algorithm implements them all, specified by three independent axes: the proposal, the update rule that reduces cost-weighted particles to a nominal plan, and the effective-sample-size threshold for resampling. We evaluate SMC-based planning on MuJoCo non-linear control, open-loop navigation through obstacle fields, 7-DoF motion planning, and non-prehensile push-T, in simulation and on a Franka FR3. Resampling combats particle degeneracy at long horizons. At a 28-knot horizon in our navigation domain it cut the collision rate by 30% over non-resampling baselines. Programmable proposals improve push-T success over perturbative baselines, improving from 1% to 100% in simulation and 10% to 90% on hardware.
comment: 8 pages, 5 figures. Submitted to IEEE Robotics and Automation Letters (RA-L). Code: https://github.com/CoMMALab/PFMP
★ A Nested Heterogeneous Multiscale Framework for Power System Dynamic Simulation
Power system dynamics involve multiple timescales, from electromagnetic transients of the network and inverter-based resources to electromechanical oscillations of synchronous generators and quasi-steady-state evolution of the operating point. Simulating these dynamics with a detailed electromagnetic transient model is computationally prohibitive, whereas existing methods reduce the model to focus on a single timescale and lose accuracy when dynamics of different timescales interact. This paper proposes a nested heterogeneous multiscale framework for highly efficient multi-timescale simulation of a power system represented by one unified detailed model. Rather than varying only the integration step, the framework adaptively shifts the simulation resolution among three nested levels so that the dynamics of interest at each stage are always captured, with the slower dynamics estimated on the fly from short detailed simulations at controlled accuracy. Theoretical analysis proves the stability of long steps during quasi-steady-state evolution and quantifies the attainable speedup. On a modified IEEE 39-bus system with frequency regulation and load restoration, the framework accurately reproduces the dynamics of all timescales and runs 15 to 18 times faster than conventional simulation of the detailed model. The accuracy and speedup of the proposed method are further verified on a 390-bus system, demonstrating its scalability to large power systems.
comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible
★ Modified CNN-Learned Surrogate Covariance With Krylov/Lanczos Subspace for DoA Estimation
Recently, learning-based approaches have been employed to improve the robustness of subspace methods in DoA estimation. A Convolutional Neural Network (CNN) was used as a data-driven framework employing an autoencoder to generate a well-conditioned surrogate covariance optimized end-to-end through a differentiable classical estimator, such as the Root-MUSIC. The complete system is designed to estimate the directions of arrival based on a spectral decomposition using the EVD of this covariance matrix. In this work, a new approach is proposed by formulating the spectral stage of the Root-MUSIC algorithm in terms of the orthogonal complement of the signal subspace, thereby eliminating the need for the noise subspace and noise eigenvalues. Furthermore, the full EVD is replaced by a Lanczos tridiagonalization with reorthogonalization, reducing the overall computational cost.
★ Real-Time Threat Classification in Multi-Agent Systems via Topology-Coupled Hidden Markov Models
Networked multi-agent systems coordinating in adversarial environments face compound cyber threats that alternate between false-data injection (FDI), denial-of-service (DoS), and stealthy decoy perturbations designed to mimic benign operational anomalies. Because the switching policy is latent and the plant dynamics evolve continuously, deterministic threshold detectors struggle to isolate deception without causing excessive false alarms. This paper presents an online, data-driven framework that couples distributed continuous time unknown input observers (UIOs) with a topology-aware Hidden Markov Model (HMM) for real-time attack detection and classification under unknown adversarial strategies. By decoupling FDI signals from augmented node states, distributed observers deployed across the network generate local augmented residuals that are discretized into emission sequences. The HMM embeds the underlying directed communication graph into the off-diagonal blocks of its transition probability matrix, explicitly capturing inter-agent attack propagation. Both transition and observation probabilities are updated recursively online via an Expectation-Maximization (EM) scheme, adapting the classifier to evolving threat behaviours without prior offline calibration. Numerical simulations on a directed network demonstrate that incorporating graph topology into the latent transition dynamics resolves the severe classification ambiguity between nominal and decoy modes, outperforming decoupled agent-independent decoders while restricting observer deployment to a sparse subset of nodes.
★ Passivity-Guaranteed Variable Impedance and Compliance Control via Online Parameter Optimization
Variable impedance control enhances the interaction capability of robot manipulators by adapting impedance parameters online. However, time-varying impedance parameters may inject energy into the system and violate passivity, particularly when tasks demand rapid stiffness changes. Existing methods address this by increasing damping or limiting the stiffness profile, which often degrades the dynamic response. In this work, we propose a quadratic programming-based parameter optimization framework that guarantees passivity by modulating the desired inertia to compensate for energy injected by stiffness variations, while preserving a critically damped system. For the case of diagonal impedance matrices, we optimize the rates of change of inertia and stiffness subject to a passivity constraint formulated as a convex quadratic program. We extend the framework to compliance control, where the desired inertia is coupled to the robot's configuration-dependent inertia matrix. We use an eigendecomposition based on Cholesky factorization to optimize the spectral weights while maintaining passivity. Experiments on the Kinova Gen3 7-degree-of-freedom manipulator validate both methods with human interaction perturbations. Comparisons with an energy tank approach and a nonpassivity-based method demonstrate that the proposed framework achieves less conservative stiffness tracking without requiring an energy reservoir or damping overhead.
comment: submitted to Control Engineering Practice
★ Deep Reinforcement Learning with Massively Parallel Simulation for Low-Thrust Multi-Revolution Rendezvous
The growing demand for on-orbit servicing and constellation management in low Earth orbit requires autonomous and reliable low-thrust rendezvous. Traditional numerical methods struggle with the long-horizon nature of multi-revolution maneuvers under perturbations, while standard deep reinforcement learning (DRL) methods are limited by low simulation throughput and poor sample efficiency. To address these challenges, this paper proposes a massively parallel DRL guidance framework. A fully tensorized, GPU-resident orbital propagator is developed to integrate $J_2$-perturbed orbital dynamics and policy optimization within a zero-copy pipeline, achieving up to a 67-fold increase in simulation throughput over conventional CPU baselines. In addition, the rendezvous problem is formulated in modified equinoctial elements and trained with a progressive curriculum learning strategy to improve convergence and generalization across increasingly challenging rendezvous scenarios. Simulations show that the proposed framework can learn robust rendezvous trajectories for complex multi-revolution rendezvous over a broad range of orbital configurations. Additional evaluations under a high-fidelity model and eclipse-induced thrust interruptions further demonstrate the robustness of the learned policy under practical mission constraints. Moreover, the trajectories generated by the proposed framework can be used as effective initial guesses for optimization-based trajectory solvers, improving the convergence of high-precision trajectory optimization. The method offers a scalable approach for real-time low-thrust guidance and trajectory optimization.
comment: 23 pages, 8 figures, 13 tables
★ Two-Timescale Trajectory Control, Task Reassignment and Resource Scheduling for Multi-UAV Cooperative Mobile Edge Computing
Uncrewed aerial vehicle (UAV)-assisted mobile edge computing (MEC) is a promising technology, leveraging the advantages of UAVs including flexible deployment, wide coverage, and reliable wireless communication. In this paper, we study the task reassignment and offloading problem in a multi-UAV cooperative operating environment, and consider the real occurrence that the computing tasks arrive more frequently than UAVs make flight decisions. In addition, we also take into account the position offset of the UAV during flight and the unavailability of the UAV due to external interference. We propose a two-timescale optimization framework, where the trajectory of the UAV is carried out on a large timescale, while the task reassignment and computing resource allocation are jointly optimized on a small timescale. The core objective is to maximize the long-term average system gain, which takes both energy consumption and task delay into account. To address the UAV trajectory optimization problem under large timescale constraints, we propose a constrained multi-agent deep reinforcement learning algorithm. For decision-making within small timescale, to jointly optimize task offloading decisions and computing resource allocation, we integrate convex optimization with a heuristic algorithm. Finally, we conduct extensive simulations to evaluate the proposed algorithms against the baseline, achieving a 24% reduction in latency and a 28% reduction in total system cost.
comment: 16 pages, 10 figures
★ Worst-Case Contagion in Financial Networks: MILP Clearing and Robust Loss Analysis
We study worst-case contagion losses in financial networks with interbank liabilities and common exposures to marketable assets. Existing linear-programming approaches provide exact stress-test values only within an insolvency-resilience margin, where all institutions remain solvent with respect to the external sector. We show that this margin is computed exactly by a finite family of linear programs, one per vertex of the uncertainty ball, whereas a single linear program based on the componentwise worst deterioration of external resources yields only a certified lower bound. We extend this framework beyond that margin in three directions. First, we derive an exact mixed-integer linear programming (MILP) formulation for the maximal clearing vector under arbitrary signed external net positions, providing a clearing oracle for robust stress testing when insolvency can occur. Second, we develop certified bounds on the worst-case loss beyond the margin, including insolvency-pattern enumeration, scenario lower bounds, fixed-pattern dual upper bounds, localized certificates, and a regime-generation procedure with monotone gap certificates. Third, we analyze the worst-case loss curve as a function of the perturbation radius. For $\ell_1$ and $\ell_\infty$ price-perturbation sets, the curve is convex, nondecreasing and piecewise linear up to the margin, with breakpoints generated by switches in worst-case shock directions, LP bases, and bank-level activity patterns. Beyond the margin, it remains nondecreasing and piecewise affine, but convexity may fail, the curve may jump upward, and the worst case need not be attained. The framework provides a certifiable MILP/LP methodology for parametric stress testing and interpretable diagnostics identifying the banks responsible for changes in systemic loss.
★ From Discrete Snapshots to Geometric Entity: Motion-Native Channel Tracking for THz Near-Field XL-MIMO
Channel tracking in mobile terahertz (THz) near-field extremely large-scale multiple-input multiple-output (XL-MIMO) systems is challenging because spherical-wave propagation couples angle and range, leading to rapid and nonlinear channel evolution under sparse pilot observations. Existing approaches typically estimate pilot-local channel or position snapshots before applying temporal tracking or smoothing, making them vulnerable to information loss during snapshot formation. To break this bottleneck, we propose a paradigm shift from tracking discrete snapshots to estimating a holistic geometric entity, namely, one shared continuous UE trajectory over the tracking window. Specifically, we parameterize the trajectory using a low-dimensional cubic B-spline representation and directly couple the per-pilot profile likelihoods through this shared trajectory before any pilot-local position estimate is finalized, thereby inducing continuous near-field channel evolution. We further characterize the efficient geometric information supplied by each pilot observation and its aggregation in the shared spline-coefficient space. Based on this formulation, we develop a regularized maximum a posteriori estimator and a quadratic-penalty lifted solver that separates parallel pilot-local geometric updates from a structured global spline update. The resulting global system is banded and, for a fixed spline degree, admits linear-complexity factorization, enabling an efficient structured implementation. Simulations demonstrate consistently improved tracking accuracy over representative temporally refined baselines in the tested medium-to-high-SNR regime, together with a favorable accuracy--runtime trade-off. These results highlight the benefit of coupling sparse pilot observations through a shared continuous trajectory rather than temporally processing independently estimated snapshots.
★ Shifted Interaction Functions: A Unified Framework for Imperfect Actuation and Sensing in Consensus Networks
Practical networked robotic systems rarely implement ideal consensus interactions exactly, since local actuation constraints, sensing biases, and heterogeneous interaction maps may shift the input-output relations used by distributed controllers. Rather than focusing solely on whether consensus occurs, such imperfections can fundamentally alter the structure of the asymptotic collective behavior, typically yielding global or clustered velocity alignment, while exact static agreement arises only in special cases. This paper develops a shifted-interaction framework for analyzing collective alignment under imperfect actuation and sensing. Shifted functions provide a compact representation of local actuator and sensor maps, allowing actuation constraints and sensing biases to be represented by horizontally and vertically shifted monotone interactions. The framework distinguishes nodal shifted actuation from edge-wise shifted sensing. For shifted actuation, a Lyapunov analysis establishes global velocity alignment under compatible actuation ranges, including bounded, non-invertible, and deadzone-type maps. For shifted sensing, a maximum-principle argument and a paired-sensitivity condition establish convergence to velocity clusters over edges with infinite accumulated sensitivity; uniform sensing sensitivity yields exponential global velocity alignment. In the linear case, closed-form expressions characterize asymptotic behavior and identify actuation as a special case of sensing. Simulations illustrate static, globally aligned, and clustered velocity-aligned behaviors.
comment: 12 pages, submitted to TCNS
★ Online Geometric Consistency Assessment of LiDAR Measurements via Recursive Bayesian Comparison
LiDAR-based localization and mapping systems produce registered point clouds that serve as geometric measurements for downstream estimation and navigation. Registration errors introduce duplicated surfaces and structural deformation into these measurements. In the absence of an independent reference, online assessment relies on geometry accumulated from earlier measurements. Such assessment requires tolerance to sampling variation and an accumulation process in which observations enter the history only after evaluation. We therefore formulate a recursive Bayesian comparison that couples geometric assessment with history maintenance. A Normal-Inverse-Wishart model represents historical voxel geometry and its uncertainty, while a weak isotropic reference adapts to current sampling. Comparing the marginal predictive support of the same current points under both models yields Normalized Drift Evidence (NDE). Assessment precedes assimilation: positive NDE triggers an alarm and preserves the preceding historical state; otherwise, the current observations update the history. Across 14 real-world sequences totaling approximately 73 min from four platform types, alarms identified all nine corrupted sequences and were absent from all five consistent sequences. In four representative scenarios with event references, first alarms occurred within 0.9 s of the reference times, with no preceding alarms. Cross-backend evaluation demonstrates consistent responses to reconstructed geometry. Ablation and controlled perturbation results establish the roles of uncertainty modeling, recursive history, displacement, and geometric support. Mean processing time is 4.2--6.1 ms per update at 1 Hz. In ten UAV trials, alarms initiated odometry isolation, landing, relocalization, and mission continuation.
★ Balancing Confidentiality and Attack Detectability in Networked Control Systems over Lossy Wireless Channels
We study the joint protection of a networked control system (NCS) against eavesdropping and data integrity attacks over a lossy wireless channel. The eavesdropper intercepts the transmitted sensor measurements and control inputs to reconstruct the system state. In contrast, the integrity attacker in this study replaces the sensor measurements with fabricated data to drive the system to undesired operating conditions. Although these two threats are typically studied separately, a wireless NCS should be protected against both simultaneously, and a defence against one affects the other. We employ an observation-encoding scheme that renders the eavesdropper's state-estimation error covariance unbounded under packet drops, and show that transmission energy governs how long this error remains bounded. In particular, lower transmission energy improves confidentiality. For integrity protection, we combine stochastic physical watermarking with a CUSUM-based sequential change detector. The proposed attack detection scheme accounts for intermittent communication and encoded measurements. The detection delay is governed by the Kullback--Leibler divergence (KLD) between the pre- and post-attack distributions of the test signal, and we show that a higher transmission energy increases the KLD and thereby reduces the detection delay. Furthermore, we derive analytical expressions for the eavesdropper's estimation error and the worst-case expected KLD, providing guidelines for selecting transmission energy to balance confidentiality and integrity protection and characterising the privacy--security trade-off. Finally, we run numerical simulations to validate our theoretical findings and demonstrate the privacy--security trade-off in practical scenarios.
comment: To be Submitted to ECC27
★ On Real Eventual Exponential Positive property of complex-valued Laplacians and their pseudoinverses
The property of real eventual exponential positivity guarantees the state trajectories of a linear time-invariant system to evolve asymptotically to right half of the complex-plane [1]. It has applications in the characterization of herdability and reachability of states in networked systems. Networks with complex-valued edge-weights arise in AC power systems and opinion dynamics. Recent work has introduced real eventual exponential positive (rEEP) Laplacians, for which the trajectories of the associated network flow eventually converge to consensus. We establish further properties of rEEP Laplacians and their pseudoinverses. Firstly, we show that the rEEP property for complex-valued matrices exhibits invariance under specific rotational transformations. Secondly, we establish that the property of rEEP is preserved under the MoorePenrose pseudoinverse operation, highlighting its robustness under algebraic manipulations. These results apply to effective impedance computation in AC networks and to consensus in multi-agent systems. Finally, we provide numerical examples to validate our theoretical results.
★ Dynamic State Estimation from Sparse PMUs via Physics-Informed Graph Neural Operators
This paper proposes a physics-informed graph neural operator for power system dynamic state estimation. From a sparse, intermittent measurement window at a few generator buses, the operator estimates the rotor-angle and speed trajectories of all machines, unmeasured ones included, in one forward pass. A graph-attention encoder processes the measurements, and the swing equations are imposed as a hard constraint or as a soft one with a learned correction that absorbs model errors. It is trained on simulated measurements, without state labels. With several phasor measurement units, masked training hides one from the input but keeps it in the loss. On four transmission systems, simulations show that the operator is the most accurate estimator that neither optimizes per scenario nor uses state labels, except on the smallest system at large model errors, where tuned Kalman smoothers are better. Networks trained on simulated states are more accurate on the two smallest systems but not on the 118-bus system, where the operator is faster and more accurate than hard-constrained moving-horizon estimation. Under model errors, it matches or outperforms an optimization-based benchmark at a fraction of its cost. The results hold for a detailed machine model with exact parameters seen only through voltage phasors.
comment: 10 pages, 10 figures, 4 tables
★ Simulating Nonlinear Resistive Networks using a Behavioral Splitting
An operator splitting method is developed for computing operating points of nonlinear resistive networks. Our behavioral formulation separates the device laws from Kirchhoff's laws and casts simulation as a zero-finding problem. Branch port resistances then provide a diagonal preconditioner for the relaxed Peaceman-Rachford iteration in scattering variables. Each iteration requires solving independent scalar equations and a linear system with a fixed weighted Laplacian matrix. Convergence is established for maximal monotone device laws when an operating point exists. Contractive branch reflections also yield a linear convergence bound independent of network topology. Numerical experiments on crossbar arrays illustrate the computational advantages.
comment: Submitted to European Control Conference 2027
♻ ★ Convergence of Byzantine-Resilient Gradient Tracking via Probabilistic Edge Dropout
We study distributed optimization over networks with Byzantine agents that may send arbitrary adversarial messages. We propose Gradient Tracking with Probabilistic Edge Dropout (GT-PD), which combines a universal self-centred projection that clips incoming messages to a radius-$τ$ ball around the receiver with a fully decentralized probabilistic dropout rule driven by a threshold-gated, dual-metric trust score in the decision and tracking channels. These identity-free defences bound adversarial perturbations while preserving doubly stochastic mixing. The projection preserves the honest network average exactly in every realization, even when honest messages are clipped. Under complete Byzantine isolation ($p_b=0$), GT-PD converges linearly to a neighbourhood of order $ασ^2/(μn_{\mathcal H})$ plus higher-order network terms, matching centralized stochastic gradient descent to first order in the step size. We also give a radius computable from problem data that ensures honest messages remain unclipped with high probability. Under partial isolation ($p_b>0$), a leaky variant, GT-PD-L, controls accumulated tracking perturbations and converges linearly to a bounded neighbourhood governed by stochastic variance and the clipping-to-leak ratio, under explicit conditions on the step size, leak, and aggregate Byzantine mixing weight. With two-tier dropout and $p_h=1$, Byzantine isolation adds no variance to honest consensus dynamics. MNIST experiments under Sign Flip, ALIE, and Inner Product Manipulation attacks show that GT-PD-L outperforms coordinate-wise trimmed mean by up to 4.3 percentage points under stealth attacks.
♻ ★ Challenges in the Proper Metrological Verification of Smart Energy Meters
The most common instruments currently used to measure active/reactive energy and power quality indicators are smart energy meters (EM). Unfortunately, the verification of such meters is currently performed under ideal conditions or with simple signal models, which do not recreate actual states occurring in the power grid and do not ensure the verification of the properties of their signal chains. This paper presents challenges in proper metrological verification of smart EM. It presents existing legal and normative requirements and scientific research directions regarding these meters. Although the meters tested comply with the normative and legal requirements, the results reveal numerous imperfections in the signal and measurement chains for the selected test signal. Based on the results of the research results, further directions have been determined in the field of smart EM.
comment: 5 pages, 5 figures, submitted to IEEE conferences: 2026 22nd International Conference on Harmonics & Quality of Power (ICHQP)
♻ ★ Clustered Randomized Smoothing for Stochastic Prediction Functions
Modern stochastic predictors can model rich, multi-modal outcome distributions. However, this expressive power comes with challenges in ensuring robust predictions $-$ a critical requirement in safety-critical domains. Randomized smoothing is a leading technique for improving robustness, particularly against adversarial perturbations. Yet, in stochastic multi-modal regression settings, randomized smoothing often fails due to mode collapse, yielding averaged predictions that do not reflect the underlying distribution. To address this limitation, we propose clustered $α$-smoothing, a framework that (1) partitions noisy samples using an arbitrary clustering algorithm, (2) applies $α$-smoothing locally within each cluster, and (3) combines the resulting predictions into a mixture distribution. By interpreting the smoothing distribution as a mixture of $α$-smoothers, we derive a lower bound on the probability that the smoothed prediction lies within a union of compact regions corresponding to distinct modes. We empirically evaluate our framework on two benchmarks, demonstrating substantial improvements over state-of-the-art methods. In stochastic trajectory prediction on a driving simulator dataset, our approach achieves, on average, a $27\%$ lower Wasserstein distance to the ground-truth distribution compared to $α$-smoothing. In quadrotor control, where modes correspond to distinct feasible paths to a target, our method reduces the collision rate by $90\%$ relative to the state-of-the-art randomized smoothing.
♻ ★ Decision-Dependent Stochastic Optimization: The Role of Distribution Dynamics
Distribution shifts have long been regarded as troublesome external forces that a decision-maker should either counteract or conform to. An intriguing feedback phenomenon termed decision dependence arises when the deployed decision affects the environment and alters the data-generating distribution. In the realm of performative prediction, this is encoded by distribution maps parameterized by decisions due to strategic behaviors. In contrast, we formalize an endogenous distribution shift as a feedback process featuring nonlinear dynamics that couple the evolving distribution with the decision. Stochastic optimization in this dynamic regime provides fertile ground to examine the various roles played by dynamics in the composite problem structure. To this end, we develop an online decision-making algorithm that both adapts to and shapes the dynamic distribution. Throughout the paper, we adopt a distributional perspective and demonstrate how this view facilitates characterizations of distribution dynamics and the optimality and generalization performance of the proposed algorithm. We showcase the theoretical results in an opinion dynamics context, where an opportunistic party maximizes the affinity of a population with evolving, polarized opinions, and in a recommender system scenario, featuring performance optimization with discrete distributions in the probability simplex.
comment: Published in the Journal of Machine Learning Research
♻ ★ On asymptotic stability of the time-varying Kalman filter for unstabilizable linear systems: an optimization perspective
This paper establishes the necessary and sufficient conditions for asymptotic stability of the time-varying Kalman filter applied to a linear time invariant system with semidefinite initial state covariance and positive definite process and measurement noise. Rather than analyze the discrete Riccati equation as in the classic literature, the equivalent state smoothing optimization problem is stated and all results are established using properties of this optimization problem. A Lyapunov-like function, termed a modified $Q$-function is derived and used for this analysis. This optimization approach removes the need for the classic but cumbersome Riccati iteration algebra and provides better generalization and application for nonlinear systems.
♻ ★ From Winding-Fault Geometry to Reliability: Estimation and Prognosis of Stator Inter-Turn Faults
This paper develops an integrated framework for estimation, prognosis, and reliability assessment of stator inter-turn faults in induction motors. The fault is characterized by its severity, defined as the fraction of short-circuited turns, and its spatial orientation. A geometric fault model shows that the resulting output signature is affine in the fault severity and exhibits a fundamental spatial periodicity. Exploiting this structure, an augmented-state particle filter jointly estimates the nonlinear electromechanical state and the unknown fault severity while quantifying posterior uncertainty. The estimated degradation is then propagated using four prognostic models: linear trend, Holt exponential smoothing, Bayesian degradation, and particle-based forecasting. Their predictions are connected to threshold-crossing remaining useful life (RUL), first-passage reliability, degradation-dependent hazard reliability, and a Weibull lifetime benchmark, thereby providing both deterministic and probabilistic health assessments. Numerical results demonstrate accurate online fault estimation and output reconstruction, characterize the effects of degradation pattern and prediction horizon on prognosis, and show consistent reliability and threshold-crossing predictions. Moreover, estimation accuracy remains comparable across the distinct fault orientations induced by the spatial periodicity. The resulting framework provides a unified connection from physics-based inter-turn fault modeling to online diagnosis, degradation prognosis, and reliability assessment.
Robotics
★ On Agentic Manipulation Policies
Recent work on robot manipulation has focused on specialist policies trained on demonstrations. We show that generalist AI models can control manipulators more reliably than specialist policies such as vision-language-action models (VLAs) and world-action models (WAMs). We systematically analyze agentic manipulation policies (AMPs), in which a generalist AI model commands the manipulator directly, adapting its actions as the task unfolds. We present AMP1, a reference harness that runs unchanged across single-arm, bimanual, and mobile manipulation. AMP1 achieves state-of-the-art results on the RoboLab-120 and CaP-Gym benchmarks, with no demonstrations or task-specific training. On RoboLab-120, AMP1 reaches an 85.6\% success rate zero-shot, compared to 49.2\% and 42.9\% for leading VLA and WAM policies. The agent succeeds zero-shot in 104 of 108 real-world trials spanning twelve tasks and three robot instances, reducing failure rates by 20x and 15x relative to strong VLA and WAM baselines. The basic harness supplies no grasping, pushing, or placing skills; the agent develops its own strategies, such as searching for hidden objects, using objects as tools, and calibrating its cameras. With the addition of a persistent memory store, the agent can also learn from experience: an augmented harness with a memory representation that is carried across episodes lowers the failure rate on RoboDojo bimanual manipulation tasks by 4.6x relative to zero-shot deployment. Our analysis suggests that AMPs are substantially more general and robust than specialist manipulation policies, providing a promising foundation for robot manipulation.
comment: Videos and code available at: https://apple-aiml-research.github.io/ml-amp/
★ Scenario Orchestration for Closed-loop Policy Evaluation of Autonomous Driving
Scenario-based simulation is central to evaluating autonomous-driving policies, but a scenario scripted before execution (i.e., with predetermined, fixed actor routes and speeds) may not preserve its intended interaction once the tested ego policy responds in closed loop. Scripted scenarios cannot adapt to unanticipated ego behavior, while learned traffic actors may remain reactive without reliably realizing a particular scenario intent. We introduce a scenario orchestrator that continuously monitors the joint traffic state and adapts non-ego actors through re-timing, re-routing, and role reassignment for creating a specific scenario intent. At each step, the orchestrator solves a constrained optimization problem that preserves the intended interaction while limiting interventions and maintaining traffic feasibility. We further distinguish scenario realization from policy failure, allowing a test to first be judged based on whether the intended interaction occurred and then by how the ego policy responded. Experiments in the CARLA simulator across multiple scenario families and driving policies show that orchestration realizes targeted interactions more reliably than scripted and learned actor-level baselines, particularly under variations in initial conditions and ego behavior. These results support closed-loop scenario orchestration as a tool for controlled and reproducible policy evaluation.
comment: 8 pages, 5 figures
★ Locating Contact on Tendon-Driven Continuum Robots From Tendon Measurements Alone via Constrained Moving Horizon Estimation
Tendon-driven continuum robots can reach through constrained spaces, but contact along their bodies is invisible to the sensors at their motors. We show that this is a structural limitation rather than a tuning problem: the map from contact forces to tendon measurements has a null space of dimension at least seven, so no estimator that leaves the contact support free can locate contact. When the obstacle geometry is known, however, non-penetration and complementarity make the contact force locally identifiable, and we prove that the resulting system with unknown contact inputs is locally exponentially strongly detectable. We present a constrained moving horizon estimator (MHE) that encodes these conditions and runs in real time as one quadratic program per sample, and we give a local, mode-wise robust stability result for it. In a finite- element benchmark of a two-section robot pressing and sliding on a wall, the estimator located contact to 0.9 mm and estimated contact force to 0.22 N, compared with 65 mm and 0.72 N for a tuned extended Kalman filter (EKF) with state projection. The estimator tolerated millimeter- level errors in the obstacle position and unmodeled wall friction, under which the projected EKF failed. Under randomized model mismatch over 40 trials, including unmodeled tendon friction, it remained better in force and location in every trial. Without contact, the MHE offered no advantage over the EKF, which confirms that its benefit comes from the contact constraints. This report extends a letter submitted to IEEE Control Systems Letters with complete proofs and additional experiments.
★ Grasp-TAGS: Task-Aware Grasp Selection via Critic Distillation for Prehensile Loco-Manipulation
Loco-manipulation, which requires mobile manipulators to coordinate locomotion and manipulation over long horizons, is a key capability for large-object rearrangement in applications such as service robotics and warehouse automation. Despite recent progress, reliably executing these tasks remains challenging because performance depends on complex robot-object and object-environment interaction dynamics. Existing approaches address this challenge either through model-based methods, which often require accurate dynamics models, or through learning-based methods, which reduce this requirement but typically assume a fixed or predefined initial grasp. However, different geometrically stable grasps can lead to substantially different downstream manipulation performance. Therefore, we propose Grasp-TAGS, a task-aware grasp selection framework for goal-conditioned large-object rearrangement. We train a unified PPO-based reinforcement learning policy across multiple initial grasp configurations and distill the learned critic into a lightweight grasp utility network that ranks candidate grasps using only the candidate grasp configuration and goal information. At inference time, we select the grasp with the highest predicted downstream utility for the downstream policy to complete the task, before even making a contact. Experiments demonstrate that Grasp-TAGS improves rearrangement success over both fixed single-grasp policies and random grasp selection, achieving an average success rate of 89.84% compared with 79.87% for the best-performing single-grasp policies, an improvement of 9.97 percentage points. Grasp-TAGS also reduces final orientation errors across all evaluated objects. For more qualitative results, refer to the supplementary webpage: https://rpm-lab-umn.github.io/Grasp-TAGS-web-page/
comment: 10 pages, 6 figures
★ Computer-Use Agents Are Embodied Generalists
In many areas of AI, general-purpose models have increasingly replaced handcrafted methods and specialized architectures. We argue that embodied AI is poised for a similar shift. Our key claim is that embodied tasks can be formulated as interaction with digital software. Specifically, we build software that renders an agent's view of a physical environment on a screen and provides controls for acting in it. A computer-use agent (CUA) can then perform embodied tasks through the same screenshots and keyboard and mouse actions that it uses to operate ordinary software. Using a single CUA design and changing only the software, our entries win first-, second-, third-, and sixth-place prizes across four ECCV workshop competitions, without any competition-specific agent tuning. These results suggest that the skills CUAs learn on software, such as visual grounding, precise clicking, and exploration, already carry over to embodied tasks, giving CUAs a far broader scope in embodied AI than previously realized. We therefore advocate that computer-use agents can be embodied generalists. Further, we advocate that, by following the training strategy that made computer-use agents general, embodied AI can converge on a single general model as well, ushering in a new era of general embodied AI. Website: https://pranjal2041.github.io/cua-embodied
comment: 14 pages, 3 figures, 2 tables
★ Seeing What Matters: Benchmarking Task-Conditioned Actionable Spatial Belief for Manipulation
When should a robot look at the cup, and when at the portafilter? Spatial-reasoning benchmarks evaluate whether a model can describe a scene, but manipulation needs something more specific: the ability to acquire the particular evidence that makes the current command executable. A view that is informative for one task can be useless for another, even at the same workstation. We introduce Seeing What Matters, a benchmark for task-conditioned actionable spatial belief in simulated tabletop manipulation. Across three domains--coffee workstation, drawer/cabinet, and microwave/appliance--the benchmark provides 860 frozen evaluation cases; expanding across methods, budgets, and inference repeats yields more than 115,000 evaluator episodes under two protocols: closed-loop actionability and same-object functional out-of-distribution generalization. Each case pairs a set of candidate RGB views with a manipulation command, shared scripted skills, and hidden postconditions, so the benchmark directly measures whether a view-selection policy acquires the observations its command actually needs. We also provide a diagnostic reference pipeline that maintains a spatial-functional belief state and ranks views by missing evidence. At primary budget settings, this pipeline reaches 98.8% closed-loop and 98.9% functional-OOD success; ablating the functional or command-binding components collapses performance to near zero. Seven vision-language models, evaluated on the same cases, reach only 26-43%. Replacing the reference pipeline's coverage metadata with frozen RGB-derived scores preserves 98.5%/98.9% success; shuffling the visual scores collapses it to 15%/11%, confirming that the result depends on image content. These gaps suggest that spatial belief for manipulation should be evaluated not by scene-level completeness, but by whether it acquires the evidence the current command requires.
comment: 22 pages (8 main + 14 supplement). Extended cross-domain evaluation and physical robot validation in progress
★ Autonomous Whole-body Loco-manipulation Using a Snake Robot IROS 2026
Snake robots can leverage their hyper-redundant morphology for both locomotion and manipulation through distributed body contact but whole-body loco-manipulation is particularly challenging since the entire body serves simultaneously as propulsive mechanism and manipulation tool Prior work on wheeled legged and snake robot platforms has decoupled these functions through dedicated appendages body segmentation or grippers We present closed-loop loco-manipulation with our snake robot COBRA without this distinction such that any part of the body may make contact We develop a contact-implicit dynamic model unifying robot-ground robot-object and object-ground interactions within a Moreau-Jean time-stepping framework and formulate a trajectory optimization over the full-order dynamics guided by a geometric planner and nominal proportional controller to autonomously push an object to a goal location during sidewinding We validate the planner and controller on hardware across varied configurations and present simulation results for a simplified trajectory optimization that resolves robot-object and object-ground contacts
comment: Presented in IROS 2026
★ A wearable input/output interface learns personalized models of haptic perception
Wearable haptic interfaces deliver physical stimuli to the skin, yet perceived locations vary widely across users and diverge from actuator locations, particularly for emergent tactile-feedback techniques. We developed a wearable haptic input/output (I/O) device integrating a 36-channel electrotactile array with capacitive touch sensing, enabling stimulation and user-reported percept locations to be registered within the same body-referenced coordinate frame. In 24 participants, the device revealed electrotactile and referred-thermal perceptual maps with notable interparticipant variability. Despite this variability, both modalities retained electrode-pair-specific spatial structure across participants. Tactile and thermal percepts also showed spatial correspondence, whose strength varied across participants. These findings motivated our human-in-the-loop calibration framework, which adapts a forward-prediction strategy to individual users and generates personalized perceptual models. These models enabled inverse rendering by selecting stimulation patterns predicted to generate a target percept for a specific user. Using these models, we demonstrate personalized haptic rendering in haptic-guided robot teleoperation, visual-to-haptic sensory substitution, interpersonal haptic communication, and virtual reality (VR). Together, this platform provides a route toward adaptive haptic interfaces that measure, model, and render according to each user's perception.
★ SURE:Spatiotemporal Uncertainty Representation for Efficient Vision-Language-Action Learning
Vision-language-action (VLA) policies have become a promising paradigm for end-to-end robot control, but their effectiveness still depends heavily on large volumes of demonstration data. In manipulation tasks, this dependence is compounded by structured spatiotemporal uncertainty in the action space: multiple nearby actions or trajectories may successfully solve the same task, while semantically equivalent demonstrations often unfold at different speeds. Standard pointwise supervision treats each demonstrated action as an exact target, imposing an unnecessary learning burden under such uncertainty. We present SURE, a data-efficient VLA training framework that explicitly models both spatial and temporal uncertainty. SURE augments a VLA backbone with a lightweight transposed-convolutional probabilistic action decoder that captures spatial uncertainty through local correlations among neighboring action bins. It further introduces Soft-DTW alignment supervision to accommodate local timing variations between predicted and demonstrated action chunks. Together, these two components reduce sensitivity to demonstrator-specific poses and execution timing, enabling more data-efficient action learning. We evaluate SURE on LIBERO, RoboTwin 2.0, and real-world manipulation tasks. Even with only 10% of the demonstration data or 20% of the training steps, SURE achieves manipulation success rates that match or surpass baseline VLAs.
★ Autonomous Particle Image Velocimetry Using Zero-Shot Sim-to-Real Reinforcement Learning
Particle image velocimetry (PIV) reconstructs a flow field from the motion of tracer particles but the reconstruction quality is bounded by how well the flow structure of interest is resolved in the images. The acquisition position is typically fixed by the operator before the measurement. We cast camera placement as a sequential decision-making problem and train a reinforcement learning agent to control a 6-DoF manipulator equipped with a camera, enabling it to locate, frame and resolve a hydrofoil wake in a water tunnel. The agent commands the camera pose and observes the scene only through the velocity field and uncertainty estimated from time-separated image pairs. It is trained entirely in a photorealistic simulation of our water tunnel, with wakes generated by Computational Fluid Dynamics (CFD) simulations of hydrofoils with varying profiles, positions and angles of attack. After zero-shot transfer to the real water tunnel, the policy reliably locates and frames the wakes of randomly placed hydrofoils. It resolves the wake at a higher resolution than a fixed camera view and with lower error in the measured flow statistics, in roughly half the time of a full scan and with far fewer image pairs.
comment: 9 pages
★ ODEAction: From Predicting Chunks to Evolving Actions
Vision-language-action (VLA) models typically generate actions in chunks, organizing continuous robot execution into separately predicted segments and thus introducing potential discontinuities at action-chunk transitions. We propose ODEAction, a chunk-free action generation paradigm that replaces action-chunk prediction with Continuous Action Evolution. Given visual observations, language instructions, and robot proprioceptive information, ODEAction learns an action transition field defined over action space, where a transition vector characterizes the evolution rate of the current action with respect to evolution time. During execution, the model takes the current action as its initial condition and integrates the learned field using an ordinary differential equation (ODE), recursively generating subsequent actions with explicit dependencies. ODEAction is implemented as an action head and does not depend on a specific vision-language backbone architecture. Experiments on LIBERO, EBench, RoboTwin, and real robots show that ODEAction improves both task performance and trajectory quality over matched action-chunking baselines, with particularly pronounced gains on fine manipulation tasks.
comment: 23 pages, 5 figures, including appendix. Technical report
★ Goodhart in the Planner: Sampling-Based Control Exploits Rewards Learned from Human Preferences
Robots increasingly choose actions by searching against a reward model learned from human preferences, in model-predictive control or best-of-$N$ selection; we measure what this does to the true objective. On HalfCheetah and Hopper with crowd-sourced labels, and on Meta-World drawer-open and plate-slide with synthetic labels and blind labels from the two authors, harder MPPI search cuts true return by up to 76%: with human labels the drawer opens in 0 of 20 runs against 10 of 10 for the cloned policy the planner starts from, while held-out label accuracy stays at 0.75-0.97. On plate-slide human labels are exploited less than synthetic ones, and on window-open search helps. True return roughly follows how far the planner's weights move from its proposals ($\sqrt{\mathrm{KL}}$) for both weighting rules we test, and best-of-$N$ with a verifier from synthetic labels falls below a single noisy action. We prove that $χ^2$-regularized MPPI at a temperature set by the scorer's error spread on the proposals is worst-case optimal, that no positive gain can be certified once that spread exceeds the scorer's own, and that an episode's return splits exactly into per-step terms, of which the guarantee covers the selection terms. Calibrated from 100 proposal pairs ranked by true return (an oracle) on start states the evaluation never uses, this rule is the safest fix we test: with each of five independent sets it succeeds in every arm run and keeps on average 86-94% of the cloned policy's return on plate-slide and drawer-open, though it is cautious where search helps. Calibration from a person's comparisons cannot detect a bias the scorer learned from that person; correcting for the pairs people cannot tell apart, or a second person, limits the damage.
comment: 19 pages, 4 figures. Equal contribution. Code and results will be released
★ LACART: Lookahead-Augmented Collision Avoidance and Reference Tracking Strategies for Safety-Critical Robots
Autonomous racing provides a demanding testbed for collision avoidance under tight geometric and computational constraints. Building on a recent conference contribution and a master's dissertation, LACART combines lookahead-based control barrier functions (CBFs), a relaxed control Lyapunov function (CLF), and quadratic programming (QP) to modify nominal vehicle commands for obstacle avoidance and reference tracking. The method handles circular obstacles and track boundaries in a common optimisation layer. Comparative simulation results against Huang \emph{et al.} \cite{r9} show similar full-track paths and lower median computation times: 0.31 versus 0.33 ms per step and 0.023 versus 0.025 ms per QP solve. The dissertation further demonstrates a 1:43-scale vehicle with camera-based estimation and wireless actuation at a reported 25 Hz control rate. We identify the conditions linking lookahead-point clearance to body clearance, illustrate hardware limitations, and propose a recorded demonstration of safety intervention and recovery. The contribution targets ground-robot collision avoidance and real-time safety monitoring.
★ DKWBC: Decoupling Motion Tracking and Balancing for Humanoid Whole-Body Control
Humanoid whole-body control requires a robot to track diverse reference motions while maintaining balance under gravity, torque limits, contact switching, and external disturbances. Existing reinforcement-learning-based trackers typically learn control outputs that jointly address motion tracking and balance. However, similar reference poses can correspond to substantially different dynamic demands, requiring a unified policy to learn motion-dependent control requirements alongside balance corrections. This can complicate learning and lead to high-frequency fluctuations in control outputs. We propose DKWBC, a dynamics-informed whole-body control framework that combines explicit model-based tracking with a learned balancing residual. Inspired by the model-based and environmental-wrench contributions to joint torques in floating-base dynamics, DKWBC uses inverse-dynamics feedforward and joint PD feedback to supply tracking torques, while a residual policy learns additional joint-torque corrections for balanced execution. By explicitly accounting for motion-dependent torque demands, this structure simplifies the policy's learning task. Experiments in simulation and on real hardware demonstrate that a single learned balancing policy generalizes across motion datasets while maintaining accurate motion tracking and balance across ground support, attached payloads, and physical interaction. Project page: https://robbyant-research.github.io/DKWBC/.
comment: 9 pages, 6 figures, 4 tables
★ Extended Model-Based Learned Inertial Odometry
Inertial odometry is a compelling approach to state estimation for agile quadrotor flight due to its affordability, low weight, and robustness to perceptual degradation. However, relying only on integrated inertial measurements is impractical, as sensor errors and time-varying biases lead to significant pose drift. Although recent advances have enabled inertial odometry for drone racing, existing approaches show limited generalization to trajectories not seen during training. This work improves generalization by adopting a body-frame representation and incorporating body-frame torque dynamics into the learning pipeline. The learned model predicts short-horizon relative displacements, which are fused with IMU measurements in an Extended Kalman Filter. Experimental results show that the proposed method outperforms the previous state-of-the-art learned inertial odometry approach for quadrotor pose estimation on unseen trajectories. An additional evaluation on four public quadrotor datasets, with whole trajectory shapes held out of training, confirms a lower velocity error on every held-out set and shows that the torque input is most informative when it is derived from the rotor signals. Code, data pipeline and trained models are publicly available.
comment: 8 pages, Published at ICUAS 2026
★ Lightweight Path-Aware Frontier Re-ranking for LiDAR-Based UAV Exploration
Autonomous 3D exploration with UAVs requires efficient target selection under limited flight time and onboard computation. In conventional Euclidean-distance-based frontier pipelines, geometric distance may poorly represent the actual collision-free travel cost in cluttered environments, leading to inefficient frontier choices. This paper presents a lightweight path-aware frontier re-ranking method for LiDAR-based UAV exploration. The method first retains a small top-N set of promising frontier candidates using a lightweight utility function and then applies time-bounded RRT* planning to estimate their feasible path costs for re-ranking. The path generated for the selected frontier is further reused for navigation to reduce redundant global planning. Experiments in three warehouse-like environments show that the proposed method consistently produces shorter exploration trajectories than conventional frontier baselines, with traveled-distance reductions of up to 14.1%, while reducing exploration time by up to 9.5% in two environments and remaining competitive in the third. The method maintains map detail comparable to the conventional OctoMap-based baseline, demonstrating an effective trade-off between exploration efficiency and mapping quality.
comment: Accepted for presentation at RIVF 2026 (20th International Conference on Computing and Communication Technologies)
★ CucumberVision at the Edge: Accuracy, Throughput, and Energy of RGB-D Cucumber Sizing on Jetson Orin Nano and Raspberry Pi 4
Non-contact fruit sizing from RGB-D imagery reaches sub-centimetre accuracy on workstation hardware, but in-field grading requires the same pipeline to run on a low-power single-board computer, and the cost of that transition is rarely reported. We deploy a curvature-aware cucumber length-measurement pipeline (YOLO26n detection, prompt-based segmentation and a medial arc spline estimator) on an NVIDIA Jetson Orin Nano and a Raspberry Pi 4, and characterize both against an RTX 3050 Ti workstation by replaying 50 identical greenhouse captures of 7 specimens, with specimen-level cluster-bootstrap statistics. Accuracy is preserved: mean absolute error (MAE) is 0.58 cm (95% CI 0.46-0.69) on both the workstation and the Jetson at matched precision, with a paired difference of zero. TensorRT FP16 compilation cuts Jetson detection latency 2.57x, giving two operating points on one board: 72 ms per fruit at 0.84 cm, or 2.74 s at 0.64 cm. Four ablations map the design space. For prompted segmentation backends, measurement accuracy is fixed by the backend and reproduces across platforms to within 0.01 cm, while its cost is fixed by the platform and varies 214x, so the two choices are separable. Whole-graph INT8 quantization of the NMS-free detection head fails structurally, returning no detections, because one output tensor carries both pixel coordinates and confidences; we show that no calibration strategy can avoid this. Excluding the head restores function at 1.37x. Energy per fruit is 42.2 J on the Jetson, and its 7 W mode consumes 73% more than its 15 W mode. The Raspberry Pi with MobileSAM costs 2.5x more per fruit than the Jetson with SAM despite drawing 47% less power, and it runs without thermal throttling under sustained load. The results define an accuracy-throughput-energy envelope with three non-dominated configurations and give rules for selecting among them.
★ Mechanical Interaction Sensing for Robotic Kiwifruit Handling with an Optoelectronic Soft Suction Cup
Robotic handling of delicate fruit benefits from compliant contact and quantitative sensing of the mechanical interaction at the end-effector. This study investigates an optoelectronic soft suction cup with a kiwifruit-specific bicomponent design for estimating interaction forces during contact with real fruit. During known-contact intervals, the optical signals accurately estimate interaction force on fruits excluded from model training: the best shallow network achieves grouped out-of-fold pooled R-squared = 0.968 +/- 0.005 (MAE = 0.138 N), with a median per-fruit held-out R-squared = 0.973 (MAE = 0.134 N). A simple monotonic mapping from the mean optical response alone achieves R-squared = 0.925 +/- 0.005, showing that substantial force-related information is already present in the optical response. Leave-one-collection-day-out validation remains strong (R-squared = 0.955 +/- 0.031), supporting robustness across acquisition sessions. As an exploratory secondary analysis, binary Soft/Firm classification from compression-test reference labels achieves moderate balanced accuracy (approximately 0.705) under grouped held-out-fruit validation, while finer three-class grading shows limited performance.
★ Can LLMs Debug Robots by Editing Their Weights?
A language model can repair a robot policy's lexical instruction grounding by editing just 65 token-embedding rows. We bring weight-space programming to robot foundation policies: GPT-6 Astra reads weights and uses a CPU workspace and budgeted simulator feedback, without gradient updates or new demonstrations. Three independent sessions on $π_{0.5}$ all improve success on 765 held-out paraphrases; the best raises success from 86.3\% to 92.0\% (48 repairs, four regressions; exact paired $p=1.31\times10^{-10}$). Among the 335 instructions containing edited tokens, success rises from 74.3\% to 87.5\%; the other 430 episodes have zero outcome changes. Matched random-direction edits do not help. Full replacement using the editor's own lexicon explains much of the gain but produces more regressions; its direct difference from the editor is not significant at 0.05. Across 13 further robot sessions, the editor declines 12 candidates; the sole retained edit does not show a significant held-out gain. Separately, benchmark metadata lowers FastWAM success from 70.0\% to 48.6\%, while the evaluated VLA's aggregate rate is unchanged. The evidence supports precise lexical repair, not general acquisition of missing robot capabilities.
★ Horizon-Coupled Obstacle Guidance for Relative-Action Diffusion Policies
Real-world robot deployment requires adapting learned policies to unseen obstacles without collecting new demonstrations or retraining. Existing policy-level guidance often applies obstacle corrections independently at individual horizon steps, which may address local obstacle violations but fail to coordinate the action sequence into a complete, task-directed detour. How obstacle responses should be propagated across the action horizon to coordinate complete, task-directed avoidance motion remains underexplored. We introduce an inferencetime obstacle-guidance framework for pretrained relative-action diffusion policies. It rolls out cumulative relative actions into a workspace trajectory, evaluates obstacle interactions along the trajectory with an SDF-based obstacle model, and propagates each obstacle response to all contributing action increments during denoising. Real-robot experiments across seven obstacleconstrained scenarios spanning four manipulation tasks show that our method achieves 105/140 (75.0%) collision-free task successes, compared with 8/140 (5.7%) for absolute-action pointwise guidance. Ablations show that the relative-action horizoncoupled formulation provides the larger observed improvement across both obstacle models.
★ MDF++: Visuo-Tactile World Action Model with End-to-end Representation Learning
Recently, world-action models (WAMs) have emerged as a powerful framework for robot manipulation. By jointly modeling action generation and future observation prediction, WAMs can act and anticipate the outcomes of the actions. However, existing WAMs typically rely on frozen pretrained encoders trained with reconstruction, which may not produce representations well suited for robot control. In this paper, we tackle the problem of learning physicallygrounded, control-relevant representations for WAMs. We propose MDF++, a visuo-tactile world-action model that learns latent representations of multisensory observations end-to-end with a diffusion backbone. While naive joint training can lead to feature collapse, MDF++ introduces a multi-task learning strategy that leverages cross-modal correlations to learn task-relevant representations. We evaluate MDF++ on a suite of contact-rich manipulation tasks, including realworld bolt screwing and nut mating, and show that it outperforms WAMs built on frozen pretrained representations by up to 50%. Additional videos are available on our website: jointmdf.github.io.
comment: Corl 2026
★ Task-Driven Closed-Loop Design Optimization of Passively Morphing Quadrotors for Contact-Mediated Narrow Gap Traversal
Aerial robots operating in cluttered environments may encounter openings narrower than their nominal width. Existing passively morphing drones reduce their width upon traversal, but they typically involve human-made designs, which can struggle to find the right balance among folding ratio, flight efficiency, and control tracking performance. We introduce a closed-loop design method that jointly optimizes quadrotor geometry and the torque-deflection response of two passively folding arms driven by the success rate in simulation for contact-mediated gap traversal. An optimization algorithm samples candidate designs that are evaluated in a contact-based simulation of the traversal task, where the task success rate is used as a feedback signal to sample progressively better designs over time. The resulting prototype passively reduces its width from 31.3 to 14.1 cm and traverses a 15 cm wide opening at a 2 m/s reference speed. It also tolerates gap approach-angle errors of up to 30 degrees and folds through a 40 cm long confined passage. Compared to a rigid twin, the soft configuration adds only 2 mm of position RMSE and less than 1% electrical power during contact-free flight. These results show that task-driven closed-loop design optimization can produce passively morphing airframes that are compact under contact, tolerant to approach error, and effectively rigid during nominal flight.
comment: Video: https://youtu.be/xGkEtl6-vCA. Code: https://github.com/ntnu-arl/soft-quadrotor-optimization
★ EGT: Execution-Grounded Traversability through Learned Closed-Loop Trajectory Distributions
Rough terrain traversability is often estimated through point-wise roll, pitch, or force-angle margins along a planned path. However, these metrics are often inaccurate and overly conservative because they ignore how terrain disturbances and low-level controller corrections can affect the vehicle's motions. To overcome these limitations, we introduce Execution-Grounded Traversability (EGT), defined as the probability that a vehicle-controller pair can execute a candidate path while meeting trajectory-level tracking and progress requirements. We estimate EGT by predicting the distribution of closed-loop trajectories using a latent-space flow matching (FM) model conditioned on multi-resolution terrain features and 3D vehicle attitude and wheel-ground contact points prior. Across 650 simulated trials on unseen terrains, EGT reduces safe-path rejection to 2.2% (versus 11.4%-39.5% for pose-margin and force-angle baselines) and unsafe acceptance to 7.7%. Integrated into a receding-horizon sampling-based planner, EGT reduces navigation failure rates by 24-42 and 30-44 percentage points over [1] and [2], respectively, while cutting goal-reaching time by up to 20.5% and 45%. Finally, we demonstrate zero-shot transfer of our simulation-trained model to real-world outdoor experiments.
★ SUGARv2: Learning Diverse Visual Humanoid Loco-Manipulation Skills from Synthesized Interaction References
Scaling up visual humanoid loco-manipulation skill acquisition requires reducing the human effort needed as tasks and spatial configurations expand, particularly in demonstration collection and task-specific training design. We present SUGARv2, a unified framework for learning diverse visual humanoid loco-manipulation skills through interaction reference synthesis and a shared training pipeline. To obtain diverse references at low cost, we adapt reusable task templates to different spatial configurations, synthesizing coordinated robot and object motions. To reduce task-specific reward design and training effort, we use reference-based imitation as a common learning objective across tasks. We first train privileged teachers to execute these references, then train visual policies through teacher distillation and reinforcement learning with the same reference-based mimic rewards, using diverse object instances and visual domain randomization to support visual generalization and sim-to-real transfer. Across five humanoid loco-manipulation tasks, SUGARv2 achieves a mean success rate of 91.0% in simulation. Deployment on a real humanoid robot further demonstrates successful sim-to-real transfer for whole-body manipulation. These results demonstrate the framework's potential for learning diverse visual humanoid loco-manipulation skills.
comment: 11 pages, 6 figures. Project page: https://tianshuwu.github.io/sugarv2-humanoid/
★ TPS-A*: Path Planning for Any-Shape Vehicles with Certified Collision Checking in Trajectory Parameter Space
Global planners for ground vehicles usually approximate the vehicle by a disk, trading safety against conservativeness, while exact-footprint checking pays for every candidate motion, pose, and footprint edge. We present TPS-A*, a lattice planner whose edges are kinematically feasible trajectories from parameterized trajectory generators, built on certified collision checking in trajectory parameter space. A grid precomputed per footprint returns, in one pass over the local obstacles, a certified free distance along every candidate trajectory, so the per-node collision cost is independent of footprint complexity and trajectory count. We prove that this grid never overestimates the free distance of the continuously swept, possibly non-convex footprint, and state the planner's guarantees: collision-free paths and bounded suboptimality relative to its lattice abstraction. Goals with or without heading are reached exactly by Reeds-Shepp shots certified as trajectories of the same generators. An exact independent test finds no collision in its paths. On 300 cluttered worlds, with the true footprint and the same goal pose for all planners, it solves every query with a collision-free goal pose in a median of three milliseconds on an AMD Ryzen 7 2700X, on par with hybrid A*, 45\% of whose paths overlap obstacles, and nearly sixty times faster than a state-lattice planner, the only other collision-free one; with position goals, eighty-nine times faster than hybrid A*. It matches hybrid A* in automated parking and on narrow doorways, docks a non-convex fork footprint that no convex model admits, and drove a real robot across a university campus among people.
comment: 12 pages, 6 figures
★ Augmentron: Scalable Multi-View Visual Augmentation for Robot Learning NeurIPS 2026
Robot policies trained on visually narrow demonstrations can become sensitive to environmental features that are unrelated to the task. Existing generative augmentation methods can introduce visual diversity, but processing every frame and camera view independently is computationally expensive and may produce inconsistent observations across cameras and timestamps. We introduce Augmentron, a geometry-aware pipeline that generates reusable scene representations and renders them through recorded camera poses. This produces temporally and multi-view consistent environ- ment and workspace variations while preserving the robot, task objects, actions, and annotations. Across three physical manipulation tasks, replacing half of the real world training dataset with Augmentron data improves model success across three tasks for both in out-of-distribution distractor objects, and out-of-distribution workspace variation evaluations. In a separate comparison against alternate augmentation techniques, Augmentron achieves the highest pooled success in all three conditions. The results show that introducing augmented data to a data training recipe can help policies generalize to dynamic real-world settings.
comment: Accepted to the 8th Robot Learning Workshop (WRL), NeurIPS 2026
★ Auditing Grasp Feasibility Predictors with Planner-Verified Counterfactuals WACV 2027
Grasp-feasibility predictors report near-perfect accuracy on Task and Motion Planning benchmarks, but high accuracy on these benchmarks does not verify that a predictor uses the collision geometry that the motion planner checks. We propose a planner-verified counterfactual (CF) test: holding the grasp pose fixed, we insert a solid obstacle into the approach corridor, add its points to the model input, and let a planner re-derive the label, which flips from feasible to infeasible. Across eight predictors spanning point clouds, voxels, depth, and object tokens, CF-pair AUROC falls from high natural AUROC (0.920-0.996) to chance (0.470-0.504), and the flip holds across four planners, two arms, and a real-object obstacle. Predictors fail at different resolutions, from a bounding box to a 12.5 cm voxel. In the point-cloud models we dissect, the obstacle either moves the representation in a direction unrelated to whether it blocks the grasp or is not registered at all. A clearance-aware read of the corridor points, trained on the unmodified benchmark with no counterfactual scenes, reaches 0.983 CF-pair AUROC, but it remains bound to its trained obstacle placements; a placement-invariant clearance representation is still open. We release the counterfactual labels, metrics, datasheet, and relabeling pipeline at https://github.com/codewithsajid/graspfc-cf.
comment: Accepted to WACV 2027. 10 pages main paper + 8 pages supplementary
★ Toward the Era of Agentic Embodied Intelligence
Foundation models increasingly guide robot actions, yet sustained autonomy requires efficient control and failure recovery. We introduce A2-MEAS, an embodied multi-agent framework built around code as a scaling interface. Learned correction codes combine corrective language, spatial evidence, execution history, and action proposals. MetaBrain provides training supervision and runtime guidance. A correction encoder and adapter absorb this knowledge into policy weights during training, while fresh codes adapt behavior with frozen weights at deployment. The action model, A2-WLA, adds motion-aware history and a latent world model to a vision-language-action policy. Surprisingly, this simple design with action pretraining and task specialization reaches 0.5755 Q-score on the 50-task BEHAVIOR 2026 public subset before online correction, 2.21x the 2025 winner's reported 0.2605 across different benchmark editions and training configurations. On all 100 BEHAVIOR 2026 public tasks, A2-MEAS achieves 0.6411 Q-score and 45.40% task success. On RoboTwin 2.0, A2-WLA achieves 93.98% average success, rising to 95.05% with the complete system. Across 12 real-robot tasks in five household settings, A2-WLA achieves 57.92% average success versus 26.67% for pi0.5. In long-horizon pour-over coffee preparation, a history-conditioned adapter raises post-failure completion at two filter-handling steps from 14% to 94% and from 8% to 82%, compared with A2-WLA retries. Our training infrastructure delivers 427.1 samples per second, 4.03x the throughput of a pi0.5 reference implementation on matched hardware. We will release the system, models, training scripts, and infrastructure to support efficient embodied learning.
★ Safe Circumnavigation on Heterogeneous Orbits: A Range-Only HOCBF-QP Framework for Nonholonomic Multi-Agent Systems
Circumnavigating a stationary target with a team of robots requires each robot to hold its own standoff distance, to stay evenly spaced from its neighbours, and to avoid collisions with the target and with each other at every instant. The task becomes arduous when the robots are assigned different orbital radii, as their sensors or roles may demand. This paper presents a safety-critical control framework for this problem. Each robot commands both its acceleration and its turn-rate, measures only its distance to the target, and exchanges information only with its two ring neighbours. The design has three features. First, a structural property of the model lets one control law serve every robot regardless of its assigned radius. Second, the acceleration drives each radius exactly to its target, while the turn-rate balances the formation with the neighbours. Third, a higher-order control-barrier-function quadratic program enforces target, geofence, and inter-robot safety on both inputs at once; it leaves the nominal controller untouched away from the constraints and stays valid while the unmeasured range rate is still being estimated. Simulations confirm these guarantees. In a comparison, the closest existing multi-orbit design, even with full state information, violates both the target standoff and the inter-robot margin, whereas the proposed controller does not.
★ Robust Readout of Cross-Point Resistive Arrays via Time-Domain Optimization
Cross-point resistive arrays are attractive for high-density tactile sensing because they reduce wiring complexity. However, conventional frequency-based readout methods can become inaccurate in robotic sensing environments, where long interconnects and measurement and environmental noise degrade the observed signals. We propose a time-domain optimization method for robust readout of cross-point resistive arrays. The method determines resistance values by fitting a time-domain observation model to the measured waveform over the interval, rather than relying solely on spectral amplitudes. By exploiting the full waveform, the proposed method improves robustness under noisy observation conditions. In simulations of a 16 x 16 array, the proposed method reduced the mean readout error by 20% compared with the baseline method. Furthermore, validation over 1,000 trials showed that the proposed method achieves more robust estimation than the conventional method, reducing the mean error and standard deviation by 24.6% and 77.6%, respectively. The improvement was also confirmed in cross-point array simulations with resistance distributions emulating tactile sensor responses.
comment: 4 pages, 1 figure, IEEE Sensors 2026 accepted
★ When to Trust the BIM: Identifiability-Aware LiDAR-Inertial--BIM Fusion under As-Built Deviations
Building information models (BIM) can provide a global geometric reference for robot localization with LiDAR-inertial odometry (LIO). However, as-built deviations from the as-designed model mean that a disagreement between a scan and the BIM may arise from a pose error, an offset wall surface (an as-built deviation or an unmodeled object in front of the wall), or a shifted room, and in some geometries these explanations produce identical observations. In this paper, we formalize this attribution problem as fault identification over three fault classes and derive a geometric criterion for when attribution is possible. Building on it, we propose an identifiability-aware method that reports an attribution only when the fault is attributable; otherwise, it retains the candidates, defers BIM updates and BIM-based pose corrections in the ambiguous directions, and recovers once new references arrive. In the pre-registered two-dimensional study, on cases whose fault is detectable but not attributable, the method's conditional error rate was 0.83%, against 31.67% for a baseline with uncertainty-based abstention at the same answer rate. On the 5,600 attributable cases, the method reported a unique attribution 5,574 times, with no incorrect attributions. In paired fault injections in a laboratory room with its IFC model and on three floors of the SLABIM dataset, it raised no off-target attribution (0 of 168 conditions in the laboratory; 0 of 1,302 on SLABIM) and attributed all eight laboratory faults predicted to be attributable, whereas both reproduced baselines raised off-target structural flags in every setting.
comment: 24 pages, 12 figures, 11 tables (including appendices)
★ DobotTrack: Target-Grounded Vision-Language-Action Learning for Embodied Person Tracking
Language-guided embodied person tracking requires a mobile agent to identify a target from a referring description and consistently follow the target under egocentric observations. We present DobotTrack, a target-grounded vision-language-action (VLA) framework that integrates auxiliary grounding supervision with efficient temporal modeling. We systematically investigate five auxiliary tasks---person segmentation, bounding-box grounding, visual-spatial classification (VSC), future DINO feature prediction, and future-frame prediction---and adopt complementary mask and bounding-box supervision to enhance target delineation and localization. We introduce Q-identity, a target-conditioned historical visual compression module that leverages target-grounding predictions to selectively preserve identity-discriminative appearance features and spatial cues in compact historical tokens. On EVT-Bench, our 0.8B-parameter model achieves an average success rate of 77.9%, exceeding OmTrackVLA by 16.9 percentage points across the three tasks. We further construct a real-world person-tracking dataset collected using a quadruped robot, comprising 126 episodes and 596,323 mask-annotated frames. The target masks enable automatic derivation of bounding boxes and visual-spatial labels encoding horizontal position and apparent image scale, providing additional grounding supervision without separate manual annotation of these labels. Together, our framework and annotation pipeline support learning target-grounded tracking policies from simulated and real-world data. Code is available at https://github.com/Dobot-Team/dobot_track.
★ ToolDex: An Ego-Exo Human Demonstration Dataset for Creative Physical Tool Use
Creative tool use is a fundamental human capability and an important step toward general-purpose embodied intelligence. It requires agents to reason about object affordances, physical constraints, and manipulation strategies, especially when standard tools are unavailable. However, progress in this area is limited by the lack of real-world datasets that capture human tool-use behavior with sufficient scale, fidelity, and diversity. Existing benchmarks are often simulation-based, focus on simplified interactions, or lack detailed observations and annotations. We introduce ToolDex, a large-scale egocentric-exocentric (ego-exo) human demonstration dataset for creative physical tool use. ToolDex captures scenarios in which humans select, adapt, and repurpose available objects as tools. Using synchronized egocentric cameras and eight exocentric cameras, the dataset provides high-fidelity multi-view observations and rich annotations for hands, tools, target objects, and their deformable interactions. Beyond data collection, we establish a benchmark suite characterizing three core capabilities: physical tool-use reasoning, sparse-view 3D and 4D reconstruction, and robot policy learning. ToolDex provides a foundation for studying human tool-use behavior and developing embodied agents that can adaptively use tools in complex real-world environments. Dataset visualizations and benchmarking results are available at https://tooluse-dataset.github.io/.
★ A Controlled Study of Models, Skills, Feedback, and Curriculum in Agentic Robotics
Agentic robotics asks coding agents to implement and iteratively improve robot policies from simulated and real-world physical interactions. We study the self-learning loop of agentic robotics through a controlled simulation study spanning nine manipulation tasks, 631 different experiment configurations, and 28K coding agent rollouts, systematically varying model capability, skill priors, feedback structure, and curriculum. The experiment data suggests that the bottleneck is often not the access to sufficient information, but interpreting it into the right policy update. Across these factors, a successful self-learning loop depends on how effectively agents attribute failures, reason task-relevant spatial and geometric structure, and selectively restructure code to improve policies.
★ FASTFUNNELS: Agile Multi Robot Navigation in Narrow Spaces via Adaptive Dynamic Funnels
We focus on the problem of high-speed multi-robot navigation in cluttered spaces with tight spatial constraints. A key challenge is balancing safety with agility, as existing methods often trade one for the other (slowing down near a bottleneck to avoid collisions versus failing to swerve in time at high speeds), especially for car-like systems. Our approach belongs to the family of formation control methods, in which Ackermann-steering agents coordinate while navigating tight spaces. However, current formation control techniques rely on rigid, pre-defined shapes and transitions that depend on initial conditions, the environment, and vehicle dynamics, and are impractical to maintain at high speeds. For example, agents passing through a small gap must first form a square, transform into a line using analytical equations, and then return to a square. In contrast, our approach is inspired by a droplet of water that continuously adapts its shape to the space it flows through without slowing down. We propose a hierarchical algorithm in which, at the high level, a reinforcement learning policy produces a dynamic, blob-like elliptical patch that morphs its shape in real time, squeezing, stretching, and orienting itself around obstacles and through tight spaces. At the low level, the robots are controlled via distributed model predictive control (DMPC) to stay inside the patch, which acts as a soft forcing function, while avoiding collisions with each other. We evaluate our approach, FastFunnels, on simulated F1TENTH racing cars across 14 scenarios with N=1 to N=6 robots, averaged over 20 trials each. FastFunnels achieves an 89% completion rate at 7.73 m/s, while all baselines either fail to complete the track or succeed only at very low speeds (0.07-1.79 m/s, about 40% completion rate).
comment: 8 pages, 4 figures, 1 table
★ VLA Handover: Failure-Prediction-Based Switching from On-Robot to Server VLA Models
Large vision-language-action (VLA) models generalize well but are difficult to run in real time on a robot, and small VLAs that run there fail more often. Remote inference over beyond-5G and 6G networks lets a robot use a large VLA on a server, but querying the server for every action chunk limits the number of robots per server. We propose VLA handover: the robot runs a small VLA (S-VLA) and switches to a large VLA (L-VLA) on a server only when the S-VLA is likely to fail. We formulate the switching as minimizing the request cost under a required success probability and, since this problem cannot be solved directly, propose a heuristic framework that retrains neither VLA. In simulation with SmolVLA and $π_{0.5}$ on three of the four LIBERO suites, VLA handover raised the success rate by 11-22 percentage points over the S-VLA alone and sent 49-77% fewer requests than server-only control at a 2-4 percentage-point lower success rate, and its completion time grew less with the network latency.
comment: 6 pages, 4 figures, 3 tables. Submitted to IEEE ICC 2027
★ Flex-RSI: Benchmarking Recursive Self-Improvement of Frontier Agents
Frontier language models are increasingly deployed as agents, and how well an agent performs depends not only on its model but also on its harness: the prompts, tools, memory, and control loop through which the model acts. When an agent is applied to a new problem, a natural question is whether it can use the feedback of the task to improve the way it works, and not merely its answer. Most existing benchmarks keep the agent's harness fixed and score the solution it produces, or target large-scale model training. We introduce Flex-RSI, a benchmark for the recursive self-improvement of agents on vision and robotics tasks under a fixed budget. In each round, an executor improves the task solution, which is kept only if it scores higher on held-out validation data, and an improver then revises the harness, including the executor and itself. Every run starts from the same general-purpose harness, and the final solution is scored on hidden test data. Evaluating closed-source and open-weight models on tasks spanning perception, reasoning, robot action, 3D reconstruction, visual creation, and infrastructure, we find that self-improvement helps most when the agent builds its own solution, that feedback recording only success or failure holds back open-weight models, and that open-weight models fall furthest behind where outputs must pass strict checks of physical stability or exact correctness. Paired runs further show that editing the harness itself contributes to the gains.
comment: Project page: https://flex-rsi.github.io. Jingdong Zhang and Jionghao Wang contributed equally
★ RevoSim: Scalable Multimodal Tactile Simulation for Dexterous Manipulation
We present RevoSim, a scalable and high-fidelity multimodal tactile simulator for the BrainCo Revo3 hand that jointly models 285 piezoresistive taxels and five visuotactile fingertips. RevoSim efficiently generates synchronized pressure, depth, RGB, and marker-displacement observations; on a single RTX 5090, its depth-conditioned marker-displacement configuration supports up to 4,096 parallel environments. We train in-hand manipulation and grasping policies entirely in simulation, establishing a foundation for future sim-to-real transfer. Project page: https://braincotech.github.io/RevoSim-web/.
★ Ordered Policy Optimization NeurIPS 2026
Proximal Policy Optimization (PPO) stabilizes policy updates by clipping the importance ratio between the new policy and the data-collecting policy. The standard rule uses one fixed clipping range for all samples. While this choice offers simplicity and effectiveness, it fails to distinguish statistically harmful extreme ratio values from large, yet tolerable changes. We study how to adapt the clipping radius to the reliability of the induced importance ratio distribution. Our case study demonstrates that reliable updates require controlling these extreme ratios, rather than merely verifying that individual ratios fall within a fixed interval. Motivated by this observation, we propose Ordered Policy Optimization (\textsc{OPO}). In each minibatch, \textsc{OPO} sorts the importance ratios, selects the extreme ratios that potentially have the largest effect on the update, and assigns them adaptive sample-wise radii instead of using the fixed clipping radius. These radii are derived from a prescribed tail profile, which specifies desired target values for the selected extreme ratios. The resulting objective remains first-order and requires only minibatch sorting and a modified policy loss. Experiments conducted on MuJoCo and Atari benchmarks demonstrate that this simple modification achieves strong performance relative to established baselines.
comment: 36 pages, 12 figures, 11 tables. Published in Proc. of the 40th Conference on Advances in Neural Information Processing Systems (NeurIPS 2026)
★ Wellness Robots and the Path to Full Autonomy
This paper introduces the autonomous wellness robot as a distinct category of socially assistive system, defined by its capacity to continuously assess and respond across the seven ICAA dimensions of wellness (see Figure 2). We propose the Care Robot Autonomy Scale (CRAS): a six-level taxonomy (L0-L5) modeled on the automotive automation standard SAE J3016 and evaluated across four wellness-aligned dimensions. We present Ryan, developed initially at the University of Denver and commercialized by DreamFace Technologies, as a case study. NIH/NIA-funded longitudinal evidence (N = 18; 887 hours) shows statistically significant cognitive improvement (SLUMS +2.61, p = 0.015) and a clinically meaningful directional reduction in depressive symptoms (PHQ-9 -2.50, p = 0.073), with no novelty effect. We position Ryan today at CRAS Level 1-2, with emerging Level 3 capabilities in some dimensions, and outline a three-phase roadmap to L5 (full autonomy) by the early 2030s.
★ HOC2R: Hand-Object Contact Trajectories for Human-to-Robot Motion Transfer
Human videos contain abundant object-manipulation demonstrations, but they do not directly provide motion cues that a robot can follow. We present HOC2R, a Hand-Object Contact trajectory extractor for Robot motion that combines frozen DINOv3 and SAM 2.1 features with learned heads for hand-object association and contact localization. HOC2R supports two modes: count-aware automation that detects, counts, and localizes visible contacts in untrimmed videos, and fixed-two localization for two-finger grippers when contact frames are known. On a 30-task point-annotated benchmark, count-aware automation improves contact-count accuracy from 58.1% to 71.4% and reduces mean localization distance from 19.81 to 10.46 pixels relative to an adapted VRB2 baseline. On seven unseen RGB-D rollouts, a five-task fixed-two checkpoint reduces mean localization distance from 67.67 to 27.02 pixels. Across six robot evaluation sequences, the 30-task fixed-two checkpoint produces four feasible trajectories and succeeds on three tasks, compared with two feasible trajectories and one successful task for each adapted baseline. HOC2R is competitive in the evaluated single-hand scenes, robust to the hand-color shift, and strongest in the evaluated multi-hand scenes. These results support HOC2R as a practical pseudo-labeling method for extracting robot-usable contact trajectories from human demonstrations.
comment: 10 pages, 10 figures, 9 tables. Accepted for oral presentation at the Australasian Conference on Robotics and Automation (ACRA 2026)
★ PRISM: Primitive Routing via In-context Skill Mixing for Retentive VLA Adaptation NeurIPS 2026
Vision-Language-Action (VLA) models promise a single language-conditioned policy adaptable to many manipulation behaviors, yet deployment additionally requires skill retention and long-horizon composition: a policy must retain previously acquired skills and reuse them inside composite tasks whose behavior changes across phases. The standard recipe absorbs every new task into one shared update, which provides neither a parameter-level locus for reusable skills nor a mechanism for invoking each skill along the trajectory. We observe that this missing structure is latent rather than absent: per-skill low-rank updates of a pretrained VLA occupy a robustly distinguishable, low-dimensional subspace, and a single composite-task LoRA concentrates roughly half of its Frobenius energy in this subspace, against a near-zero matched random-rotation control. Building on this geometry, we introduce PRISM, which represents reusable skills as parameter-level primitives: each skill family is stored as a dedicated low-rank action-expert adapter in a primitive bank, primitives are augmented with skill-specific low-rank residuals and composed at inference by a per-chunk VLM-conditioned router that reads the prefix representation to track task progress, and the router is supervised by per-chunk labels from a deterministic, task-conditioned, action-derived oracle. During composite training, the vision encoder, the language LoRA, the skill residuals, the router, and the shared action projections and time MLP are updated; every primitive is held frozen, so the base per-skill action-expert adapters are preserved by construction. On RoboCasa365, PRISM achieves 56.43% atomic-skill success and reaches 20.00% on long-horizon composite tasks, while a standard $π_{0.5}$-LoRA two-stage fine-tune reaches 17.86% atomic and 17.50% composite success. We additionally validate PRISM on an AgileX PIPER real-robot setup.
comment: 19 pages, 6 figures, 4 tables. Accepted at NeurIPS 2026
★ HomeLoop: A Simulation Framework and Benchmark for Robot-IoT Coordination in Everyday Domestic Assistance and Safety
Recent progress in embodied robotics is bringing robots closer to deployment in homes, as vision-language models (VLMs) are demonstrating significant advances in reasoning, making VLM-driven robots promising home assistant agents. These agents will often be introduced into homes that already have cameras, Internet of Things (IoT) devices, and residents with wearables. To fulfill their potential, they need to integrate with existing smart infrastructure and reason through ambiguous scenarios: a smoke alarm may come from a pan about to ignite, a pan already on fire, a resident cooking or a faulty sensor, and each calls for a different response. To that end, we introduce HomeLoop, a simulation framework and benchmark built on Isaac Sim that combines robot bodies, networked devices and wearables, simulated residents and visitors, faulty sensors, and hazards that evolve over time. Our benchmark is composed of 20 tasks across five families grounded in real-world household hazards and needs, and designed so that the same alert can have different causes. A runtime monitor scores each episode with temporal-logic checks on the final outcome and the actions along the way, and every action carries a predefined cost. We evaluate six open-weight and proprietary VLMs. The best achieves a family-balanced score of 61.9%, and across models we see a pattern of over-reaction: failed episodes involving prohibited interventions outnumber those involving missed required interventions by more than two to one, with models often shutting off the stove before investigating the alarm's cause. HomeLoop is an extensible foundation intended for use in research on robot-IoT coordination, long-term memory, proactive assistance, and privacy-safety trade-offs. HomeLoop will be released with 2,100 home layouts that can be automatically furnished in diverse configurations, five robot bodies and 20 networked device types.
★ Polarization calibration of fisheye cameras using skylight in the wild ICRA 2026
Skylight polarization provides directional cues for robotic navigation, but fisheye lens effects and analyzer imperfections can degrade polarization measurements across the field of view. We present a joint lens-analyzer calibration method that explicitly models incidence-dependent lens transmission together with analyzer gains and angular offsets in a unified eight-parameter formulation. Given known geometric calibration, rotations between camera frames can be estimated using visual features and cross-frame directional polarized light consistency can be used for polarization calibration. Therefore, our calibration approach requires only a short handheld outdoor sequence under natural skylight, without dedicated polarization calibration equipment or a predefined sky polarization model. On real-world outdoor sequences, the proposed method achieves a weighted cross-frame angle-of-linear-polarization (AoLP) mean absolute error below 0.5° across all incidence angles, with a 73.4% reduction in the 80°-90° incidence angle region, and generalizes to an independent sequence with an overall mean absolute error of 0.466° without refitting. Preliminary solar-azimuth experiments using a camera mounted on a quadruped robot further show improved alignment with the visible solar center, supporting the potential of calibrated AoLP measurements for relative-heading estimation and future polarization--inertial navigation. The calibration software tool will be released upon acceptance of this paper.
comment: 8 pages, 10 figures, submitted to ICRA 2026
★ FATHOM: World-Model-Guided Diffusion Action-Trajectory Selection for Sparse-Marker AUV Tracking
Reliable relative tracking is fundamental to cooperative autonomous underwater vehicles (AUVs), yet remains challenging under nonlinear vehicle dynamics and degraded underwater visual conditions. Active light markers improve robustness to appearance variation, but provide deliberately sparse observations that can become incomplete or corrupted by occlusion, reflections, and detection noise during dynamic motion. To address the resulting partial-observation control problem, this paper proposes FATHOM, a world-model-guided diffusion action-trajectory selection framework for sparse-marker AUV tracking. FATHOM first integrates structured marker observations, inertial measurements, action history, and temporal dynamics into a predictive tracking belief using a multimodal task-supervised recurrent state-space model. A diffusion action prior then generates multiple expert-like multi-thruster trajectories, whose future tracking consequences are predicted through the learned latent dynamics. Rather than directly executing a generated trajectory, FATHOM selects the most favorable candidate under an explicit receding-horizon objective and continually updates the decision from onboard observations. Extensive offline analyses verify the learned predictive belief, multi-step world-model prediction, diffusion trajectory generation, and predictive selection. More importantly, comprehensive real-AUV experiments, including strong pose-based comparisons and component ablations, demonstrate robust autonomous tracking across nominal and challenging motions with naturally occurring partial marker visibility, while maintaining real-time onboard operation without motion-capture feedback.
★ Localizing and Editing the Goal Representation of a Vision-Language-Action Model
Operators can state safety constraints for robots as prohibitions such as "do not put the bowl on the stove". We study whether OpenVLA follows such prohibitions and where it represents the goal it executes. The 10 tasks of LIBEROGoal share one scene, so we can label the goal that each rollout executes. We combine closed-loop rollouts with activation patching, linear probes, activation steering and runtime monitors. On LIBERO-Goal, negation had no detectable effect on which task OpenVLA executed. The model executed the forbidden task in 79% of negated rollouts and completed 78% of plain tasks. A goal probe gives similar read-outs under "not X" and "X". Activation patching shows that the first action depends mostly on the instruction tokens before layer 12 and on the last prompt token from layer 16 onward. A fixed steering vector at the instruction tokens at layer 8 redirected 12 of 18 rollouts. Fixed vectors and prefill patches at layers 12 to 20 redirected none of 158. From layer 16, only patches at every decoded action token redirected the robot (15 of 18 rollouts at layer 24). When the instruction changes during a rollout, the layer-32 goal read-out detects non-compliance better than failure detectors, and an external probe on image and instruction features performs comparably. These results support enforcing prohibitions outside the instruction and monitoring goal read-outs against the specification. Goal edits in OpenVLA should act before layer 12 or patch every action token.
★ Lodos: A Lightweight Simulator Designed for Contact-Rich Interactions with Detailed Meshes
In recent years, massively parallelized rigid-body simulation has played a central role in robotics research, enabling rapid progress in areas such as reinforcement learning, stochastic optimal control, and randomized smoothing-based trajectory optimization. Particularly in contact-rich settings, the scalability of such simulations in large part depends on well-optimized representations of collision geometry, often requiring a coarse approximation of the underlying surfaces using primitives such as spheres, capsules, cylinders, and boxes. Whenever the complexity of the surfaces or the precision demands of the application render such approximations insufficient, it is instead necessary to employ mesh-mesh collisions, which impose a significant reduction in throughput. This paper proposes a simulation framework designed from scratch to address this bottleneck, with novel routines for broadphase and narrowphase collision detection as well as the subsequent contact dynamics computation. As experimentally validated, the framework matches existing well-established simulators in terms of physical faithfulness, and can achieve approximately an order of magnitude higher throughput in the detailed mesh-mesh collision setting that it specializes in. The source code is available at https://github.com/martius-lab/lodos.
★ Lodos-v2: An Efficient Primal Formulation of the Exact Frictional Contact Problem
Lodos-v2 is a GPU-based simulation framework for massively parallel rigid-body dynamics with contact. It is specifically designed to achieve high throughput in scenes involving collisions among large numbers of detailed meshes. Its predecessor, Lodos-v1, had two main limitations. First, it solved the standard convex relaxation of cone-complementarity-based contact dynamics, thereby inheriting well-known physical artifacts, including lift-off and artificial compliance in the normal direction, as well as creep in the tangential direction. Second, its contact-group reparameterization scheme, which rescales spring-damper forces using a single factor per collision pair, could not be integrated with other constraints required by a full-fledged simulator, such as joint limits and welds. Lodos-v2 addresses both limitations through two main contributions. First, it introduces an efficient algorithm for solving the exact frictional contact problem by sequentially solving a cascade of convex relaxations in primal form within an outer proximal minimization loop. Second, it extends the contact-group reparameterization scheme of Lodos-v1 to the primal formulation of cone-complementarity-based contact dynamics, enabling its integration with the other constraints of a full-fledged simulator. Experiments demonstrate that these contributions result in throughput improvements over established rigid-body simulation frameworks. The source code is available at https://github.com/martius-lab/lodos.
★ What Limits mmWave Human-State Sensing for Robot Perception?
Assistive robots and search-and-rescue platforms require reliable human state sensing in smoke, darkness, and privacy-sensitive environments, where cameras and LiDAR may be unreliable Millimeter-wave radar offers an alternative that works without ambient light, penetrates smoke, and does not capture visual appearance. However, the compact arrays that fit robot size, power, and cost budgets have limited apertures, producing joint-localization errors on the order of decimeters at a few meters, large enough to compromise the safety margins and tracking accuracy a robot depends on. Existing pipelines reduce pose error through frame fusion, multiple viewpoints, return selection, or stronger estimators, but examine one technique at a time, without relating its effect to a shared measurement limit or separating gains from added information from gains from estimator completion. We address this gap with a framework that models the human and environment as one time-varying scattering field and states a Cramer-Rao type error floor for joint localization as a function of bandwidth, aperture, signal-to-noise ratio, fused frame count, phase coherence, and visibility of retained returns. We then use existing frame fusion, multi viewpoint, return selection, and pose estimation methods as matched probes of this floor rather than as competing pipelines. Under the evaluated conditions, experiments on three datasets and a live deployment show these strategies move joint localization error in the predicted directions, for example lowering mean per-joint position error from about 97 to 65 mm with a second viewpoint, while a stronger estimator mainly lowers overall error by completing joints the measurement cannot resolve. These results give the robotics community a principled basis for deciding whether to prioritize observation design or estimator capacity when deploying reliable mmWave human sensing.
★ STRIDE: Learning Humanoid Locomotion in Minutes with Acceleration-Based Rewards
Learning locomotion directly on a humanoid robot is a promising direction to avoid the sim-to-real gap. However, locomotion rewards in Reinforcement Learning (RL) typically track the base velocity, which cannot be measured onboard without an external motion capture system or well-calibrated state estimator. This is why we introduce \method{}, which learns humanoid locomotion from minutes of samples with a reward that only requires the onboard Inertial Measurement Unit (IMU). Instead of tracking the base velocity, STRIDE rewards the robot for following a commanded acceleration profile. To learn efficiently from this reward, we use the off-policy RL algorithm XQC and combine a high gain setting for the joint position controllers with left-right symmetry augmentation. In simulation, STRIDE learns to walk on the Unitree G1 and Booster T1 humanoids within 14 minutes of samples. Compared to learning without these two components, it reduces the falls during training by 81%, while without them the robot learns only in over an hour.
★ ActPrune: Action-Grounded Visual Token Pruning for Efficient Vision-Language-Action Models
Vision-language-action (VLA) models perform versatile manipulation tasks from language instructions, but their inference cost limits how fast a robot can respond in closed-loop control. Visual token pruning reduces this cost; however, prior methods estimate token redundancy from encoded visual embeddings or attention, only after the visual encoder has processed the entire observation. We propose \ours, a training-free token pruning method that moves the pruning decision ahead of the vision encoder by grounding it in a geometric prior derived from the robot's own action. The action chunk predicted at the previous control step is already available before the current observation is encoded, and it specifies the end-effector and camera motion between consecutive observations. The same action affects the two standard camera views asymmetrically: in the fixed third-person view, \ours retains patches along the projected end-effector trajectory together with semantic anchors, while in the wrist view, it prunes patches that remain unchanged after compensating for the action-induced camera motion. Extensive experiments on two simulation environments (LIBERO and VLABench) and four real-robot tasks demonstrate that \ours achieves up to 1.61 $\times$ speedup and 59.5\% FLOPs reduction with negligible degradation in success rates.
★ It Takes Two to Tangram: Multi-Agent Object Centric Reinforcement Learning for Assembly
Geometric assembly requires fine spatiotemporal coordination to place parts without causing collisions or deadlocks under tight clearances. To address this, we present an object-centric reinforcement learning framework that treats each part as an agent and learns a shared policy for continuous assembly control. A graph neural network represents the pieces, their target poses, and their spatial relationships, and predicts translational and angular velocities for each component. Per-agent proximal policy optimization with a centralized graph critic directly learns motion and temporal coordination within a single feedback policy. We study this formulation through planar Tangram assembly, where pieces must slide and rotate into a given configuration while remaining on the assembly surface. Procedural generation of target configurations and randomized initial arrangements support training and evaluation across diverse assembly problems. Our experiments isolate the roles of relational structure and multi-agent optimization, and compare learned control with sampling-based planning. We demonstrate concurrent assembly in simulation and transfer the policy to sequential execution on a real robot, as our object-centric formulation decouples learned assembly coordination from low-level physical control. Project page: https://tangrambench.org/
comment: This work has been submitted to the IEEE for possible publication
♻ ★ ObjView-Bench: Rethinking Difficulty and Deployment for Object-Centric View Planning NeurIPS 2026
Object-centric view planning is a core component of active geometric 3D reconstruction in robotics, yet existing evaluations often conflate object complexity, planning difficulty, budget assumptions, and physical reachability constraints. As a result, conclusions drawn from idealized view-planning evaluations may not reliably predict performance under realistic reconstruction settings. We introduce ObjView-Bench, an evaluation framework for rethinking difficulty and deployment in object-centric view planning. First, we disentangle three quantities underlying view-planning evaluation: omnidirectional self-occlusion as an object-side attribute, observation saturation difficulty, and protocol-dependent planning difficulty defined through a set-cover formulation. This separation supports controlled dataset construction, analysis of slow-saturation objects, and a case study showing that planning difficulty-aware sampling can improve learned view planners. Second, we design deployment-oriented evaluation protocols that reveal how budget regimes and reachable-view constraints alter method behavior. Across classical, learned, and hybrid planners, ObjView-Bench shows that difficulty, budget, and reachability constraints substantially change method rankings and failure modes. Project page: https://objview-bench.github.io/
comment: Accepted to NeurIPS 2026
♻ ★ Mechanisms of Misgeneralization in Physical Sequence Modeling NeurIPS 2026
Generative sequence models are often trained to plan motion in physical domains, from robotics to mechanical simulations. When constructing a dataset to train such a model, engineers may curate demonstrations to specify how trajectories should be distributed over a physical quantity like travel distance or mechanical energy. For example, a roboticist building a maze navigation agent might choose demonstrations whose travel distances cover a fixed range uniformly, hoping to constrain the agent's expected power usage. We find that standard deep learning can violate this intent: each generated trajectory can seem plausible on its own, but the aggregate distribution over the physical quantity is wrong. We call this failure physical misgeneralization. In this work, we identify and formalize the problem, develop and validate an account of its mechanism, and use that account to compare and design mitigations. Using controlled synthetic tasks, we show that physical misgeneralization arises when local errors typical of the model class propagate through the physical measurement to shift the recovered distribution. We estimate these errors with a data deviation kernel, and we use it to predict which physical quantity values gain or lose mass under real diffusion sequence models trained on both toy and applied trajectory generation tasks. Finally, our mechanistic interpretation helps identify which mitigations are structurally promising. We first compare interventions that modify the dataset and generative interface, then show that the mechanistic understanding we develop throughout this paper suggests a novel way to construct data representations informed by the kernel. We show that our proposed mitigation, which transforms the coordinate system under which the generator learns and samples, effectively reduces quantity drift.
comment: 40th Conference on Neural Information Processing Systems (NeurIPS 2026). kentonishi.com/physical-misgeneralization
♻ ★ Beyond Benchmarks: Continuous Edge Inference for Traffic Sign Recognition on Embedded Vehicular Hardware
Traffic sign recognition (TSR) systems deployed on embedded vehicular hardware face deployment effects that image-based benchmark evaluation does not capture, including frame-to-frame prediction instability in streaming video and load-dependent throughput under sustained operation. We present Edge-TSR, a continuous edge inference system for TSR on the NVIDIA Jetson Orin Nano that combines sparse detection, multi-object tracking, region-level fine-grained classification, and a lightweight track-aware temporal stabilization mechanism adding only 0.07~ms per sampled frame. Evaluating on the source videos of the INTSD benchmark, we find that the state-of-the-art TSR models lose 20-30 percentage points of mAP@50 when moving from image-based to streaming evaluation. Edge-TSR achieves the best streaming classification performance among all evaluated baselines, and temporal stabilization improves accuracy by 10.58 percentage points over per-frame inference. We characterize latency, throughput, and thermal behavior across sampling intervals and four driving conditions, showing that throughput is load-proportional (20 ms per classified region) and that every-frame inference is limited by throughput rather than thermal throttling. A 55-minute, 26 km vehicular deployment demonstrates sustained online operation at 16.18 FPS within safe thermal limits without cloud offload. We also release the annotated 50,732-frame streaming evaluation dataset and full implementation.
♻ ★ ManiTaskGen: A Comprehensive Task Generator for Benchmarking and Improving Vision-Language Agents on Embodied Decision-Making
Building embodied agents capable of accomplishing arbitrary tasks is a core objective towards achieving embodied artificial general intelligence (E-AGI). While recent work has advanced such general robot policies, their training and evaluation are often limited to tasks within specific scenes, involving restricted instructions and scenarios. Existing benchmarks also typically rely on manual annotation of limited tasks in a few scenes. We argue that exploring the full spectrum of feasible tasks within any given scene is crucial, as they provide both extensive benchmarks for evaluation and valuable resources for agent improvement. Towards this end, we introduce ManiTaskGen, a novel system that automatically generates comprehensive, diverse, feasible mobile manipulation tasks for any given scene. The generated tasks encompass both process-based, specific instructions (e.g., "move object from X to Y") and outcome-based, abstract instructions (e.g., "clear the table"). We apply ManiTaskGen to both simulated and real-world scenes, demonstrating the validity and diversity of the generated tasks. We then leverage these tasks to automatically construct benchmarks, evaluating high-level embodied decision-making through a grounded skill interface for agents built upon existing vision-language models (VLMs). Furthermore, we propose a simple yet effective method that utilizes ManiTaskGen tasks to enhance embodied decision-making. Overall, this work presents a cross-scene task generation framework facilitating both benchmarking and improvement of embodied decision-making agents. Our code is available at https://manitaskgen.github.io.
comment: Project Website: https://manitaskgen.github.io/
♻ ★ SGM-SLAM: Scene Graph Matching for Data-Efficient Distributed SLAM
We introduce a data-efficient distributed Simultaneous Localization and Mapping (SLAM) framework for multi-robot operation in GPS-denied and communication-constrained environments. The framework is designed for teams of robots equipped with LiDAR, cameras, and inertial sensors, where robots may operate from independently initialized local coordinate frames and share only limited spatial overlap. Our approach constructs a scene graph by using fused RGB-LiDAR point clouds to generate both a semantically segmented point cloud layer, and a layer of discrete bounded objects, to accompany estimated robot trajectories. Scene graph matching is performed collaboratively through exchanging and matching object data with neighboring robots. To maximize communication efficiency, we utilize a multi-step data exchange and optimization process. We demonstrate the effectiveness and communication efficiency of the proposed approach using real-world datasets containing limited trajectory overlap, viewpoint variation, repeated structures, and degraded visual conditions.
♻ ★ Adaptive Collision Sensitivity for Efficient and Safe Human-Robot Collaboration
What is considered safe for a robot operator during physical human-robot collaboration (HRC) is specified in corresponding HRC standards (e.g., ISO 10218-2:2025). The regime that allows collisions between the moving robot and the operator, called Power and Force Limiting (PFL), restricts the permissible contact forces. Using the same fixed contact thresholds on the entire robot surface results unnecessary productivity losses, as the robot needs to stop even when impact forces are within limits. Here we present a framework that decides whether the robot should interrupt motion based on estimated collision force computed individually for different parts of the robot body and dynamically on the fly, based on the Effective Mass (EM) of each robot link. We performed experiments on a simulated and real collaborative robot arm (UR10e) with sensitive skin (AIRSKIN) for collision detection and isolation. To demonstrate the generality of our method, we added experiments on simulated KUKA LBR iiwa robot, where collision sensing draws on joint torque sensing. On a mock pick-and-place scenario with both transient and quasi-static collisions, we show an increase in productivity over 45% from using the standard approach. The method is applicable to any robot for which the EF can be calculated.
comment: Accepted to Advanced Intelligent Systems
♻ ★ The effect of limb kinematics on the speed of a legged robot on granular media
Achieving effective locomotion on diverse terrestrial substrates can require subtle changes of limb kinematics. Biologically inspired legged robots (physical models of organisms) have shown impressive mobility on hard ground but suffer performance loss on unconsolidated granular materials like sand. Because comprehensive limb-ground interaction models are lacking, optimal gaits on complex yielding terrain have been determined empirically. To develop predictive models for legged devices and to provide hypotheses for biological locomotors, we systematically study the performance of SandBot, a small legged robot, on granular media as a function of gait parameters. High performance occurs only in a small region of parameter space. A previously introduced kinematic model of the robot combined with a new anisotropic granular penetration force law predicts the speed. Performance on granular media is maximized when gait parameters minimize body acceleration and limb interference, and utilize solidification features of granular media.
♻ ★ Learning-augmented robotic automation for real-world manufacturing
Industrial robots are widely used in manufacturing, yet most manipulation still depends on fixed waypoint scripts that are brittle to environmental changes. Learning-based control offers a more adaptive alternative, but it remains unclear whether such methods, still mostly confined to laboratory demonstrations, can sustain hours of reliable operation, deliver consistent quality, and behave safely around people on a live production line. Here we present Learning-Augmented Robotic Automation, a hybrid system that integrates learned task controllers and a neural 3D safety monitor into conventional industrial workflows. It retains the core industrial automation backbone while introducing learning only where adaptability is needed. We deployed the system on an electric-motor production line to automate deformable cable insertion and soldering under real manufacturing constraints, a step previously performed manually by human workers. With less than 20 minutes of real-world data per task, the system operated continuously for 5 hours 10 minutes, producing 108 motors without physical fencing and achieving a 99.4% pass rate on product-level quality-control tests. It achieved near-human takt time with reduced cycle-time variability while consistently producing high-quality solder joints comparable to those of experienced operators. These results establish a practical pathway for extending industrial automation with learning-based methods.
♻ ★ Spectral Alignment in Forward-Backward Representations via Temporal Abstraction NeurIPS 2026
Forward-backward (FB) representations provide a powerful framework for learning the successor representation (SR) in continuous spaces by enforcing a low-rank factorization. However, a fundamental spectral mismatch often exists between the high-rank transition dynamics of continuous environments and the low-rank bottleneck of the FB architecture, making accurate low-rank representation learning difficult. In this work, we analyze temporal abstraction as a mechanism to mitigate this mismatch. By characterizing the spectral properties of the transition operator, we show that temporal abstraction acts analogously to a low-pass filter that suppresses high-frequency spectral components. This suppression reduces the effective rank of the induced SR while preserving a formal bound on the resulting value function error. Empirically, we show that this alignment is a key factor for stable FB learning, particularly at high discount factors where bootstrapping becomes error-prone. Our results identify temporal abstraction as a principled mechanism for shaping the spectral structure of the underlying MDP and enabling effective long-horizon representations in continuous control.
comment: Accepted at NeurIPS 2026 (camera-ready). Project page and code: https://erfi.github.io/spectral_alignment/
♻ ★ Accuracy potential of visual localization exploiting high-end street-level imagery
Many applications, such as surveying, autonomous navigation, robotics and augmented reality, require accurate pose information with respect to a reference frame. Visual localization can complement GNSS positioning, whose availability and accuracy are often limited. However, the accuracy potential of visual localization has not yet been systematically investigated against survey-grade requirements, and no publicly available large-scale outdoor dataset provides ground-truth poses accurate enough for such an investigation. We address both gaps. First, we present the FHNW Muttenz dataset. It covers a contiguous 10 km street network, mapped in two mobile mapping campaigns about 1.5 years apart, and comprises high-resolution reference imagery as well as query sequences from four cameras in five scenes. The two campaigns are co-registered to each other with an accuracy of 5 mm. Co-registration of the query images to the reference imagery yields ground-truth poses accurate to 6-9 mm in position and well below 0.1° in orientation. All images are georeferenced with an accuracy of about 2.5 cm. Second, we use this dataset to systematically evaluate sparse feature-based visual localization with a modular pipeline built from established components, using the high-resolution reference imagery directly as the scene representation. For daytime imagery and pre-calibrated cameras, the median pose errors with respect to the reference imagery are 1-5 cm in translation and 0.03-0.1° in rotation. Given an accurately georeferenced scene representation, visual localization not only approaches the accuracy of survey-grade GNSS positioning but also provides full orientation information. The dataset is publicly available at https://fhnw-muttenz-vl-dataset.github.io/
comment: 28 pages, 6 figures, 22 tables. v2: substantially revised and restructured; experiments recomputed after a joint re-estimation of the query camera poses and calibration and a correction to the feature extraction. Dataset: https://fhnw-muttenz-vl-dataset.github.io/
♻ ★ RADAR Perception for Dynamic Obstacle Avoidance onboard small-scale Quadrotor UAVs
Fast dynamic obstacle avoidance (DOA) on uncrewed aerial vehicles (UAVs) demands not only low-latency control and actuation but also reliable, long-range perception to detect obstacles and estimate their motion in time to maneuver. We present, to the best of our knowledge, the first mmWave RADAR-based perception-and-control system for fast onboard DOA. We derive and analyze latency and spatial bounds that relate sensing range, relative speed, and control delay, yielding sufficient conditions for successful avoidance. We further extend prior latency analyses to include the decision latency of the tracker and trigger to evaluate them per trial. Our computationally efficient system combines a lightweight interacting multiple models (IMM) tracker with a control-barrier functions (CBF) controller that directly outputs evasive accelerations. In 300 throw experiments spanning three object sizes and varying visibility (light and dark), it achieves mean position errors of at most 0.15 m, 0.93m, and 0.87m in the x, y, and z directions, with a similar spread in 90 experiments in smoke. An onboard implementation on a Raspberry Pi 4B demonstrates real-time feasibility with an end-to-end processing latency of approximately 14ms per RADAR frame and a system latency of 75ms from first detection to the avoidance command. The UAV successfully avoided obstacles in all 20 tethered indoor flights and all 39 untethered outdoor avoidance maneuvers across three ball sizes, during both hover and forward flights at up to 5m/s with measured closing speeds up to 14.6m/s.
comment: Preprint. This work has been submitted for possible publication and is currently under review
♻ ★ Revisiting Temporal Regularization for Smooth Control in Deep Reinforcement Learning ICRA 2027
Deep Reinforcement Learning policies can produce nonsmooth action oscillations that hinder deployment on physical robots. Existing architectural and penalty-based approaches seek spatial smoothness by directly reducing sensitivity to changes in state inputs, but their broad constraints can degrade task performance as stronger smoothing is pursued. Temporal regularization instead constrains action differences along observed transitions, but has been considered unable to provide the spatial smoothness needed under observation noise. We revisit this assumption by proving that the temporal penalty bounds the expected action differences between current states sharing a next state, revealing a spatial effect that empirically extends to spatial smoothness. Building on this finding, we propose Conditioning for Action using only Temporal Smoothness (CATS), which combines a temporal penalty with linear ramp-up. We highlight temporal regularization's ability to provide spatial smoothness while better preserving task performance than explicit spatial regularization. Through linear ramp-up, CATS allows the policy to learn rewarding behavior before progressively smoothing its actions, improving return preservation and both temporal and spatial smoothness. Experiments in both simulation and the real world show that CATS substantially reduces action oscillation without degrading task performance, with little computational overhead.
comment: Submitted to ICRA 2027. Project page: https://dmach1ne.github.io/CATS/
♻ ★ Evolving Drone Morphologies for Agile Flight
The design of multicopter drones has remained almost the same since their inception. While conventional designs work well in many cases, they may not be optimal in specific tasks. This paper revisits rotary drone design by exploring which body morphologies are best suited for different objectives and constraints. An evolutionary algorithm is used to produce drone morphologies that performs better than standard configurations at thrust-to-weight ratio, maneuverability and size. To generate a range of drones with performance trade-offs between them, the non-dominated sorting genetic algorithm II is used. The evolved drone bodies outperform a standard multicopters in at least one of the three objectives. The Pareto front extrema show improvement of 489.2%, 23% and 4.8% in maneuverability, thrust-to-weight ratio and size, respectively, when compared to a standard quadcopter. The improvement in maneuverability is attributed to the tilt angles of the motors, while the increase in thrust-to-weight ratio is due to the higher number of propellers. Experiments show that the performance improvement transfers to a real world drone racing task, where the evolved drone outperforms the standard drone in terms of number of gates passed,flight speed and lap times.
comment: 29 pages, 12 figures
♻ ★ Improving Imitation Learning Efficiency for Manipulation through Geometric Prior Pretraining
Applying an imitation learning policy to a new manipulation task usually requires collecting new demonstrations and retraining the model, which makes sample efficiency a practical concern. Pretraining on large-scale robot datasets is effective in this respect, but such datasets are costly to collect and train on, while data augmentation techniques typically require a new round of data generation and retraining for each task. A complementary question is what useful prior can be provided to a policy at negligible cost before any task-specific data are collected. In this study, we construct a geometric visual pretraining dataset in which each scene contains only a plane, an object, and a hand, and trajectories are generated automatically. The scenes contain neither textures nor backgrounds; pretraining primarily exposes the policy to the geometric relationship between the hand and the object. Furthermore, representing the hand as a cube avoids tailoring the dataset to a specific robot morphology. We evaluate this geometric prior using ACT on three simulated robots across five manipulation tasks each, as well as on three real-world robot tasks. Across many of these robot--task combinations, fine-tuning from the geometric prior achieves higher success rates in the early stages of training than training from scratch while using only a small number of task demonstrations. These results suggest that even highly simplified geometric scenes can provide a useful initialization that transfers across robots and to real-world tasks when task data are limited.
comment: 8 pages, 6 figures, 2 tables
♻ ★ NoContactNoWorries: Estimating Contact through Vision and Proprioception for In-Hand Dexterous Manipulation IROS
Perceiving physical contact is fundamental to dexterous manipulation. While robots often rely on dedicated hardware tactile sensors, humans exhibit a remarkable ability to infer contact by integrating visual information with an innate sense of their body's pose and movement. Inspired by this embodied perceptual skill, we investigate whether a robot can learn to infer contact from vision, an approach that also offers a scalable alternative to tactile hardware specifically for binary contact estimation, which faces practical challenges in cost, fragility, and integration. We present NoContactNoWorries, a transformer-based multimodal framework that fuses RGB-D vision with the robot's proprioception to infer binary contact states as a pseudo-tactile signal for hand-object interactions. We validate by training a single contact prediction model on multiple objects and show that the inferred contact signal supports downstream reinforcement learning agents for in-hand object reorientation, generalizing to novel objects. Experiments in both simulation and on a real-world robot validate our approach, highlighting the feasibility of inferring contact from vision and proprioception. Project Page: https://soham2560.github.io/no-contact-no-worries/
comment: Accepted to IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS) 2026
♻ ★ Hold-Out Self-Validation Cannot Certify the Accuracy of Photogrammetric Reconstructions
Photogrammetric reconstructions used for inspection are often delivered without an independent accuracy check. Hold-out self-validation is a possible substitute. Images are withheld, their orientations are re-estimated from the rest of the block, and agreement is taken as evidence of quality. We test it on ten GNSS-referenced aerial captures from eight sites, on ETH3D and EuRoC, and on 30 IMC 2025 scenes. The standard form re-localises each withheld image only against points seen by at least two retained images. Its score saturates. It stayed at 1.00 in all 40 thinned reconstructions while camera-position error varied by a factor of up to 5.9 within one capture. It also scored 1.00 on a corrupted model with only one of 226 cameras within 5 m. Reprojection error ranked some of the worst models best. A few lost cameras can raise the least-squares RMSE to the size of the block, so we also evaluate with a robust alignment. Taking the correspondences from the verified matches lowered the score below 0.95 in 14 of 18 corrupted runs, against 4 for the standard form and 10 when its failed resections count as errors, with all seven nominal runs at 1.00. Mapping the block again without the withheld images also responds, but mapping the same corrupted data twice without withholding gave a similar disagreement, so the signal is mainly that a corrupted reconstruction is not reproducible. Neither test can see an error that every reconstruction repeats, and neither can replace check points.
comment: 12 pages, 2 figures, 6 tables. v5: rewritten and shortened for journal submission; ten public GNSS-referenced aerial captures instead of five; robust alignment and re-matched, re-mapped and repeat-mapping hold-out variants added; conclusions revised accordingly. Code and result tables: doi:10.5281/zenodo.21737747
♻ ★ Modular magnetic soft robotic catheter enables visualized intervention in the pancreatobiliary tree
Diagnosis and therapy in the pancreatobiliary tree are limited by access. Guidewires reach deep into the ducts but are blind, whereas cholangioscopes provide vision but are too large to readily enter the pancreatic body and tail, the cystic duct, or the gallbladder. At the millimeter scale of these ducts, every function added to a steerable catheter competes with steering for cross-section and adds bending stiffness. We present a 1.47-mm modular magnetic soft robotic catheter in which each function is built into the magnetic body, onto its surface, or onto the footprint of the camera, and we measured what each function costs in bending under bedside fields. Imaging and fiber Bragg grating (FBG) shape sensing accounted for most of this cost, whereas a compliant gripper, magnetic anchoring balloons, and a micrometer-thick ultrasound-triggered release shell together reduced the tip angle at 8 mT only from 70° to 64°. The anchoring balloon increased the gripping force about threefold without stiffening the catheter during navigation. In live pigs, the robotic catheter performed operator-initiated task-autonomous cannulation of the native papilla without sphincterotomy and was teleoperated under onboard vision 75 mm into the pancreatic duct and, without a guidewire, through the cystic duct into the gallbladder, whose lumen it imaged. Anchoring, grasping, and payload release were each demonstrated in vivo. Adding each function at a measured cost to steering allows a soft robot small enough for these ducts to intervene as well as navigate.
comment: 31 pages, 7 figures, 1 table, 7 supplementary figures
♻ ★ VLD: Visual Learned Distance as an Interface for Reinforcement Learning Navigation
Training end-to-end policies from image data to directly predict navigation actions for robotic systems has proven inherently difficult. Existing approaches often suffer from either the sim-to-real gap during policy transfer or limited action-labeled training data. To address this problem, we introduce Visual Learned Distance (VLD), a scalable framework for image-goal navigation that decouples perception learning from policy learning. We first train a self-supervised distance-to-goal predictor on about 3,000 hours of internet-scale and simulated video, providing a distance signal that a reinforcement learning (RL) policy can minimize. The RL policy is trained separately in simulation using privileged geometric distances perturbed by GeoNoise, a learned noise model that reproduces the viewpoint- and overlap-dependent errors of the trained distance predictor. At deployment, the policy consumes VLD predictions zero-shot, without retraining, inheriting ``where to go'' from large-scale visual training while retaining the robust navigation behaviors learned in simulation. This error modeling is key to the transfer: with VLD inputs, GeoNoise-trained policies reach $58.7\%$ success on HM3D and $76.2\%$ on Gibson, whereas policies trained with generic temporally correlated noise collapse to $4.9\%$ and $11.5\%$. On a physical robot, the VLD-driven policy succeeds in $88\%$ of trials, compared with $60\%$ for a strong end-to-end image-goal baseline that outperforms it in simulation. Our decoupled design thus provides an alternative and scalable path toward reliable navigation policies that transfer from simulation to the real world.
♻ ★ WorldBridge: Bridging Video and Action for Task Generalization in World Action Models
Achieving the broad task generalization of embodied models in physical environments remains a central challenge in robot learning. We study this challenge in World Action Models (WAMs) through two complementary capabilities: video generalization, which predicts the visual evolution of new tasks, and action generalization, which translates these predictions into executable behavior through a video-to-action bridge. By varying video and action supervision for tasks outside an initial training set, we examine their respective contributions to task transfer. On RoboTwin 2.0, using only in-distribution video-action supervision yields a 25.07% success rate on out-of-distribution tasks. Introducing videos of these tasks without their action labels raises success to 55.90%, while adding just one action-labeled demonstration per task further improves it to 70.70%. These findings suggest that broader video coverage enables task transfer by leveraging existing inverse-dynamics capabilities, while even sparse action supervision further strengthens those capabilities. Guided by this perspective, we introduce WorldBridge, a WAM with separate video and action experts connected through structured attention and trained with joint flow matching. WorldBridge incorporates visual memory at two temporal scales and action forcing, alongside a unified data processing pipeline that standardizes and filters heterogeneous robot trajectories and egocentric hand-derived supervision. Across multiple manipulation benchmarks, WorldBridge achieves state-of-the-art performance among the compared methods, including 96.23% on RoboTwin 2.0 full data setting and 80.9% on clean to random setting, 82.8% on RoboCasa GR1 TableTop, and 57.6% on DOMINO Level-1. Our study highlights the complementary roles of video prediction and action grounding in developing robots that generalize across tasks.
comment: Authors are listed in alphabetical order. Project page: https://worldbridge-jd.github.io/
♻ ★ A Straight Line Is Not a Camera Pose: Auditing Short-Clip Pose Benchmarks
The absolute trajectory error (ATE) is the standard measure of camera-motion accuracy, from SLAM to feed-forward 3D reconstruction. On the common Sintel protocol, an image-free constant-velocity line has a lower mean ATE than 10 of 11 recent models re-run with official code and than 85 of 110 published methods. On Lightspeed it beats 5 of 9 models and 10 of 11 published methods, while on TUM, ScanNet, DyCheck and ADT every model beats it. Two properties of ATE's alignment explain this. First, it maps any moving constant-velocity prediction onto the line that best fits the true path, so the line scores the path's regression residual. A residual signal-to-noise ratio (SNR) criterion tells when moving a prediction towards its constant-velocity fit lowers ATE. With the alignment's first-order refit, it accounts for the outcome of all but one of 4010 model-clip evaluations, including all 504 on three benchmarks registered before any model ran. The true path's straightness flags Sintel and Lightspeed. Second, the alignment ignores where cameras look: rotating a trajectory by 30 degrees against its orientations leaves ATE unchanged yet worsens the static-point reprojection of every model on Sintel, TUM and ScanNet, and the line reprojects worst on Sintel by geometric mean. Fitting the alignment's rotation to the predicted orientations gives pose-anchored ATE (PA-ATE), a one-function change at about ATE's cost (37 vs. 34 $μ$s per clip, and 61 vs. 66 $μ$s with the rotation error). It is never below ATE and, as we prove, equals the translational relative pose error over all frame pairs divided by $\sqrt{2}$ when the orientations agree. It ranks every image-free reference below every model on Sintel, follows the reprojection error where ATE does not, and largely preserves the models' order. Its rotation error also exposes Lightspeed ground-truth orientations that the images contradict.
comment: 39 pages, 9 figures
♻ ★ RiskFlow: Fast and Faithful Safety-Critical Traffic Scenario Generation
Safety-critical traffic scenario generation is essential for evaluating autonomous driving systems under rare but high-risk interactions. Existing diffusion-based methods offer strong controllability in closed-loop generation, but their iterative denoising process is computationally expensive and may accumulate sampling and guidance errors over long rollouts, causing unrealistic motion artifacts such as abnormal acceleration and off-road behavior. To address these issues, we propose RiskFlow, a closed-loop safety-critical traffic generation framework that reformulates future behavior generation as one-step flow transport in the action space during training. At test time, RiskFlow generates safety-critical interactions through a selective guidance procedure in which time-to-collision-based critical-agent ranking selects high-risk agents and an adversarial interaction objective directly refines their output actions. Vehicle-dynamics rollout and map-aware regularization then translate guided actions into feasible trajectories while discouraging off-road behavior. By intervening on selected agents in the output space, RiskFlow avoids repeated corrections of intermediate denoising states, thereby mitigating accumulated guidance errors and accelerating closed-loop scenario generation. Experiments on the nuScenes dataset under closed-loop evaluation show that RiskFlow achieves a strong adversariality-realism trade-off across multi-agent and long-horizon settings. Compared with representative baselines, RiskFlow consistently improves realism while maintaining competitive safety-critical generation capability, and substantially reduces inference time for evaluation.
♻ ★ STAR: Steering Action Representations with Proprioception for VLA Inference
Vision-language-action models (VLAs) map language instructions and visual observations to robot actions, showing strong capabilities across robotic manipulation tasks. However, in our probing experiment, we find that action representations near the action output do not fully preserve task-relevant visual grounding. Motivated by this late-layer grounding gap, we introduce STAR, for STeering Action Representations, a training-free inference method for pre-trained VLAs. STAR applies an exponential moving average over late-layer residual updates, allowing later action representations to inherit object-aware update directions from earlier layers without model updates or architectural changes. It further uses recent proprioceptive states to adapt the mixing strength, using conservative mixing when execution dynamics change rapidly. Experiments on robot manipulation benchmarks show that STAR improves late-layer object awareness and increases average task success rates by 4.0 percentage points. Our code is available at https://github.com/PKU-VaLuE-Lab/STAR.
♻ ★ Self-Distilling Future Video Representations for World Action Models
World Action Models (WAMs) have emerged as a promising approach to learning generalist robot policies using rich spatiotemporal priors from pre-trained video diffusion models. A recent line of WAMs improves inference efficiency by conditioning a separate action decoder on intermediate future video representations extracted from a single denoising step, avoiding costly iterative denoising required for video generation. This design makes action prediction depend directly on the quality of these future representations. In this paper, we introduce Future REpresentation learning via self-Distillation with temporal context (FRED), a framework that trains WAMs to improve the quality of future video representations from noise in a single denoising step. Specifically, FRED uses self-distillation to predict future representations from noise, using representations extracted from ground-truth future observations as targets. We further condition the video model on historical observations, providing temporal context for predicting future representations better. Across diverse real-world manipulation scenarios, FRED consistently outperforms competing WAMs. It achieves particularly large gains on long-horizon and out-of-domain tasks under zero-shot environment generalization, and substantially improves human-to-robot skill transfer, reaching 50.0% task success on tasks absent from robot training data and demonstrated only in human videos, compared with 19.4% for the best competing WAM.
comment: 27 pages; Project page: https://huiwon-jang.github.io/FRED/
♻ ★ VOMMI: Collecting and Leveraging Portable Demonstrations for Mobile Manipulation
Portable mobile-manipulation demonstrations can help alleviate data scarcity for embodied intelligence, but obtaining reliable, low-cost, and robot-free motion supervision from RGB observations remains challenging. Existing approaches often rely on teleoperation or specialized devices equipped with additional sensing hardware, while directly using estimated visual odometry (VO) trajectories can introduce inconsistencies due to accumulated drift and imperfect motion supervision. We present the Visual-Odometry-Conditioned Mobile Manipulation Interface (VOMMI), a portable demonstration collection and learning framework that connects portable RGB demonstrations to vision-language-action (VLA) post-training through offline trajectory reconstruction and online visual-motion conditioning. VOMMI synchronizes body and hand views to capture navigation context and local object interactions without requiring human-robot kinematic correspondence calibration. R2-VO refines offline demonstration trajectories using sparse geometric anchors and produces causal local-motion tokens over multiple prediction horizons for online policy conditioning. An action-group residual adapter incorporates these tokens only into the base branch. Experiments use a 500-trajectory portable for each task, with 75 trajectories held out for RGB-VO evaluation, and 200 robot demonstrations as references. Our policy, post-trained only on portable demonstrations, achieves 18.2% lower base-velocity error than a policy trained with robot-collected demonstrations, while maintaining comparable end-effector translation accuracy. Offline reconstruction reduces absolute trajectory errors for the body and hand streams by 24.6% on average relative to the best evaluated baseline for each stream. The complete system improves the mean success rate by 8.3 percentage points over OpenPI 0.5 across three real-robot tasks.
comment: 9 pages, 6 figures
♻ ★ Multi-bounce Drum Roll with Optimized Active Tricks to Leverage Soft Embodiment
This paper presents a soft robotic drummer for accurate and efficient drum rolls. High-frequency drum rolls require the "multi-bounce technique," where a drumstick bounces multiple times with a single stroke. In robotic reproduction of this technique, the body's elasticity is key, while the fine motion during the stroke is also crucial for maximizing the potential of that elasticity. Therefore, we design two tricks: i) Tap-Pull (TP) trick to increase the number of rebounds by adding a pulling motion after impact; and ii) Micro-Pulse (MP) trick to keep the drumming volume by injecting small oscillations during the stroke. Due to the nonlinear complexity of soft embodiment, both tricks are efficiently tuned using Bayesian optimization in a data-driven manner for accomplishing the respective objectives quantified. We evaluated the optimized behaviors with soft and rigid end-effectors. As a result, the soft TP achieved the highest bounce count (12.25 per stroke) with uniform intervals. The soft MP suppressed the volume decay, yielding 6.8-times higher acoustic efficiency compared to the rigid MP. These results indicate that the proposed tricks with the combination of elasticity and optimization can make robots play excellent drum rolls.
comment: 7 pages, 6 figures. Published in the 2026 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM 2026). Supplementary video: https://youtu.be/TBzHv8jMz4Y
♻ ★ MosaicIMU: Composing Carrier Experts for Generalizable Neural Inertial Odometry
Robust inertial odometry is essential for various carriers when external sensing is unreliable. Learning-based methods reduce integration drift by capturing local motion priors, but these methods often remain tied to a particular carrier, limiting generalization across heterogeneous platforms. We present MosaicIMU, a carrier-conditioned Mixture-of-Experts (MoE) pretraining-and-adaptation framework for generalizable neural inertial odometry. MosaicIMU uses a prototype-based router to compose carrier-specific expert features, decodes local velocity and uncertainty constraints, and integrates them with a history-aware EKF. For unseen domain adaptation, it freezes the pretrained base model and learns a new lightweight expert residual branch. For edge-deployment, it further reuses the router to select informative online samples for efficient incremental updates. Experiments show that MosaicIMU consistently outperforms learning-based baselines, reducing average ATE and RTE-10s by 40% and 34%, respectively. These results highlight that MosaicIMU provides a scalable pretraining-to-deployment paradigm for generalizable and adaptive neural inertial odometry.
comment: Accepted to the Conference on Robot Learning (CoRL 2026) as a Spotlight paper
♻ ★ RigidFormer: Learning Rigid Dynamics using Transformers NeurIPS 2026
Learning-based simulation of multi-object rigid-body dynamics remains difficult because contact is discontinuous and errors compound over long horizons. Most existing methods remain tied to mesh connectivity and vertex-level message passing, which limits their applicability to mesh-free inputs such as point clouds and leads to high computational cost. Efficiently modeling high-fidelity rigid-body dynamics from mesh-free representations therefore remains challenging. We introduce RigidFormer, an object-centric Transformer-based model that learns mesh-free rigid-body dynamics with controllable integration step sizes. RigidFormer reasons at the object level and advances each object through compact anchors; Anchor-Vertex Pooling enriches these anchors with local vertex features, retaining contact-relevant geometry without dense vertex-level interaction. We propose Anchor-based RoPE to make attention aware of the objects' spatial layout while respecting the unordered nature of objects and anchors: object-token processing is permutation-equivariant, and the mean-pooled anchor descriptor is invariant to anchor reindexing. RigidFormer further enforces rigidity by projecting updates onto the rigid-body manifold using differentiable Kabsch alignment. On standard benchmarks, RigidFormer outperforms or matches mesh-based baselines using point inputs, runs faster, generalizes to unseen point resolutions and across datasets, and scales to 200+ objects; we also show a preliminary extension to command-conditioned articulated bodies by treating body parts as interacting object-level components. Project page: https://frank-zy-dou.github.io/projects/RigidFormer/
comment: NeurIPS 2026. Project page: https://frank-zy-dou.github.io/projects/RigidFormer/
♻ ★ MATE: Asynchronous Multi-Rate Diffusion Policy Composition
Diffusion policies have shown strong potential for robotic imitation learning, and recent extensions incorporate additional modalities to improve manipulation performance. However, these modalities often differ not only in information content but also in sensing rates and inference latencies. Existing multimodal diffusion policies typically rely on synchronous fusion or manually designed multi-frequency architectures, which either slow down high-frequency feedback or limit extensibility to new modality combinations. We propose LAG-Fusion, a latency-aware guidance fusion framework for asynchronous multimodal diffusion policy composition. LAG-Fusion allows modality-specific policies to operate at their native inference rates and contribute denoising guidance whenever available. To make asynchronous composition consistent, we derive a reference-frame rebasing rule for diffusion variables under relative action representations, enabling delayed guidance to be aligned before fusion. We instantiate LAG-Fusion in contact-rich manipulation by composing a low-frequency vision policy with a high-frequency force policy. Experiments under heterogeneous modality latencies show that LAG-Fusion improves policy responsiveness and task performance over synchronous fusion and specially designed force-aware baselines.
♻ ★ DexHOT: Learning Hand-Object Configuration Transfer for Dexterous Manipulation
Dexterous manipulation involves not only moving objects, but also arranging the hand around them to maintain useful contacts and enable subsequent actions. Yet many learning-based systems specify goals primarily through object poses or trajectories, leaving hand-object configurations to emerge implicitly. We introduce DexHOT, which formulates dexterous manipulation as hand-object configuration transfer: moving between feasible configurations that jointly specify hand articulation and object pose relative to the wrist. DexHOT learns a reusable relation-conditioned transfer primitive from synthetic feasible configuration pairs, without human reference motions, and composes long-horizon behaviors through closed-loop tracking of configuration subgoals. On DexHOTBench with 120 objects, a single policy achieves 80.5% transfer success on seen objects and 69.4% on unseen objects; on a real dexterous hand, it achieves 61.5% success over 200 real-robot transfers and 90% task success across six demonstration-guided tasks. These results show that explicit hand-object goals provide an expressive and composable interface for dexterous manipulation. More details and videos are available at https://dex-hot.github.io/.
comment: 9 pages, 4 figures
♻ ★ BLaFA: Bridging Language to 3D Functional Affordances for Dexterous Robotic Grasping in Tool Use
In industrial assembly, equipment maintenance, and everyday tool-use scenarios, robots need not only to grasp objects stably but also to establish task-appropriate contact configurations according to task intentions and tool functionality. However, existing methods lack explicit mappings from open-vocabulary instructions to 3D functional contact structures and dexterous grasp configurations, limiting robots' adaptability to diverse tools and functional tasks. To address this limitation, we propose BLaFA (Bridging Language to 3D Functional Affordances for Dexterous Robotic Grasping), a structured framework for language-driven functional dexterous grasping. First, a Knowledge-guided Language Parsing (KLP) module leverages structured knowledge to transform natural-language instructions into semantic sextuples of manipulation constraints. Second, a 3D Gaussian Splatting based hierarchical semantic representation and a Triangular Functional Point Localization (TriLocation) module are developed to localize 3D functional contact keypoints under task-tool topology constraints. Finally, a 3D Keypoint Grasp Matrix Transformation (KGT3D+) module maps the localized keypoints and semantic constraints to executable wrist poses and finger-level joint and force commands. Evaluations on 100 open-vocabulary language instructions, 14 task-tool combinations covering four functional intents, and real-world robotic experiments in representative scenes demonstrate that BLaFA can effectively translate natural-language instructions into task-aligned 3D functional contact structures and corresponding dexterous grasps, thereby enabling language-driven robotic tool use across diverse tasks. The source code will be made publicly available. Code will be publicly available at https://github.com/PopeyePxx/BLaFA.
comment: Code will be publicly available at https://github.com/PopeyePxx/BLaFA
♻ ★ Think Fast, Plan Selectively: Adaptive Deliberation for Efficient Data-Driven MPC
Data-driven model predictive control (MPC) combines learned world models with online trajectory optimization, achieving strong performance in continuous control. However, the per-step cost of sampling and evaluating hundreds of candidate trajectories restricts deployment to control frequencies well below what real-time robotics demands. Motivated by the dual-process theory of human cognition, which distinguishes between fast, intuitive processing (System 1) and slower, deliberative reasoning (System 2), we ask whether every decision requires the same degree of computational deliberation. We propose Fast-TD-MPC, a lightweight framework that adaptively routes between fast policy execution and test-time planning, reserving costly deliberation for states where it is most needed. Fast-TD-MPC delivers competitive task performance across 103 continuous control tasks while achieving up to ~4x faster inference. Under external disturbances, Fast-TD-MPC selectively falls back to planning, maintaining robustness comparable to the original planner.
♻ ★ CompVLA: A Variable Compliance Vision-Language-Action Model for Contact-rich Manipulation
Contact-rich manipulation, requiring robots to regulate not only motion but also how they yield to external forces, has emerged as the next frontier for Vision-Language-Action (VLA) models. However, existing VLAs output purely kinematic commands, degrading performance on real-world contact-rich tasks. In this paper, we introduce CompVLA, a unified VLA framework that jointly predicts motion and a stiffness matrix from RGB and language inputs. Our approach augments the conventional architecture with a dedicated Compliance Expert, which outputs time-varying stiffness and virtual displacement profiles executed via geometric impedance control. We demonstrate that CompVLA achieves the highest average success rate across diverse contact-rich tasks, outperforming both vanilla and compliance-aware VLA baselines, with ablations confirming each component is essential.
comment: Conference on Robot Learning (CoRL) 2026
♻ ★ Sketch-MoMa: Teleoperation for Mobile Manipulator via Interpretation of Hand-Drawn Sketches ICRA
To use assistive robots in everyday life, a remote control system with common devices, such as 2D devices, is helpful to control the robots anytime and anywhere as intended. Hand-drawn sketches are one of the intuitive ways to control robots with 2D devices. However, since similar sketches have different intentions from scene to scene, existing work needs additional modalities to set the sketches' semantics. This requires complex operations for users and leads to decreasing usability. In this paper, we propose Sketch-MoMa, a teleoperation system using the user-given hand-drawn sketches as instructions to control a robot. We use Vision-Language Models (VLMs) to understand the user-given sketches superimposed on an observation image and infer drawn shapes and low-level tasks of the robot. We utilize the sketches and the generated shapes for recognition and motion planning of the generated low-level tasks for precise and intuitive operations. We validate our approach using state-of-the-art VLMs with 7 tasks and 5 sketch shapes. We also demonstrate that our approach effectively specifies the detailed motions, such as how to grasp and how much to rotate. Moreover, we show the competitive usability of our approach compared with the existing 2D interface through a user experiment with 14 participants.
comment: Accepted to IEEE International Conference on Robotics and Automation (ICRA) 2025. Project Page: https://toyotafrc.github.io/SketchMoMa-Proj
♻ ★ Closed-Form 6-DoF Pose Estimation of Medical Tools via Gradiometer-Based Electromagnetic Localization System
Electromagnetic tracking \added[]{is promising for} remote navigation of endoluminal \added[]{medical tools}. However, existing six-degree-of-freedom (6-DoF) pose recovery approaches often require pre-calibrated workspace field maps or iterative nonlinear optimization. This letter presents a Gradiometer-Based Electromagnetic Localization System (GELS), a closed-form tracking framework that uses a compact magnetometer array as an embedded quasi-gradiometer to estimate local magnetic fields and gradient tensors. These quantities are mapped by the Euler homogeneous relation to displacements between source and array, from which Procrustes registration over multiple sources recovers the array orientation and position using at least three noncollinear sources. The algorithm requires known source positions and array geometry, but no pre-calibrated workspace field maps, initial pose guesses, or calibrated source magnetic moments. The recovered \added[]{array} pose also enables a demonstration of sublevel dipole localization by serving as a mobile magnetic reference frame. Benchtop experiments across sensor array configurations and excitation modes validate the proposed closed-form 6-DoF localization pipeline, with a median solver runtime of \SI{172.00}{\micro\second}. A perturbation analysis identifies inter-sensor inconsistency and dipole model mismatch as dominant accuracy limits, informing future sensor array and source design to improve pose accuracy.
Multiagent Systems
★ A Separation Principle for Cooperative Multi-Agent Reinforcement Learning
We study cooperative multi-agent reinforcement learning (MARL) problems in which agents evolve under decoupled noisy dynamics but share a population objective. A canonical example is a mobility operator that orchestrates a fleet of vehicles to meet customer demand. The stochastic and time-varying nature of these problems calls for learning approaches, yet existing MARL algorithms do not scale: learning assignment and routing jointly over the full state-action space becomes prohibitive as the number of agents grows. Formulating the problem over the space of probability measures, we prove a separation principle: the population cost-to-go function is upper bounded by an optimal transport problem whose transportation cost is the target-conditioned single-agent cost-to-go function. This simplifies the multi-agent problem to learning a single-agent policy and coordinating the agents via optimal transport. Building on this principle, we introduce SALT (Separation-based Assignment and Learning via optimal Transport), a family of algorithms that learn a target-conditioned single-agent policy using any standard reinforcement learning algorithm, and then coordinate the agents via optimal transport. Since learning occurs at the single-agent level, SALT scales to fleet sizes where MARL baselines become intractable and transfers without retraining to fleet sizes unseen during training. In a vehicle-delivery experiment on the southern Manhattan road network, where MARL baselines are computationally prohibitive, SALT improves travel time by 15-21% over a shortest-path routing heuristic. In stochastic grid worlds, SALT reduces average travel times by 23-61% relative to MARL baselines and scales to population sizes where these methods fail.
★ Incentive Pathologies in a Live LLM Agent Arena: An Empirical Study of Concentration, Reward-Rule Implementation Discrepancy, and Network Dilution
We audit one live LLM agent arena (AITRAP; 22 autonomous agent seats, ~96,000 reasoning nodes) over nine days (Oct 16-25, 2026), during which the platform deployed two bundled incentive changes: a progressive tax reform (V2.9.3, Oct 19) and a seed-inversion with explorer-side action update (V3.0, Oct 21). These serve as before-and-after markers, not cleanly isolated interventions. Four exploratory predictions are tested against ten data frames (F1-F10) using the Gini coefficient, Top-K concentration share, code-level reconstruction of the continuation-bonus mechanism, and the Mann-Kendall trend test. The Gini analysis is inconclusive at 95% confidence (bootstrap intervals straddle 0.7). The Top-4 node-activity-score share falls from 97.2% (Oct 19, pre-V2.9.3) to 49.4% (Oct 25, post-V3.0; 56.7% under a merge-seat assumption). The continuation bonus, announced as depth-based x10, resolves at code level to a four-condition anti-abuse gate paying x2.85 in the typical case and permanently blocking the self-continuation it superficially rewards; the gate was active ~40 hours before the V3.0 announcement. The events-per-node ratio declines significantly (Mann-Kendall S=-45, z=-3.94, p=8.3e-5) from 7.36 to 7.069. We discuss implications for incentive designers, the announcement-implementation gap, and public peer review as an event record.
comment: 29 pages, 2 figures, 4 tables; data available at Zenodo DOI 10.5281/zenodo.23691548
★ Enabling Privacy-Preserving Natural Language Access to Industrial Smart Meter Data
Energy management officers and grid operators increasingly rely on smart meter data for anomaly detection, performance monitoring, and strategic decision-making. However, extracting actionable insights from large-scale, heterogeneous telemetry databases requires advanced SQL and data engineering expertise, creating a critical bottleneck for non-technical personnel. Furthermore, the strict data privacy regulations and security constraints inherent to critical energy infrastructure often prohibit the routing of sensitive consumer data through cloud-based Large Language Models (LLMs). To address these challenges, we present OpenMeter, a schema-agnostic Multi-Agent System (MAS) designed to translate natural language queries into executable code and human-readable insights. This architecture lowers the technical barrier for day-to-day utility operations by empowering non-expert personnel to interact with complex databases through intuitive natural language dialogue, transforming raw telemetry into a highly valuable tool for routine grid workflows and strategic decisions. Multi-scenario ablation studies over this environment demonstrate that full multi-agent orchestration shows its largest observed accuracy gains on resource-constrained local hardware (+6.1 pp EX-R over the strongest single-agent configuration); at frontier scale, individual components retain value but should be enabled selectively, indicating that the cost of full orchestration is justified specifically where privacy demands local execution.
comment: 9 pages, 1 figure, 6 tables
★ ReSUM: Reasoning-Driven Multi-Agent Framework for Query-Focused Summarization
LLMs excel at language tasks, but long-meeting QFS remains difficult due to complex speaker dynamics, sparse key information, and limited data. Meetings involve disfluencies, long transcripts and complex discourse. QFS have high risk of information omission, hallucination, even for advanced models such as GPT-4-mini and Gemini. Existing end-to-end methods compress retrieval, evidence extraction, and summary generation into a single pass, failing to handle the complex reasoning required for targeted meeting summaries. We propose ReSUM, a modular multi-LLM framework coordinated by a planner that orchestrates retrieval and summarization agents. By distilling teacher reasoning into query- and plan-conditioned retrievers and summarizers, ReSUM enhances long-context robustness, minimizes irrelevant exploration, and generates more faithful and comprehensive query-focused summaries. Experiments show consistent gains on query-focused meeting summarization benchmarks.
comment: 10 pages, 4 figures
★ Recursively Self-Improving Electromagnetic Spectrum Agents: Architecture and Operation
Electromagnetic spectrum agents (ESAs) must contend with changing environments and task requirements during sustained autonomous operation, yet their existing capabilities may not remain suitable for new operating conditions. This article proposes recursively self-improving electromagnetic spectrum agents (RSI-ESAs) that use operational feedback to update their capabilities and build on prior improvements. We present their architecture and editable components, and describe a self-improvement process encompassing problem diagnosis, candidate generation, validation, and inheritance. By retaining effective task versions and using modification records and diagnostic knowledge to guide subsequent updates, RSI-ESAs can accumulate task capabilities and improvement experience across successive rounds. A shared-spectrum scheduling case study illustrates improved task performance and capability accumulation over multiple improvement rounds under the tested conditions. Finally, we discuss open challenges for broader tasks and long-term operation, outlining research directions for the continued capability development of ESAs.
★ From Emergence to Reversibility: Toward Reversible Multi-Agent Systems
Multi-agent systems become capable through institutional mechanisms, such as rules that route information to agents that act on it. These mechanisms are also natural places to restrict the system. Yet as agents learn, share memory, delegate, and repair failures, the collective can find other paths to the same effects: a restriction on the original mechanism still operates but no longer stops them, while routine performance and compliance checks pass. We propose \textbf{Reversible Multi-agent Systems} as a research agenda: collectives that grow while an authorized supervisor can still restrict specified effects, keep them restricted as agents adapt, and reconstitute useful cooperation under renewed authority. What is reversed is the ability to produce a specified effect; restrictions must hold even if learned knowledge persists. The goal is to preserve effective intervention, not a particular mechanism: at every step of growth, some authorized intervention must keep a specified prohibited effect rare while maintaining required service. We identify three research problems: tracking what capabilities depend on, handing off control before an old mechanism loses its effect, and separating legitimate repair from unauthorized reconstruction. We outline how to evaluate control across growth paths, back control claims with evidence, and allocate the rights to restrict and restore.
★ Sharing the Road: LLM-Based Bargaining in Mixed-Autonomy Traffic
Mixed-autonomy traffic involves heterogeneous driving populations competing for shared road space. Previous studies have shown the existence and emergence of collective rationality using game-theoretic and reinforcement learning approaches, which rely on explicitly specified behavioral assumptions or reward functions for driving agents. However, how bargaining behaviors between driving agents evolve and lead to collective outcomes based on richer agent characteristics and language-based reasoning remains less understood. To fill this gap, we develop a bargaining framework based on large language models (LLMs) in which population-level autonomous vehicle (AV) and human-driven vehicle (HV) agents negotiate road-space allocations under heterogeneous objectives and private information. Through controlled experiments varying traffic composition, agent objectives, and traffic-domain knowledge, we examine bargaining behavior, coordination, and resource utilization. Results show that negotiated allocations respond to relative traffic density without strictly following proportional sharing, while density-balanced states exhibit greater coordination difficulty. Fairness-oriented objectives substantially improve agreement rates but do not necessarily yield equally high road-space utilization. Furthermore, domain knowledge affects bargaining outcomes differently across objective configurations, shaping both coordination and the role-dependent effects of fairness. These findings demonstrate the potential of LLM-based bargaining for modeling and designing interactions among driving agents in mixed-autonomy traffic.
★ Phantom Consensus: Measuring Social Reasoning in Multi-Agent LLM Decision-Making NeurIPS 2026
Multi-agent LLM systems increasingly aggregate judgments across model instances, but reasoning about other agents can itself introduce failure. We study a setting in which LLMs shift their public decisions after observing ambiguous social signals while reporting inaccurate estimates of what other agents privately judged, even when the group's recorded private judgments do not support the apparent consensus. We call this failure phantom consensus and study it as a pluralistic-ignorance-style pattern in controlled LLM collectives. To measure this phenomenon, we introduce PHANTOM-Bench, a benchmark that separates isolated private judgments, public decisions after ambiguous social signals, elicited estimates of the group's private judgment, transparency-based recovery, and controls for neutral public text and explicit vote-following. Across nine closed- and open-weight models, ambiguous public signals consistently degrade decision quality and can turn correct private group judgments into incorrect public collective outcomes. Public accuracy drops by as much as 47.8 percentage points, and collective rationality failures reach 47.3% in the most susceptible setting. We observe substantial errors in reported group-private beliefs alongside public-stage failures, while control conditions reveal model-dependent differences in susceptibility to ambiguous and explicit peer signals. Revealing the agents' recorded private decisions repairs many failures without revealing the gold label, although recovery varies across models. Our results show that evaluating multi-agent LLM systems requires measuring not only final agreement, but whether that agreement reflects independently measured private judgments or an apparent consensus.
comment: Accepted at NeurIPS 2026, SocialAgent Workshop
★ AIM: Adaptive Interaction in Multi-Agent Debate for Multimodal LLM Inference NeurIPS 2026
Multi-Agent Debate (MAD) improves reasoning on multimodal tasks, but its accuracy and token efficiency depend on how agents interact during debate. A largely overlooked design choice is which interaction modalities agents use to exchange evidence. Existing methods rely on fixed text- or graph-based interaction, which fails to convey fine-grained multimodal evidence and thus degrades accuracy while inflating token cost due to verbose descriptions. To address this, we propose Adaptive Interaction in Multi-Agent Debate (AIM), an adaptive routing framework that dynamically selects the optimal modality combination for inter-agent interaction in multimodal MAD. Beyond text and graph, AIM introduces task-dependent modality interaction, where agents exchange targeted regions of interest from the task-specific multimodal input, such as image regions, audio segments, or spatio-temporal video clips. To determine the best combination for each instance, AIM first generates a structured baseline response and extracts interpretable routing features that capture which evidence the instance requires, how confident the baseline agent is, and how much new evidence each modality would add beyond the baseline response. A lightweight router then uses these routing features to select a modality combination that avoids both under-fusion (i.e., agents miss decisive evidence) and over-fusion (i.e., redundant modalities inflate token cost) risks. Across eight benchmarks spanning three multimodal task families, \name achieves the highest accuracy. Combining modalities improves accuracy by up to 6.4\% over the best single-modality baseline, and adapting the combination per instance adds up to 4.2\% over the most accurate fixed combination of each benchmark with up to 53.4\% fewer tokens.
comment: Accepted in The Fortieth Annual Conference on Neural Information Processing Systems (NeurIPS 2026)
★ ODD+LLM: Specifying LLM-Agent Behavior for Reproducible Social Simulation NeurIPS 2026
The ODD protocol is the de facto standard for describing agent-based models, but it lacks vocabulary for the components that determine LLM-agent behavior: prompt assembly, memory serialization, parsing, and retry logic. We propose ODD+LLM, a backward-compatible extension adding five documentation blocks grounded in the CoALA cognitive-architecture framework. To show that the gap matters, we conduct a multiverse analysis [Steegen et al., 2016] of a published LLM naming-game simulation, varying six specification choices a reimplementer must make. Variants that are defensible readings of the paper stay within statistical noise of our baseline reconstruction. Missing one instruction that the paper states but is easy to overlook (that the model answers as an outside observer rather than as a player) raises the share of runs settling on the convention agents individually favor from 0.35 to 1.00 and makes convergence three times faster, in a reimplementation that uses a different model from the original. We complement the demonstration with a reconstruction-sufficiency review of 28 published LLM-agent simulations, finding that the details of how prompts are built and how model outputs are parsed are undetermined in 77% of cases, and that no paper fully specifies its main LLM call. ODD+LLM gives the field a standard home for information it already produces but currently scatters across code repositories, supplementary files, and undocumented defaults.
comment: 18 pages, 2 figures, 6 tables. SocialAgent Workshop @ NeurIPS 2026
★ Auditable Human-Agent Teams for Autonomous X-ray Experimentation
Closed-loop autonomous experimentation can accelerate scientific discovery, but implementation effort limits accessibility. Agentic systems may lower this barrier, although sustained operation on physical instruments remains largely unexplored. Here, using coding-agent infrastructure grounded in validated scientific methods, we demonstrate autonomous synchrotron X-ray experiments on nanoparticle superlattices under evolving, human-defined objectives. Complete human-agent and agent-agent transcripts enable an audit that reconstructs decision provenance and characterizes the division of cognitive labor. Across three experiments, 61-67% of messages involved interpretation or judgment, while the share of methodological and scientific contributions increased from 46% in the setup-focused first experiment to 56-61% in later experiments. Provenance showed that 59-68% of error corrections involved multiple participants and that about half of all decisions incorporated the input of another participant. These results may guide design principles for multi-agent scientific systems and inform future agent harnesses, model training, and accessible autonomous experimentation.
♻ ★ Aïra: Rethinking AI Research Assistants for Interdisciplinary Science
Scientific discovery increasingly depends on interdisciplinary teams whose members contribute distinct expertise, conceptual frameworks, vocabularies, assumptions, and standards of evidence. Today's AI research assistants are largely designed to support individual researchers through literature review, writing assistance, coding, and data analysis. While these capabilities improve personal productivity, they provide little support for the collaborative reasoning required to integrate knowledge across disciplines. In this vision paper, we argue that AI research assistants should evolve from tools that optimize individual workflows to systems designed for interdisciplinary teams. We introduce aïra, an AI research assistant built around this idea. Rather than focusing solely on summarization or question answering, aïra identifies disciplinary perspectives, translates terminology, highlights assumptions, and synthesizes collaborative research opportunities. This paper offers a vision describing the design principles underlying aïra, presenting its system architecture, illustrating its outputs through interdisciplinary research meetings, and outlining future research directions for AI systems that support collaborative scholarship.
comment: Accepted to AIAS 2026
♻ ★ Generalized Intention Modeling in Multi-Agent Reinforcement Learning NeurIPS 2026
Modeling an opponent's intent is critical for effective decision-making in non-cooperative, competitive, and general-sum multi-agent reinforcement learning. Existing opponent modeling methods encode intent using an embedding derived from episode information chosen a priori, such as the opponent's next action or a future environment state, and use this to guide the ego-agent's behavior. These approaches assume that the chosen information is universally representative of intent; however, we show empirically that this is not the case as intentions are often task- and environment-dependent. To address this, we introduce a task-adaptive opponent modeling framework that learns a performance-driven mixture of multiple intent representations. We further introduce a new intention representation that maximizes mutual information with the ego-agent's future returns, thereby capturing opponent information that is most directly relevant to performance. Our approach consistently matches or exceeds the performance of state-of-the-art baselines across diverse tasks and yields insights into when and why different opponent modeling strategies succeed.
comment: Accepted at NeurIPS 2026
♻ ★ The Consensus Trap: Collaborative Interleaved Generation to Defend against a Corrupted Majority
Majority voting offers a simple way to combine answers from multiple large language model (LLM) agents. When a majority follows instructions favoring the same incorrect answer, however, agreement reinforces the error. We call this failure the "consensus trap": a minority may identify the error, yet its reasoning cannot change a vote over completed answers. We introduce chunk-interleaved Round-Robin (RR) collaboration, in which agents take turns extending one unfinished reasoning trajectory. A minority of agents with correct reasoning can insert corrections that subsequent speakers read before the answer is complete. RRMaj combines several such trajectories by a final vote. We characterize a majority barrier for outcome-level voting over final answer labels and derive a local sufficient condition for expected progress when the minority's cumulative corrections outweigh opposing corrupted updates. Across model families and verifiable reasoning benchmarks, RRMaj substantially improves accuracy under majority corruption while retaining comparable average performance under minority corruption in the main advisory-instruction setting. RR also strengthens full-response debate. Scoring the shared reasoning between turns shows that, on average, continuations from uncorrupted agents increase support for the correct answer over the incorrect target. Parallel local serving adds modest latency in the measured deployment. These findings show how language shared during reasoning, and the timing of that exchange, can help a minority correct a group's answer.
♻ ★ Classification Aggregation with Multiple Memberships
People face the problem of classifying objects into categories while aggregating their opinions. We axiomatically study such a classification aggregation problem with multiple memberships, meaning that each individual assigns one category to every object, whereas the resulting classification may assign more than one category. Under the natural settings and axioms (i.e., Independence and Supportiveness), each of our new axioms -- Responsibility, Split Stability and Rejection Stability -- characterises what we call the junta rules. The rules assign a category to an object exactly when some member of a fixed coalition does so. The coalitions that can secure an assignment form a grill, a topological structure implied by an ultrafilter. Hence, the study parallels the literature on Arrow's theorem characterised by the decisive coalition structures. An infinite population setting is also investigated to clarify the differences among the axioms.
comment: 33 pages
♻ ★ Measuring Weak-to-Strong Legibility of Reasoning Models NeurIPS 2026
Language models are increasingly trained to "reason" before answering users' queries, outputting hundreds of intermediate tokens before a final answer. While reasoning is designed with direct model capabilities in mind, we argue that reasoning language models (RLMs) should also be assessed by interaction effects between their reasoning traces and weaker models. In contexts like safety monitoring and distillation, the performance of weak models increasingly depends on legibility--how thoroughly and precisely a reasoning trace externalizes an RLM's actions. Existing efficiency-based metrics for legibility fail to capture "thoroughness", instead focusing on conciseness. Thus, we introduce transfer utility, a method for measuring trace legibility derived from interactions where weak models finish tasks given incomplete traces. Evaluating 84k traces from 12 RLMs across three tasks, we find that reasoning traces generated by the highest-performing and the most efficient models rank the lowest for transfer utility. We also find evidence that transfer utility predicts the ability of weak monitors to verify procedurally dense traces for math or logical problem-solving, though the effect moderates when verifying fact-driven traces. Finally, we show that open-source reward models decouple from transfer utility signals on correctness. Together, these findings surface status quo trade-offs in weak-to-strong legibility, motivating an interaction-first framework for future applications.
comment: Accepted to NeurIPS 2026
♻ ★ False Belief Dissemination, Persistence and Mitigation in Multi-Agent Social Networks Through the Lens of the Mandela Effect
Multi-agent social networks are emerging as communities in which agents communicate, retain memories, and publish information that others can find as evidence. These interactions create opportunities for false claims to spread and remain influential beyond their original sources, analogous to the Mandela effect in human societies, in which many people recall the same false memory. In this paper, we systematically study false-belief dynamics through the lens of the Mandela effect, focusing on dissemination, persistence, and mitigation. Specifically, we build a controlled framework connecting agent communication, private memory, and a shared simulated web to mimic information dissemination in practical social networks, supported by 540 fabricated claims spanning 8 domains and 4 plausibility levels. For dissemination, we measure how widely and quickly claims gain acceptance across models and communication topologies. For persistence, we withdraw agents promoting fabricated claims while retaining existing articles and memories, and track subsequent acceptance. For mitigation, we compare preventive measures applied before implantation with corrective interventions introduced after claims have spread. Experimental results show that evaluated models readily accept and disseminate fabricated claims, with dissemination reach as high as 96.0%. Dissemination reach varies with claim plausibility, domain, and network size and composition, while communication topology shapes dissemination speed across model families. Acceptance barely declines after fabricators withdraw, and targeted corrective interventions outperform the preventive measures without eliminating false beliefs. These findings identify persistent false beliefs as a safety challenge for multi-agent social networks and highlight the need to address both their initial dissemination and the accumulated evidence that sustains them.
comment: 40 pages, 6 figures, 13 tables
♻ ★ When Disclosure Is Compulsory: Evaluating Ride-Hailing Rebalancing Policies with Self-Interested LLM Driver Agents
The EU Platform Work Directive, to be transposed by December 2026, requires ride-hailing platforms to state in plain language the main parameters of their automated decisions and how a worker's behaviour affects them. Such rules are exactly what a large language model (LLM) can read, so drivers will be able to let one decide for them. We compare two main families of rebalancing policy once their rules are public: a reactive policy (R) that sends idle drivers to cover requests just lost, and an anticipatory policy (A) that plans moves on a demand forecast. Each driver is an LLM agent maximizing its own net income, given no rules (N) or the rules at the directive's level of detail (L). We test two LLMs, Claude Haiku 5.5 and Jev, which understand the rules equally well, in a three-zone model and in Manhattan. With the rules disclosed, drivers follow the platform more closely: in Manhattan, the share of relocation offers they decline falls from about four in five to one in four under both policies. Yet the effects differ by policy. For the platform, disclosure costs R 3.7% of its profit in Manhattan, as R now pays for moves drivers used to make for free, while A's profit is unchanged; the two policies, which tie on profit under full compliance, end 6.1% apart. For drivers, disclosure raises income in the small model under both policies and LLMs; for users, it raises the service rate there by 10 to 18 percentage points. In Manhattan, self-interested drivers also make A a win-win-win against full compliance, raising driver income, platform profit and service rate by 10 to 12% each. Rebalancing policies that perform equally well under full compliance can thus show opposite results under mandatory disclosure. The study offers a framework to evaluate such policies under the new requirement and is, to our knowledge, the first to do so with LLM agents.
♻ ★ Game-Creation-Agents: Autonomous Game Production
Game-Creation-Agents (GCA) turns a natural-language request into a runnable Phaser 4 browser game. The pipeline first builds a multimodal game design document that couples mechanics and scene contracts to grounded visual references. It then projects that document into a runtime scaffold, machine-readable contracts, and deterministic tests. Eight specialized agents next carry out production: scenes are implemented serially, assets are produced in parallel, and one reduced manifest joins the two branches into a single build. Build-linked real-UI checks and a code-blind critic then verify that build and decide whether the game may ship. Each blocker returns to the agent that owns the affected artifact for a bounded repair, and the game is released once all the gates pass. We evaluate finished games separately, through their rendered interfaces alone, over prompt-matched tasks run under a common protocol. GCA achieves stronger requirement satisfaction, core mechanics, content depth, and functional visuals than a comparable open pipeline. In a side-by-side human study, raters also score GCA's games higher on every dimension and prefer them overall.
comment: Project: https://github.com/Tencent/Game-Creation-Agents
♻ ★ Beyond Reasoning: RL-Policy Guided LLM Inference for Efficient Strategy in Liar's Poker
AI researchers have long focused on poker-like games as a testbed for environments characterized by multi-player dynamics, imperfect information, and reasoning under uncertainty. While recent breakthroughs have matched elite human play at no-limit Texas hold'em, the multi-player dynamics are subdued: most hands converge quickly with only two players engaged through multiple rounds of bidding. In this paper, we present Solly, the first AI agent to achieve elite human play in reduced-format Liar's Poker, a game characterized by extensive multi-player engagement. We trained Solly using self-play with a model-free, actor-critic, deep reinforcement learning algorithm. Solly played at an elite human level as measured by win rate (won over 50% of hands) and equity (money won) in heads-up and multi-player Liar's Poker. Solly also outperformed large language models (LLMs), including those with reasoning abilities, on the same metrics. Solly developed novel bidding strategies, randomized play effectively, and was not easily exploitable by world-class human players. In one extension, when Solly's policies were provided to LLMs as a domain-specific guide for inference, LLM performance improved and token costs decreased. This result suggests that low-cost RL agents can be used to boost LLM efficiency and reasoning capabilities.
Systems and Control (EESS)
★ Geometric Pairwise Classification of Affine Constraints over Ellipsoidal Regions
This paper presents a deterministic geometric procedure for analysing affine constraints over fixed ellipsoidal regions, motivated by chance-constrained set approximation. Analytical tests classify inequalities as always violated, never violated, or cutting. Pairwise tests determine whether two inequalities can be strictly violated at a common point, yielding a compatibility matrix. The tests follow from quadratic optimisation and the Karush--Kuhn--Tucker conditions, with explicit treatment of tangencies, parallel normals, and degenerate inequalities. The formulation applies in arbitrary finite dimensions and provides geometric information for subsequent bounds on simultaneous sampled violations and safe-region approximation.
comment: 6 pages, 1 figure
★ Generalized H matrix stability for power systems
The small-signal stability of a power system model with arbitrary loads is investigated, with a focus on locally constant power loads in lossless networks. A central element of the proposed analysis is the matrix H, which represents the partial derivatives of generator powers with respect to their relative angles. The local stability of any equilibrium point is determined by the properties of H if the loads consume locally constant power. Specifically, an equilibrium point is asymptotically stable if H is positive definite and unstable if H is full-rank and not positive definite, while several quantitative relationships between the eigenvalues of H and the system Jacobian matrix are also established. The main contribution is that loads are allowed to be arbitrarily located within the transmission network, thereby generalizing the existing H matrix stability criterion which is valid only if loads are located at the same buses as the generators. The advantage of the proposed criterion is demonstrated through case studies involving systems with up to 2869 buses. Accurate stability assessments are maintained, where alternative methods, such as the NRPS model, are found to be overly conservative. Additionally, a computational acceleration by a factor of 340 relative to Jacobian-based methods is achieved.
★ Autonomous Whole-body Loco-manipulation Using a Snake Robot IROS 2026
Snake robots can leverage their hyper-redundant morphology for both locomotion and manipulation through distributed body contact but whole-body loco-manipulation is particularly challenging since the entire body serves simultaneously as propulsive mechanism and manipulation tool Prior work on wheeled legged and snake robot platforms has decoupled these functions through dedicated appendages body segmentation or grippers We present closed-loop loco-manipulation with our snake robot COBRA without this distinction such that any part of the body may make contact We develop a contact-implicit dynamic model unifying robot-ground robot-object and object-ground interactions within a Moreau-Jean time-stepping framework and formulate a trajectory optimization over the full-order dynamics guided by a geometric planner and nominal proportional controller to autonomously push an object to a goal location during sidewinding We validate the planner and controller on hardware across varied configurations and present simulation results for a simplified trajectory optimization that resolves robot-object and object-ground contacts
comment: Presented in IROS 2026
★ A Cross-Domain TARA-to-NAC Rule Mapping Methodology: From Automotive OEM Networks to Energy Storage Systems
Threat analysis and risk assessment under ISO/SAE 21434 ends with cybersecurity goals, but existing methods give little structured guidance for turning those goals into enforceable network access control rules. Compliance standards such as IEC 62443, ISO/SAE 21434, and ISO/IEC 27001 are usually applied as parallel efforts rather than as one enforcement architecture. This paper presents a six-step methodology that maps cybersecurity goals from a threat analysis and risk assessment to network access control rules, each placed at a named enforcement point and traceable to the threat records behind it. It also presents a unified compliance framework that treats IEC 62443 and ISO/IEC 27001 as a fixed substrate and domain-specific regulation as an interchangeable overlay. The methodology is applied to an automotive original equipment manufacturer plant network, addressing multi-tier supplier authorization and legacy operational technology devices, and then to utility-scale battery energy storage, where North American Electric Reliability Corporation Critical Infrastructure Protection standards and ISO/IEC 27019 replace the automotive-specific standard while the substrate stays constant. In the automotive example, 37 threat entries and 24 cybersecurity goals yield 23 rules covering 11 of the 13 in-scope goals; the energy example yields six rules. Three derived rules have no equivalent in the cited standards. Both examples are desk-based: the methodology is applied to them, not tested on them. The result is a reusable pattern for translating risk assessment into enforcement across critical infrastructure sectors, with empirical validation left as future work.
comment: Preprint, 23 pages, SAE format
★ Adaptive PD-like Fault Tolerant Controller for a Class of Nonlinear Systems
The article presents a nonlinear adaptive control strategy for a class of feedback-linearizable systems subject to process faults and disturbance. The controller is derived through Lyapunov analysis and incorporates an adaptive proportional gain within a PD-like structure, preserving simplicity while providing nonlinear adaptive behavior. A residual-based gating mechanism activates the adaptation only when the tracking error exceeds a prescribed threshold, preventing unnecessary gain updates during nominal operation and improving robustness to noise. The proposed design yields uniform global ultimate boundedness of the closed-loop signals, with explicit expressions for the steady-state tracking bound. Practical tuning guidelines for the adaptive parameters and Lyapunov matrix selection are provided to ensure consistent gating behavior. Simulations on a precision positioning system illustrate effective compensation of process faults, rapid gain escalation during degraded conditions, and smooth reversion to nominal performance, demonstrating that the method enhances fault tolerance while retaining the structural simplicity of PD control.
★ Nonlinear Fault Tolerant Control of Wind Turbine Pitch Systems: A Comparative Study
Electro-hydraulic pitch systems account for about one fifth of wind-turbine failures, motivating the development of resilient control strategies for fault mitigation. The study considers a large set of robust nonlinear control designs of different feedback principles and architectures, and assesses their fault-tolerant capabilities in comparison to each other and against the industrial standard, i.e. the Proportional-Integral (PI) controller. The implementation and tuning of each scheme is based on a high-fidelity simulator to ensure nominal performance, whereas each controller is deployed unchanged on an experimental apparatus that includes a full-scale electro-hydraulic pitch system. The comparative evaluation of the considered fault-tolerant solutions is carried out through several tests that cover nominal operation and twenty-five representative fault scenarios, including sensor, process, and actuator faults. The analysis highlights the trade-off between performance preservation under faults and increased actuator chattering. The experimental results show that introducing a dynamic input-reconstruction module into the design of the nonlinear controllers significantly reduces actuator chatter without compromising their fault-tolerant properties.
★ LACART: Lookahead-Augmented Collision Avoidance and Reference Tracking Strategies for Safety-Critical Robots
Autonomous racing provides a demanding testbed for collision avoidance under tight geometric and computational constraints. Building on a recent conference contribution and a master's dissertation, LACART combines lookahead-based control barrier functions (CBFs), a relaxed control Lyapunov function (CLF), and quadratic programming (QP) to modify nominal vehicle commands for obstacle avoidance and reference tracking. The method handles circular obstacles and track boundaries in a common optimisation layer. Comparative simulation results against Huang \emph{et al.} \cite{r9} show similar full-track paths and lower median computation times: 0.31 versus 0.33 ms per step and 0.023 versus 0.025 ms per QP solve. The dissertation further demonstrates a 1:43-scale vehicle with camera-based estimation and wireless actuation at a reported 25 Hz control rate. We identify the conditions linking lookahead-point clearance to body clearance, illustrate hardware limitations, and propose a recorded demonstration of safety intervention and recovery. The contribution targets ground-robot collision avoidance and real-time safety monitoring.
★ A Novel Algorithm to Simulate Conductance-based Neuronal Networks
Conductance-based models are central to neurophysiology and neuromorphic engineering, yet their simulation scales poorly to large networks. We study and analyze an iterative solver that operates in signal space. Each iteration reduces to $N$ decoupled scalar linear time-varying differential equations, one per neuron, which are solved in parallel and in closed form. The solver admits two complementary interpretations: a variable metric forward-backward splitting method, rooted in operator theory, and a waveform relaxation quasi-Newton method, rooted in circuit simulation. We prove uniform and superlinear convergence, with a bound that reflects the conductance structure of the dynamics. The algorithm is illustrated on the classical Hodgkin--Huxley neuron and on networks of up to 5000 neurons.
★ Beyond Feasibility: Efficient and Robust On-Device Localization with UWB Concurrent Transmissions
Concurrent transmissions (CTX) enable an ultra-wideband (UWB) tag to range to, or self-localize from, multiple nodes with a single frame reception, significantly improving update rate and energy efficiency compared to traditional approaches. Prior work demonstrated the concept feasibility via prototypes whose real-world applicability is however hindered by many limitations, notably including their inability to directly execute on UWB tags and to cope with interfering UWB signals. We address these limitations in two ways. First, we redesign the core CTX techniques to enable a fast and efficient execution on resource-constrained MCUs without sacrificing accuracy. Second, we leverage the new features in the IEEE 802.15.4z standard to achieve robustness to interference from other UWB devices and enable tags to dynamically select which anchors to use, two crucial and unexplored capabilities in CTX-based systems. We quantitatively evaluate our approach experimentally in two real-world testbeds with different characteristics. We first analyze our new techniques individually, offering an in-depth discussion of design choices and performance trade-offs. Then, we embed all our techniques into a CTX-based system enabling a tag to self-localize entirely on-device, unlocking gains up to 44% in runtime and 85% in energy efficiency compared to traditional approaches.
comment: Submitted to the IEEE Transactions on Mobile Computing
★ Data-driven representation of continuous-time linear time-invariant behaviors via Hankel integral operators
We present a data-driven representation of the finite-horizon behavior of deterministic controllable continuous-time LTI systems. We first introduce a notion of persistency of excitation for continuous-time signals that does not require the signal to be differentiable. We then show that, when the input is persistently exciting in this sense, the closure of the range of a Hankel integral operator built from a single input-output trajectory equals the behavior of the system over a finite time horizon. This result provides a nonparametric representation of system trajectories directly in terms of measured data, without requiring an explicit parametric model of the system.
comment: 9 pages, 1 figure
★ Characterization of the Frequency-Current Curve of First-Order High-Pass Neurons
A spiking model, called the High-Pass (HP) model, has recently been proposed as a minimal excitable primitive. This model is composed of a static activation function, usually a sigmoid, in positive feedback with a high-pass filter. The HP model enters a tonic spiking regime whenever its input lies in the high-gain zone of the sigmoid, where the sigmoid's slope exceeds 1. In neuroscience, the frequency-current (FI) curve is the standard way of characterizing the tonic spiking regime of a neuron. So far, the FI curve of the HP model has been characterized analytically only for specific sigmoids, and has otherwise been computed through simulation. These simulations are computationally heavy, since simulating neurons requires solving a stiff system of differential equations. This paper bridges this gap by analytically characterizing the FI curve of the HP model with a first-order high-pass filter, for a broad class of sigmoid functions. The FI curve of the first-order HP model can be computed by solving two input-independent equations and evaluating two input-dependent integrals, with no need to simulate the model itself. A spiking primitive is said to be Type I (resp. Type II) if tonic spiking starts at a zero (resp. non-zero) frequency. This paper also shows that, for the HP model, the type depends solely on the local shape of the sigmoid at the outer edges of its high-gain zone. A sharp corner, where the derivative approaches a value strictly smaller than 1 from the outside, yields a Type I primitive. A locally twice differentiable corner, with a bounded second derivative and a derivative approaching 1 at the edge, yields a Type II primitive.
comment: 8 pages, 4 figures
★ An Inspection Game for Monitor Design Against GNSS Timing Attacks in Synchronous Islanding
Phasor measurement units (PMUs) take their time from GNSS, so a spoofer that slowly drags a receiver clock rotates the reported phase without touching the power waveform. The threat is acute in synchronous islanding, where a controller keeps an islanded generator in phase with the grid, using one PMU on each side of an open breaker, so that the two can be reconnected at any moment: a differential clock bias of about $0.56$\,ms can make the controller hold the generator $10^\circ$ out of phase while it reports perfect synchronism. We cast the defense as a zero-sum inspection game. The defender, as inspector, chooses its timing monitors and audits the clocks periodically; the attacker, as inspectee, drags the two clocks to an unsafe bias by the next audit; the payoff is the evidence, the Kullback--Leibler divergence between the probability laws of the defender's record under attack and under normal operation. Our central result is that the evidence equals the cost of the cheapest innocent explanation of the attacked record. This single identity predicts which monitors work, turns the attacker's best response into a strictly convex quadratic program, and turns the defender's design into a convex program with a saddle point. Pairing explanations, which bound the evidence from above, with test statistics, which bound it from below, certifies audit periods in which every successful attack leaves at least a threshold of evidence. On a 50\,Hz island, these evidence-based periods, at the matched-test threshold, are up to 8.4 times as long as those of a rule that checks each monitor separately, and the case study favors placing the better oscillator on the utility side of the breaker.
★ An Optimal Power-Flow Grouping Topology for Single-Isolation-Stage Multi-Output Solid-State Transformers
Electric-vehicle dc fast-charging stations and data centers require multiple high-power dc outputs with independently varying demands. Conventional multi-output solid-state transformers typically establish a common dc bus and use downstream dc/dc converters for port-level regulation and, where required, galvanic isolation. A single-isolation-stage solid-state transformer can eliminate this additional high-power interface stage by directly forming multiple isolated outputs from its submodule dc/dc stages. However, because one submodule is generally insufficient for an individual output port, multiple submodules must be connected in parallel, while inter-submodule balancing circuits redistribute power under unbalanced loading. The submodule grouping therefore determines the internal balancing-power paths. Sequential grouping can create long transfer paths and accumulated balancing power, increasing current stress and losses. This paper proposes an optimal power-flow grouping topology that interleaves submodules assigned to different output ports, thereby localizing power redistribution and reducing the power processed by the balancing circuits. Analytical models quantify the required balancing power and its reduction relative to sequential grouping. A coordinated submodule-bypass strategy is also developed, and the remaining output-power capability under different fault scenarios is evaluated. A six-submodule prototype validates the proposed topology. Under the same highly unbalanced operating conditions, the proposed topology reduces the maximum submodule dc-link voltage difference by 63\% and the peak balancing current by 14.5%, while improving the measured efficiency by approximately 5.6% at rated power. Stable post-fault operation is also experimentally demonstrated.
comment: 13 pages, 19 figures. Submitted to IEEE for possible publication
★ DC-Link Voltage Regulation in a Single-Isolation-Stage Multioutput Solid-State Transformer for Wide-Range DC Outputs
Single-isolation-stage solid-state transformers with balancing circuits eliminate an additional downstream dc--dc conversion stage. However, the remaining isolated converters must directly accommodate a wide output-voltage range. This paper investigates variable dc-link voltage operation of a single-isolation-stage multioutput SST to ease this requirement. Coordinated cascaded H-bridge modulation and embedded four-switch balancing circuits support intentionally unequal submodule dc-link voltages, allowing the isolated-converter input voltages to be adjusted according to the output requirements. The achievable voltage range is determined by relating the required balancing power to the BC transfer capability within the available CHB switching windows, subject to modulation and peak and rms current limits. The analysis quantifies the range extension provided by BC operation beyond modulation alone. For a single-phase-shift dual-active-bridge converter, input-voltage following within this range reduces the calculated peak current and extends the ideal zero-voltage-switching region compared with fixed input. Experiments on a six-submodule SST prototype demonstrate unequal dc-link voltage operation and inter-submodule power exchange. Experiments result on DAB demonstrate reduced current stress and improved efficiency with voltage following.
comment: Submitted to IEEE Transactions on Power Electronics (TPEL) for possible publication
★ Reconciling European Electricity Data with Diffusion Priors Learned from Observations Alone
The ENTSO-E Transparency Platform publishes generation, load, and cross-border power flows for European bidding zones. The values carry undocumented errors and violate energy conservation, and data reconciliation consists in correcting them to satisfy that balance. In this work, we formulate data reconciliation as Bayesian inference. Classical methods implicitly take a uniform prior, which we replace with a diffusion model estimated from the observations alone by empirical-Bayes expectation-maximization. The resulting posterior is sampled with a guided process whose projection step makes every sample satisfy the balance exactly. On synthetic systems of six to thirty-five bidding zones, built from ENTSO-E resource-adequacy data, the diffusion prior gives the most accurate reconciliation, increasingly so as the system grows. Its credible intervals are calibrated or conservative, and it remains the most accurate under a misspecified noise covariance. Finally, when the generation of entire zones goes unreported, the classical methods fail while the learned prior recovers it from cross-zone structure.
★ Controlled Excitation for Identification of Nonlinear Systems with Saturated Outputs
We consider nonlinear systems whose measured output is the product of an unknown matrix and a known saturation function of the unmeasured state. Generating excitation that allows the matrix to be recovered from the output is challenging when the dynamics are largely unknown. We exploit saturation to construct this excitation. Under a geometric accessibility condition, we derive explicit open-loop controls that drive the state into regions where the saturation takes prescribed values. For componentwise saturations, this condition holds generically with as few as two independent control directions. The controls require no state measurement and only partial knowledge of the dynamics, with no information on the structure of its nonlinear part. The resulting regressor is known in advance, and the unknown matrix is recovered exactly by a closed-form regression. Under stronger structural assumptions and with sufficiently many independent outputs, an output feedback tracks prescribed reference trajectories within the unsaturated region to arbitrary accuracy and permits exact state recovery. Suitably chosen references then also enable exact identification of linearly parameterized (nonlinear) dynamics. We analyze robustness to measurement noise and illustrate the method numerically in the presence of both measurement and dynamical noise.
★ Real-Time Nonlinear Mixed-Integer Model Predictive Control With Horizon-Tail Relaxations
Model predictive control (MPC) of nonlinear systems with discrete decisions leads to mixed-integer nonlinear programs (MINLPs) whose combinatorial complexity often prevents real-time deployment. In this paper, we propose a decomposition-based MPC method that reorders and reduces computations to shorten the delay between receiving a state measurement and applying the control input. Before the measurement arrives, we solve a continuous relaxation of the MINLP at a predicted state and construct a mixed-integer quadratic program (MIQP). Once the measured state is available, we solve the MIQP that reconstructs the discrete controls and simultaneously performs one sequential quadratic programming step toward the corresponding nonlinear solution. To reduce the combinatorial complexity in the horizon length, we either fix the integer controls or relax their integrality over the prediction horizon tail, while retaining an integer-feasible first control input. We derive error bounds relating the nonlinear objective values obtained with the horizon-tail approximations to the optimal value of the original mixed-integer problem. In a data center cooling example, the proposed method reduces mean feedback computation time by 94% relative to a state-of-the-art MINLP method with a total closed-loop cost difference of less than 1%.
comment: Code available at: https://github.com/nosnoc/advanced-minmpc
★ Circuit Theory-Informed AI for Condition Monitoring in Power Electronics
Artificial intelligence for power electronics is evolving toward a deeper comprehension of system intrinsic behaviors and dynamics. As a novel integration of neural computation and electric circuits, spike-based neuromorphic computing offers promising potential in high energy efficiency and adaptability for data-driven applications. Building on the latency-coding principles, this article proposes a framework for data-light condition monitoring that relies solely on three-phase AC current measurements. The AC current signals are encoded as membrane potentials of RC-based spiking neurons, which are inherently compatible with spiking neural networks that can effectively facilitate the estimation of device parameters. Case studies are presented to validate the performances, while constraints and further optimizations are also evaluated and discussed, and future extensions of the methodology are provided.
comment: The manuscript has been accepted for publication in the Proceedings of 2026 IEEE Design Methodologies Conference (DMC 2026)
★ Safe Circumnavigation on Heterogeneous Orbits: A Range-Only HOCBF-QP Framework for Nonholonomic Multi-Agent Systems
Circumnavigating a stationary target with a team of robots requires each robot to hold its own standoff distance, to stay evenly spaced from its neighbours, and to avoid collisions with the target and with each other at every instant. The task becomes arduous when the robots are assigned different orbital radii, as their sensors or roles may demand. This paper presents a safety-critical control framework for this problem. Each robot commands both its acceleration and its turn-rate, measures only its distance to the target, and exchanges information only with its two ring neighbours. The design has three features. First, a structural property of the model lets one control law serve every robot regardless of its assigned radius. Second, the acceleration drives each radius exactly to its target, while the turn-rate balances the formation with the neighbours. Third, a higher-order control-barrier-function quadratic program enforces target, geofence, and inter-robot safety on both inputs at once; it leaves the nominal controller untouched away from the constraints and stays valid while the unmeasured range rate is still being estimated. Simulations confirm these guarantees. In a comparison, the closest existing multi-orbit design, even with full state information, violates both the target standoff and the inter-robot margin, whereas the proposed controller does not.
★ Fair Selection of Generalized Nash Equilibria: A Comparability Perspective
Generalized Nash equilibrium (GNE) problems model self-interested agents coupled through their costs and shared constraints, and arise throughout control applications that allocate a scarce common resource. Such problems typically admit a continuum of equilibria, so any implementation must \emph{select} one. This selection is not neutral: it decides how the cost of the shared constraint is distributed among the agents, a decision often expected to be fair. The variational GNE (v-GNE), which charges all agents a uniform shadow price, is the conventional choice and is frequently advocated as fair. Through the lens of \emph{comparability} from social choice theory, we show that the GNE set itself requires no inter-agent comparability of costs, whereas the v-GNE implicitly assumes cardinal unit comparability, one of the stronger notions. We then introduce the fair GNE (f-GNE), which selects an equilibrium by explicitly optimizing a fairness metric matched to the comparability at hand, and give constructive routes to compute it for single-constraint and linear-quadratic games. An electric-vehicle charging game serves as a running example throughout this self-contained tutorial.
comment: Book chapter, 25 pages, 4 figures. To appear in Socio-Inspired and Socially Aware Control for Complex Systems, Springer
★ Supporting Data Center Grid Integration via Co-Optimized Battery Energy Storage Sizing and Capacity Allocation
Interconnecting large data centers can be delayed by grid requirements on demand flexibility, ramping limits, and ride-through performance. This paper proposes an optimization-based framework to size a Battery Energy Storage System (BESS) that accelerates data center grid integration by providing BESS capacity required to meet selected flexibility and grid-support requirements. The model co-optimizes BESS power rating and energy capacity while allocating capability across three services: (i) demand response over a specified duration, (ii) mitigation of high-magnitude, high-frequency load fluctuations through reserved active power with zero net state-of-charge change, and (iii) voltage support via reactive power capability at the point of common coupling. Minute-level operational scheduling enforces state-of-charge feasibility and load ramp constraints, and incorporates a controllable queue of delay-insensitive computing tasks to enhance flexibility. A case study using a real data center load profile demonstrates the proposed method and quantifies the BESS power-energy requirements and operational trade-offs needed to satisfy the multi-service constraints, providing practical guidance for interconnection planning and deployment.
★ Grid-Oriented Probabilistic Load Forecasting for AI Data Centers
Probabilistic load forecasts for AI data centers need to represent both uncertainty under changing workloads and temporal dependence to support grid operations. This paper develops an open, grid-oriented benchmark using measured GPU power from a 448-GPU AI cluster. Forecasts are evaluated through four stages: probabilistic prediction, online uncertainty calibration, temporal scenario generation, and horizon-wide reserve sizing. Following a change in cluster utilization, a quantile LightGBM forecast achieves only 63.4% coverage for a nominal 90% interval, while online regime-conditioned calibration increases coverage to 87.3%. Even with well-calibrated marginal forecasts, independent sampling produces unrealistic ramp distributions and excessive reserve procurement. Reconstructing temporal dependence from historical forecast errors reduces the discrepancy in ramp statistics and lowers realized reserve cost by 7.4-10.6% across three forecasting tasks. These findings show that marginal forecast quality alone is insufficient for operational decisions and motivate joint evaluation of calibration, temporal structure, and decision outcomes.
★ Extending Grid Foundation Models for Phasor-Domain Dynamic Simulation
Grid Foundation Models (GridFMs) are emerging as a powerful framework for learning and transferring knowledge across power grids and grid analysis tasks. This work extends GridFMs from steady-state analysis to dynamic simulation. We present DynGridFM, which uses component-wise tokenization and a structured latent operator to learn shared dynamic representations across heterogeneous grids and enable transfer to unseen systems. Pretrained on nine grids, DynGridFM is adapted to a held-out 48-machine grid, where fine-tuning on 64 trajectories achieves accuracy comparable to training from scratch on 128. On a single GPU, it achieves 924x to 1232x speedup over numerical simulators on one CPU core.
★ An Ultra-Low-Noise Tunable TIA in 45nm Silicon Photonics for Electro-Optical Ultrasound Data Acquisition
This paper presents the design and experimental characterization of a 32-channel, WDM-compatible electronic-photonic receiver array fabricated in the GlobalFoundries 45nm monolithic silicon photonics process. Optimized for miniaturized, millimeter-scale electro-optical ultrasound data acquisition probes for broadband photo-acoustic imaging (PAI), the chip utilizes integrated micro-ring resonator (MRR) demultiplexers and photodiode (PD) arrays to decode multi-wavelength acoustic signals outside the body. This work focuses specifically on the comprehensive characterization of the receiver front-end, evaluating both the optical WDM filter bank and the transimpedance amplifier (TIA) performance. Monolithic co-integration minimizes critical photodiode-to-TIA interface parasitics. The TIA incorporates a DC-offset cancellation loop that compensates for photodiode-induced offset currents while providing tunable low-frequency cutoff control over a range of 30 kHz to 1.5 MHz. This tunability enables effective suppression of flicker noise and improves overall signal integrity. The front-end achieves the peak transimpedance gain of 144 dB$Ω$ while maintaining an ultra-low input-referred noise current density of $1\text{ pA}/\sqrt{\text{Hz}}$, demonstrating a highly scalable architecture for high-sensitivity intravascular and endobronchial imaging.
★ Sensing-Based Communication: A Non-Orthogonal Passive Reflection Framework for ISAC
While integrating uplink without disrupting the primary downlink remains a critical challenge for existing reflective communication systems, this paper proposes a non-orthogonal passive integrated sensing and communication (ISAC) mechanism termed Sensing-based Communication. Originating from and validated within an optical retro-reflective system, this mechanism possesses an underlying wave-agnostic architecture with the potential to be generalized across other physical modalities. It leverages incident downlink signals for parasitic modulation via passive tags. By relying solely on fundamental on-off keying (OOK) reflection, the tag unifies the reflective sensing channel and the uplink communication channel, and simultaneously achieves both sensing and duplex communication. Consequently, our approach introduces uplink capability without sacrificing the downlink data rate or consuming additional spectrum resources. At the receiver, we design a joint decoding architecture combining cross-correlation computation and successive interference cancelation (SIC) to fully exploit the degrees of freedom in broadband signals for multi-user interference suppression. We derive closed-form expressions for the interference and bit error rate (BER) assuming zero-mean Gaussian signals. Using an experimental testbed, we validate the pivotal theoretical finding, the scaling relationship of SINR with signal bandwidth and cross-correlation time--where linear regression yields a strong goodness-of-fit with $R^2 = 0.9688$. Furthermore, evaluations under noise-like orthogonal frequency division multiplexing (OFDM) waveforms exhibit high consistency between theoretical derivations and simulation data ($R^2 = 0.9372$, mean absolute percentage error, $\text{MAPE} = 14\%$), confirming the model's accuracy and waveform generalizability.
★ Mobility Information Capacity of Multi-Robot Systems: Safety-Aware Maneuver Design
Multiple robots operating in shared workspaces must adjust their speed, direction, and trajectories to coordinate with one another and respond to changing tasks and disturbances. The available maneuver freedom depends on the dynamics, uncertainty, and safety margins, even for a fixed fleet size and workspace. However, robot count, workspace occupancy, and task throughput do not quantify how much the fleet can vary its motion or how reliably these variations can be observed. We develop a mobility information capacity (MIC) framework that jointly characterizes these two aspects. Mobility information is defined as the mutual information between a joint maneuver sequence and its noisy trajectory observation, and MIC maximizes this quantity under maneuver-resource constraints. For a linear Gaussian maneuver-to-observation model with feedback dynamics, we characterize the finite-horizon information benchmark and derive its stationary long-horizon spectral limit, accounting for heterogeneous mobility gains, colored uncertainty, and temporally coupled maneuver costs. We further map workspace-boundary, inter-robot separation, and actuator chance constraints into coupled maneuver-covariance restrictions and derive conditions under which covariance optimization reduces to resource allocation across mobility modes. Finite joint maneuver codebooks are then constructed through individual risk screening and noise-whitened trajectory separation, with bounds linking sequence recovery error and achieved information to the capacity benchmarks. An aerial-robot simulation study evaluates spectral convergence, safety-aware allocation, finite-codebook identification, and fleet-size scaling. The resulting framework complements occupancy and task-throughput measures by quantifying the maximum observable maneuver freedom supported by a mobility system.
comment: 12-page main article and 4-page supplementary material
★ Dynamic Traffic Modeling using Equivalent Circuit Models
This paper presents an equivalent circuit model (ECM) for dynamic traffic modeling by exploiting analogies between electrical and transportation variables. Vehicle concentration is represented by capacitor voltage, while traffic flow is modeled using controllable current sources. The proposed formulation is shown to preserve the conservation and sending/receiving relationships of the spatially discretized Lighthill Whitham Richards traffic model. The framework is extended to restricted inflow, traffic signals, and two road merging and diverging scenarios. Validation against microscopic Simulation of Urban MObility (SUMO) simulations, involving traffic signals, two-road merging, and single-road diverging scenarios, demonstrates that the proposed ECM successfully reproduces the principal macroscopic traffic dynamics, including queue formation, backward congestion propagation, vehicle accumulation, saturation, and subsequent traffic recovery.
★ Agibot: An Open-Source Modular Swerve-Drive Robot Platform for Agricultural Robotics Research and Education
Agricultural production faces persistent labor shortages and increasing demand for automation, yet progress in agricultural robotics remains constrained by the limited availability of affordable, flexible, and accessible research platforms. This paper presents Agibot, an open-source, modular, low-cost agricultural robotics platform designed for autonomy research and experimentation, with potential applications in hands-on robotics education. The platform integrates four-wheel independent swerve drive, swappable computing, distributed embedded control, multimodal sensing, and configurable manipulation within a unified sense-plan-act-verify framework supporting perception, control, navigation, manipulation, and autonomous task verification. The platform was evaluated through representative row-crop navigation and manipulation tasks under varying lighting and soil conditions. Experiments demonstrated reliable lane following, autonomous row-to-row navigation, target identification, seed dispensing, plant manipulation, and closed-loop verification of task success. Its modular architecture supports independent testing, modification, and validation of autonomy components for rapid prototyping and experimentation. These proof-of-concept tasks, controller-tuning studies, and integrated trials demonstrate the platform's research capabilities, while its accessible and reconfigurable design supports its use for hands-on robotics education. By combining affordability, modularity, open-source accessibility, configurable manipulation, and end-to-end sense-plan-act-verify autonomy, Agibot provides a practical platform for agricultural robotics research, technology development, and workforce training.
comment: 14 pages. Submitted to IEEE Transactions on AgriFood Electronics (TAFE)
★ Sample-Covariance Regularization for Data-Driven Predictive Control from Frequency Response Functions under Colored Noise
In this work, we investigate how sample covariance obtained from frequency-response functions (FRFs) can be exploited to improve performance of data-driven frequency-domain predictive control (FreePC) in the presence of colored measurement noise. In case of periodic multisine excitation, this covariance is obtained directly from repeated periods and provides a frequency-dependent measure of uncertainty without requiring an additional parametric noise model. To incorporate this information into both indirect and direct predictive control, we develop a frequency-domain subspace predictive controller (FD-SPC) and establish its equivalence to the direct FreePC formulation in the noiseless setting. Regularization terms for both formulations are then motivated by expected prediction-error criteria and constructed from the sample covariance of the FRF estimates. Across all tested noise models and data sizes, regularization consistently reduces the median realized control cost relative to the corresponding controller without covariance regularization, with reductions of up to 35%.
★ Control of Linear Dynamical Systems with Operational Constraints
This paper presents a method for designing servo-controllers that enforce operational constraints on multi-input multi-output linear time-invariant dynamical systems. The proposed method introduces a control augmentation that complements a baseline controller to ensure forward invariance of state/output trajectories within prescribed box constraints. The derivations of the control design are based on the Nagumo Theorem, the Comparison Lemma, and principles from min-norm optimal control akin to the theory of Control Barrier Functions. Importantly, an explicit piecewise-linear feedback control law is derived, which provides continuous enforcement of constraints without online optimization, enabling rigorous stability and robustness analysis via classical linear systems theory. Extensions to handle estimation errors in output feedback and integral action in proportional-integral control are provided. Theoretical results are validated through flight control simulation studies that demonstrate input/output constraint satisfaction, robustness to disturbances, and closed-loop stability. The proposed method offers a mathematically rigorous, practical, and analyzable alternative to common heuristic or optimization-intensive constrained control strategies.
★ Open-Source Multi-Fidelity EMT Modeling of Data Center Loads for Power System Studies
Characterizing data center-grid interactions under rapid load fluctuations requires electromagnetic transient (EMT) models that capture fast converter and control dynamics. This paper develops a modular multi-fidelity EMT model of data center loads in PSCAD. The modular design supports flexible assembly of subsystem models for different power supply configurations. The multi-fidelity design incorporates switching models, average-converter models, and reduced-order models to balance simulation accuracy and computational efficiency according to analysis needs. The model also provides a systematic controller tuning procedure and supports control mode switching. Comparative simulations demonstrate how this data center load model responds to both grid-side and load-side disturbances and assess the accuracy and computational efficiency of models with three fidelity levels. The model is publicly available on GitHub.
★ Quantum Natural Muon: Spectral-Natural Optimization for Variational Quantum Circuits
Training parameterized quantum circuits requires optimization methods that account for two distinct structures: the information geometry of quantum states and the matrix or tensor structure induced by circuit parameterization. We introduce Quantum Natural Muon (QNM), a spectral natural-gradient framework that applies polar normalization to circuit-semantic parameter blocks after damped quantum Fisher preconditioning. This ordering is essential: quantum Fisher preconditioning and blockwise polar normalization do not commute in general, so QNM is not an order-free composition of quantum natural gradient (QNG) and Muon. We show that Frobenius-matched polar mixing preserves active singular subspaces while raising a spectral floor and improving active conditioning, and that the resulting direction remains a controlled perturbation of QNG. For its distance-aware realization (DA-QNM), we further establish alignment, descent, and nonconvex stationarity guarantees for the normalized update, together with perturbation bounds under gradient and QFIM estimation error. Experiments on variational quantum eigensolving, weighted QAOA-MaxCut, and supervised quantum classification isolate the roles of quantum geometry, circuit-semantic spectral structure, and radius control. The strongest gains arise when the spectral blocks match the circuit structure, supporting the central principle that quantum-state and structured spectral geometries provide complementary information for variational quantum optimization.
♻ ★ A Polynomial-Time Algorithm for the Minimal Input/Output Node Cardinality Disturbance Decoupling Problem over Networks via Output Feedback
For networked systems, we propose a polynomial-time algorithm for solving the minimal input/output node cardinality disturbance decoupling problem via output feedback. The algorithm computes an optimal solution, namely, a pair of minimum-cardinality sets of control input and measurement output nodes that achieve disturbance decoupling, by solving a min-cut/max-flow problem on an extended network.
comment: MATLAB functions implementing the results presented in this article are available in the GitLab repository: https://gitlab.liu.se/lucle71/node-based-ddp or in Zenodo: https://doi.org/10.5281/zenodo.23476722
♻ ★ Beyond Benchmarks: Continuous Edge Inference for Traffic Sign Recognition on Embedded Vehicular Hardware
Traffic sign recognition (TSR) systems deployed on embedded vehicular hardware face deployment effects that image-based benchmark evaluation does not capture, including frame-to-frame prediction instability in streaming video and load-dependent throughput under sustained operation. We present Edge-TSR, a continuous edge inference system for TSR on the NVIDIA Jetson Orin Nano that combines sparse detection, multi-object tracking, region-level fine-grained classification, and a lightweight track-aware temporal stabilization mechanism adding only 0.07~ms per sampled frame. Evaluating on the source videos of the INTSD benchmark, we find that the state-of-the-art TSR models lose 20-30 percentage points of mAP@50 when moving from image-based to streaming evaluation. Edge-TSR achieves the best streaming classification performance among all evaluated baselines, and temporal stabilization improves accuracy by 10.58 percentage points over per-frame inference. We characterize latency, throughput, and thermal behavior across sampling intervals and four driving conditions, showing that throughput is load-proportional (20 ms per classified region) and that every-frame inference is limited by throughput rather than thermal throttling. A 55-minute, 26 km vehicular deployment demonstrates sustained online operation at 16.18 FPS within safe thermal limits without cloud offload. We also release the annotated 50,732-frame streaming evaluation dataset and full implementation.
♻ ★ A Comprehensive Comparison of Reinforcement Learning Algorithms in Autonomous Driving
This paper presents a comprehensive comparison of four representative Reinforcement Learning (RL) methods in autonomous vehicle (AV) decision-making, including Fitted Value Iteration (FVI), Rollout Classification Policy Iteration (RCPI), Deep Q-Network (DQN), and a discrete-action adaptation of Deep Deterministic Policy Gradient (D-DDPG). The four methods are applied to the same AV decision-making problem at an unsignalized intersection and are compared in terms of optimality, safety, training stability, and training and decision cost. Observations suggest that RCPI provides the closest estimation to the exact solution and has the most stable training curve; however, it is data-intensive and requires the largest volume of data. D-DDPG is the most data-efficient one, but at the price of the training stability. Based on the observations, we provide suggestions for the algorithm selection for AV decision making.
♻ ★ Can sunlight replace LEDs in vertical farms? A critical assessment of optical-fiber daylighting strategies
Vertical farming is a promising approach for increasing food production in controlled environments while reducing land use, water consumption, and dependence on external climatic conditions. However, its large electricity demand, mainly driven by artificial lighting, still limits its energetic and economic sustainability. This study evaluates the techno-economic potential of an optical-fiber daylighting system designed to transport concentrated sunlight directly to the crop growing zones of a vertical farm. A validated transient vertical farm model was coupled with Tonatiuh ray-tracing simulations to assess a single-axis solar-tracked Fresnel concentration system. Five optical fiber typologies and three daylighting strategies were compared in terms of crop production, specific electrical energy consumption, and light cost, and benchmarked against light-pipe and rooftop photovoltaic solutions. The optical-fiber system achieved an annual useful-PAR delivery efficiency of 19-27%, depending on fiber typology. By distributing daylight across all rack shelves and limiting heat gains through a UV-IR filter, it reduced electricity consumption by 35-89%, compared with 12% for the light-pipe configuration. The daylight-only strategy achieved the lowest SEEC, equal to 1.81 kWh kg^-1, but caused a 52% crop productivity reduction. Hybrid LED-optical-fiber strategies avoided this penalty and achieved electricity savings of about 50%, with a minimum SEEC of 3.16 kWh kg^-1. However, current optical fiber costs make the system economically unattractive, requiring an estimated capital cost reduction of about 90% to become competitive. Rooftop photovoltaics covered 22% of annual electricity demand and achieved an 11-12-year payback, making PV the most economically viable rooftop solar option under current market conditions.
♻ ★ A Realistic Discrete Event Simulation model for Ambulance Location and Deployment within a regional Emergency Medical Service
The objective of Emergency Medical Services (EMSs) is to promptly respond to calls from citizens for first aid, providing pre-hospital care and, if necessary, to transfer patients to an appropriate Emergency Department (ED) by ambulance. The efficiency of such a system strongly depends on the deployment of ambulance home bases, i.e., locations where ambulances and their crews are strategically positioned, ready to respond to emergency calls. This paper presents a general Discrete Event Simulation (DES) model designed to capture the stochastic behaviour and workflow of regional ambulance emergency systems. The proposed model incorporates and integrates information collected from different sources, reproducing very accurately the operation of the ambulance system, thus allowing a more comprehensive and realistic analysis. To show the applicability and reliability of the proposed general model, a case study provided by the Azienda Regionale Emergenza Sanitaria - ARES 118 (an Italian Regional Emergency Medical Services Authority - ARES~118}) is presented. It concerns a territory within the Lazio region of Italy, including a medium-size city along with sparsely populated areas. The reported results about scenario analyses highlight how the model we propose can be fruitfully used by the managers to improve effectiveness and quickness of the entire regional EMS system.
comment: 41 pages, 12 figures
♻ ★ Dynamic Centrality Measures for Water Distribution Network Hydraulics
Water distribution networks (WDNs) are susceptible to various failures, including but not limited to human errors, cyber-attacks, and network modifications, necessitating the vulnerability analysis of WDNs. Graph-theoretic centrality measures---as a main class of centrality measures---aim to rank the network components solely based on their influence (i.e., criticality) on the WDN topology in the case of input changes, while overlooking the WDN dynamics. To overcome such a limitation, this paper uses a control-theoretic centrality measure to identify the network's most and least influential pipes in WDNs, by simultaneously incorporating the dynamics and topology of the WDN. First, given a WDN modeled by nonlinear differential-algebraic equations (NDAEs) consisting of transient flow dynamics as differential equation (DE) and conservation of water mass as algebraic equation (AE) and considering the pipe flow rates as states of the state-space (SS) representation, we extract a linearized system modeled by linear ordinary differential equations (LODEs) around the equilibrium (i.e., steady) flow rate vector. Second, treating pipe flow rates as SS nodes (states), we introduce a node centrality-based measure, namely \textit{vulnerability vector (VV)}, to rank the network pipes based on their influence on the dynamics and topology of the WDN in the case of input changes. In particular, the network's most and least influential pipes can be identified through such a centrality-based approach. This enables water engineers to understand the WDN's vulnerability better and effectively prioritize the maintenance and operational efforts on the most influential pipes within the WDN.
♻ ★ RADAR Perception for Dynamic Obstacle Avoidance onboard small-scale Quadrotor UAVs
Fast dynamic obstacle avoidance (DOA) on uncrewed aerial vehicles (UAVs) demands not only low-latency control and actuation but also reliable, long-range perception to detect obstacles and estimate their motion in time to maneuver. We present, to the best of our knowledge, the first mmWave RADAR-based perception-and-control system for fast onboard DOA. We derive and analyze latency and spatial bounds that relate sensing range, relative speed, and control delay, yielding sufficient conditions for successful avoidance. We further extend prior latency analyses to include the decision latency of the tracker and trigger to evaluate them per trial. Our computationally efficient system combines a lightweight interacting multiple models (IMM) tracker with a control-barrier functions (CBF) controller that directly outputs evasive accelerations. In 300 throw experiments spanning three object sizes and varying visibility (light and dark), it achieves mean position errors of at most 0.15 m, 0.93m, and 0.87m in the x, y, and z directions, with a similar spread in 90 experiments in smoke. An onboard implementation on a Raspberry Pi 4B demonstrates real-time feasibility with an end-to-end processing latency of approximately 14ms per RADAR frame and a system latency of 75ms from first detection to the avoidance command. The UAV successfully avoided obstacles in all 20 tethered indoor flights and all 39 untethered outdoor avoidance maneuvers across three ball sizes, during both hover and forward flights at up to 5m/s with measured closing speeds up to 14.6m/s.
comment: Preprint. This work has been submitted for possible publication and is currently under review
♻ ★ Data-Driven Continuous-Time LQR: Linking Reinforcement Learning and Behavioral System Theory
Behavioral system theory and reinforcement learning (RL) have independently developed data-driven approaches to the continuous-time LQR problem, yet their respective methods have largely evolved in separation. We connect these two adjacent lines of work by studying their data-driven parameterizations, namely a closed-loop (CL) parameterization from behavioral system theory and an integral RL (IRL) parameterization. The CL parameterization characterizes the closed-loop system via a matrix that satisfies equality constraints. While this parameterization has been extensively studied for discrete-time systems, we adapt key results to the continuous-time setting and develop a policy iteration (PI) scheme, derive an algebraic Riccati equation, and introduce an alternative convex program. The IRL parameterization utilizes off-policy data to perform policy evaluation, which is used for PI or value iteration. Within the IRL framework, we derive a policy gradient flow and propose convex programs for the LQR problem. Finally, we provide a unified treatment of these parameterizations that enables a systematic understanding of existing approaches, and clarifies their structural relationships.
comment: Submitted to IEEE TAC
♻ ★ Safety Blind Spot in Remote Driving: Considerations for Risk Assessment of Connection Loss Fallback Strategies
As part of the overall goal of driverless road vehicles, remote driving is a major emerging field of research of its own. Current remote driving concepts for public road traffic often establish a fallback strategy of immediate braking to a standstill in the event of a connection loss. This may seem like the most logical option when human control of the vehicle is lost. However, our simulation results from hundreds of scenarios based on naturalistic traffic scenes indicate high collision rates for any immediate substantial deceleration to a standstill in urban settings. We show that such a fallback strategy can result in a SOTIF-relevant hazard, raising the question whether this design decision is acceptable. Therefore, from a safety perspective, we call this problem a safety blind spot, as safety analyses in this regard seem to be very rare. In this article, we first present a simulation on a naturalistic dataset that shows a high probability of collision in the described case. Second, we discuss the severity of the resulting potential rear-end collisions and provide an even more severe example by including a large commercial vehicle in the potential collision.
comment: This work has been submitted for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible
♻ ★ Voltage Sensitivity-Based Fair Allocation of Operating Envelopes in Distribution Networks
Operating envelopes (OEs) define the admissible active and reactive power injections of distributed energy resources while respecting distribution network limits. In unbalanced low-voltage feeders, OEs based only on voltage magnitude and thermal limits may overestimate available flexibility because they neglect power quality requirements. This paper develops a three-phase unbalanced AC optimal power flow framework to compute coupled active and reactive power OEs while enforcing limits on the voltage unbalance factor (VUF). It also proposes a fairness mechanism based on voltage sensitivities to allocate available flexibility among multiple photovoltaic (PV) units. The proposed mechanism is compared with lexicographic max--min fairness as a benchmark for equitable resource allocation. Case studies on unbalanced test feeders evaluate how VUF constraints change the feasible operating region of PV units. The results show how VUF constraints affect feasible flexibility and demonstrate that the proposed voltage sensitivity-based mechanism is nearly six times faster than lexicographic max-min fairness.
♻ ★ Probabilistic Sequential Safety for Stochastic Dynamical Systems
In this letter, we investigate a notion of probabilistic safety for a stochastic dynamical system. Standard safety notions are based on the idea that an unsafe region of the state-space must be avoided at each instant. In contrast, we study a sequential safety notion which is based on avoiding a tuple of target regions of the state-space. The proposed safety notion is inspired by many practical systems, ranging from modeling jailbreak attacks in large language models (LLMs) to predicting hazardous chemical reactions. In solving the sequential safety problem, we solve a sequential reachability problem for a stochastic dynamical system.
♻ ★ An Enhanced Submodule for Modular Multilevel Converters With DC Fault-Clearing Capability
Modular multilevel converter (MMC) has been successfully applied in various power electronic systems owing to its high efficiency, scalability, and superior output performance. Although the half-bridge submodule (HBSM) is widely used in MMCs for its structural simplicity, it is incapable of handling direct-current (DC) short-circuit faults. The diode-clamp submodule (DCSM) addresses this limitation by providing DC fault ride-through capability. However, because its two identical capacitors are connected in series, the equivalent capacitance is halved. To overcome this drawback, an enhanced SM is proposed in this paper. For the same energy storage capacity, the proposed SM reduces the total required capacitance by 75% compared with the DCSM. In addition, the proposed SM requires one fewer diode than the DCSM, thereby lowering the overall MMC capital cost. The topology and the operating modes of the proposed SM are described in detail, and its functionality is experimentally validated. The results demonstrate that the proposed SM can suppress DC fault currents and restore normal operation without additional external protection devices.
comment: 9 pages, 17 figures
♻ ★ Defense Filters for Robust Learning of Partially Observed Dynamical Systems
Robust estimation of unknown system dynamics from partial observations under attack is a core challenge in learning for control, increasingly critical in multi-agent networks subject to adversarial data poisoning. Existing methods either require highly restrictive assumptions on the adversary or yield weak error bounds. In this paper, we consider identifying partially observed linear dynamical systems subject to adversarial attacks with an attack probability $p < 0.5$ at each time. We propose a defense-filtering framework leveraging knowledge of the observation matrix to construct a defense filter satisfying full column rank and subspace separability conditions, which enables the evaluation of a single-attack scenario against observed data. Using this framework, we analyze the constrained $\ell_2$-norm estimator and establish finite-sample error bounds for recovering the Markov parameter matrix up to a linear transformation. This is the first approach to provide provable system identification guarantees via a defense filter for partially observed systems subject to harsh environments where the attack probability approaches $0.5$.
comment: 8 pages, 1 figure
♻ ★ A Measure-Theoretic Formulation of Behavioral Systems
Ensemble control specifies probability laws on admissible trajectories, including their temporal dependence. Starting from an arbitrary closed behavior, we organize its laws into a framework that connects standard probability-measure geometry, occupation-measure control, and data-driven trajectory parametrization. Classical convexity and extreme-point properties describe the feasible laws, while classical Bellman duality governs additive control objectives. On compact state and input spaces, full-path constrained duality for continuous trajectory functions gives primal attainment and zero gap whenever feasible. For controllable linear time-invariant systems, the noise-free Fundamental Lemma yields an equivalent law-level Hankel factorization, with a canonical pseudoinverse lift. The factorization transfers trajectory-law constraints exactly to coefficient space and connects them to covariance control. Explicit noisy-Hankel bounds quantify subspace approximation for second-moment-bounded laws under known rank and Gaussian output measurement noise. Numerical studies include a nonlinear reset problem whose nonadjacent correlation constraint requires temporal memory, covariance-constrained optimization, and noisy-data policy evaluation. Known transition kernels provide a further structural perspective through history-conditional consistency.
comment: 44 pages, 4 figures. Single-column manuscript
♻ ★ Learning-Oriented Fault-Tolerant Control for UAVs with Time-Varying Fault Estimation
The increasing deployment of unmanned aerial vehicles (UAVs) in safety-critical applications has created a growing demand for control strategies that remain reliable in the presence of actuator and sensor faults. This paper presents a fault-tolerant control framework that combines constructive mathematical modeling, proportional--integral (PI) fault estimation, and learning-oriented control. A nonlinear quadcopter model is first developed using both the Newton--Euler and Euler--Lagrange formulations, from which a discrete-time linear model is derived for observer and controller design. A PI observer is then proposed to estimate actuator and sensor faults, with exponential estimation-error convergence established for constant faults in the absence of disturbances and ultimate boundedness established for slowly time-varying faults under bounded disturbances. The estimated fault signals are incorporated into an active fault-tolerant control architecture together with a Deep Deterministic Policy Gradient (DDPG) controller, which is trained using only the yaw channel and subsequently applied to improve closed-loop tracking performance. Numerical simulations illustrate the observer's estimation performance under constant and slowly time-varying faults, including bounded disturbances, and the tracking performance of the integrated controller under multiple actuator and sensor fault scenarios. The proposed framework combines model-based fault estimation with learning-based control to improve fault accommodation and tracking performance in UAV systems.
comment: This is an extended version of the paper published in the proceedings of IEEE CoDIT
♻ ★ Lipschitz-Guided Machine Learning Framework for Accelerating Transient Stability Analysis of Flexible Traction Power Supply Systems
The rapidly varying and moving train loads in electrified railways impose stringent requirements on the transient stability of flexible traction power supply systems (FTPSSs). However, the complex dynamic interactions among numerous traction substations (TSs) make it challenging for existing transient stability analysis (TSA) methods to achieve both high accuracy and computational efficiency. To bridge this gap, this paper proposes a Lipschitz-guided machine learning framework that leverages Lipschitz continuity to restructure the transient stability verification of FTPSSs. By replacing computationally intensive satisfiability modulo theories (SMT) solvers with a deterministic algebraic check on a finite mesh, the proposed method ensures the Lyapunov conditions between mesh points without simplifications of the FTPSS dynamics. Validated on FTPSSs with three to five TSs through extensive case studies, the proposed framework accelerates training by up to 12 times compared with SMT-based neural Lyapunov methods and certifies the five-TS system. Furthermore, the proposed framework achieves up to a 33.5-fold increase in the estimated region of attraction (ROA) compared with linear matrix inequality.This advancement provides an efficient TSA solution for droop-controlled FTPSSs.
comment: 10 pages, 9 figures
♻ ★ Governing Well in the Algorithmic Age: The Foundations of Digital Statecraft
The digital substrate - data, algorithms, infrastructure, platforms, applications - is being governed without adequate conceptual foundations. The ability and legitimacy required to govern this substrate, and to govern with it, are simultaneously misaligned, contested, and structurally absent. We introduce digital statecraft as the organising concept for this emerging field, arguing that 'digital' reconstitutes the statecraft question rather than merely extending its domain. The concept operates on two dimensions - statecraft over digital systems, concerning the authority and capacity of the state in relation to the digital substrate itself, and statecraft with digital systems, concerning the deployment of algorithmic tools as instruments of governing authority. And it rests on two foundational requirements, technical coherence and legitimate authority, that are genuinely in tension. We derive ten principles of digital statecraft from these foundations, each naming a condition whose absence produces an identifiable and structural governance failure: public interest first, human-machine complementarity, governability by design, systemic coherence, hybrid institutions, adaptive governance, human centricity and civic agency, accountable and traceable authority, judgment across time, and the non-delegable core. This article takes the state as the starting point, the institutional form that developed historically in response to the problem of effective and legitimate public governance, and the only current candidate for which the full set of legitimacy conditions is institutionally available. But the digital statecraft programme holds open a deeper question than just whether states can reform themselves: governing well in the algorithmic age may require rethinking the boundaries, scale, and affiliative basis of statehood itself.
comment: 27 pages
♻ ★ Scalable Optimization for Mobility-Aware Coordinated Electric Vehicle Charging in Power Distribution Networks
Rapid growth in electric-vehicle (EV) charging demand is placing increasing stress on power distribution networks (PDNs), whose hosting capacity is often limited and spatially uneven. Beyond demonstrating that coordination can help, this paper answers an open question that is central for planners: What is the maximal achievable benefit of EV charging demand flexibility from spatial and temporal shifting in reducing overload-driven distribution upgrades at a regional scale? We introduce MAC (Mobility-Aware Coordinated EV charging) to establish a credible upper bound, which entails rethinking charging flexibility around individual mobility, fusing travel itineraries with feeder-level hosting-capacity data, and solving population-scale optimization with spatio-temporal coupling to certified near-optimality. (i) MAC expands feasible scheduling by coupling charging decisions over the full mobility horizon. Instead of enforcing per-session energy recovery, it only requires the EV state-of-charge (SOC) to remain sufficient for upcoming trips. (ii) MAC is computationally scalable via an iterated price response (IPR) scheme. Each iteration posts a locational-temporal price, collects the fleet's best responses in parallel, and updates the price from the observed capacity shortage. Custom batched subproblem solvers remove the per-iteration bottleneck of solving millions of best responses. In a future-oriented 30% EV adoption scenario for the San Francisco Bay Area, MAC almost eliminates overload-driven upgrade needs relative to unmanaged charging. Comparing across the baseline spectrum, mobility-aware flexibility alone removes most of the overload, outperforming even fully coordinated session-based charging, and coordination on top suppresses most of the remainder. Both levers are thus essential, and the resulting best-case benchmarks provide references for PDN planning and operations.
comment: 20 pages, 8 figures, 3 tables
♻ ★ Interconnections of Dissipative Networks Through a Dynamic Scattering Controller
The problem of modular scattering-based design of dissipative networks is addressed. Under basic assumptions imposed on subsystems, design of a dynamic scattering-based interconnection controller is proposed which guarantees that the resulting interconnection is internally stable and possesses the same set of basic properties as the individual subsystems. This enables iterative application of the developed technique, which allows for building of large-scale dissipative networks in a modular fashion. The proposed method provides substantial flexibility in the controller design as well as in the choice of input and output signals used for interconnections.
comment: 17 pages, 10 figures
♻ ★ Conformalized Data-Driven Reachability Analysis with PAC Guarantees
Data-driven reachability analysis computes reachable sets directly from noisy data. Existing matrix-zonotope reachability methods require a known noise bound, and their nonlinear extensions require a Lipschitz constant. We propose Conformalized Data-Driven Reachability, which calibrates the open-loop prediction error of a least-squares model with training-conditional split conformal prediction and propagates the initial set and the input set with zonotopes. The resulting sets contain the whole trajectory with a probably approximately correct (PAC) guarantee that requires only independent and identically distributed trajectories, and the number of calibration trajectories does not depend on the horizon. The prediction error depends on the initial states and the controller only through the model error. The method extends to measurement noise with a known quantile and to nonlinear systems through local affine models. In experiments, the sets keep an average coverage above 99% when the initial states and the controller change, whereas sets calibrated directly on the states lose coverage.
Robotics
★ OopsieRecover: Learning Transferable VLA Corrections from Semantic Interventions
Vision-Language-Action (VLA) models can perform a wide range of robotic manipulation tasks, but often fail during deployment. Teaching them to recover from these failures requires substantial corrective data: existing approaches learn to recover from failures by directly associating corrective actions with the observations in which they were demonstrated. For instance, DAgger-like fine-tuning can require tens to hundreds of interactions, yet the resulting policy may still fail to generalize the learned corrections beyond the demonstrated scenarios. We hypothesize that semantics can capture the underlying reasons for failures and their corrections, enabling policies to learn from only a handful of corrective demonstrations and generalize those corrections to novel objects, and scene variations. We propose OopsieRecover, a framework that uses abduction to infer the intent behind a small set of corrective demonstrations and encodes the inferred knowledge as dense spatial representations and conditional costs. These costs capture when a behavior is undesirable and steer the policy toward more desirable action regions. We experiment across 6 tasks and variations in object poses, scenes, and related tasks, and find that only a handful of corrective demonstrations can enable a VLA to overcome failures across a broad range of novel situations, without requiring new corrective data for each variation. Please refer to https://robin-lab.cs.utexas.edu/oopsierecover for videos.
comment: 9 pages, 5 figures
★ Latent RL Priors for Data-Efficient Dexterous Manipulation
Learning dexterous manipulation from limited visual demonstrations is challenging because policies must acquire both perceptual grounding and precise control, while reinforcement learning (RL) in simulation can acquire rich control behaviors from large-scale interaction with privileged information. We introduce \textbf{Latent RL Prior}, a two-stage framework that reuses this control knowledge for data-efficient visual policy learning. During privileged pretraining, we distill large-scale RL rollouts into a flow-matching action decoder conditioned on proprioception and latent task context. During visual adaptation, we freeze the decoder and learn a visual interface that maps images to its latent context using action supervision alone, without privileged labels or latent targets. The same decoder can further be shared across arms by learning coordinated visual contexts for bimanual manipulation. Across three simulated tasks, our method outperforms full-parameter fine-tuning of $π_{0.5}$ and GR00T N1.7 in the low-data regime; on door lifting and bimanual assembly, it surpasses full-data baselines using $10\times$ and $5\times$ fewer demonstrations, respectively. Further analysis shows that direct visual-to-latent conditioning improves adaptation performance and training stability over routing visual predictions through the pretrained state encoder. Robustness to unseen mass variation and greater linear accessibility of contact-force information in decoder representations further provide evidence of retained RL-acquired control knowledge. Project website: https://yixiaowang7.github.io/latent-rl-prior-page/.
★ VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
Models for physical AI tasks, such as robotics and autonomous driving, need not reason solely in language. They can use visual plans--points, bounding boxes, and trajectories--to represent spatial goals and anticipate motion. However, autoregressive vision-language models generate these plans one coordinate token at a time, making richer predictions increasingly costly. We introduce VFlash, a method for fast, accurate visual planning through Flash Primitive Decoding. VFlash exploits the known structure of visual primitives to predict their coordinates jointly, using a lightweight drafter that generates an entire primitive in one forward pass. The layout adapts to the primitive's type, dimensionality, and length, supporting compact boxes and long, variable-length trajectories. Crucially, spatial accuracy need not require reproducing an autoregressive model's exact token sequence. VFlash therefore trains on annotated primitives and directly accepts structurally valid predictions without target-token agreement. Across the evaluated 2D and 3D robot trajectory tasks, VFlash consistently reduces prediction errors relative to autoregressive decoding. Preliminary single-rollout measurements indicate approximately 25x faster generation for 3D point motion forecasting. A world action model conditioned on VFlash plans achieves 70.2% average success across four MolmoSpaces task suites, compared with 62.6% for the baseline.
★ Failure Prediction and Classification in Vision-Language-Action Models Via Calibrated Latent Probing
Vision-Language-Action models (VLAs) enable general-purpose embodied intelligence, but their performance is brittle under distribution shift, where unanticipated failures can damage equipment or endanger people. Deploying VLAs in real-world systems therefore requires a runtime monitor that is timely, calibrated, and informative while being efficient enough to run on the edge. Existing methods fail to fulfill these requirements. We present VLA-Tug, a lightweight probe over the latent layers of a VLA backbone that improves early failure detection and bounds the missed-detection and false-alarm rates, with minimal inference overhead. We show that supervising on the pre-failure segment of a trajectory increases early failure detection at matched false-positive rates by up to 30% over the strongest learned baseline. Under shift to unseen tasks, the bounds on missed-detection and false-alarm rates remain valid in up to 86% of runs, versus at most 21% for baselines. Unlike existing methods, VLA-Tug predicts the failure mode to enable corrective actions, raising top-3 accuracy on unseen tasks by up to 44% over a majority baseline. VLA-Tug enables real-time monitoring where the well-timed and informative probe supports lightweight steering that improves task success. We release a dataset of 3,180 annotated manipulation episodes to support future work on latent failure detection in VLAs at https://vla-tug.github.io.
comment: Code, videos, and dataset available at https://vla-tug.github.io
★ T3R: Training-Free Two-Stage Token Refinement Towards Efficient and Robust VLA Models
Vision-Language-Action (VLA) models enable end-to-end robot control, but inference is increasingly expensive. Each decision processes hundreds of visual tokens, often across multiple cameras, leading to substantial computation while allowing irrelevant background content to distract action prediction under distribution shift. We argue that these issues stem from two complementary forms of redundancy: many visual tokens are spatially irrelevant, while attention over the remaining tokens is often misallocated. Based on this observation, we propose T3R, a Training-Free Two-Stage Token Refinement framework. In the first stage, segmentation-guided token pruning removes irrelevant visual patches to reduce computation and eliminate distracting context. In the second stage, saliency-guided attention biasing reallocates attention toward action-critical visual regions and language instructions within the retained tokens. Together, pruning produces a cleaner candidate set, while attention biasing further emphasizes the most decision-relevant evidence, yielding a better accuracy-efficiency frontier than either stage alone. On LIBERO-Spatial, T3R removes 85% of scene-camera visual tokens while essentially matching the unpruned success rate, and improves success by up to 7 percentage points under out-of-distribution object-placement perturbations. The timing comparison attributes 7.8 ms per inference call to pruning and attention-bias application. As input resolution increases, LLM-forward profiling shows up to a 3.3x speedup over the unpruned baseline. Evaluations across LIBERO, SIMPLER, and RoboTwin with three VLA backbones show the strongest accuracy-efficiency benefits in multi-camera settings. These results demonstrate that training-free token refinement is a practical approach to efficient and robust VLA inference.
★ Social Humanoid: Socially Responsive Human-Humanoid Interaction
Humanoid robots are typically trained as solitary agents, leaving them unable to coordinate physically with human partners who naturally anticipate and adjust to one another. To bridge this gap, we present Social Humanoid, a hierarchical system that generates and executes socially responsive whole-body motion on a 29-DoF Unitree G1 conditioned on a human partner's motion. Its high-level policy performs flexible sequence modeling of human and humanoid motion through a shared tokenized representation, generating reactive humanoid motion and anticipating the partner's motion when they are occluded, while a low-level whole-body tracking controller executes the generated motion on the robot. To train the system, we introduce a two-phase adaptive retargeting procedure that retargets one interaction partner to the robot while morphing the other partner to preserve contact dynamics, reachability, and relative body scale. We apply this procedure across five motion-capture datasets to produce a new Human-Humanoid Interaction (HHI) benchmark of 260 hours of human-humanoid interactions. Our results demonstrate that Social Humanoid produces coordinated, human-aligned behavior across diverse interaction types in simulation and the real world. Project page: https://von31.github.io/social-humanoid/
comment: CoRL 2026, Project page: https://von31.github.io/social-humanoid/
★ A Multimodal Haptic Interface for Simultaneous Softness and Thermal Modulation
Haptic interactions, such as touching an object in virtual reality, involve detecting a plethora of sensations through the skin. Whilst tactile displays exist to simulate and manipulate all of these cues individually, little progress has been made constructing devices that can control multiple cues simultaneously. This paper presents a tactile display technology that offers both softness and temperature change. A prototype display, based on particle jamming and the thermal transfer of heated and cooled water, is evaluated in terms of its ability to change both stiffness and temperature. Mechanical evaluations demonstrate controllable stiffness increase of 300%, whilst thermal characterisation showed strong surface uniformity and a range of achievable temperatures from 10°C up to 50°C, with closed-loop control maintaining a 0.2°C error from a setpoint. This work advances the design of multimodal tactile displays with applications to perceptual research, medical simulation, and affective haptics.
comment: Published at EuroHaptics 2026
★ SCALE: Real-Time Latent Communication for Cyber-Physical Agents
Latent communication enables agents to collaborate efficiently by directly exchanging intermediate model representations rather than raw sensor data or text messages. Existing approaches, however, are primarily designed for LLM agents and do not consider the bandwidth constraints and heterogeneous task deadlines faced by cyber-physical agents. In this work, we propose SCALE, a real-time latent communication method tailored for cyber-physical agents. Motivated by the spatio-temporal continuity of the physical world, SCALE matches tokens across consecutive latent messages using similarity scores to identify reusable information and avoid redundant transmission. It then profiles the semantic gain and communication cost of transmitting different amounts of latent information. Based on these profiles, SCALE employs a real-time scheduler that jointly determines which pending messages to transmit and how many latent tokens to send, subject to available bandwidth and task deadlines. We evaluate SCALE across three cyber-physical benchmarks and a physical spider robot case study under diverse network conditions and heterogeneous task deadlines. Our results show that SCALE significantly improves both task performance and on-time delivery in bandwidth-constrained networks. Our code is available at: https://anonymous.4open.science/r/SCALE-4E80/.
★ SLA: Scaling Spatial-Language-Action Model via Continual Pre-training
Reliable robotic manipulation requires both semantic understanding and precise control in three-dimensional space. However, vision-language-action (VLA) foundations largely inherit visual representations from 2D vision-language pre-training, while spatial supervision is often introduced only during task-specific adaptation. Developing spatial understanding as a transferable foundation-model capability therefore calls for learning from broader robot experience. We present SLA, a Spatial-Language-Action Model that scales implicit spatial alignment to continual pre-training and maintains it during downstream post-training. Initialized from a pretrained pi05 policy, SLA jointly optimizes action prediction and alignment between intermediate visual embeddings and spatial representations from a frozen 3D perception model. We curate and unify heterogeneous robot datasets to support spatial learning across diverse embodiments, tasks, and scenes. To accommodate this diversity, we introduce feature-dependent gates that adaptively aggregate visual embeddings from multiple VLA layers for spatial supervision. The teacher further exploits partially available depth and camera calibration alongside RGB observations, enabling effective use of heterogeneous geometric annotations. Probabilistic state dropout encourages reliance on visual cues and mitigates overfitting to demonstration trajectories. All auxiliary components are used only during training, allowing direct reuse of pretrained VLA weights and preserving the original inference architecture.
comment: Project page: https://SLA.github.io/ ; Github: https://github.com/OpenHelix-Team/Spatial-Forcing/tree/master/SLA
★ MOMASPLAT: Dynamic Novel-View Demonstration Augmentation for Mobile Manipulation
Visuomotor policies trained through imitation learning are highly sensitive to changes in camera viewpoint. This is particularly problematic for mobile manipulation (MoMa), where robots rarely execute manipulation from the exact same base pose used during demonstration collection due to navigation uncertainty, localization drift, and environmental constraints. Recent approaches use novel-view synthesis (NVS) to augment demonstrations with unseen viewpoints, but they assume static scenes and therefore fail to re-render the manipulated object during the post-contact interaction phase, leaving the most critical parts of the demonstration unaugmented. We present MOMASPLAT, a novel-view demonstration augmentation framework for mobile manipulation that synthesizes full manipulation trajectories from unseen robot base poses, including dynamic object interactions. Our key insight is that standard MoMa data collection naturally provides the multiview observations needed to reconstruct both the static scene and the manipulated object in 3D. MOMASPLAT combines 3D Gaussian Splatting, feed-forward 3D foundation models, and kinematics-based action retargeting to separately reconstruct the scene and manipulated object, render them from novel viewpoints, and generate geometrically consistent synthetic demonstrations paired with transformed actions. We evaluate MOMASPLAT on five real-world mobile manipulation tasks involving both rigid and articulated objects across three scenes, and validate the fidelity of its synthesized demonstrations against ground truth in simulation. MOMASPLAT substantially improves robustness to unseen base poses, more than doubling the average success rate of the strongest baseline and maintaining non-trivial success at large base displacements where all baselines fail, while reducing the need for largescale real-world data collection.
comment: Project page: https://robin-lab.cs.utexas.edu/momasplat
★ VisionSIM: Beyond Idealized Sensors - A Multi-Modal Simulator for Robot Perception
Robotic perception increasingly relies on heterogeneous sensor suites combining cameras with emerging modalities (event, SPAD, iToF/dToF, and thermal). Yet existing simulators idealize these sensors, hiding critical modality-specific failure modes that limit real-world deployment. We present VisionSIM, a unified, physically grounded simulation platform that elevates sensor realism to a first-class concern by modeling the raw physical measurement process for diverse conventional and emerging modalities. We expose fine-grained control over physical sensor parameters, reproducing failure modes overlooked by general-purpose simulators: pileup and glare for LiDAR; phase wrapping, saturation, and motion artifacts for iToF; refractory drops for event cameras; and history-dependent emission for thermal cameras. This enables principled cross-modality comparison on identical scenes and trajectories. We release the platform and a large indoor dataset with synchronized raw measurements and rich ground-truth annotations.
★ SHORE: Stability-Hybrid On-Policy Reinforcement Learning for Online Bin Packing
Online 3D bin packing with a processing buffer requires choosing which box to pack, its orientation, and its placement while keeping the stack stable under unknown future arrivals. A collapse ends the episode and forfeits the remaining volume. Existing methods use geometric rules or hard stability masks, limiting how risk can influence the joint choice of box, orientation, and placement. We propose Stability-Hybrid On-Policy Reinforcement Learning (SHORE), which combines a learned stability probability of a placement with an on-policy estimate of its return conditional on stability. A 3D U-Net learns stability from executed outcomes and simulated alternatives, revealing outcomes of actions underrepresented in early rollouts. A discrete on-policy actor--critic uses stability predictions in both its action values and policy prior, with gates that limit the effect of an untrained or changing model. In deployment, SHORE does not require additional search or simulation overhead. In Isaac Gym, SHORE reaches 73\% space utilization by 0.4M environment steps, while the same learner without stability modeling reaches that level at about 1M. On the expanded 60-box set, SHORE achieves 0.734 space utilization versus 0.588 for the GOPT state-of-the-art baseline, with a higher placement-failure frequency (1.03\% vs. 0.63\%), trading more risk for greater packed volume. The results show faster learning and higher utilization on the more demanding realistic box arrival set. Ablations suggest that simulated alternatives provide useful early supervision, while executed outcomes alone sustain later performance. We also demonstrate the method on a real robot to verify the simplicity of integrating the method to real bin packing pipelines.
★ Radio-to-Pose Information for Gauge-Drift Correction in Cooperative Localization
Cooperative localization uses odometry and inter-robot measurements to estimate the motion of a robot team. These measurements describe only relative geometry, but the team's estimated position and orientation can drift together over time, i.e., gauge drift. In this work, we study correction of this gauge drift using a radio attenuation map registered in the same world coordinate frame used for robot poses. Specifically, we propose Radio-to-Pose Information (RPI), an interface that converts power residuals and their pose derivatives into a Gaussian information matrix and vector pair. RPI builds on the insight that received power is sensitive to shared displacement in the world map. The pose estimator uses the interface together with odometry and inter-robot measurements to update poses and minimize uncertainty. We characterize the shared motions detectable from the map and observed links. For an uncertain shared map, RPI keeps correlations across robots and time and reproduces joint Gaussian pose estimates and covariances. Across $1000$ simulated Gaussian trials, RPI matches the results of joint estimation. Across 32 paired trials with a registered map, RPI reduces common-position RMSE from $0.542$m to $0.140$m in a $50\times50$m arena compared with a baseline that uses radio only to weight relative measurements.
★ USB Gym: A Gymnasium Testbed for Closed-Loop Ultrasound View Matching
Background and Objective: Our main goal is ultrasound guidance: moving a probe until its image matches a target view. Published methods for each part of a guide are evaluated in separate simulators. We asked whether these methods, combined into one guide, are good enough. Methods: We built USB Gym, a testbed whose standard engine simulates ultrasound from public patient CT, with a network drawing organ textures from CT labels. Its guide has three replaceable parts (slots), an organ segmenter, a controller and a reward, which received 20 published methods, each run on the same 60 episodes of six held-out patients. In each slot, a published method replaced ours only if detectably better on these patients. The better of this combination and our original guide on 20 new CTs is the benchmark baseline. A pose oracle, reading the exact probe pose available only in simulation, set the reference. The results below are exploratory. Results: On these CTs, the probe started on average 16.0 mm from the target view; the pose oracle brought it to 1.5 mm. Of the results known when the combination was fixed, only an image-similarity reward (SSIM) was slightly but detectably better, so the combination was our guide with that reward. This combination was the benchmark baseline, ending at 13.7 mm (16% of the oracle's progress). Conclusions: Combining untuned published methods with ours did not guide the probe near the target view; the task remains under-explored. The testbed accepts new methods; no guide was tested on hardware.
comment: 75 pages; supplementary material (62 pages) as an ancillary file. Submitted to Computer Methods and Programs in Biomedicine. Code: https://github.com/tohnperfect/USBGym
★ GPWM: Modeling the World as 3D Particle Motion
We introduce GPWM, a Generative Particle World Model that predicts how 3D scenes evolve by generating the motion of particles and rigid bodies over time. Unlike video world models that generate scene appearance, GPWM operates directly in 3D physical space. A single representation and architecture cover rigid, deformable, cloth, and granular materials, in both passive physics and robotic interaction. GPWM is an object-aware multi-resolution transformer that combines flow matching with diffusion forcing to generate coherent physical trajectories far beyond its training horizon. We train and evaluate GPWM on ParticleWorlds, a 25.5-million-frame corpus spanning diverse geometries, materials, interactions, and robot embodiments. GPWM outperforms the best prior learned-dynamics baseline in each of six physical domains, reducing prediction error by 31% on average. Breaking from the domain-specific setting of prior methods, a single Unified GPWM models these heterogeneous regimes with one set of weights, largely outperforming per-domain baselines and even surpassing our own specialist models in two domains. GPWM further generalizes to unseen geometries, physical properties, object counts, and rollout horizons. Beyond prediction, GPWM enables contact-rich planning by learning a generative prior over joint actuator-object motion. Instead of searching over unconstrained actions, planning searches the model's latent noise for interactions that achieve a desired object-space goal. With the same world model, noise-space planning substantially outperforms action-space search on pushing and picking, where complex contact sequences are hard to discover. Executed open-loop in simulation, the resulting plans achieve goal-directed manipulation without reference action trajectories or goal-specific policies.
comment: 39 pages, 27 figures, 7 tables. Robert Ren and Matthew Bronars contributed equally (ordered by coin flip). Project page: https://robertren1122.github.io/gpwm/
★ Learning the Task or Learning the Verifier? Rethinking Criteria Exposure in Embodied Self-Improvement
Self-improving embodied agents are increasingly being developed and compared on robot simulation benchmarks. Several recent works expose the benchmark's success criteria to the learning agent, a practice we call Criteria Exposure (CE). These criteria, originally written to verify policies trained by imitation learning, often demand more than the corresponding language instructions, so a score from such a verifier may not faithfully reflect whether the instructed task was completed, especially for robot-control programs written by an agent. As embodied coding agents report ever higher benchmark scores, a natural question arises: when a coding agent is exposed to the benchmark's criteria, does it generate code that better performs the instructed task, or code that simply overfits to the verification criteria? We conduct controlled experiments with a self-improving agent with and without CE, and find that with CE the agent quickly produces code with a higher success rate under the given criteria, but this advantage disappears when the criteria are changed slightly without departing from the original instructions. More importantly, the skills the agent develops under CE transfer poorly across tasks. We further propose instruction-faithful criteria and show that they better reflect the performance of agent-generated programs.
comment: 22 pages, 8 figures, 10 tables. Project page: https://dagroup-pku.github.io/PyRUA-Lean/RSIJudge
★ Transformer-Based Trajectory Prescription for sEMG-Optimizing Robot-Mediated Rehabilitation
Robot-mediated rehabilitation has the potential to deliver high-intensity, individualized therapy, but existing systems (which have not consistently improved outcomes) typically prescribe kinematic trajectories rather than explicitly targeting the neuromotor activations thought to support recovery. To address this gap, and ultimately enable activation-optimizing rehabilitation robot behavior, we present a personalizable transformer-based model mapping individuals' muscle activations (as measured via surface electromyography) to prescribed contact force trajectories when interacting with an end-effector rehabilitation robot. The model uses a transformer encoder-decoder leveraging rotary positional embeddings (RoPE) to generate a latent representation of trajectory-relevant and user-specific neuromotor factors, and augments trajectory prediction with flow matching to produce a distribution over plausible prescriptions biased toward the training distribution. Using data from 16 healthy participants completing 15 isometric trajectory tracking tasks, we evaluate generalization across held-out tasks and users, as well as the feasibility of user-specific fine-tuning within a therapy-session timescale. The model achieves mean trajectory prediction errors of <8% on held out tasks and users, with effectively instantaneous deterministic inference and sub-second sampling of trajectory distributions. An initial test using the model during live rehabilitation tasks demonstrates that, on average, participants reach the target neuromotor activations with just 15% error accumulated throughout the task. These results provide a proof of concept for data-driven, activation-targeted trajectory prescription in robot-mediated rehabilitation and establish a foundation for future validation in clinical populations. Data and code have been open-sourced as part of the OpenRobotRehab project on SimTK.
comment: Accepted for presentation and publication in the 2026 Conference on Robot Learning (CoRL). PMLR, 2026
★ Aligning Policy Encoders with a Shape Teacher
3D-aware representations can improve robotic manipulation by providing geometric information that is robust to changes in viewpoint, object position, and occlusion. However, using 3D inputs at deployment requires additional sensing and processing, making image-based policies more practical. We ask whether an image-based policy can acquire 3D shape information during training while retaining a purely image-based interface at deployment. We propose shape-teacher alignment, a training-time mechanism that uses a pretrained 3D shape representation as an alignment target for an image-based policy encoder. Specifically, we use Point-MAE features from individual objects as targets for corresponding image patches, transferring geometric structure to the policy without changing its deployment inputs or architecture. On LIBERO, shape-teacher alignment produces image representations that are more structured around object shape and corresponding parts, while substantially accelerating policy convergence and improving success rates. For pretrained policies, convergence is accelerated by 4.7x and 5.6x on the Object and Spatial suites, respectively, without requiring the shape teacher or depth input at deployment.
★ Fault-Geometry-Aware Control Allocation for Emergency Quadrotor Landing under Single-Motor Loss of Effectiveness
Partial single-motor loss of effectiveness (LoE) changes the remaining control-authority geometry of a quadrotor and can change which control allocator performs best during emergency landing[cite: 11]. We compare three allocators in Crazyflie software-in-the-loop simulation: bounded weighted least squares (reported as PINV), a finite-candidate residual allocator (QP-lite), and the same residual allocator with scoring parameters tuned offline by the Cross-Entropy Method (CEM)[cite: 11]. For this study, fault geometry is represented by the known failed-motor index[cite: 11]. At $η=0.496$ remaining effectiveness, matched development trials select CEM for M1, M3, and M4 and PINV for M2; the M3 selection is a one-trial margin (29/30 versus 28/30)[cite: 11]. After freezing this routing map, the policy achieves 117/120 safe first-contact touchdowns on fresh seeds and 115/120 in a separate randomized-order integrated-supervisor evaluation[cite: 11]. For the critical M2/PINV branch, Firth regression gives a nominal $η_{50}\approx0.4956$ with a within-model 95% bootstrap interval of approximately [0.4955, 0.4956][cite: 11]. One-factor-at-a-time $\pm10\%$ plant perturbations shift $η_{50}$ by up to 0.05495 for thrust coefficient and 0.04974 for mass; the most frequent violated first-contact criterion also changes from vertical speed in many conditions to angular rate for mass -10% and thrust-coefficient +10%[cite: 11]. At fixed $η=0.496$, PINV has a higher M2 safe count than frozen CEM in 9/11 tested plant states and ties in 2/11; the two ties are both 0/30[cite: 11]. The experiments therefore distinguish motor-conditioned allocator selection from sensitivity to plant-model mismatch[cite: 11].
comment: 6 pages, 7 figures, 1 table
★ RoboARCade: Benchmarking Agentic Robot Control
Improvements in multi-modal capabilities have enabled agents to act directly as robot policies, a setting we refer to as agentic robot control (ARC). We introduce RoboARCade, a benchmark of long-horizon manipulation tasks across three embodiments designed to test in-context adaptation, memory, and reasoning for precise control. The tasks build on existing open-source benchmarks, which we make harder by widening the range of initial conditions. RoboARCade is easy to use and extend: all tasks run on a single simulation backend, MuJoCo, each task is a self-contained data pack, and agents connect through interchangeable interfaces, so new tasks, agents, and interfaces can be added independently. The benchmark separates leading agents: GPT-6 Astra achieves 49.8% compared with 43.6% for Claude Fable 5.1, leaving substantial room for improvement. Comparing two interfaces, we find that a richer visual interface adds only 1.5 to 6.8 points at 1.7 to 2.3 times the cost. Failures stem mostly from choosing a poor manipulation strategy or executing a good one imprecisely, rarely from misunderstanding the scene, and agents incorrectly claim success on 24% to 39% of attempted subtasks. Agents can also improve without human input: one successful demonstration in context raises Astra's score from 53.5% to 73.0% on a five-task subset, and a prompt the agent writes and revises itself reaches 72.0%. Code, videos and full traces are available at roboarcade.org.
★ La-JEPA: Compact Robot Policies with Latent World Modeling
World-action models adapt pretrained video generators, and vision-language-action models build on pretrained vision-language backbones; both tie policy architecture and inference cost to large transformers designed for generation rather than control. It remains unclear how much these generation-oriented choices affect control. In this work, we introduce Latent Action JEPA (La-JEPA), a compact world-action model whose policy transformer is trained from scratch in the feature space of a pretrained V-JEPA 2.1 encoder. Following Fast-WAM, La-JEPA learns to predict future features jointly with actions during training and generates only actions at inference. Among policies without embodied pretraining, La-JEPA achieves state-of-the-art performance in clean and randomized RoboTwin 2.0 scenes and on LIBERO-Plus with correct instructions, while using far fewer parameters than the strongest competing policies. It runs substantially faster than Fast-WAM at comparable performance and outperforms the closest JEPA-based methods, LeapBot-WA and JEPA-WAM, on LIBERO-Plus and RoboTwin 2.0 without their pretrained policy backbones. On four real-robot tasks, its average success is comparable to pi0.5 and above non-embodied-pretrained baselines. Through a design study of the choices behind La-JEPA, we show that JEPA features outperform video VAE latents, most clearly out of distribution, and that training-time future prediction choices significantly impact the policy even when future prediction is skipped at inference. We correct the LIBERO-Plus instructions, as the officially released evaluator corrupts them for most variants, and re-evaluate several strong open baselines. Without a generative pretrained policy backbone, La-JEPA is better and faster than far larger policies, suggesting that robot foundation models may benefit from architectures designed for control rather than inherited from generation.
comment: 30 pages, 6 figures. Code: https://github.com/mt-clemente/lajepa
★ Detecting Silent Faults at the Environment-Trainer Boundary in Reinforcement Learning
Robot reinforcement learning increasingly runs environments outside the trainer: in another process, on another machine, in another language. Many faults at this boundary raise no error. We make the boundary a wire protocol and ask what notices a fault in it. On Pendulum, HalfCheetah, Hopper and a 7-joint arm (Pusher), we injected up to 23 faults into what crosses the wire and 2 into the trainer's log, from one value to every value. Unless every value was wrong, a learning-curve threshold caught at most one run in ten, and a comparison with ten fault-free seeds at most 21%. A clean evaluation of all 1,060 final policies agreed with the curves: of the 728 boundary-fault runs both curve checks passed, 723 scored no more than three standard deviations below fault-free seeds, though nearly all ended on other weights and, for reward faults, logged other returns. Checks of the run itself, not of its outcome, saw every fault. A step ledger kept on both sides and compared step by step caught and located every fault that crossed, on Pendulum, Hopper and Pusher on one machine and on Pendulum across two, with no fault-free run and, on Pendulum, a median 3% more training time. An episode record kept at the environment caught the faults in the trainer's log. We list such faults found in practice, in Gymnasium 1.3 and our own code and, reported by others, in five multi-agent libraries, and recommend a step ledger and episode counts kept at the environment.
comment: 7 pages, 6 figures, 6 tables. Accepted to the CoRL 2026 Workshop on Everything Beneath the Policy. Code, specification, and every experiment's protocol, logs and findings: https://github.com/PlugRL
★ Educational robotics beyond ROS: a custom library for direct transfer of Webots controllers in the Turtlebot3
Robot Operating System (ROS) is becoming a standard in robotics courses at higher education, but its installation, configuration and distributed programming model can introduce a significant entry barrier in the new landscape of AI-based master and degree studies, where many students do not have a strong informatics background. This paper presents an educational experience that enables students to program a TurtleBot3 Burger directly in Python, while preserving a direct transfer path from simulation to the physical robot. The approach is based on a custom library that mirrors the Webots API, allowing the same controller code to be executed in the simulator and on the real TurtleBot3 with minimal modification. The library provides access to the robot motors, wheel encoders, inertial sensors and LiDAR, and is complemented by scripts that automate deployment and execution on the physical platform. It was used in a master-level course on introductory autonomous vehicles, and initial student feedback suggests that the approach reduces the perceived complexity of ROS-related infrastructure and helps learners focus on algorithmic logic and real-world behaviour.
★ H-TRIP: Exact Many-to-Many Distance Index for Multi-Robot Exploration
Frontier-based exploration systems repeatedly evaluate geodesic travel costs between robot poses and candidate viewpoints, and tour-based planners consume the full pairwise distance matrix built from them. Recomputing these matrices with one graph search per source scales with the explored map size, whereas distance caches that refresh only locally affected or newly created frontiers can retain stale costs between surviving pairs when newly discovered free space creates a global shortcut. We present H-TRIP (Hierarchical Tropical Rank-one Incremental Propagation), an exact distance index grounded in Kleene algebra over the min-plus semiring for monotonically growing occupancy grids. H-TRIP partitions the known map into a spatial hierarchy with fixed boundary ports. Each unknown-to-free revelation is decomposed into efficient rank-one min-plus edge insertions at the affected leaf, and only decreased port-to-port summaries are propagated along its ancestor chain. A shared hierarchical lift evaluates the dense many-to-many matrix of shortest distances for all active query points on the occupancy map. We prove that sequential insertions preserve exact shortest-path distances on the grid. Benchmark experiments confirm exact agreement with reference searches on fully built maps and show that dense many-to-many queries at a few dozen viewpoints are up to two orders of magnitude faster than sequential k x BFS. On exploration runs with sixteen robots, in-loop queries on the growing occupancy graph are 2.21x-8.59x faster than co-scheduled k x BFS on maps with side 256, 512, and 1024 cells.
★ H-BRICK: Hierarchical Boundary Reachability Index with Compressed Kleene Closure for Directed Grid Graphs
Repeated directed reachability queries arise in online path planning, task-agent allocation, and swarm control on static grid maps, where reachability is asymmetric and full transitive closure is storage-prohibitive. We introduce H-BRICK, a Hierarchical Boundary Reachability Index with Compressed Kleene closure for exact cell-to-cell queries on directed grid graphs. H-BRICK partitions the graph into tiles, computes a local Boolean Kleene closure within each tile, and projects it onto the tile's boundary ports. The resulting directed port-to-port summaries are then composed recursively across shared interfaces up to a single root covering the graph. We prove algebraically that the construction is exact: in the Boolean Kleene algebra of binary relations, every composed summary equals the boundary projection of the regional reflexive transitive closure, and every cell-to-cell query agrees with reachability in the original graph. Experiments on directed grid graphs across diverse map topologies compare H-BRICK against directed search and component-based baselines, and compare its storage requirements against full transitive closure. A batched reachability evaluation further demonstrates that sharing endpoint lifts across many-source/many-target queries amortizes hierarchical traversal and, at k=64, yields a 5.0x-22.9x throughput improvement over the scalar query loop. The results show that hierarchical boundary compression is most advantageous in query-intensive spatial graphs where online search traverses large reachable regions and the hierarchy maintains compact boundary interfaces.
★ Reinforcing End-to-End Autonomous Driving Safety via Negative-Aware Planning in a Unified World-Action Model
World-action models (WAMs) jointly model future traffic states and driving actions for end-to-end autonomous driving in intelligent transportation systems. However, their action learning remains dominated by expert-trajectory imitation, leaving unsafe alternatives close to expert trajectories insufficiently constrained. To improve WAM robustness in safety-critical scenarios, we propose NegWAM, a safety post-training framework for unified WAMs that learns explicit safety boundaries from near-expert failure trajectories. NegWAM constructs negative trajectories that are geometrically close to the expert trajectory yet carry safety risks, uses them as repulsive anchors to steer predictions away from dangerous behavior, and encodes them as scene-conditioned rider tokens to learn scene--trajectory compatibility. During training, NegWAM jointly optimizes the flow-matching and near-expert negative objectives, while the auxiliary rider branch is discarded at inference, incurring no additional overhead. On NAVSIM v2, NegWAM achieves 40.4 EPDMS on navhard, a 4.1% relative improvement over the strongest compared scorer-free WAM baseline, and achieves 93.3 EPDMS on navtest. These results demonstrate that NegWAM achieves strong performance in safety-critical scenarios while preserving routine-driving performance. Zero-shot closed-loop evaluation on HUGSIM further demonstrates cross-benchmark generalization. The code is available at https://github.com/Zizizi-hao/NegWAM.
★ An Integrated Open-Source Stack for Vision-Based Autonomous Navigation of Wheeled Rovers
Vision-based rover autonomy requires integrating perception, localization, planning and platform-specific control under limited sensing and computing resources. We present an open-source ROS 2 stack for following directional arrows, reaching a terminal cone and returning autonomously to the start. It connects YOLOv8n detection and geometric direction, range and bearing estimation with RGB-D SLAM, wheel-inertial state estimation, kinematics-aware planning, behavior-tree mission management and STM32 actuation. Standard interfaces separate reusable autonomy from the hardware layer; a proportional path-following plugin supports low-speed rover motion. Public resources include software, configuration, a labeled dataset, trained weights and evaluation logs. On 478 held-out images, the detector achieves mAP@0.5 of 0.720 under the Ultralytics evaluator and outperforms a classical OpenCV baseline under a shared evaluation protocol. The reported perception latency is 25 ms on a Jetson Orin NX. Three missions, one indoor and two outdoor, demonstrate marker-guided navigation and autonomous return without operator motion commands or reported collisions. Field results document integration failures and corrective choices; repeated instrumented trials remain necessary to quantify physical navigation errors and reliability.
comment: Accepted at RiTA 2026
★ STEP: Spatiotemporal Training with Explicit Prediction for Perceptive Humanoid Locomotion
For perceptive humanoid locomotion, the relevance of terrain geometry depends on the robot's body state and how both evolve under its actions. Existing work has advanced terrain-aware sensory fusion and cross-modal prediction, improving how visual and proprioceptive information is combined and related. However, feature aggregation alone does not specify which cross-modal relationships should be learned, while prediction between simultaneous observations does not explicitly constrain their action-dependent evolution. We present Spatiotemporal Training with Explicit Prediction (STEP), a one-stage framework that jointly supervises cross-modal relationships and action-dependent state evolution within a shared control representation. Multiple learned queries aggregate depth and proprioceptive tokens into a compact policy state. Bidirectional masked prediction uses each visible modality to predict latent features of the omitted modality, explicitly supervising terrain-body relationships. An action-conditioned predictor maps the current policy state and executed action to the next fused state, with an exponential-moving-average target defined directly in the policy representation space. Simulation comparisons show overall improvements in terrain traversal over evaluated fusion alternatives and ablations, while depth-dropout and delay experiments demonstrate the benefits of both predictive objectives under degraded visual input. Deployed on a 23-DoF Noetix E1 humanoid with onboard sensing, STEP succeeds in 98 of 100 hardware trials across stairs, a 40\,cm platform, a 70\,cm gap, stepping stones, and backward stair traversal without rearward terrain visibility.
★ Leave It Better Than Found: Value-Guided Cooperation in Sequential Tasks
We consider the problem of sequential task continuation, where one robot acts in a workspace that has been affected by a previous robot. Current approaches in multi-robot cooperation rely heavily on co-training or co-optimization, assumingthat the robots have access to collective interaction data during the policy training or planning process. However, as increasingly varied robotic embodiments and policies are deployed and operate in the same real-world environment, learning policies for cooperation becomes infeasible due to the combinatorial explosion of possible robot combinations that need to be accounted for. To address this challenge, this paper considers the problem of ad hoc sequential cooperation between robots that have never before operated together in a shared environment. We demonstrate that robots can explicitly communicate their environmental preferences through sharing their value functions, enabling coordination without the need for prior joint training. To ground this exchange and prevent overconfidence in out-of-distribution states, we further introduce a k-nearest neighbors (KNN)- based uncertainty filter. Our method yields an increase of up to 48.5 pp compared with naive or even selfish agents, in task success rates across simulated and real-world experiments, demonstrating that value function transmission between agents is useful not merely as an internal learning tool, but as an effective basis for ad hoc coordination. Website: https://proroklab.github.io/PAVE/
★ Terrain-Indexed Motion Priors and Geometric Support for Extreme Humanoid Parkour
We present a progressive imitation framework that learns terrain-appropriate humanoid parkour skills and subsequently adapts their hand--knee support. Human demonstrations provide traversal behaviors, but their contact patterns need not suit a robot's proportions or the available support surfaces. Our framework changes the role of demonstrations across three stages: privileged imitation produces physically executable robot references; terrain-indexed adversarial motion priors guide skill acquisition; and geometric support guidance with a reduced motion-prior weight enables contact adaptation. Candidate-based precontact attraction and actual-contact evaluation guide support without prescribing a contact sequence. The deployed policy selects behaviors from onboard depth, proprioception, and velocity commands without an external skill label. On the Noetix E1, the policy achieves 96.34\% hybrid success in simulation and transfers without real-world policy fine-tuning, completing 29 of 30 individual obstacle trials and four of five hybrid courses. Support-adaptation ablations show higher traversal success for the complete adaptation stage on the tested terrains, while qualitative observations show changes in contact placement. These results support acquiring skills from demonstrations before relaxing their influence to accommodate robot-specific support configurations.
★ HITCH: Teaching a Vision-Language Model to Operate a Humanoid through a Whole-Body Harness
Enabling vision-language models (VLMs) to control humanoid robots requires both reliable whole-body interfaces and knowledge of how to use the body that carries over across tasks. We present HITCH (Harness Interfaces To Command Humanoids), a harness in which perception returns the 3D position of any point the VLM selects in its image, and execution turns commands for the pelvis, torso, hands, walking, and gaze into whole-body motion with a learned tracker. We find that both measuring the scene and executing commands accurately raise task success. We then fine-tune a VLM on trajectories in which teacher policies solve five base tasks through HITCH, so that it learns how to operate the humanoid. Without task-specific trajectories, the fine-tuned model completes 51% of SIMPLE episodes and 42.5% of HumanoidArena episodes, and adding five trajectories per SIMPLE task raises its SIMPLE success to 95%. On a real G1, it carries a box and walks to a goal without real training data.
comment: Project Page: https://hitch.axell.top/
★ Development of Anthropomorphic Arm for collaborative and Home Service Robot CHARMIE
Service Robots are becoming a reality and are progressing by leaps and bounds. Alongside this avenue of robotics is also the field of prosthetics, which is evolving rapidly. These topics are hot cakes in the industry and are widely developed. Some have one quality, and some have others, but are making a shoreline towards success. The development of arm and hand as individual or as a package is becoming common as more and more advanced collaborative robots come to the surface and join their human counterparts in the real world. This dissertation presents an open-source 3D-printed arm which was assembled and programmed on two fronts. The first is the control of the hand by utilising teleoperation with a camera and ML-based hand pose estimation. Secondly, it is the control via kinematics utilising the Denavit-Hartenberg method to define the motion and to find the position of the end effector in 3D space.
comment: 20 pages, 16 figures
★ Symmetry-Aware Surrogate-Assisted Testing of UAVs
We investigate a symmetry-aware surrogate-assisted testing approach for UAV autopilots. The proposed approach exploits geometric symmetries between flight missions by trans- forming obstacle configurations into a shared canonical coordi- nate frame. This enables a neural-network surrogate trained on a single reference mission to be reused across arbitrary missions, reducing the amount of high-fidelity simulation data required for training. The surrogate model is trained to predict the minimum clearance distance between a UAV and surrounding obstacles and is used to guide test generation towards safety-critical scenarios. We evaluate this approach using the Aerialist UAV test bench and benchmark its performance against techniques from the 2024 to 2026 UAV Testing Competitions. Results show that the approach achieves an average critical-test-case hit rate of 42% across the evaluated missions, compared with approximately 10% for the constrained-random baseline used to generate the training data. This approach also achieves high input diversity and competitive benchmark performance. However, our findings reveal limitations in transferring surrogate predictions across missions with differing flight-path characteristics, highlighting the boundaries of geometric canonicalization. To support reproducibility and future research, we provide the trained surrogate model and a dataset comprising more than 2,500 high-fidelity simulations.
★ A Unified Fractal Impedance Control Framework
Physical human--robot interaction requires controllers capable of maintaining intrinsically bounded interactions with the environment. The Fractal Impedance Controller (FIC) has been proven to guarantee these safety characteristics by design. This paper introduces the robotic implementation and experimental validation of its most advanced architecture: the Harmonic-Passive Motion Paradigm (H-PMP), an evolution of the Passive Motion Paradigm designed to mimic human-like interaction. Furthermore, we present a unified framework for FIC deployment, supported by an open-source software suite featuring ROS2, MuJoCo, and physical hardware integrations. Through three experiments conducted on a Kinova Gen3 arm (trajectory tracking, a wall press with sudden loss of contact, and learning-by-demonstration) we compare the H-PMP against earlier FIC architectures. The results demonstrate that the H-PMP achieves a seamless trade-off between admittance and impedance behaviours. This makes it ideal for deployment in unstructured environments, enabling the robot to transition between interaction modes with no computational overhead.
★ Reversible Dynamics, Asymmetric Actions: Parking as a Trajectory-Level Affordance
Drivers are often told to back into a tight parking space, not drive in nose first. The advice looks paradoxical, because a slow car is dynamically reversible: reverse its speed and replay the steering backward, and it retraces a feasible path. Played backward, though, reverse-in shows a car driving forward out of the space. Reverse-in and forward-in are different tasks: they share a start position but end facing opposite ways. We separate three clearances between car and obstacles. Static fit $F$ is the clearance the parked car keeps at its best acceptable destination pose. Eventual access $A_\infty$ is the clearance it can keep all the way there with unlimited corrections. Resource-bounded access $A_K$ is the clearance that survives a budget of $K$ reversals, switches between forward and reverse. They satisfy $F\ge A_\infty\ge A_K$. In one fully specified scene (one car, bay, aisle and start), a certified argument proves $A_\infty=F=0.5000$ m for both tasks. In this scene, all certified forward/reverse asymmetry therefore arises at finite $K$: it lies in how many corrections each task needs, not in the clearance either can eventually keep. With one reversal, an explicit path gives reverse-in at least $0.4410$ m clearance. A certified reachability argument caps forward-in's one-reversal clearance at $d_0<0.1720002$ m. With two reversals, forward-in gets at least $0.2833$ m. To keep a clearance strictly above any $d$ with $d_0\le d<0.2833$ m, reverse-in thus needs exactly one reversal and forward-in exactly two when both tasks move forward first. If both move in reverse first, the counts are two and three. The contrast depends on the start position (Section 4.15). Proposed experiments test whether observers distinguish fit, eventual access and budgeted access. They compare these with simpler cues, such as how the turning body sweeps, and learned manoeuvre labels.
comment: 79 pages, 15 figures
★ Time-Indexed Skill Success Models for Latency-Aware Skill Selection and Timing in Dynamic Manipulation
Robots that manipulate moving objects act on observations that are outdated when the chosen skill starts, and the best skill can change within a few hundred milliseconds. Skill competence models, however, predict success for a launch at the moment of observation. We propose time-indexed skill success models, which predict whole-task success when a skill is launched after a given delay. Trained on rollouts branched at a grid of launch delays, they let a receding-horizon planner select both the skill and its launch time under fixed or random latency. In a MuJoCo study in which a UR5e captures a sliding puck with three skills, the planner exceeded a static competence model by 2.8 to 6.2 percentage points at 100 to 300 ms of latency, including when that model was evaluated on point or sampled forecasts of the launch state. Across five complete runs (three training seeds and two distribution shifts), all 80 paired 95% intervals against four baselines excluded zero. On delayed launches, the time-indexed model was calibrated (expected calibration error 0.009 against 0.239), and pausing the world during latency overstated success by up to 23 points. On a real UR5e that captures a rolling ball after 0.2 s of injected latency, a static model trained in a hardware-matched simulator predicted 99% success on a prespecified subset of 35 delayed launches that it rated at least 0.8. Of these, 63% succeeded (95% interval 46 to 77%), and the time-indexed model predicted 56%. In this single-skill, predictable regime, a matched physics-based forecast was also well calibrated.
comment: 16 pages: 9-page letter with 4 figures and 2 tables, followed by 7 pages of supplementary material (14 tables, 6 figures). Submitted to IEEE Robotics and Automation Letters
★ GuidedWAM: Training-Free Energy Guidance for World Action Models in Autonomous Driving
World action models (WAMs) for autonomous driving couple the driving policy and the visual world model in a single generative network whose objectives are fixed post-training. Conditioning on a new driving objective requires retraining. We study energy guidance as a general mechanism and present GuidedWAM, a model-agnostic framework for injecting driving objectives into frozen WAMs at sampling time. Our framework constructs each objective as differentiable energies on the sampler's intermediate clean-action estimate, which needs only forward access to the frozen model, and is stabilized by sampling-time guidance scheduling, per-energy normalization, and velocity-bounded correction. On the NAVSIM v2 hard-split, guidance improves all three frozen, publicly released WAMs by 1.8 to 3.6 EPDMS, averaged over three sampler seeds, and adds less than 10% to the sampling time per step. These results position energy guidance as a practical post-training control channel for WAMs.
★ XM4D: 4D Reconstruction through Certifiable Geometric Optimization
Accurate 4D reconstruction remains challenging: jointly estimating camera motion, scene geometry, and non-rigid deformation entails highly nonconvex optimization, making global optimality difficult to guarantee. We present XM4D, a framework for accurate and efficient 4D reconstruction built upon certifiably globally optimal geometric optimization. By separating the estimation of camera motion from non-rigid deformation, we formulate both stages as structured quadratically constrained quadratic programs (QCQP) that admit semidefinite programming (SDP) relaxations and verifiable global optimality certificates. For camera motion estimation, we lift static point tracks into 3D using foundational depth prediction models and formulate a scaled bundle adjustment problem that jointly estimates camera poses, depth scales, and static landmarks. By analytically eliminating translations and landmarks, we reduce the problem to a QCQP over scaled rotations. For non-rigid reconstruction, we represent dynamic motion using an embedded deformation graph (EDG) and show that its tracking, rigidity, and temporal regularization objectives similarly reduce to a rotation-only QCQP. Both problems admit SDP relaxations that we solve efficiently with a GPU-accelerated Burer-Monteiro solver based on Riemannian optimization. The recovered geometry further initializes dynamic Gaussian splatting for photometric refinement. Extensive experiments across seven benchmarks demonstrate accurate camera motion estimation on both static and dynamic scenes, efficient optimization that scales to thousands of frames, and state-of-the-art novel-view rendering quality.
★ Structure from Collision via Hypothesis-Space Reasoning WACV 2027
Knowledge of the hidden interior structure of an object, including the presence and location of cavities, is essential for predictive simulation, robotic manipulation, and nondestructive inspection. However, such cavities leave no direct signature on the static visible surface, making collision-induced deformation one of the few cues used to infer their presence. Structure from Collision (SfC) addresses this problem by recovering the hidden interior structure of an object from multiview observations of a collision event. SfC is challenging for two reasons: hidden cavities are only indirectly observable through surface motion, and voids correspond to the absence of material, making direct optimization using standard density representations difficult. The density-based SfC baseline inherits both difficulties. Surface-matching gradients do not directly indicate the cavity location, while density gradients can be weak or uninformative inside or near void regions. Therefore, we reformulate SfC as hypothesis-space reasoning, representing hidden cavities as explicit structural hypotheses whose variables encode available structural assumptions rather than relying on implicit density updates. A global stage performs gradient-free optimization over this hypothesis space using forward-only material point method (MPM) simulation. A local stage then refines the selected hypothesis as a Lagrangian particle system through gradient-based optimization using differentiable MPM simulation. On the SfC benchmark, our method improves hidden-structure reconstruction and future prediction over the density-based SfC baseline. Interior-topology stress tests and robustness analyses characterize its scope and limitations. The project page is available at https://www.kecl.ntt.co.jp/people/kaneko.takuhiro/projects/sfc-hypothesis/.
comment: Accepted to WACV 2027. Project page: https://www.kecl.ntt.co.jp/people/kaneko.takuhiro/projects/sfc-hypothesis/
★ AffordJEV: Affordance Grounding by Confirming Candidate Regions
Affordance grounding asks which visible part of an object performs a requested operation, a basic capability for embodied manipulation and human--robot interaction. Most existing methods follow a System Two paradigm: a multimodal language model first writes a chain of thought and then turns its conclusion into a box or points for a segmenter. This incurs considerable inference cost, gives the semantic decision little direct control over the extent of the final region, and makes mistakes hard to correct. Yet locating an affordance is essentially a perceptual judgment that need not be argued out in words. We therefore propose, in which frozen segmenters propose candidate regions and a System One model decides directly which of them perform the operation, generating no text and reading each decision from a single forward pass. Our contributions are twofold. (1) An inference framework that confirms candidate regions level by level, from object to part to sub-part, with evidence passed between the levels in both directions so that the levels correct one another. (2) A System One model trained for this framework: existing System One models are built for general-purpose decisions, judge one candidate per request and are ill-suited to the framework's region questions, so ours learns directly from the requests the framework itself produces and judges all candidates of a level at once. Trained only on InstructPart and evaluated with definition-aligned operation words, outperforms chain-of-thought methods in domain and in zero-shot transfer to UMD and IIT-AFF, is better overall than existing System One models run in the same framework, and needs fewer judge requests. To our knowledge, is the first approach to bring System One models to affordance grounding.
comment: 11 pages, 3 figures, 5 tables
★ TactileLab: Sim-to-Real Dexterous Robot Manipulation with Efficient Shear-Sensitive Tactile Simulation
Robotic simulators that represent shear within tactile sensing are becoming increasingly important for contact-rich manipulation. However, simulating contact shear with high-fidelity deformable or soft-body simulation is computationally demanding, limiting scalability to large-scale reinforcement learning. Here we introduce \textit{TactileLab}, a scalable multimodal tactile simulation framework built on IsaacLab, with a particular focus on a new optical-flow-based tactile representation termed \textit{simulated tactile flow}. Instead of relying on soft-body physics to model shear deformation, TactileLab uses rigid-body simulation to efficiently estimate dense tangential motion at the contact interface, producing tactile flow fields that encode shear-relevant contact dynamics. This flow field complements geometric features from a contact depth field. We further introduce a novel policy architecture for jointly processing long-horizon proprioceptive signals and rich tactile observations. Across multiple contact-rich tasks, policies trained using simulated tactile flow learn efficiently at scale and exhibit emergent contact-aware behaviours. To facilitate sim-to-real policy deployment, we then propose a novel real-to-sim tactile translation method that maps real tactile image streams to simulated tactile flow and depth fields, together with a new tactile data collection method, and validate it across diverse tactile sensor designs. Using this pipeline, we demonstrate real-world deployment on tasks encompassing dual-arm and dexterous in-hand manipulation. Overall, TactileLab provides an end-to-end sim-to-real framework for scalable tactile policy learning, enabling diverse shear-sensitive manipulation across heterogeneous tactile robotic systems.
★ Learning Robotic World Models with Semantic and Visual Feedback
Robot world models predict action-conditioned future visual states to support robotic planning and decision making. However, visual reconstruction does not fully capture task progress, semantic evaluation alone provides limited constraints on fine-grained visual errors, and homogeneous predictions limit comparative feedback. We present RoboImagine, a framework for post-training autoregressive robotic world models through candidate-future comparisons with semantic and visual feedback. Diverse search promotes diversity among candidate trajectories under shared visual observations, action sequences, and task instructions. A frozen multimodal evaluator provides task-relevant semantic preferences through pairwise video comparisons in both presentation orders, complemented by pixel- and perceptual-level visual consistency feedback that constrains local structural and appearance errors. These signals drive policy optimization through group-relative advantages. Fixed initial context and branch-specific rolling histories support long-sequence generation with bounded context and consistent conditioning between candidate generation and policy optimization. Experiments with separately trained models on RT-1, BridgeV2, and RoboCasa cover real-world and simulated manipulation. On RT-1, under matched initialization, update counts, and evaluation protocols, RoboImagine improves PSNR by 1.26 dB and reduces LPIPS by 11.93% relative to continued supervised fine-tuning. Ablations show that diverse search improves prediction quality and that combined semantic and visual feedback yields higher external multimodal evaluation scores than visual feedback alone while maintaining comparable visual fidelity. Rolling histories further improve long-horizon prediction. These results support improving visual fidelity, task relevance, and long-horizon prediction through candidate comparison and complementary feedback.
comment: 27 pages, including supplementary material. Project and code: https://github.com/Alexander-wu/RoboImagine . Data and weights: https://huggingface.co/easylearning/RoboImagine
★ Generate-then-Rank: Learning Symbolic Goal Interpretation for Embodied Agents
Goal interpretation is to translate a natural language instruction into a formal symbolic goal that a planner can execute. It is critical for embodied agents because any error here propagates to every downstream planning and action step. Doing it well requires both generating suitable candidates and reliably selecting among them. We present a two-stage generate-then-rank approach to Goal Interpretation in the Embodied Agent Interface (EAI). First, supervised fine-tuning adapts a Large Language Model (LLM) to EAI's output convention. Second, Candidate-Output Ranking from Errors (CORE) learns preferences over complete outputs from the model's own generated alternatives. We conducted experiments on the household scenes of the Embodied Agent Interface Benchmark. Multiple open source LLMs including DeepSeek-V3 chat, DeepSeek-Reasoner, and Qwen2.5-7B and Qwen2.5-72B were evaluated. The proposed method outperformed the baseline models including LLMs with simple prompting, chain-of-thought, and knowledge graph based approaches.
comment: 14 pages, 2 figures. Accepted at the 10th Euro-China Conference on Intelligent Data Analysis and Applications (ECC 2026)
★ A Sequential Framework for Urban Drone Delivery: Network Design, Vehicle Sizing, and Energy and Cost Assessment
A sequential engineering framework is presented for urban drone delivery, linking hub location, route planning, mission-specific vehicle sizing, and energy and operating cost assessment. Aggregate origin-destination demand is assumed known a priori. For each hub configuration, the maximum required link distance is minimized, and feasible routes that return to their departure hubs are generated. Route frequencies are determined using mixed-integer linear programming to satisfy delivery demand. The resulting range requirement guides motor-propeller selection and battery sizing to minimize vehicle takeoff mass. Flight distances and frequencies, cruise performance, and vehicle dimensions are combined to estimate monthly energy consumption, fleet size, hub area, staffing, and operating cost. The framework is applied to district-level Seoul delivery data. Increasing the number of hubs from one to four reduces the required range from 30.6 to 21.9 km and vehicle takeoff mass from 8.12 to 6.12 kg. Despite a 21.5% increase in total flight distance, monthly flight energy consumption decreases by 13.1%, from 15.2 to 13.2 MWh. Battery replacement is the largest vehicle-related operating expense, while operating cost per delivery increases from USD 1.18 to USD 2.20 because of higher rent and labor costs at the hubs. Among the designs evaluated, range-matched vehicles achieve the lowest operating cost in the two- and four-hub networks; a longer-range vehicle is least costly in the three-hub network. Sensitivity analyses further examine the effects of delivery demand and operating assumptions. The results demonstrate that reductions in vehicle mass and energy consumption do not necessarily lead to lower system operating cost.
comment: 16 pages, 16 figures
★ Flexible Piezoelectric Tactile Sensing System for Intelligent Object Recognition
Tactile sensing provides information about object properties that can improve robotic grasping and manipulation. However, accurate object recognition during the early stages of grasping remains challenging, particularly under strict real-time constraints. This study presents a flexible, distributed piezoelectric tactile sensing system for early-stage object recognition. The system was experimentally evaluated using six everyday objects and three recognition approaches: feature-based classification using support vector machines (SVMs), raw-signal classification using 1D convolutional neural networks (1D-CNNs) and convolutional recurrent neural networks (C-RNNs), and spatiotemporal heatmap classification using a 2D-CNN. The effects of sensor count and temporal window size on recognition accuracy and real-time feasibility were systematically investigated. Increasing the number of sensing elements improved recognition performance across all approaches. The best-performing method, based on spatiotemporal heatmaps and a 2D-CNN, achieved F1-scores of 0.9796, 0.9846, and 0.9903 using 16, 24, and 32 sensors, respectively. The optimal configuration, comprising 32 sensors and a 100-ms temporal window (50 samples), was deployed on an embedded edge platform, achieving an end-to-end recognition latency of approximately 101 ms from contact onset and an energy consumption of 63.8 microjoules per inference. These results demonstrate the potential of distributed piezoelectric tactile sensing for accurate, energy-efficient, and real-time object recognition during robotic grasping.
★ Wedge-Jamming Mechanism for String Configuration
Variable stiffness mechanisms that achieve both flexibility and rigidity are important technologies for robotic systems. Recently, wire-based jamming mechanisms have attracted significant attention. In the wire-based jamming mechanisms, modules are chained by a wire, and the overall stiffness is controlled by changing the friction between the modules through wire tension. These mechanisms have been widely studied due to their simple structure and controllability. However, conventional wire-based jamming mechanisms mainly focus on holding torque along the joint axis, and the resistance against disturbances in other directions has not been sufficiently explored. In this study, we propose a wedge-jamming mechanism that introduces wedge-shaped contact surfaces into the modules of wire-based jamming structures. By changing the wedge angle, the holding characteristics against disturbances in different directions can be adjusted. Experimental results show that an out-wedge configuration improves resistance to vertical disturbances, while an in-wedge configuration enhances resistance to torsional disturbances. These results demonstrate that the anisotropic holding performance of wire-based jamming mechanisms depend on wedge geometry.
comment: 8 pages, 16 figures, conference
★ Seeing Beyond the Next Step: World-Model-Guided Human-Like Navigation in Multi-Agent Scenes NeurIPS 2026
Selecting a next motion in a populated scene is not just a matter of avoiding what is currently in the way: an action can be locally feasible yet drift onto the road instead of the crosswalk, cut into a stream of crossing pedestrians, or bottleneck a narrow passage. These failure modes are invisible from the current state alone, and acting on them requires reasoning about what each candidate action will cause. World models offer exactly this, namely action-conditioned predictions of the future, but rolling them out at every decision step is a cost deployable navigation systems cannot pay, and the field has largely traded foresight for speed. We argue this is a false dichotomy. The natural place for a world model in human-aware navigation is at training time, as a preference oracle that distills foresight into a fast policy, rather than at inference time, as an online planner. We pair a discrete visual action prior with two task-aligned world models, one for nearby social dynamics and one for future scene semantics, that score candidate actions by their predicted consequences and refine the policy through group-relative preference distillation. At deployment, both world models are removed and the policy acts in a single forward pass. Across five held-out evaluation scenarios, this yields navigation that is simultaneously safer, more socially and group-coherent, and more goal-directed than strong reactive and learned baselines, with naive observers rating the resulting motion as more human-like. Taken together, our findings point to a different approach for bringing world-model foresight into deployable agents: not accelerating rollouts, but amortizing them into the policy so that the cost of looking ahead is confined to training. Project page: https://seoneun.github.io/-NeurIPS-2026-Seeing-Beyond-the-Next-Step/
comment: Accepted to NeurIPS 2026. 37 pages, 13 figures, 12 tables
★ Dual-Arm Tabletop Rearrangement with Movable Stacking and Gap-Aware Push Transfers
Object rearrangement in cluttered tabletop scenes is challenging, especially in wide workspaces where dual-arm systems improve coverage and efficiency. Limited free space often necessitates buffering, while objects near or across the shared reachability region of the manipulators must be transferred under careful coordination to avoid inter-arm collisions. Conventional inter-arm transfer typically relies on a fixed handoff configuration, which constrains where and how objects are exchanged, enforces spatiotemporal synchronization of the arms, and reduces flexibility in dense layouts. This paper presents a dual-arm rearrangement framework that combines movable stacking with Gap-Aware Push Transfer for asynchronous cross-boundary exchange. The method scans the shared boundary to identify collision-free transfer gaps, places an object near a selected crossing point, and then pushes it into the receiving workspace without requiring simultaneous interaction between the two arms. A joint Monte Carlo Tree Search (MCTS)-based planner coordinates both manipulators while reasoning over buffering and transfer decisions, and execution cost is used after planning to assess the resulting plan. Experiments show that the proposed method shortens joint execution sequences and becomes increasingly advantageous over a fixed handoff baseline as scene density increases, achieving lower execution cost in denser scenes while remaining competitive in lighter ones.
★ AffectiveVLA: Safe and Intent-Aware VLA Control for Human-Humanoid Social Interaction
Social touch interactions, such as handshakes, fist bumps, and hugs, require humanoid robots to infer human intent, generate contact-rich motions, and maintain safe physical contact. We present AffectiveVLA, an intent-gated Vision-Language-Action (VLA) pipeline deployed on a Unitree G1 humanoid with dexterous hands. A lightweight RGB-based model recognizes interaction intent from two onboard cameras and converts it into a language prompt for the VLA policy. Low-confidence predictions suppress motion execution, reducing unintended activations. During physical contact, interaction-specific compliance profiles adjust arm impedance, while an independent voice-triggered mechanism initiates a latched safe-stop procedure. Using VR teleoperation, we collected approximately 1,300 multimodal interaction episodes and fine-tuned four VLA models: GR00T, RLDX-1, Being-H0.5, and UnifoLM-VLA-0. On-robot evaluation identifies GR00T as the best-performing standalone model. Integrating intent recognition increases its macro-F1 score from 0.75 to 0.90, while the intent model achieves a p95 inference latency of 6.27 ms. These results demonstrate that explicit intent gating improves the reliability of VLA-based social interaction without compromising real-time operation.
★ Fast and Accurate Hand Estimation in Pixels, Interaction and World Space
Monocular hand mesh estimators commonly predict rig parameters and project the mesh to obtain 2D keypoints. We find that directly predicted 3D joints agree better with 2D predictions, and fitting a mesh to predicted points reduces mesh error. We present FAH (Fast and Accurate Hand Estimation), a family of models built on SAM 3D Body's hand branch through fine-tuning and feature distillation. The models predict corresponding 2D and 3D joints and surface points, fit MANO or MHR-Hand, and optionally estimate anatomical parts and contact without object geometry at inference. Causal refinement reduces acceleration error, and camera estimation places the hands in world space. Experiments show improvements in image-space alignment, hand geometry, contact estimation and world-space tracking over the evaluated baselines. The default ViT-L model runs at 65 fps with mesh fitting on an RTX 4070 Ti SUPER, or 49 fps with parts and contact enabled. The smallest ViT-S model reaches 125 fps without mesh fitting, parts or contact. Code: https://github.com/noahcao/FAH.
comment: 10 pages of main text plus 15 pages of supplementary material; 5 figures; 12 tables; code available at https://github.com/noahcao/FAH
★ Not Losing Your Way: Mitigating Visual-Inertial Odometry Drift through On-Demand Visual Feature Creation ICRA 2027
Visual-inertial odometry (VIO) can degrade during aerial inspection in repetitive or poorly textured environments. We propose an active environment-modification framework for unmanned aerial vehicles (UAVs) relying on cameras and an inertial measurement unit only, without prior maps or additional localization sensors. Inspired by the fairy tale of Tom Thumb, the UAV creates visual features on demand by releasing ground markers or marking walls through physical contact. An information-based indicator triggers interventions using temporal confirmation and hysteresis. Evaluation includes three simulated environments and a 900-run factorial campaign. Lower cruise speed and active marking are associated with reduced divergence, while accuracy among non-divergent runs depends on speed-strategy interactions. Corridor experiments, with five runs per configuration, use OptiTrack for flight control while VIO-based marking decisions remain active online. The combined framework reduces the reported position root mean square error by approximately 35% relative to baseline VIO, supporting active feature creation for drift mitigation.
comment: 8 pages, 4 figures. Submitted to the 2027 IEEE International Conference on Robotics and Automation (ICRA 2027)
★ PhySA: Decoupling Aesthetic and Physical Rewards for Humanoid Locomotion
Humanoid locomotion policies trained without an explicit smoothness objective tend to produce jerky, high-energy gaits that stress actuators. Vision-language models (VLMs) offer an appealing reward source: given a rendered rollout, a VLM returns an aesthetic quality score on [0,10] without task-specific feature engineering. We find, however, that aesthetic preference and physical smoothness point in opposite directions: across four production VLMs, clips with higher joint acceleration receive higher aesthetic scores (mean Pearson r=+0.66). We study PhySA (Physical Smoothness and Aesthetics), which keeps the two on separate, independently removable reward channels: a sparse, asynchronous VLM score and a phase-triggered correction channel that adjusts physical penalty weights from simulator state. Ablations attribute the smoothness gain to the correction channel. On MuJoCo Humanoid-v5 at 2.6M steps, PhySA lowers joint acceleration by 26 to 32% relative to vanilla PPO, Text2Reward, and Eureka (three seeds), at the cost of shorter episodes. In a single extended run it lowers joint acceleration by 59.5% relative to Prompt2Policy, while a three-seed replication shows a smaller, non-significant reduction (7.2%). The gain does not transfer to IsaacLab or the Unitree G1, even after rescaling the smoothness term.
comment: 24 pages, 12 figures, 11 tables. Accepted to the CoRL 2026 Workshop on R2RL: Roadmap to Sample-Efficient Real-World Reinforcement Learning
★ Energy-Aware Open-Vocabulary Detection Scheduling for Low-Power Robotic Autonomy
Open-vocabulary object detection (OVD) lets robots search for run-time object queries, but fixed-rate inference spends embedded compute on low-value frames during idle motion, map revisits, or stable viewpoints. This paper presents an energy-aware OVD scheduler for low-power mobile robots. The scheduler uses odometry, map change, and detector confidence to switch among low-, mid-, and high-power YOLO-World-S modes on a Jetson Orin Nano. We evaluate the energy-quality trade-off on an AgileX LIMO using 40 real navigation trials: two target objects, five placements per target-policy pair, and four perception policies. Across bottle and cup trials, the scheduler records 8/10 mission success compared with 7/10 for each fixed-rate baseline, while reducing mean Jetson input power (a Jetson-side power-draw metric) by 5.2% relative to an 8 Hz high-rate baseline and lowering achieved OVD call rate by 34.0%. It does not minimize mission-window Jetson input energy because longer missions can erase instantaneous power savings. A separate 60 s stationary microbenchmark isolates the perception workload: the scheduler lowers mean power from 8.045 W to 7.301 W and energy from 484.7 J to 438.1 J relative to the 8 Hz baseline. These results support robot-state-aware OVD scheduling as a promising power-management mechanism with competitive task reliability, while showing that mission-window Jetson input energy must be read together with task progress and runtime.
comment: 6 pages, 8 figures, accepted for publication in ICARCV 2026
★ Which Objects Stay? Learning Hierarchical Persistence Beliefs for Lifelong Open-Vocabulary Object Maps
Lifelong object maps must estimate whether an object remains at its previously observed location after it leaves view. Existing persistence filters rely on class-based survival priors, making them difficult to apply reliably in open-vocabulary settings. We introduce HiPer, a hierarchical persistence filter that derives persistence beliefs directly from data rather than semantic categories. For a new object, HiPer retrieves visually and geometrically similar training instances and estimates an initial survival rate from their observed histories, without requiring class labels. This estimate initializes a conjugate Beta belief over the object's survival probability, which is updated in closed form using detections and missed observations under Rosen's persistence-filter framework. When an object is re-identified at a new location, its persistence belief is transferred with the object, enabling the map to learn object-specific mobility over time. Experiments on 120 held-out simulated sequences show that HiPer consistently achieves the best performance among competing methods. Retrieval-based priors outperform CLIP-derived label priors, and object-specific online learning provides further gains. Results on 47 held-out 3RScan environments demonstrate that the proposed retrieval-based prior also outperforms state-of-the-art baselines in real-world settings.
★ GraspBench: Execution-Aware Cross-Robot Evaluation of 6-DoF Grasp Generation in MuJoCo
Modern 6-DoF grasp generators are often evaluated using grasp scores, dataset-level metrics, or single-robot demonstrations, yet proposal quality alone does not determine whether a grasp is executable by a particular robot. We present GraspBench, a controlled MuJoCo protocol for evaluating complete grasp-generation pipelines after robot-specific inverse-kinematics filtering, motion planning, and fixed approach-close-lift execution. GraspBench evaluates six configured pipelines across four robot embodiments, five diagnostic objects, and a structured tabletop pose grid, yielding 540 scenarios per method and 3,240 method-scenario evaluations in total. Missing outputs are conservatively counted as failures, and success follows the binary lift criterion produced by each method's execution runner. GraspGen achieves the highest overall success rate (73.1%), followed by EconomicGrasp (63.5%), while robot- and object-conditioned analyses reveal substantial variation across embodiments and object geometries. GPG produces the largest native candidate sets but achieves 44.3% success, indicating that proposal volume alone does not guarantee robot-executable grasps. Cluster-aware paired intervals support the principal method differences while not estimating repeated-execution uncertainty. The results characterize complete configured pipelines on fixed scenarios rather than intrinsic model quality or stochastic repeatability, showing that grasping performance depends jointly on the generator, robot embodiment, scene, and execution protocol.
comment: 7 pages, 6 figures, accepted for publication in ICARCV 2026
★ Real-Time Event-Based LED Tracking and Pose Estimation with PWM-Encoded Markers
Event cameras are neuromorphic vision sensors with pixels that independently and asynchronously respond to brightness changes at microsecond resolution, supporting state estimation under rapid motion, motion blur, and high dynamic range (HDR) illumination. This also enables capture of high-frequency blinking patterns from light-emitting diodes (LEDs). Based on this, we propose a real-time event-based framework for LED tracking and six-degree-of-freedom (6-DoF) relative pose estimation using pulse-width modulation (PWM)-encoded LED markers. Multiple LEDs share a common blinking frequency and encode identities through distinct PWM duty cycles, reducing frequency-resource requirements and computational cost. Our dual-rate asynchronous tracking framework combines low-rate LED detection with high-rate raw-event updates, maintaining robust LED identity association and accurate pixel-level tracking under rapid motion. LED identities and positions establish reliable 2D-3D correspondences for real-time metric 6-DoF relative pose estimation. Extensive evaluations on a self-collected dataset demonstrate robustness under large motion flow and transient image-plane LED overlap. Furthermore, we develop a coordinated robotic platform comprising a quadrotor and a Mecanum-wheeled mobile robot, demonstrating the proposed framework's potential for closed-loop drone guidance through hovering and target-following experiments under time-varying HDR illumination, with onboard deployment on an NVIDIA Jetson Orin NX. The dataset and implementation will be released to facilitate further research.
comment: This work has been submitted to the IEEE for possible publication
★ Data-Driven Configuration of Protective Separation Zones in Collaborative Robot Cells from Operator Occupancy SC
The protective separation zone of a collaborative robot cell is commonly dimensioned for a worst case that disregards where the operator is actually present during the task. This paper presents a camera-based, data-driven method that configures this zone from observed operator occupancy. Operator positions accumulated over several production shifts are aggregated into an occupancy heatmap, from which a protective field is derived that trades off the achievable robot velocity against the required safety distance, following the speed and separation monitoring principle of ISO/TS 15066. The resulting configuration is advisory, that is, reviewed within the risk assessment rather than applied autonomously to the robot controller. The method, which builds on a pose-based perception pipeline with activity recognition, was evaluated in two operational cells at two industrial partners, where the protective field could be reduced by approximately 17% and a productivity increase of 9% was measured. However, adoption proved to depend on more than the demonstrated gain alone.
comment: 7 pages, 4 figures. Accepted for publication at the 2026 IEEE Stuttgart Conference on Automotive Production (SCAP). \c{opyright} 2026 IEEE
★ Whole-Body Elasto-Geometrical Calibration of Wheeled Humanoid Robots Using a Shared-Waist Dual-Chain Model
Wheeled humanoid robots carry both arms on one actuated waist, usually without torque sensors, so the accuracy of both end-effectors depends on a single waist model. After a first stage that estimates the tracker and target extrinsics, we identify the waist and both arms in one problem from the target positions on both flanges, measured by an optical tracker. The kinematic model is complete, with waist-base, parallel-axis, and branch parameters; arm compliances are driven by measured torques and waist compliances by gravity torques of the robot's mass model; and Gaussian priors with evidence-maximizing scales stabilize the estimate. Independent and sequential single-chain calibration are restrictions of this problem, and its waist information is the sum of the single-chain Schur complements. On two robots with different waists, each without and with a payload, calibration reduces the mean errors on held-out poses from 11.3-31.1 mm to 0.50-1.11 mm. For every training-set size from 20 poses to all, the method has 17-63% lower mean errors than adaptations of two published whole-body calibration methods, mainly through the waist compliance. Independent calibration yields two waist models; using either for both arms raises the other arm's error to up to 3.21 mm.
comment: 10 pages, 6 figures, 5 tables
★ Unified Overview of Safety Certificate Functions: Techniques from Hamilton--Jacobi Reachability and Control Barrier Functions
Safety is a central concern in modern autonomous systems. Safety certificates---functions whose existence guarantees that a system meets its safety specification---have long been used as a principled tool for rigorously reasoning about safety. Methods for synthesizing such certificates are scattered across research areas, with little guidance on when to use a particular approach. This tutorial provides a holistic treatment of safety certificate synthesis centered on two widely used families: Hamilton--Jacobi (HJ) reachability analysis and control barrier functions (CBFs). For each family of methods, we first introduce its general formulation, along with classical and learning-based tools for certificate synthesis. Next, we introduce two benchmark problems consisting of vehicle systems of differing complexity to assess the certificate quality of different synthesis methods and their effectiveness in safety-critical control. We use these benchmarks to compare various methods and provide practical guidance for selecting an approach.
★ Semantics-to-Contact: A Stagewise Framework for Robust Contact-Rich Manipulation NeurIPS 2026
Contact-rich manipulation requires precise control under complex contact dynamics while remaining robust to diverse visual conditions. However, real-world reinforcement learning often overfits to the visual appearance of training scenes. We propose Semantics-to-Contact (S2C), a stagewise framework for visually robust contact-rich manipulation. The insight is to decompose the problem into two stages: semantic focusing and contact refinement. In the first stage, a vision-language model localizes task-relevant regions across visually diverse scenes, providing coarse spatial guidance and reducing the burden of exploration. In the second stage, a real-world human-in-the-loop residual reinforcement learning policy learns fine-grained contact behaviors within the localized region using force feedback. To further improve visual robustness, we introduce an object-centric visual augmentation strategy that randomizes background appearance while preserving the robot and task-relevant objects. Experiments on six contact-rich real-world manipulation tasks exhibit that S2C consistently improves success rates over strong baselines and maintains reliable performance under significant visual and positional variations. These results demonstrate that stagewise semantic focusing and contact refinement provide a practical path toward visually robust real-world contact-rich manipulation. Our demos can be seen in https://edem-ai.github.io/S2C/.
comment: NeurIPS 2026
★ Embodied RAG++: Retrieval-Augmented Generation From What Exists to What Works in Embodied Scenes
To serve a request in a human environment, a robot system must find an object that satisfies a functional requirement and a robot that can feasibly reach it. Scene graphs record which objects exist and where, but they often omit capabilities documented only in product manuals. Retrieval-augmented generation methods can read such manuals, yet they rarely reason jointly over object capabilities, physical instances, robot states, and dynamic connectivity. We introduce E2W-Bench, the Exists-to-Works Benchmark, which links dynamic hybrid scene graphs with real product manuals to evaluate capability-grounded robot--object assignment under availability, mobility, vertical-reach, and connector constraints. We also present Embodied RAG++, a spatially grounded retrieval framework that scopes retrieval to the manuals linked to candidate objects, caches extracted capabilities per product model, and resolves robot--object feasibility through deterministic spatial reasoning. On E2W-Bench, Embodied RAG++ reaches a pair accuracy of 0.90 with Qwen2.5-32B and 1.00 with Llama3.3-70B, compared with at most 0.30 and 0.55 for the baselines, while requiring the shortest preprocessing time among the retrieval-based methods and tying for the lowest total token count.
comment: accepted to SII 2027
★ Training-Independent Validation of a Learning-Based Controller for Stunt Maneuvers
Learned controllers for vehicle maneuvers are usually validated under the conditions and objective used to train them, where they appear highly reliable, yet the parameters that change on real roads are often not those the model was trained on. We argue that validation must be independent of training: it should vary physical parameters that training kept fixed and use acceptance criteria absent from the training objective. A learning-based control policy, G-Maneuver, with 90.11% success is tested for stunt-level maneuvers. A looser terminal-heading score ranked the worst setting as the best. Because real stunt tests are safety-critical, a simulation-to-reality-to-simulation (Sim2Real2Sim) loop uses the simulation to select safe zones for real-vehicle trials with 16.9% acceptance. Automotive safety analysis frameworks are used to identify parameters for validation using coverage tests. We demonstrated that the kinematic stunt-driving indicators, namely heading change, sideslip, and speed-normalized yaw rate, match between simulation and the real world, while force and roll indicators do not, which identifies the indicators that can support safe deployment. A video of the experiments is available at https://youtu.be/dN1Oes8pNIo.
comment: 9 pages, 7 figures, 2 tables. Video: https://youtu.be/dN1Oes8pNIo
★ LMGN: A Unified Policy Learning Model for Last-Mile Grasp-Navigation
Legged robots equipped with manipulators have shown significant potential in the fields of daily life and industry in complex environments. However, existing methods mainly focus on global navigation or manipulation independently, while overlooking the last-mile navigation problem, where the robot must jointly determine the base placement, approach direction, and grasp execution to achieve reliable object grasping, and different manipulation stages require distinct control behaviors. To address these challenges, we propose Last-Mile Grasp-Navigation (LMGN), a unified policy learning model for Quadruped robot loco-manipulation. An Explicit Distance and Directional Reachability (ED-DR) representation is first introduced to explicitly represent the features of grasp-compatible base placement and approach direction. We then develop a semantically aligned Hierarchical Mixture-of-Experts (HMoE) policy that models stage-aware behaviors across the stages of base adjustment, pre-grasp alignment and grasp execution, and object lift. Finally, a new optimization method that jointly aligns navigation and manipulation, stage-dependent expert routing, and action-level consistency is proposed to optimize the HMoE policy. Extensive experiments on the Habitat and Isaac Gym simulators demonstrate that LMGN consistently outperforms state-of-the-art methods, denoting that integrating grasp-aware reachability with stage-aware policy learning significantly improves the success rates of last-mile navigation and target grasp for robots.
★ Concertina: Embodying Manipulation and Cognition in One Diffusion Vision-Language Model
Most vision-language-action (VLA) policies decouple manipulation from cognition by bolting an external action mechanism onto a vision-language model (VLM)--either quantizing actions into discrete tokens whose errors bear no stable correspondence to physical displacements, or delegating continuous trajectory generation to a separate action expert whose gradients compete with or are blocked from updating the VLM backbone. We argue that manipulation and cognition should instead be embodied in the same model, shifting from a decoupled VLA to an Embodied Vision-Language Model (eVLM). We introduce Concertina, an eVLM that places continuous trajectory representation, blockwise generation, and physical-coordinate supervision directly on the computation and parameter path of a shared diffusion VLM backbone. Each control timestep is encoded as one continuous latent action token in the backbone hidden space, and the shared Transformer reconstructs temporally ordered action blocks via the same corrupt-and-reconstruct operator used for multimodal pretraining. Active action projections account for only 0.21% of parameters in the 4.46B model with zero separate action-generator layers, compared with 16.5%-35.1% dedicated action-expert capacity in representative continuous VLAs. Using our proposed Action Interface Audit, we confirm that cross-time trajectory coupling emerges in the middle and late layers of the shared backbone rather than at the action interfaces. On standard benchmarks, Concertina achieves 98.15% average success on regular LIBERO, 85.9% on the 10,030-task LIBERO-Plus benchmark, and 4.142 average completed chain length on CALVIN ABC-D. Code: https://github.com/lose4578/Concertina Hugging Face: https://huggingface.co/collections/Traly/concertina
★ Fast and Reliable Open-Vocabulary Object Search with Absence Detection from Robot Memory
Mobile manipulators deployed in everyday human environments, such as homes for assistive care or flexible automation, often need to locate specific objects on demand from accumulated observations. Equally important, they must reliably determine when a target is absent before committing to navigation or interaction. Existing approaches demand either slow exhaustive re-processing or expensive 3D representations, and are evaluated on recall and accuracy rather than specificity, leaving their ability to abstain on missing targets unmeasured. We find existing systems that search using text or visual embedding almost never abstain on absent objects achieving at most 9\% specificity. To support accurate search at speed, we present a two-stage architecture that decouples fast candidate retrieval from verified selection, enabling both high recall and reliable absence detection from sequential RGB observations with pose. We evaluate eight retrieval-selection configurations plus an exhaustive search baseline across ScanNet apartment scenes, including significant numbers of absent objects. Adding a visual verification step to caption-based retrieval matches or exceeds exhaustive search reliability while requiring an order of magnitude fewer detector calls. We reproduce ScanNet scores on a physical robot operating entirely on-board with a single consumer GPU running in real time.
comment: 7 pages, 7 figures
★ Kinematic analysis of a double-closed-loop four-bar chain-based leg for a multimodal quadruped-pentarotor robot
This paper presents the kinematic modeling and singularity analysis of a three-degree-of-freedom, double four-bar closed-loop quadruped leg mechanism. This leg is essential for transitioning between the two operational modes; therefore, it is examined in both joint and task spaces, leading to the development of its Jacobian matrix, which is vital for designing the gaits required for terrestrial mode and mid-air aerial reconfiguration (for aerial mode). Various gaits are devised for the system's motion in the MATLAB simulation, and their practical implementation on the single-leg test rig is demonstrated.
comment: Journal paper submitted to Robotica with ID:ROB-2026-0373
★ FREE-LIVO: Physically Decoupled Yet Tightly Coupled LiDAR-Inertial-Visual Odometry
LiDAR-Inertial-Visual Odometry (LIVO) integrates complementary LiDAR, camera, and IMU measurements to achieve robust state estimation. However, most existing methods assume a fixed relative pose and accurate time synchronization between sensors. Toward modular and heterogeneous robotic systems, this paper presents FREE-LIVO, a tightly coupled odometry method that requires neither a rigid LiDAR-camera rig nor LiDAR-camera time synchronization. The proposed method uses a pretrained image feature matcher to establish correspondences between a camera image and a dense LiDAR intensity image generated from a single LiDAR scan using a generative model. From the resulting 3D point-to-pixel correspondences, we construct reprojection factors that directly constrain LiDAR and camera poses at different acquisition times. These factors are jointly optimized in a single factor graph together with constraints from point cloud registration, visual feature tracking, and the IMU measurements associated with each sensor. On six sequences from the public M3DGR dataset, FREE-LIVO achieves an average absolute trajectory error (ATE) RMSE of 0.45 m without provided LiDAR-camera extrinsic parameters and exhibits minimal accuracy degradation under time synchronization errors of up to 0.5 s. Furthermore, experiments with the LiDAR and camera mounted on separate robots demonstrate that the method can jointly estimate the trajectories of independently moving sensors.
★ Closed-Loop Physics-Based Manipulation Evaluation at a Commodity Price: What 4.4 L4 GPU hours at $3.50 Buys You IROS 2026
Running evaluations of contact-rich robotic manipulation at statistical scale has largely been possible only at well-resourced labs, highlighting the need for affordable and efficient options for broader audiences. We present a reproducible evaluation harness wrapper based on a previous study at 200-rollout scale on serverless commodity GPUs, at a cost of approximately $3.50 USD and 38 minutes of wall-clock per replication (4.4 L4-GPU-hours across eight parallel shards). Using the harness, we report a partially controlled negative result on action chunking with sparse data: three independently trained chunked checkpoints produce zero genuine successes across 780 rollouts, while three single-step checkpoints range from 2.6% to 11.8% success rate. A per-step-replanning control partially isolates chunk prediction, rather than open-loop chunk execution as the mechanism, with the failure localized to the training formulation and multi-step prediction. This work further democratizes physics-based policy development and evaluation through our reusable harness.
comment: 4 pages, 2 figures. Accepted to IROS 2026 Workshop on Building Scalable Infrastructure for Robot Learning: From Data Scaling to Real-World Deployment
★ MessyNav: Zero-Shot Navigation via Manipulation in Messy Environments
Real-world environments are messy, with paths often blocked by everyday objects like chairs and toys, requiring robots to perform interactive navigation by manipulating obstacles to clear a path. Existing methods rely on extensive simulation-based learning or per-scene instructions specifying which objects to avoid or manipulate. They struggle to generalize across novel objects, diverse environments, and varied contexts such as a cart of fragile glassware that should be avoided while an empty one can be pushed. We formalize this problem as open-world interactive navigation and present MessyNav, a zero-shot framework based on vision-language models (VLMs). At its core, MessyNav performs explicit VLM-based affordance reasoning before task planning to decide whether to swipe, pick up, or avoid an object and produce a high-level plan. Evaluated on a mobile manipulator across real-world indoor and outdoor environments, MessyNav improves success rate by 27 percentage points (pp) over the strongest baseline in real-robot trials and achieves statistically significant gains in a larger-scale offline evaluation, without task-specific training.
comment: CoRL 2026
★ Kinematic Hypothesis Context: Embodiment-Conditioned Learned Video Coding for Bandwidth-Limited Robot Policy Uplinks
Robots increasingly run large vision-language-action policies on remote servers, making the camera uplink the bottleneck. Existing video codecs spend bits on content the receiver could predict: the robot's own motion. We introduce the kinematic hypothesis context, a zero-rate conditioning signal that both ends build from joint states, the robot model and the camera calibration. Ego-motion homographies, per-link robot reprojections, an end-effector plane and a background memory warp the previous reconstruction into hypotheses that condition a learned video codec. We prove that history-only and global-pose conditioning with a limited receptive field cannot access this information under large motion, and that coding gains become task gains at the policy's success cliff. On LIBERO, the context saves 23.9% bit rate over a capacity-matched history-conditioned codec and 40-56% over x264, x265 and SVT-AV1, and needs 23.2% fewer bits than DCVC-RT at equal policy action deviation; trained to lower rates, it matches DCVC-RT in PSNR. In closed loop at equal rate, OpenVLA-OFT success nearly doubles from 22.8% to 43.1% at 6 kbps (72.6% with lower-rate training, 23.6 points above DCVC-RT), pi_0.5 success rises from 55.5% to 77.4%, and the gain grows under packet loss. The context transfers across three codec hosts and to real robot data.
comment: 15 pages, 12 figures. Preprint; supplementary material and code to be released upon formal publication
★ Dynamic Watermarking and Action-Conditioned Latent Dynamics for Secure Vision-Based Controllers
Replay attacks are difficult to detect in vision-based control systems because previously recorded observations can remain plausible while the physical system follows a different trajectory. This paper develops an adaptive dynamic-watermarking framework for detecting replay attacks against vision-based controllers. A private Gaussian watermark is added to the nominal control input, and a control input-conditioned latent dynamics network (LDN) predicts the corresponding evolution in a compact latent space learned from image observations. Attack evidence is obtained from the discrepancy between the predicted latent response and the latent representation of the received image observation. A reinforcement learning policy adjusts the watermark covariance using the latent state and detector confidence, thereby balancing attack detectability, watermark energy, and control distortion. A Lagrangian penalty on task-specific safety costs is included during training to limit the adverse effect of the excitation. The framework is evaluated on a vision-guided UR5 manipulation task in which the robot reaches a target while avoiding a moving human arm under nominal and replay-attacked observations.
comment: 8 pages, 3 figures, Submitted to ACC 2027 conference
★ An Ordering Optimization for Fruit-Vegetable in High-Density Environment based on Genetic Algorithm
This paper presents a robotic harvesting method that combines RGB-D fruit localization, GA-based harvest ordering, and waypoint-based robot motion in dense and unstructured crop environments. The global RGB-D camera provides 3D fruit positions for ordering, while the end-effector-mounted camera provides close-range observation during execution. The GA determines the harvesting sequence, and static or dynamic waypoints define the intermediate robot motion. The method was evaluated in a controlled laboratory testbed and in four-, five-, and six-fruit field conditions. GA solutions were compared with exhaustive enumeration, and both waypoint offsets were examined by parameter sensitivity analysis. Continuous execution reduced unnecessary Home-return motion, while the dynamic waypoint changed with the remaining fruit distribution without changing the GA sequence. In the field evaluation, the dynamic-waypoint continuous strategy produced 40 collision events and a 70.7% target-approach success rate, compared with 55 collision events and 65.3% for the static-waypoint continuous strategy.
comment: 39 pages, 11 figures
★ Efficient Diffusion Policy Fine Tuning with Latent Noise Representation Bridging NeurIPS
Pre-trained diffusion and flow policies have emerged as powerful backbones for robotics, yet adapting them to novel data via reinforcement learning (RL) remains computationally expensive, sample inefficient and lacks theoretical guarantees. Prior approaches that fine-tune the initial noise bypass costly backpropagation through time but struggle with the "noise aliasing" problem: there does not exista natural link between the noise space value and the action space value. This paper resolves these challenges by introducing the Latent Noise Linear Markov Decision Process (LNL-MDP). We derive the Bridge Equation for LNL-MDPs which eliminates the noise aliasing problem by aligning values with a shared representation between the noise and the action space. Furthermore, our LNL-MDP fine tuning framework falls under the FineTuneRL theory with near-optimal guarantee. Leveraging these insights, we propose Latent Representation Bridging for Diffusion (LARBRID), an algorithm that efficiently learns the representation to optimize the initial distribution that steers the pre-trained model to act optimally in the environment. It yields up to 2-3$\times$ performance increases on challenging control problems across a diverse suite of robotic locomotion and manipulation tasks. Code can be found at: github.com/Maincode-Research/LARBRID.git.
comment: Conference on Neural Information Processing Systems (NeurIPS), 2026
★ VREgo: Consumer VR Headsets for Learning Robot Manipulation from Egocentric Demonstrations
VREgo trains manipulation policies from demonstrations recorded on consumer VR headsets; we instantiate it with EgoKit on one such headset, the PICO 4 Ultra. On-device head and hand tracking and stereo object localisation allow the operator to begin a demonstration at the table. We adapt HumanEgo's preprocessing to estimate grasp thresholds per recording, update object positions after release, and propagate segmentation masks between keyframes. We evaluate policies in simulation on two tabletop tasks recorded by one operator. For cube placement, we compare PICO and Project Aria recordings of the same 41 layouts. With 40 training demonstrations, the PICO policies placed the cube on the tray in 67, 74, and 60 of 100 evaluation layouts across three training seeds. The corresponding Aria results were 73, 78, and 0. For a two-stage stacking task recorded with the PICO alone, the baseline placed the block on the tray in 94 of 100 layouts and met the end-of-episode criterion for both placements in two. In one diagnostic episode, repeated replanning deferred release beyond the eight executed steps. Demonstration replays also showed that the open gripper could displace the cube before grasping it. After adapting the training trajectories and adding task-specific execution rules that use simulator object positions, the system met both placement criteria in 93.8% of episodes across four evaluation noise seeds. The same 100 layouts informed rule development and evaluation. VREgo is available at https://vrego.chuange.org/.
★ CRISP: Constraint-Resolved Interactions between Subsystems for Predictive Whole-Body Cooperation
This paper presents a layered control framework that recasts high-dimensional system dynamics as a composition of dynamically interconnected subsystems, with dynamic dual-arm manipulation on the humanoid serving as a representative application. The humanoid is decomposed into a single rigid body (SRB) representing the torso and floating-base dynamic models representing the articulated arms and lower-body. The rigid connections between arm and torso and the closed kinematic chain formed between the end-effectors during cooperative manipulation are represented as holonomic constraints. The corresponding Lagrange multipliers parametrize the interaction wrenches transmitted at the shoulders and between the end-effectors. The high-level nonlinear model predictive control (NMPC) jointly optimizes the subsystem state trajectories, arm torques, ground reaction forces (GRFs), and interaction wrenches, enabling online wrench design and optimization while preserving the dynamic coupling among the independently modeled subsystems. The optimized arm torques are applied directly for cooperative dual-arm manipulation, whereas the prescribed GRFs are mapped to lower-body joint torque for whole-body coordination. We validate the proposed framework through numerical simulations and hardware experiments on a G1 humanoid robot. The results demonstrate optimal interaction wrenches consistent with simulated contact measurement, cooperative dual-arm trajectory tracking, and stable whole-body coordination while executing the NMPC in online manner.
comment: 7 pages, 8 figures
★ An Empirical Study of Reinforcement Learning for Keypoint Positioning of Deformable Linear Objects
Deformable linear object (DLO) manipulation is challenging because high-dimensional configurations, nonlinear contact dynamics, and diverse material and geometric properties produce a broad range of object responses that are both difficult to model and data-intensive to learn. This work studies reinforcement learning for planar manipulation of a one-end-grasped DLO, focusing on training efficiency, observation design, robustness, and zero-shot sim-to-real transfer. We compare three Proximal Policy Optimization training stacks and find that the GPU-parallel MuJoCo Playground stack enables faster convergence and more consistent performance than CPU-based alternatives under the same wall-clock budget. Relative keypoint observations and an asymmetric actor-critic formulation further improve learning efficiency. Policies trained with observation noise, delay, domain randomization, and disturbances achieve simulation success rates of 96.4% for single-point tracking, 98.0% for disturbance recovery, and 88.8% for two-point tracking. Using markerless RGB-D keypoint estimation, the policies transfer without real-world fine-tuning, succeeding in 9/10, 10/10, and 8/10 physical trials, respectively.
★ VICTR: Value-Informed In-Context Learning for Vision-Language-Action Models
Recent advances in robot foundation models have highlighted the potential of in-context learning (ICL) for adapting robot behavior from demonstrations without further fine-tuning. Previous work has used retrieval-based post-training to equip pretrained vision-language-action (VLA) models with the capability for ICL by teaching them to condition action predictions on retrieved visually-relevant demonstration chunks. However, the effectiveness of retrieval hinges on selecting context relevant to the current execution stage, which visual similarity alone may not capture. We introduce VICTR, a framework enabling in-context learning in VLAs via progress-informed context retrieval built around an In-Context Value Function Estimator (IC-VFE). IC-VFE conditions on a reference demonstration video, a language instruction and the current observation to estimate task-relative progress. By conditioning progress estimation on a reference demonstration, IC-VFE can align the current state with a demonstrated execution even for tasks unseen during training. VICTR uses the estimated progress to select demonstration chunks at corresponding stages of execution and supplies them to a $π_{0.5}$ policy post-trained for in-context action generation. On unseen out-of-domain real-world manipulation tasks, VICTR raises the success rate from 6.2% for a fine-tuned $π_{0.5}$ baseline to 23.8% and more than doubles the task progress rate, from 28.9% to 55.5%. Project page with code, data and videos: https://ai4ce.github.io/VICTR/
comment: 11 pages, 8 figures, 4 tables. Project page: https://ai4ce.github.io/VICTR/
★ National and State-Level Approaches to Automated Vehicle Certification: A Comparative Framework for State Operational Governance
Automated vehicle (AV) deployment in the United States is expanding without a unified federal certification framework, leading to substantial variation in how states authorize and oversee AV testing and deployment. This paper examines how U.S. states regulate Society of Automotive Engineers (SAE) Level 4-5 automated driving systems (ADS), identifies common certification approaches, compares them with international practices, and evaluates the implications for North Carolina's future regulatory strategy. Publicly available state statutes, administrative regulations, agency guidance, and federal policy documents were reviewed. Ten states, selected to capture the range of regulatory approaches, were analyzed using a structured framework covering operational scope, governance and oversight, federal interaction, and implementation outcomes. Certification practices in the European Union, China, Japan, and Australia were reviewed for comparative context. Three state certification models were identified, reflecting increasing regulatory involvement: baseline legal compliance, administrative self-certification with ongoing oversight, and permit-based authorization requiring formal agency review before deployment. North Carolina currently aligns with the baseline compliance model. International frameworks are converging toward scenario-based testing, operational-design-domain-specific approval, safety-case documentation, cybersecurity assurance, and lifecycle oversight, although direct transferability to the U.S. is limited. The paper develops a three-model typology for comparing state AV certification approaches and integrates state practices with international frameworks, providing a consistent basis for evaluating differences in regulatory involvement, governance complexity, and institutional responsibilities across jurisdictions.
comment: Transportation Research Board Annual Meeting 2027
★ ForPC: Force-Aware Whole-Body Control for Precise Humanoid Cart Pushing
Humanoid robots are expected to take over strenuous material handling in manufacturing, logistics, and healthcare, where operating existing wheeled carts would let them move heavy loads without modifying the workplace. However, humanoid cart pushing remains challenging because the bimanual grasp closes a force-coupled loop in which the contact wrenches that drive the cart also disturb the robot's balance, and payload changes alter the coupled dynamics. We propose ForPC, a force-aware whole-body control framework that regulates cart velocity through two-stage reinforcement learning. A locomotion prior is first trained under varied hand loads and upper-body postures to provide balance support. A force latent and a state latent are then inferred from onboard observations, without external force sensors, and fed to a residual policy that coordinates bounded arm, waist, and leg adjustments with base-velocity commands. Multi-objective rewards penalize bimanual internal opposition, grasp slip, and actuator saturation, encouraging the policy to apply contact force on demand without human demonstrations. Sim-to-sim experiments show that ForPC reduces cart-velocity RMSE by up to 68.2% relative to a locomotion-only baseline (Loco-only) and cross-track RMSE by up to 74.2% on the multi-bend route. Trained solely in simulation, the policy also transfers to a physical Unitree G1 without real-world training or fine-tuning and, with no payload-specific adjustment, pushes a cart carrying up to 30 kg at 0.5 m/s. These results suggest that humanoid robots could eventually replace humans for efficient material handling in daily life.
comment: 14 pages, 7 figures, 4 tables. Project website: https://roboforpc.github.io/. Corresponding author: Cheng Cheng
★ CS-Pi: Constraint-Satisfying Flow Policy Sampling
A flow-matching policy deployed in the field must often satisfy a constraint that was absent from its demonstrations, such as avoiding an obstacle it has never seen. Retraining for every new constraint is impractical, so the constraint must be enforced at inference time. Enforcing it must not discard what the policy learned: an action the policy would rarely sample may avoid the obstacle yet fail the task. We therefore want actions that are both safe, satisfying the constraint, and plausible, remaining likely under the policy. Gradient guidance, the most common approach, steers noisy states, so it must evaluate its cost on the policy's running estimate of the clean action; when the policy is uncertain between passing left or right of an obstacle, this estimate is their average and can pass straight through it. We instead build on rejection and projection. Rejection redraws actions until one is safe: every output is plausible, but the wait is intractable when safe actions are rare. Projection moves an action to the nearest safe one: it is efficient, but the result need not be plausible. Our key insight is that flow-matching sampling lets us combine the two, keeping projection's efficiency and rejection's plausibility. CS-Pi, our draw-project-renoise sampler, projects each draw to the nearest safe action, adds noise back scaled to how far the projection moved it, and re-runs the unmodified policy until a draw is safe. Replacing only the projection step extends CS-Pi to continuous costs. Across a 2D diagnostic, a 7-DoF shelf pick-and-place, an orientation preference, and a zero-shot Franka deployment, CS-Pi reaches 81% safe success on the shelf and 55% on hardware, against 55% and 40% for projection, while staying plausible.
comment: Accepted to CoRL 2026. Project page: https://sumo048.github.io/CS-pi_site/
★ HaNDF: Object-Conditioned Geometric Neural Hand Distance Fields NeurIPS 2026
We propose, HaNDF, object-conditional neural distance fields as data-driven priors for modeling the plausible hand-object configurations, in the product manifold of hand articulations. While recent unconditional generative priors showed success in modeling human pose, hands are versatile articulated bodies often found in interaction with other objects. This makes it crucial to develop a generative prior which can model the plausible hand poses conditioned on the object under interaction. To this end, our HaNDF proposes a geometry-aware, object-conditioned neural distance field, representing the plausible articulations in the zero-level set of a conditional field. We leverage tools from Riemannian geometry to (i) represent plausible hands in the zero level-set of an object-induced neural field, and (ii) project a given pose onto this field. Using HaNDF, we can also optimize for a plausible condition, determining the object under interaction. Our extensive evaluations demonstrate that our HaNDF provides a flexible and powerful prior for hand-object interaction, suitable for applications such as pose refinement, reconstruction under occlusion, and physically plausible manipulation synthesis.
comment: Accepted at NeurIPS 2026
★ Meituan-Robotics-0: A Vision-Language-Action Foundation Model for Desktop Manipulation
We present Meituan-Robotics-0 (MR0), a vision-language-action foundation model and embodied-agent system for stationary-base desktop manipulation. MR0 combines 147 million vision-language samples with 15,060 hours of curated robot trajectories spanning 12 embodiments. A three-stage training recipe aligns multimodal representations with robot behavior through discrete action-token pre-training, transfers these priors to continuous cross-embodiment control through flow-matching mid-training, and adapts the policy through real-world post-training. We introduce Rollback, which pairs replaced policy segments with human corrections to provide positive and negative supervision for CFG-DAgger without value learning or advantage labeling. A unified end-effector interface supports heterogeneous embodiments, while asynchronous action-chunk merging maintains smooth execution under variable inference latency. An event-driven agent maintains persistent task events and reassesses plans when progress or feasibility changes, enabling cross-task retrieval and conditional task resumption. MR0 achieves 93.6% success on RoboTwin 2.0 and ranks fifth in the RoboDojo VLA track. Under matched HG-DAgger post-training, MR0 achieves higher average success rate and progress score than $π_{0.5}$ across three real-robot tasks. On Ethernet insertion, CFG-DAgger trained with Rollback data reaches 70% success and a progress score of 0.90. Qualitative agent deployments demonstrate cross-task event retrieval and recovery from temporarily unsatisfied task conditions.
comment: 25 pages. Project page: https://meituan-robotics.github.io/Meituan-Robotics-0/ ; Code: https://github.com/Meituan-Robotics/Meituan-Robotics-0
★ Q-OIL: Optimistic Exploration in Learning from Human Interventions
As robots are deployed in the real world, they will inevitably make mistakes. Human interventions provide valuable feedback for improving their behavior, enabling them to adapt online to new scenarios. However, studying human-in-the-loop robot learning is difficult: real-world experiments are slow and expensive, so most algorithm development relies on simulated human models. We show that common human modeling assumptions do not match how real people intervene in practice. Through analyzing intervention data from human participants, we find that humans tend to intervene when agent progress stalls, and that their corrective actions are often suboptimal. Motivated by these findings, we introduce a progress-based model of human intervention that achieves higher rank correlation with real-user algorithm rankings, enabling realistic algorithm benchmarking and development in simulation. Building on these insights, we propose Q-OIL, a method that adds optimistic bonuses to intervention transitions in reinforcement learning and softly guides exploration toward human corrections. We find that decoupled critics are critical for stability because they prevent the bonuses from backing up to pre-intervention states. As a result, Q-OIL learns more efficiently than competitive baselines, achieving 10-50% higher success in simulation and on real robots, and improving pretrained VLAs on two bimanual manipulation tasks. As we ground both evaluation and algorithms in realistic human behavior, this work provides a practical path toward scalable robot learning from interventions.
comment: Project website: https://weirdlabuw.github.io/qoil
♻ ★ SAGE: A hybrid vision-language and geometry framework for work-zone pathway prediction under temporary traffic control
Temporary traffic control redirects vehicles through work zones, requiring automated vehicles to connect an interpretation of the visible devices to a continuous passing path. Geometric predictors provide several plausible paths, yet selecting the one consistent with the temporary control remains difficult. We propose SAGE (Semantic-to-Action Grounded Execution), a read-decide-execute-check framework that separates path construction, path preference, and control compliance within a shared candidate space. The central idea is to use a vision-language model's (VLM's) trajectory as a preference over intact geometric candidates. A role-decoupled generator constructs the candidates, and rank fusion combines each candidate's geometric score with its agreement with the VLM trajectory. Supervision structured around the Manual on Uniform Traffic Control Devices (MUTCD) trains SAGE-VLM to state the lane status and hazard side before its passage decision and trajectory. The predicted hazard side enters image-space predicates derived from MUTCD provisions, which support a final intervention bounded in score and displacement. On 2,306 real-world ROADWork validation episodes, the candidates include a path within 10 px of the human demonstration in 87% of episodes, yet selection regret accounts for 53% of the geometric ranker's error. SAGE reduces average and final displacement errors by 8.3% and 12.0% relative to the ranker and hard-constraint violations by 37%. The trajectory opinion improves all four tested generator-selector combinations. SAGE is comparable to state-of-the-art VLM modules in accuracy, with a faster-responding VLM, and outperforms four other adapted driving VLMs in displacement error and safety measures. Project page: https://hhjj233.github.io/SAGE-page/.
comment: 26 pages, 9 figures, 9 tables. Project page: https://hhjj233.github.io/SAGE-page/
♻ ★ ProCompNav: Proactive Instance Navigation with Comparative Judgment for Ambiguous User Queries NeurIPS 2026
In human-robot interaction, agentic robots should resolve ambiguous user requests through proactive clarification while keeping user burden low. We study this challenge in interactive instance navigation, where an agent must identify and reach the user-intended target among similar objects. Existing approaches ask open-ended questions and select a candidate that strongly matches the accumulated user-provided information. However, this process can require lengthy user responses and lead the agent to make premature decisions without distinguishing the target from distractors. To address these limitations, we propose Proactive Instance Navigation with Comparative Judgment (ProCompNav), a two-stage framework that first constructs a candidate pool and then identifies the target through Recursive Comparative Judgment (RCJ). RCJ iteratively narrows the pool by selecting an attribute-value pair that divides the candidates, asking the user a binary question, and removing inconsistent candidates, without requiring an attribute unique to the target. ProCompNav substantially improves Success Rate and reduces Response Length on an interactive navigation benchmark, and achieves the highest Success Rate among evaluated methods on a non-interactive benchmark. In two human studies, participants preferred ProCompNav's binary questions to open-ended questions and its batched delivery to interleaved interaction. Our results highlight the need to jointly consider task performance and user-friendly interaction strategies when designing agentic robots.
comment: Accepted to NeurIPS 2026 (Oral), Project page: https://tree-jhk.github.io/procompnav/ Code: https://github.com/ml-postech/ProCompNav
♻ ★ OpenTac: An Open Ecosystem for Customizable, Scalable, and Transferable 3D Tactile Sensing
We present OpenTac, a novel sensor and a full-stack ecosystem for tactile skins that are highly customizable, scalable, and readily deployed in downstream robot control and learning applications. Our core contribution is twofold. First, we develop a scalable resistive tactile skin with a structured deformable overlay that converts tangential loading into spatially varying pressure. Differential responses across neighboring "taxels" support decoding of local shear vectors, and the common-mode response encodes normal force. Second, we provide a highly customizable sensor design tool that is able to automatically generate fabrication files for these tactile skins from 2D outlines and developable 3D surfaces, including from abstract descriptions such as text. We develop an ecosystem around the OpenTac sensor, including the CAD design tool, a sensor simulator, cross-sensor transfer, and fast/adaptive data acquisition. We demonstrate the broad applicability of the OpenTac-CAD sensor design tool by designing and fabricating more than 30 sensors across a variety of robot embodiments. We characterize the three-dimensional tactile fields produced by OpenTac with a commercial, high-precision force/torque sensor. We demonstrate the broad applicability of 3D tactile sensing across contact-rich feedback control and manipulation policy learning on four real-world robot embodiments. Additionally, adaptive acquisition enables kilohertz-rate partial readouts, and our readout circuitry enables high dynamic range (HDR) acquisition, measuring, for example, a 3.7 kg weight stack and a 20 g weight on the same sensor in the same frame. We provide all hardware, software, and result videos at opentac.github.io
comment: 15 pages, 15 figures. Project website, open-source hardware and software, and demonstration videos: https://opentac.github.io
♻ ★ GUARD: Sensitivity-aware Uncertainty Attribution for Safe Traditional and Learning-based Robot Control
This paper presents the framework GUARD (Guided robot control via Uncertainty attribution and probAbilistic kernel optimization for Risk-aware Decision making) that combines a technique from probabilistic state estimation with an uncertainty identification method to derive critical thresholds for robot control algorithms. The first part focuses on complementing a method for probabilistic kernel optimization (PKO) by a sensitivity-aware uncertainty attribution technique. Next, the estimated uncertainties and their sources are forwarded to the robot controllers to adapt safety margins and trigger the initialization of safe backup trajectories. To validate GUARD's applicability and performance, experiments are conducted on a traditional model predictive control (MPC) algorithm and a reinforcement learning (RL) based controller. The results demonstrate that GUARD reduces collision rates by 44-67% in the RL-based evaluation, while decreasing backup policy activation by 50-75% in the medium- and high-uncertainty profiles. In the MPC simulation, GUARD successfully avoids collisions. By unifying the probabilistic kernel optimization algorithm from robot state estimation with the uncertainty estimation method for learning-based control, GUARD brings together traditional techniques and learning for safe and efficient robot applications.
comment: 9 pages, 9 figures, 2 tables. Substantially revised and expanded version with updated title and authors. Extended validation using model predictive control and reinforcement learning-based control
♻ ★ Networked Admissibility-Preserving Control for Directed Safe Coordination
This paper addresses safe coordination of scalar agents with constrained input dynamics over fixed rooted digraphs. We propose a networked Admissibility-Preserving Control (APC) architecture for a common moving output corridor under heterogeneous asymmetric input bounds, wherein the Admissibility-Preserving Input Realization (APIR) generates the physical inputs. In logarithmic barrier coordinates, the closed-loop dynamics form an exact cascade of exponentially decaying realization errors and directed consensus dynamics. We derive direction-specific compatibility conditions that guarantee uniform corridor and input margins, bounded commands, unique complete solutions, and exponential consensus for compact initial sets. The closed-form expression for the agreement value quantifies the effect of the initial APIR mismatch. Under the corresponding compatibility conditions, partial pinning guarantees safe reference tracking whenever the root component contains at least one informed agent. Simulations illustrate the coordination guarantees.
♻ ★ PhysTrace: Interpretable Temporal Evaluation of Physical Consistency in Video Generative Models
World action models increasingly rely on pretrained diffusion/flow-based video generators assumed to capture physical dynamics for prediction, planning, and action generation. Existing evaluations assess plausibility, probe representations, or measure predictive surprise, yet physical consistency is history-dependent: preceding events constrain what can physically occur later. We introduce PhysTrace, a training-free framework that uses MLLMs to formulate history-conditioned physical expectations and tracks their agreement with observed video windows through temporal video--language alignment, rather than relying on direct plausibility judgments or task-specific probes. On IntPhys2, PhysTrace achieves 67.2% possible--impossible discrimination accuracy, outperforming the evaluated direct MLLM baselines. We further introduce reference-guided masked denoising for controlled reconstruction from mixed possible--impossible video latents. To support these reconstructions and evaluate temporal localization as an additional capability, we introduce IntPhys2-Timing, adding violation intervals to IntPhys2, with PhysTrace achieving a Hit Rate of up to 77.4%. On these reconstructions, human evaluations reveal model-dependent differences in physical consistency and fewer physically impossible outcomes at higher initial noise levels, trends broadly captured by PhysTrace.
comment: 19 pages, 8 figures, 7 tables, including supplementary material
♻ ★ RAYEN: Imposition of Hard Convex Constraints on Neural Networks
Despite the numerous applications of convex constraints in Robotics, enforcing them within learning-based frameworks remains an open challenge. Existing techniques either fail to guarantee satisfaction at all times, or incur prohibitive computational costs. This paper presents RAYEN, a framework for imposing hard convex constraints on the output or latent variables of a neural network. RAYEN guarantees constraint satisfaction during both training and testing, for any input and any network weights. Unlike prior approaches, RAYEN avoids computationally expensive orthogonal projections, soft constraints, conservative approximations of the feasible set, and slow iterative corrections. RAYEN supports any combination of linear, convex quadratic, second-order cone (SOC), and linear matrix inequality (LMI) constraints, with negligible overhead compared to unconstrained networks. For instance, it imposes 1K quadratic constraints on a 1K-dimensional variable with only 8 ms of overhead compared to a network that does not enforce these constraints. An LMI constraint with 300x300 dense matrices on a 10K-dimensional variable can be guaranteed with only 12 ms additional overhead. When used in neural networks that approximate the solution of constrained trajectory optimization problems, RAYEN runs 20 to 7468 times faster than state-of-the-art algorithms, while guaranteeing constraint satisfaction at all times and achieving a near-optimal cost. Finally, we demonstrate RAYEN's ability to enforce actuator constraints on a learned locomotion policy by validating constraint satisfaction in both simulation and real-world experiments on a quadruped robot. The code is available at https://github.com/leggedrobotics/rayen
comment: Accepted at the International Journal of Robotics Research
♻ ★ VIGOR: Zero-Shot Visual Generalization via Latent-Space Consistency in Model-Based Reinforcement Learning NeurIPS 2026
Model-based reinforcement learning (MBRL) achieves strong sample efficiency by planning in learned latent dynamics but degrades substantially under unseen visual distractions such as background, lighting, or camera changes. Unlike model-free RL, where encoder perturbations affect only single-step predictions, MBRL suffers from a two-level vulnerability: distractions push encoder outputs out of distribution, and these errors compound through recursive latent rollouts. We propose visual generalization via latent-space consistency in model-based RL (VIGOR), which enables zero-shot generalization to unseen distractions while retaining its backbone's sample efficiency. VIGOR comprises three interdependent components: (i) asymmetric weak-to-strong augmentation, which pairs weak-only and weak-to-strong latent views within a batch; (ii) dynamics-level consistency, which enforces augmentation-invariant transition predictions via direct latent regression; and (iii) encoder-level stabilization, which prevents encoder drift under the cross-augmentation supervision of (ii). On the DeepMind Control Suite (DMC) and Robosuite, VIGOR instantiated on TD-MPC2 outperforms state-of-the-art model-free and model-based baselines in zero-shot generalization, surpassing the second-best baseline by 3.4% on DMC and 43.6% on Robosuite, with sample efficiency comparable to its backbone and fewer environment interactions than the other baselines. Ablations show that VIGOR is augmentation-agnostic: augmentations from distinct perturbation families preserve strong generalization, confirming that latent-space consistency, not the augmentation choice, drives robustness. A second instantiation on DreamerV3, a structurally distinct recurrent state-space model, also improves zero-shot generalization on both benchmarks, indicating backbone-agnostic benefits. Code and videos: https://vigor-gen.github.io/.
comment: Accepted at NeurIPS 2026. Revised version with additional experiments on DreamerV3 and updated discussions
♻ ★ AnyDexRT: Calibration-Efficient Dexterous Hand Retargeting with Few-Shot Human Guidance
Teleoperation is a key interface for controlling dexterous robotic hands and collecting demonstrations for imitation learning. Its effectiveness largely depends on kinematic retargeting, which maps operator hand motions to feasible and intuitive robot hand motions. Existing methods often require hand-crafted objectives, precise calibration, or global shape matching between human and robot hand spaces, making them sensitive to hand-specific tuning and less reliable across different dexterous hands. We propose AnyDexRT, a calibration-efficient retargeting method for intuitive dexterous teleoperation across human-like dexterous hands. AnyDexRT combines self-supervised fingertip correspondence learning with few-shot human guidance to anchor the mapping in task-relevant regions, and further refines pinch-related poses using a contact classifier. Experiments on diverse dexterous hands and real-world teleoperation tasks show that AnyDexRT improves retargeting quality, reduces manual tuning, and provides more intuitive and efficient control than prior retargeting methods. Project website: https://chenxi-wang.github.io/projects/anydexrt
comment: Correct parameter; simplify anchor collection. See Appendix E for details
♻ ★ WoVR: World Models as Reliable Simulators for Post-Training VLA Policies with RL
Reinforcement learning (RL) promises to unlock capabilities beyond imitation learning for Vision--Language--Action (VLA) models, but its requirement for massive real-world interaction prevents direct deployment on physical robots. Recent work attempts to use learned world models as simulators for policy optimization, yet closed-loop imagined rollouts inevitably suffer from hallucination and long-horizon error accumulation. Such errors not only degrade visual fidelity, but also mislead policy optimization by providing unreliable learning signals. We propose WoVR, a reliable world-model-based RL framework for post-training VLA policies. Instead of assuming a faithful world model, WoVR explicitly regulates how RL interacts with imperfect imagined dynamics. It improves rollout stability through a controllable action-conditioned video world model, reshapes imagined interaction to reduce effective error depth via Keyframe-Initialized Rollouts, and maintains policy--simulator alignment through World Model-Policy co-evolution. Extensive experiments demonstrate that WoVR enables stable long-horizon imagined rollouts and effective policy optimization, achieving superior LIBERO performance and consistent real-world gains across multiple robotic platforms. These results show that world models can serve as practical simulators for RL when hallucination is explicitly controlled. Additional visualization results are available at https://wovr-rlinf.github.io.
comment: 25pages, 11 figures
♻ ★ PartiGen: Part-Aware Representations for Language-Conditioned Humanoid Motion Generation
Translating language into humanoid motion requires coordinating body parts, determining motion duration, and producing references suitable for robot execution. We introduce PartiGen, a part-aware latent diffusion framework that incorporates anatomical structure into a shared whole-body motion representation. Part-specific attention processes anatomically masked frame features, and its outputs are fused to combine local limb information with whole-body context. Conditioned on text and motion history, the model generates motion primitives autoregressively. Forward-kinematics supervision encourages kinematic consistency, while a learned end-of-sequence predictor selects a frame-level endpoint after generation. On HumanML3D retargeted to the Unitree G1, PartiGen achieves a segment-level FID of 3.165, a 16.8% reduction over TextOp, and improves R@1 by 1.4 percentage points. In the duration study with 16-frame primitives, endpoint prediction reduces mean absolute duration error from 12.471 to 3.080 frames compared with a fixed-length cutoff. We further evaluate robot execution using a pretrained, frozen GRIT tracker: 409 of 500 generated references (81.8%) pass the posture-based stability screen in MuJoCo. Following simulation screening and manual selection, 26 of 27 attempted motions are judged successful on the real G1.
♻ ★ Memory before Goal: Role of Pre-History Memory for Humanoid Loco-Manipulation
Humanoid loco-manipulation in dynamic scenes is non-Markovian: cues vanish as objects change or are hidden, or leave the view as the robot turns and walks, yet whole-body vision-language-action (VLA) policies see only the present. We present WholeBodyMem, an object-guided perceptual memory that keeps the pre-history of object events seen before the robot acts on its goal. SAM3 tracks the task objects, and their events causally select key frames within an adaptive budget, which the action expert reads by cross-attention together with object tokens encoding appearance, geometry and time. We also contribute HuMeGo, a humanoid memory benchmark with six simulated and six real-world Unitree G1 tasks. Keeping the right frames rather than more frames, WholeBodyMem surpasses memoryless and uniform-memory baselines, raising its base policy's success from 29.1% to 72.6% in simulation and from 36.7% to 88.3% on the real robot, while also improving the standard SIMPLE tasks (70.2% to 86.9%).
comment: 8 pages, 4 figures. Project page: https://memory-before-goal.github.io. GitHub: https://github.com/crystalccy1/MemoryBeforeGoal
♻ ★ RoboCurate: Harnessing Diversity with Action-Verified Synthetic Data for Robot Learning
Synthetic data generated by video generative models has shown promise for robot learning as a scalable pipeline, but it often suffers from noisy action labels due to imperfect video generation or inaccurate action prediction. Recently, vision-language models (VLMs) have been leveraged to validate video quality, but they cannot directly evaluate the generated actions themselves. To tackle this issue, we introduce RoboCurate, a novel framework for action-verified synthetic robot data that addresses inconsistencies between generated videos and annotated actions. Specifically, RoboCurate executes the actions predicted by an inverse dynamics model (IDM) in a simulator and verifies action consistency by measuring the motion agreement between the simulator rollout and the generated video. To better leverage such a framework, we further expand the scene diversity of synthetic data beyond the available dataset through off-the-shelf generative models with structured editing axes. We then use a VLM to pre-filter physically implausible generated videos before verifying their action labels. Across diverse manipulation scenarios, RoboCurate yields substantial relative improvements in success rates compared to using only real data, achieving +90.3% on GR-1 Tabletop (300 demos), +29.1% on DexMimicGen in the pre-training setup on simulation benchmarks, and +179.9% and +42.9% in the co-finetuning setup on challenging real-world ALLEX and OpenArm humanoid benchmarks, respectively.
comment: CoRL 2026 G2A Workshop Accepted (Oral); Project page is available at https://seungkukim.github.io/robocurate/
♻ ★ CoDiMAD: Diffusion-Based Privileged Distillation for Communication-Free Multi-Robot Coordination
Decentralized multi-robot coordination under partial observability remains challenging, especially when agents act solely on local sensing without communication. Privileged policy distillation provides a promising approach by transferring knowledge from a globally informed oracle to sensor-constrained students. However, in multi-agent systems, the same local observation may correspond to multiple global configurations requiring qualitatively different cooperative actions, making the conditional action distribution inherently multi-modal. Standard deterministic distillation collapses these modes to their mean, often yielding invalid or hesitant actions. To address this issue, we propose CoDiMAD, a three-stage framework that trains a privileged oracle with MAPPO, constructs an offline dataset of local-observation-oracle-action pairs, and distills the oracle into decentralized students parameterized as conditional denoising diffusion probabilistic models. By approximating the conditional oracle-action distribution through the diffusion reverse process, CoDiMAD samples decisive actions from coherent coordination modes rather than averaging across them. Theoretical analysis characterizes the mode-averaging failure of deterministic distillation and the distributional recovery property of diffusion-based distillation. Experiments on three cooperative tasks show that CoDiMAD consistently outperforms direct local MARL and deterministic distillation baselines. The source code will be made publicly available upon acceptance.
♻ ★ Evo-DexBench: Benchmarking Generalization across Related Tasks for Dexterous Manipulation
General-purpose dexterous manipulation requires policies to transfer knowledge from related tasks to new tasks with only a few target demonstrations. Existing benchmarks often provide too many target demonstrations and leave the differences between source and target tasks uncontrolled. As a result, they cannot clearly distinguish knowledge transfer from task memorization. Our key insight is the Core-Gen principle. Core tasks are related tasks that share a functional interaction and provide source knowledge, while a Gen task is a novel target within the same functional family that has only a few demonstrations. This setup tests whether a policy can reuse functional interaction knowledge. Based on this principle, we propose Evo-DexBench, which controls both source-task relatedness and target-task exposure. It comprises 30 tasks in six functional families, three everyday scenes per task, and 1,248 multimodal demonstrations collected via VR teleoperation with simulated Shadow Hands equipped with 123 taxels. Experiments show that, among six representative policies, even the best-performing method achieves an average success rate of only 24.4%, highlighting the challenges existing methods face in transferring functional interaction knowledge. Further analysis shows that, across the two tasks, joint Core + Gen training improves performance by 12% and 38%, respectively, over training on Core followed by fine-tuning on Gen, indicating that joint training makes better use of Core knowledge. With matched Core data, 8 Gen demonstrations achieve performance comparable to training on 16--32 Gen demonstrations alone. On all three tested Gen targets, functionally matched Core data yields greater gains in success rate than mismatched data. We release the Evo-DexBench dataset and benchmark to support the evaluation of dexterous manipulation policies' generalization across related tasks.
comment: v2: Extended experiments to all 30 tasks in six families with an added QwenOFT baseline (best average SR updated to 24.4%); added a target-data efficiency study; extended tactile ablation to all 30 tasks; updated abstract, figures, and appendix
♻ ★ GigaBrain-WBC-0.5: A Behavior World Model for Robust Humanoid Whole-Body Tracking with Environment Interaction
General-purpose motion trackers enable humanoid robots to follow diverse whole-body motions while maintaining balance, but are trained only on flat ground, failing to exploit bipedal mobility over complex terrain. Cross-terrain controllers, meanwhile, are task-specific or accept only low-dimensional locomotion commands. We introduce InterTrack, the first behavior world model (BWM) for robust whole-body tracking with environment interaction. Its Transformer jointly predicts the next action, state, and behavior distribution, learning environment-conditioned dynamics. To scale interaction training data, an automatic annotation pipeline reconstructs 3D support geometry from retargeted motions. At deployment, the policy handles commands implausible in the current environment in a "best-effort" manner. Quantitatively, InterTrack achieves an 81.3% success rate on terrain interaction (4.3 times the best evaluated baseline) and a 99.3% fall-recovery rate, while also improving free-space tracking and outperforming three leading tracking baselines across all of these regimes. To our knowledge, we provide the first demonstration of real-time cross-terrain whole-body teleoperation on a humanoid robot, alongside object interaction, stable responses to missing supports, and robust recovery from falls.
comment: Technical report. Project page: https://shepherd1226.github.io/gigabrain-wbc-0.5/ Code: https://github.com/open-gigaai/giga-brain-wbc Data: https://huggingface.co/datasets/open-gigaai/GigaBrain-WBC-0.5-TerrainData
♻ ★ Why Koopman Structure Does Not Help Differentiable World Models for Humanoid Whole-Body Tracking
We test whether Koopman structure helps a differentiable world model used to train a simulated Unitree G1 whole-body tracking policy by backpropagation through imagined rollouts, replacing the MLP world model of an otherwise unchanged pipeline. It does not: return is at best equal (0.823 versus 0.830), one-step error on identical data is 1.3 to 1.6 times the MLP's, and training is about 15% slower. Lifted rollouts diverge or fail to train policies; of the schemes tested, re-lifting at every imagined step trains best. Re-lifting never uses the lifted linear dynamics and leaves a model affine in the action, with state-dependent offset and slope; an action-affine MLP also predicts worse, so the accuracy cost belongs to this class. At a fixed state, the dynamics' curvature in the action explains only part of it. Across states, the policy's actions span a curved region, so an affine model fits policy data only as the tangent at the policy's action, the operating point: on G1, the MLP's tangent there is as accurate as the MLP; at the mean action, 31 times worse. The offset must encode, from the state alone, where the policy acts, whereas an MLP reads the action. Supplying the policy's action removes most of the remaining gap on G1, Ant and Humanoid. The only tested online repair approaching the MLP feeds the action into the nonlinear lift, abandoning the structure. For this use, the MLP world model remains the better choice.
comment: 8 pages, 3 figures, 2 tables. Preprint
♻ ★ EmboTeam: Asymmetric State Grounding for Heterogeneous Multi-Robot Collaboration
Heterogeneous multi-robot collaboration over long horizons requires both global task planning and robust coordination during execution. Existing plan- ners using large language models (LLMs) can generate task decompositions or symbolic plans, but often execute them as rigid action sequences, making them fragile to local state mismatches and synchronization failures. We propose Em- boTeam, a neuro-symbolic framework for asymmetric state grounding. An LLM generates structured state expectations, which are verified and coordinated through symbolic search using Planning Domain Definition Language (PDDL) and then grounded into a team-level parallel behavior tree. Robot-specific subtrees per- form precondition checking, skill execution, effect validation, local replanning, and blackboard-based synchronization. We further introduce MACE-THOR, con- taining 42 heterogeneous multi-robot tasks across 8 indoor layouts. EmboTeam improves task success rate from 12% to 55% and goal condition recall from 32% to 75% over LaMMA-P, demonstrating the effectiveness of synchronized reactive grounding for collaboration over long horizons.
comment: Accepted to the 33rd International Conference on Multimedia Modeling (MMM 2027)
♻ ★ VTAM: Efficient Visuo-Tactile World-Action Models for Contact-Rich Manipulation without Tactile Pretraining
Tactile vision-language-action models (VLAs) and tactile world-action models (WAMs) are a promising new paradigm for contact-rich manipulation, but existing models either underutilize touch, rely on large-scale tactile pretraining, or incur high inference latency due to iterative video generation. We present VTAM, an efficient visuo-tactile world-action model for contact-rich manipulation that requires no tactile or action pretraining. RGB and tactile observations share a frozen video VAE and the pretrained video transformer weights. VTAM first adapts the video backbone to jointly predict future visual and tactile latents, then fine-tunes it end-to-end with a randomly initialized action expert that predicts actions, proprioceptive states, and a light flow-force proxy computed from tactile optical flow. Trained only on task demonstrations, VTAM outperforms all baselines on four real-robot tasks across two platforms with different tactile sensors, outperforms all baselines without tactile pretraining in average success on eight UniVTAC simulation tasks, and is comparable to tactile VLAs and WAMs pretrained on over 30K hours of largely visuo-tactile robot data (83.4% vs. 84.5% for N0-TWAM). Without full video denoising at inference, VTAM is approximately 6x and 16x faster than N0-TWAM and LingBot-VA without compilation, with compiled latency comparable to Fast-WAM. Ablations show that feeding frozen-VAE tactile features directly to the action expert, bypassing tactile world modeling, fails on every tested task, that Stage I visuo-tactile world-model fine-tuning improves UniVTAC success on most tasks, and that flow-force proxy keeps the policy from fully ignoring touch. The tactile encoder transfers zero-shot across GelSight Mini and Xense Photon sensors.
comment: 12 pages. Project page: https://plan-lab.github.io/projects/vtam/
♻ ★ Keeping Jerk in Check: Jointly Trained Perception-Gated Experts for Smooth Terrain Transitions
Perceptive legged locomotion across discontinuous terrain requires shared locomotion knowledge, task-specific maneuvers, and smooth transitions between them. Existing approaches train specialized policies independently and either switch among them with a hierarchical selector or distill them into a monolithic policy. Both fix the constituent behaviors before they are composed, and both are assessed through aggregate course outcomes that leave behavior at terrain boundaries uncharacterized. We instead propose CTS-MoE, which formulates multi-terrain locomotion as multi-task reinforcement learning and jointly trains a perception-gated dense mixture-of-experts actor with task-specific critics: soft routing composes expert knowledge into continuously varying behavior, while separated critics isolate heterogeneous objectives so that specialization survives joint optimization. A concurrent teacher-student formulation transfers this composition from privileged terrain information to onboard depth, so that at deployment routing is inferred from perception alone, with no terrain label, selector, or distillation stage. Against hierarchical and DAgger-distilled baselines, the policy raises connected-course success to 93.3% from 73.3% and 53.3%, lowers mean action jerk by about a third across the course, and avoids the switching transient that hard selection introduces at boundaries. On a Unitree Go1, it completes an extended outdoor course in 10/10 trials and transfers to four outdoor terrain types. Project Website: https://cts-moe.github.io/ .
♻ ★ Synthesizing Dexterous Robot Demonstrations from Human Videos via Virtual Try-On
Robot foundation models have shown remarkable performance in robot control, yet adapting them to a new task still relies heavily on teleoperated demonstrations. Egocentric human videos offer a scalable substitute, but overlaying rendered robots on human videos or retargeting human poses into robot actions leaves significant visual or action gaps for high-DoF robot embodiments. To address this limitation, we propose RoboTryOn, which treats human-to-robot translation as virtual try-on: try-on keeps the person and swaps the garment, RoboTryOn keeps the demonstrated motion and swaps the human for the robot. Specifically, RoboTryOn converts a human video into a video of the target robot specified by a reference image, while preserving the original task, objects, and background. To maintain visual consistency, we translate selected keyframes autoregressively and use them to guide video synthesis with motion conditioned on the human hand skeleton. Additionally, we use an inverse dynamics model (IDM) that incorporates depth features to infer action labels from synthesized videos for policy learning. Our experiments across multiple humanoid embodiments show that policies trained with our synthetic data improve the average success rate on novel tasks, including challenging bimanual manipulation, from 2% to 60%, without any real robot demonstration of those tasks. Moreover, training with our synthetic data matches the performance of policies trained solely on teleoperated demonstrations at approximately a third of the collection cost, demonstrating the cost efficiency of our approach.
comment: 29 pages, 15 figures, 13 tables. Project page: https://ryan-seungyong-lee.github.io/RoboTryOn
♻ ★ Vision-Language Models Are Joint Denoisers in Robotic Action Generation
Diffusion-based vision-language-action (VLA) models cascade pretrained vision-language models (VLMs) with diffusion action experts, but typically restrict the VLM to providing static conditioning. This separation limits how much pretrained perception and reasoning can guide action generation, especially in out-of-distribution settings. We introduce JADE, a Joint Action DEnoiser that enables the VLM and action expert to jointly refine actions. Our key insight is that structured action latents allow the VLM to read the current noisy action together with visual observations and language instructions. At every denoising step, the VLM contextualizes the noisy action and guides the action expert through gated cross-attention, so that its guidance evolves with the action being generated. We further supervise the VLM to recover clean action latents from noisy ones, which gives it a direct action-learning signal and couples perception with control. Additionally, joint denoising enables JADE to generate actions with fewer sampling steps, helping offset the additional cost of VLM participation. An adaptive step-size schedule further balances task performance and inference latency. On RoboDojo, JADE achieves an average success rate of 12.67%, the highest among the evaluated $π_{0.5}$-based methods, and is competitive with recent VLA and WAM foundation models pretrained at scale. It also improves performance on RoboTwin 2.0 and raises the average success rate across four real-world bimanual tasks by 20.8 percentage points compared with $π_{0.5}$.
comment: formatting error
♻ ★ CubeDAgger: Interactive Imitation Learning for Dynamic Systems with Efficient yet Low-risk Interaction IROS2026
Interactive imitation learning makes an agent's control policy robust by stepwise supervisions from an expert. The recent algorithms mostly employ expert-agent switching systems to reduce the expert's burden by limitedly selecting the supervision timing. However, this approach is useful only for static tasks; in dynamic tasks, timing discrepancies cause abrupt changes in actions, losing the robot's dynamic stability. This paper therefore proposes a novel method, named CubeDAgger, which improves robustness with less dynamic stability violations even for dynamic tasks. The proposed method is designed on a baseline, EnsembleDAgger, with three improvements. The first adds a regularization to explicitly activate the threshold for deciding the supervision timing. The second transforms the expert-agent switching system to an optimal consensus system of multiple action candidates. Third, autoregressive colored noise is injected to the agent's actions for time-consistent exploration. These improvements are verified by simulations, showing that the trained policies are sufficiently robust while maintaining dynamic stability during interaction. Finally, real-robot scooping experiments with a human expert demonstrate that the proposed method can learn robust policies from scratch based on just 30 minutes of interaction. https://youtu.be/kBl3SCTnVEM
comment: accepted for publication in IROS2026
♻ ★ OpenNWM: A Generalist Navigation World Model with Latent Action Pretraining
Navigation world models predict future observations conditioned on navigation actions to support planning, but their generalization across real-world environments remains constrained by the limited coverage of action-labeled data. Action-free navigation videos provide diverse visual experience, yet lack the recorded actions required for action-conditioned training. We introduce OpenNWM, a generalist navigation world model with latent action pretraining. It learns universal latent actions from large-scale action-free videos through visual prediction pretraining, and further aligns them with physical motions via action-conditioned post-training. To support video pretraining, we curate NavAnywhere, a large-scale action-free navigation video dataset comprising 17.5 million visual observations from 15 heterogeneous sources. It captures navigation by ground robots, humans, and drones, with diverse viewpoints and motion patterns across five broad scene types: residential areas, public and commercial spaces, urban settings, parks, and natural landscapes. Experiments demonstrate improved visual prediction on in-domain and out-of-domain benchmarks, together with improved navigation performance. These results show that latent-action pretraining enables diverse, action-free videos to improve dynamics learning and generalization for open-world navigation.
comment: Project page: https://openspace-lab.github.io/OpenNWM/ Code: https://github.com/OpenSpace-Lab/OpenNWM
♻ ★ Event-Grounded Sparse Autoencoders for Vision-Language-Action Policies NeurIPS 2026
Vision-Language-Action (VLA) policies translate language and visual inputs into robot actions, where their hidden representations directly shape closed-loop behavior. However, mechanistic interpretability tools from language and vision-language models do not transfer cleanly to VLAs: outputs are robot actions rather than human-readable tokens, and validating internal changes requires expensive closed-loop rollouts. We propose an event-grounded interpretability pipeline that anchors SAE feature analysis to behavioral events rather than text contexts. We extract keyframes from end-effector trajectories and cluster them within each task using visual, robot-state, and temporal cues. The resulting events guide SAE feature ranking, with optional VLM annotations providing descriptions of the events. To our knowledge, our pipeline is among the first to ground SAE-based VLA analysis in closed-loop behavioral events. Across two simulation architectures and a real-robot study, event-grounded ranking yields the strongest causal effects on OpenVLA and transfers to the continuous action chunks of $π_{0.5}$. SAE is a sparse but imperfect intervention basis: usability varies with architecture and intervention site, and aggressive intervention reveals safety and interpretability limits. Overall, event-grounded SAE analysis emerges as a practical starting point for behavior-anchored VLA interpretability, motivating future work on SAE analysis beyond end-effector motion, finer-grained closed-loop evaluation, and safe interventions for high-stakes VLA deployments.
comment: NeurIPS 2026 camera-ready version
♻ ★ ZipDrift: Generating Multimodal Short-Horizon MPC Costs with One-Step Drifting Models
Short-horizon model predictive control (MPC) is fast but myopic. ZipMPC compensates by learning a context-dependent correction of the short-horizon cost that imitates a long-horizon expert. We observe that the optimal correction is often multimodal: when a slower opponent can be passed on either side, good corrections of opposite sign coexist, and the expert's choice between them hinges on tiny cost differences. Deterministic regression then averages modes or inherits inconsistent expert tie-breaking. We propose ZipDrift, which trains a one-step drifting generator over MPC cost corrections, tilted toward low closed-loop cost and learned only from cost evaluations of candidate corrections, without differentiating through the optimiser. A few generated candidates are ranked by a learned critic, and the model-based MPC then enforces dynamics and constraints. In a kinematic overtaking benchmark derived from ZipMPC, more than half of the contexts are bimodal and the manual short-horizon MPC crashes in 28% of them. With an eight-dimensional stagewise correction, evaluated on 1000 fresh contexts and five training seeds, ZipDrift with eight candidates crashes in 0.74% of episodes, compared with 3.4% for an idealised ZipMPC-style regressor and 2.1% for critic-ranked random candidates with the same budget. A mixture density network (MDN) with the same selection is equally good (0.76%). The benefit of drifting lies in sample quality: in a pre-specified test on further unseen contexts, a single drifting sample crashes in 8.7% of episodes against 13.2% for the MDN, and ZipDrift needs four candidates instead of eight to fall below 1%. With many candidates, all methods degrade as selection exploits errors of the critic.
comment: 7 pages, 3 figures, 5 tables. Preliminary simulation study
♻ ★ World Models as Adversaries: Multi-Agent Self-Play Fine-Tuning for Robust Motion Planning NeurIPS 2026
Robust motion planning in dense traffic requires autonomous vehicles to interact in rare and safety-critical scenarios that are underrepresented in naturalistic driving data. Although adversarial training offers a feasible solution, existing methods often rely on external scenario generators, heuristic perturbations, or simulator-heavy rollouts, which makes them difficult to integrate with modern autoregressive planners. Here, we cast adversarially robust planner learning as a constrained min-max game and propose Adversarial World Modeling (AWM), a theoretically grounded multi-agent self-play fine-tuning framework. Since solving the exact game is intractable, AWM introduces a principled decoupled solver. In the inner minimization, the planner's predictive world model is converted into a role-conditioned adversary that learns sparse, scene-adaptive attack coalitions via counterfactual credit assignment. In the outer maximization, the ego planner optimizes a regret-aware robust best response against the frozen AWM, utilizing tail-risk weighting and reference-anchored trust regions to improve hard-case recovery while preserving nominal driving behavior. Experiments on the nuPlan and InterPlan benchmarks demonstrate that our method generates transferable adversarial interactions and yields a robust planner that achieves competitive closed-loop performance in both nominal and highly interactive long-tail scenarios. Theoretical analysis justifies the decoupled solver and the main optimization components.
comment: Accepted at NeurIPS 2026
♻ ★ SONIC-K1: Porting a Universal Humanoid Motion Tracker to a Small Humanoid on a Single GPU
We present SONIC-K1, to our knowledge the first open-weights whole-body motion tracker for the Booster K1 humanoid, obtained by adapting the SONIC framework to this smaller robot. The policy is trained from scratch on a single shared GPU with a small fraction of the compute reported for SONIC. In MuJoCo simulation, it tracks nearly all held-out motion clips and performs well on external benchmarks under SONIC's success criterion. The first release ran on the real K1 but shook its legs on steps and turns, and simulation showed a sensitivity to actuation delay. We therefore post-trained it with the robot's deployment torque limits, randomized actuation delay and gain randomization. The post-trained v1.0 stays robust under real-robot torque limits and large actuation delays, where the first release often failed, and in our runs on the real K1 it walks and turns without the earlier leg shaking. These results show broad motion-tracking capability on a modest training budget. Code, policy weights and evaluation tools are publicly available.
comment: 4 pages, 3 figures. Project page: https://kyry.me/sonic-k1
♻ ★ PFM-HR: Pose Flow Matching for Humanoid Robots
Motion priors improve reinforcement learning for physics-based humanoid tracking, but temporal priors require ordered motion clips, while pose priors provide limited guidance for policy-induced pose transitions. We present Pose Flow Matching for Humanoid Robots (PFM-HR), a reusable flow matching prior trained directly on large scale unordered pose data. PFM-HR introduces the Pose Geometry Score (PGS), which quantifies how joint coordinate changes during rollouts align with the local geometry of pose variation captured by the prior. Using PGS to modulate the tracking reward guides policy exploration toward structured pose changes while keeping the prior frozen across tracking tasks. Experiments demonstrate that PFM-HR improves both single motion and general motion tracking, especially for highly dynamic motions.
comment: 7 pages
♻ ★ When appearance and motion disagree: evaluating class-motion consistency against disguised uncrewed surface vessels
Defense systems for maritime assets must distinguish approaching benign craft from uncrewed surface vessels (USVs) with sufficient warning to act. Shore cameras using computer vision can separate hostile vessels from benign traffic, but these systems may be attacked using an adversarial hull patch or physical disguise. We evaluate adversarial hull patches and find they suppress detection in up to 29% of attempts but never cause misclassification as benign. Physical disguise, however, can bypass both a camera check and a radar presence check. To evaluate defenses, we develop a hybrid replay protocol from automatic identification system (AIS) tracks that combines recorded benign motion with direct or off-axis attack run-ins. We find that for recreational-labeled vessels attacking via direct run-in within 4 nautical miles, an asset-relative closing-rate check flags 94-95% at 15-20 knots, compared with 12% for a track-only speed check. The checks are complementary on off-axis approaches, constraining the attacker to remain below both thresholds and extending the time required to reach the asset. Flagging on either check has a 4.9% held-out false-alarm rate overall and 12.3% on recreational traffic. The replay-based evaluation protocol supports comparative test and evaluation of defense methods.
♻ ★ EgoKit: Towards Unified Low-Cost Egocentric Data Collection with Heterogeneous Devices
Egocentric video is increasingly used as a data source for robot learning, activity understanding, and embodied AI research, but collecting it at scale remains fragmented in practice: each candidate host device, such as an Android phone, iPhone, iPad, smart glasses, or extended reality (XR) headset, exposes a different SDK, a different policy on raw camera access, and different limitations on external USB cameras and on-device tracking. Synchronized ego-view and wrist-view capture is therefore typically obtained by either committing to a single proprietary platform or building one-off rigs that do not transfer across devices. To address this gap, we present EgoKit, a toolkit that exposes the same egocentric recording workflow across six heterogeneous host devices. Across all supported devices, EgoKit presents the same recording interaction and produces locally stored video with a uniform log format; on XR headsets, it additionally logs head pose and OpenXR-standard 26-joint hand tracking aligned to the video streams. The companion accessories, including two wrist cameras with mounts, a head strap, and a USB-C hub, add wrist-view capture to any supported host without custom hardware fabrication. To validate that EgoKit data can train robot policies, we replicated HumanEgo with 41 demonstrations recorded by EgoKit-PICO: in simulation, policies trained on 20 or 40 of them placed a cube on a tray in 60-74% of 100 unseen layouts, about as often as policies trained on Project Aria recordings of the same layouts. EgoKit is available at https://egokit.chuange.org/.
♻ ★ Closed-Loop Object-Informed Control for Non-Prehensile Robot Manipulation
Non-prehensile robot manipulation is challenging due to discontinuous, long-horizon interactions between the robot and the objects it manipulates. Sampling-based model predictive control methods are effective with discontinuous contact but face challenges with finding promising trajectories in long-horizon planning. We propose a closed-loop object-informed (CLOI) method that splits the problem into object-level planning to find long-horizon object poses that lead the object to its goal, and robot-level planning to select robot actions that follow those poses. We use model predictive path integral (MPPI) control to solve the subproblems and couple their solutions through consensus on the object poses using the alternating direction method of multipliers (ADMM). The object plan is revised toward robot-realizable object trajectories, while the robot plan is aligned with the object poses the task requires. In planar pushing tasks with obstacles using an xArm6 manipulator, CLOI increases the success rate by 35% in simulation and 43% on hardware, compared to standard MPPI given the same computational budget.
♻ ★ Diffusion-Steering-Based MPC with a JEPA World Action Model
World action models (WAMs) predict actions and future observations, typically from a reconstruction-based representation that carries noisy, redundant information which can complicate downstream predictions. We introduce LeWAM, a bidirectional transformer for forward, backward, inverse dynamics and policy prediction, on a decoder-free JEPA latent trained end-to-end through all four modes. We see the following benefits: 1) Alignment: linear probes read robot and object state from LeWAM's latent better than from a regular Le World Model (a forward-only JEPA world model), while the latent ignores visual distractors as well as LeWM does and far better than a reconstruction-based WAM. 2) Acting: Closed-loop evaluations of LeWAM match a regular flow-matching policy trained on the same encoder at matched size, while also providing a world model. 3) Planning: Sampling raw actions when planning with WAMs lets MPC exploit dynamics-model inaccuracies; planning in the noise space of the policy head instead improves the closed-loop performance of these WAMs.
comment: 14 pages, 6 figures
♻ ★ CLAW: Continuous Latent Action World Models via Adversarial Regularization
We introduce CLAW, a fully end-to-end self-supervised framework for learning a world model jointly with continuous latent action representations directly from action-free videos. Our approach leverages adversarial latent regularization to prevent the latent from encoding future observations, so that it captures the structure of the action. The diffusion-based world model then models environment dynamics as a distribution over future observations rather than as a single deterministic rollout. Both components are learned from observation sequences alone without any action annotations. By simultaneously training the Latent Action Model and world model, CLAW learns to reason about how inferred actions induce environment transitions from visual observations alone. We show that the resulting latent action world model supports both imitation learning from observation and goal-directed planning. In imitation learning, latent actions extracted from raw videos enable behavior cloning. For planning, CLAW generates sequences of latent actions and maps them to executable actions to reach desired goals. Extensive experiments across diverse tasks and embodiments demonstrate that CLAW produces semantically meaningful latent action representations, supports effective action transfer, and enables planning and imitation from observation, outperforming existing methods.
comment: 9 pages, 27 pages of supplementary material
♻ ★ RoboFQE: Rollout-Free Evaluation of Visual Robot Policies from Offline Trajectories
Evaluating robot policies before deployment typically requires producing new trajectories through real-world execution, simulators, or learned world models. These rollout-based approaches impose infrastructure requirements; simulator and world-model evaluators additionally face prediction errors that can compound over the evaluation horizon. In this paper, we study off-policy evaluation as an alternative to rollout-based robot-policy evaluation methods. Specifically, we introduce RoboFQE, an evaluator that estimates the discounted expected return of a target policy from offline robot trajectories without estimating a behavior-policy density, combining fitted Q-evaluation with a pretrained vision foundation model and learned terminal success annotation to bridge the gap between standard OPE benchmarks and continuous-control visual manipulation. Controlled experiments across simulated and real-world manipulation tasks show that RoboFQE produces accurate discounted-return estimates and preserves policy rankings even on long-horizon manipulation tasks where video world-model evaluators are degraded by autoregressive prediction errors. Ablation experiments show that pretrained vision encoders substantially outperform from-scratch alternatives, particularly on long-horizon tasks, and that learned terminal success annotation enables evaluation without ground-truth reward labels with only slight reduction of accuracy.
♻ ★ Real-Time Execution with Autoregressive Policies
Real-time execution, enabled by asynchronous inference that ensures both smooth action trajectories and fast reactivity, is critical for realistic deployments of large-scale Vision-Language-Action models. However, recent work on real-time execution primarily focuses on variants of diffusion policies, even though it is more critical for autoregressive policies given their slower rollout speed in synchronous inference. In contrast, we demonstrate that autoregressive policies can achieve real-time execution by adjusting the tokenization horizon and applying constrained decoding, thereby guaranteeing strict latency bounds that enable multi-trajectory decoding to maximize performance. Across simulated and real-world environments, we find that the autoregressive policy consistently outperforms its equivalent-level flow-matching policy counterpart while achieving significantly improved task completion speeds from synchronous inference. Coupled with the inherent advantages of autoregressive policies, such as faster convergence and better generalizability in instruction-following, these results confirm that autoregressive policies can remain a competitive policy type supporting real-time execution.
comment: Published as a main conference paper at CoRL 2026
♻ ★ Behavior Graphs: Unsupervised Graph Structures for Interpreting Robot Deployment Data
Data flywheels promise continuous improvement via insights from deployment data. Understanding how the deployed model succeeds and fails can steer subsequent data collection towards patching failures and reinforcing successes. However, robot deployments create large volumes of high-dimensional rollout data, which is challenging to understand and confidently ground data collection decisions in. To make deployment data easier to analyze and act on, we propose behavior graphs, which compress policy rollouts into a single interpretable, directed graph. We construct the graphs by learning to identify similar behavior modes across rollouts, connecting the behaviors temporally, and finally linking them to outcomes. In a user study on three real-world tasks, where participants analyzed rollouts to choose a data collection strategy, 16 of 18 preferred behavior graphs to rollout videos, reaching comparable conclusions with 40.3% lower perceived temporal demand and 37.6% lower effort. Beyond interpretability for deployment data, behavior graphs are capable progress predictors: across 43 datasets, they improve the rank correlation between predicted progress and true temporal order by 22.9% and reduce mean absolute error by 53.5% relative to the strongest baseline. As the number of deployed robots grows, behavior graphs lend structure to high-dimensional policy rollouts and make deployment data easier to interpret and reason about.
comment: 8 pages, 6 figures, 3 tables. Project website: https://behavior-graphs.com/
♻ ★ XIT: Exploration and Exploitation Informed Trees for Active Gas Distribution Mapping in Unknown Environments
Mobile robotic gas distribution mapping (GDM) provides critical situational awareness during emergency responses to hazardous gas releases. However, most systems still rely on teleoperation, limiting scalability and response speed. Autonomous active GDM is challenging in unknown and cluttered environments, because the robot must simultaneously explore traversable space, map the environment, and infer the gas distribution belief from sparse chemical measurements. We address this by formulating active GDM as a next-best-trajectory informative path planning (IPP) problem and develop XIT (Exploration and Exploitation Informed Trees), a sampling-based planner that grows a single shared tree toward multiple frontier goals, coupling exploration of unresolved regions with informative gas sampling en route. XIT draws a batch of samples from an Upper Confidence Bound (UCB) information field derived from the current gas posterior and expands the tree using a cost that trades off travel effort against information acquisition. To enable plume-aware exploration, we introduce the gas frontier concept, defined as unobserved regions adjacent to high gas concentrations, and propose the Wavefront Gas Frontier Detection (WGFD) algorithm for their identification. High-fidelity simulations demonstrate the benefits of XIT in terms of GDM quality and efficiency, and a scaled real-world experiment demonstrates its practicality. Although developed for active GDM, XIT is applicable to other robotic information-gathering tasks in unknown environments that face the exploration and exploitation trade-off.
comment: Accepted for publication in IEEE Transactions on Robotics
♻ ★ EvoPlan: Evolutionary Neuro-Symbolic Robot Planning with Spatio-Temporal Guarantees
LLM-based robot planners are fluent but cannot guarantee that their plans are executable or safe. Classical PDDL planners can guarantee symbolic validity, but only once the problem is fully specified, and they make poor use of an LLM's ability to read context and repair plans. We present EvoPlan, a neuro-symbolic framework whose guarantees are symbolic validity and satisfaction of a constraint learned from data. The planner works with locally-hosted open-weight models, so it does not require a cloud service. EvoPlan has three parts. First, an offline procedure that mines a single global Signal Temporal Logic (STL) mobility constraint from demonstrations. Perturbed demonstrations supply the missing negatives without encoding any rule, and evolutionary search fits the constraint, which captures driving rules or social-navigation preferences within a modelled taxonomy of norms. We use the mined constraint to shield driving and navigation policies on Bench2Drive and HA-VLN-CE. Second, an evolutionary PDDL planner: an LLM proposes and repairs plans, programmatic validators decide which ones survive, and the validated portion of the plan grows over iterations. On open-world ALFWorld and NL-PDDL Blocksworld, it approaches classical planning when the goal vocabulary agrees with the action model, and it stays robust when the two disagree and classical planning fails. Third, a constrained execution loop: Nav2 uses the costmap to ground the plan into waypoints, which are checked against the mined constraint. Violations and execution failures update the PDDL problem for re-planning. We demonstrate the full pipeline in Gazebo and on a Clearpath Jackal robot. More details and videos are on the project page: https://nbhavyasai.github.io/evoplan/
comment: Accepted at CoRL 2026. Project page: https://nbhavyasai.github.io/evoplan/
♻ ★ DexTeleop-0: Force-Aware Bimanual Dexterous Teleoperation with Ego-Centric Perception towards Shared Autonomy
Fine-grained, bimanual dexterous manipulation remains a foundational challenge in robotics. Embodiment gaps hinder accurate kinematic mapping, and visual tracking alone does not regulate contact forces. To address these limitations, we propose a tactile-driven adaptation strategy designed to enable fine-grained manipulation on top of teleoperation pipelines. Instantiated within our bimanual dexterous framework, DexTeleop-0, this strategy introduces a real-time optimization loop that adds robot-side contact corrections to human tracking commands using fingertip feedback. By estimating contact positions and forces, the system dynamically computes localized corrections using a contact-stiffness approximation and the kinematic Jacobian, with per-contact vector force tracking, soft predicted-contact-moment regularization, and bounded hand-joint residuals. We evaluate this tactile-driven adaptation strategy across both simulated environments and real-world hardware. Controlled simulation replay yields final-stage success rates of 91% for Ball Assembly and 62% for Stir in Cup, versus 88% and 49% for force tracking alone. Fixed-input replay on two hand types, seven-operator teleoperation, and external-force traces further assess contact assistance. The open-sourced project page can be found at https://henryhcliu.github.io/dexteleop-0.
comment: 16 pages, 9 figures, 6 tables
♻ ★ Open-HumanWAM: An Open Recipe for Scaling Human and Robot Experience for Embodied Foundation Models
Egocentric human video offers a scalable source of manipulation priors for robot learning, yet translating raw data scale into generalizable robot policies demands rigorous data curation and training recipes. We introduce Open-HumanWAM, the first fully open recipe for scaling egocentric pretraining to 50,000 hours, anchored by Open-HumanWAM-50K---a curated egocentric video corpus densely annotated with hand motion and natural language. Leveraging this corpus, we systematically investigate pure human pretraining via closed-loop simulation evaluations and uncover an empirical log-linear scaling law linking pretraining data hours to downstream robot success rates. Grounded in these findings, we develop HumanWAM-Vanilla, a world--action model pretrained exclusively on 50k hours of egocentric human video. Upon robot post-training, HumanWAM-Vanilla reaches 92.08% success on RoboTwin 2.0 Clean2Clean and 68.88% on Clean2Random, alongside a 14.17% mean success rate and a 19.00 mean progress score on RoboDojo. To push the capability frontier further, we explore joint human--robot pretraining with HumanWAM-Goal, a visual-goal-conditioned world--action model. Hybrid data scaling unlocks broad zero-shot generalization across both simulation and physical benchmarks, propelling post-trained RoboTwin 2.0 performance to new state-of-the-art levels: 92.70% on Clean2Clean and a substantial leap to 84.60% on Clean2Random. On physical robots, HumanWAM-Goal further exhibits emergent embodied in-context learning, dynamically adapting its manipulation behavior to unseen visual goals without parameter updates. To support reproducible scaling research in embodied AI, we open-source the Open-HumanWAM-50K corpus, data curation and pretraining pipelines, alongside the pretrained and post-trained HumanWAM checkpoints, inference code, and simulation evaluation suites.
comment: GitHub: https://github.com/DAGroup-PKU/HumanNet/tree/Open-HumanWAM Project website: https://dagroup-pku.github.io/open-humanwam/
Multiagent Systems
★ SCALE: Real-Time Latent Communication for Cyber-Physical Agents
Latent communication enables agents to collaborate efficiently by directly exchanging intermediate model representations rather than raw sensor data or text messages. Existing approaches, however, are primarily designed for LLM agents and do not consider the bandwidth constraints and heterogeneous task deadlines faced by cyber-physical agents. In this work, we propose SCALE, a real-time latent communication method tailored for cyber-physical agents. Motivated by the spatio-temporal continuity of the physical world, SCALE matches tokens across consecutive latent messages using similarity scores to identify reusable information and avoid redundant transmission. It then profiles the semantic gain and communication cost of transmitting different amounts of latent information. Based on these profiles, SCALE employs a real-time scheduler that jointly determines which pending messages to transmit and how many latent tokens to send, subject to available bandwidth and task deadlines. We evaluate SCALE across three cyber-physical benchmarks and a physical spider robot case study under diverse network conditions and heterogeneous task deadlines. Our results show that SCALE significantly improves both task performance and on-time delivery in bandwidth-constrained networks. Our code is available at: https://anonymous.4open.science/r/SCALE-4E80/.
★ Radio-to-Pose Information for Gauge-Drift Correction in Cooperative Localization
Cooperative localization uses odometry and inter-robot measurements to estimate the motion of a robot team. These measurements describe only relative geometry, but the team's estimated position and orientation can drift together over time, i.e., gauge drift. In this work, we study correction of this gauge drift using a radio attenuation map registered in the same world coordinate frame used for robot poses. Specifically, we propose Radio-to-Pose Information (RPI), an interface that converts power residuals and their pose derivatives into a Gaussian information matrix and vector pair. RPI builds on the insight that received power is sensitive to shared displacement in the world map. The pose estimator uses the interface together with odometry and inter-robot measurements to update poses and minimize uncertainty. We characterize the shared motions detectable from the map and observed links. For an uncertain shared map, RPI keeps correlations across robots and time and reproduces joint Gaussian pose estimates and covariances. Across $1000$ simulated Gaussian trials, RPI matches the results of joint estimation. Across 32 paired trials with a registered map, RPI reduces common-position RMSE from $0.542$m to $0.140$m in a $50\times50$m arena compared with a baseline that uses radio only to weight relative measurements.
★ Trilemma: How Agents choose between Honesty, Confidentiality, and Task Completion
Personal AI agents often act on underspecified instructions and draw on context such as their user's messages and calendar to decide how to proceed. This context can contain private preferences that interfere with the task, with other people's interests, or with confidentiality norms. Prior work has studied deception under goal conflicts and the leakage of private information, but largely as separate problems. We study both problems together in a simulated company Slack workspace in which four employees delegate sprint staffing to personal agents. Two agents learn from private messages that their employee avoids working with a particular colleague. Because the other agents request justifications for task preferences and the only true justification is confidential, the agents cannot simultaneously remain honest, preserve confidentiality, and complete the staffing. We evaluate seven open-weight models and Claude and GPT models. Of the 135/140 open-weight agents that pursue the avoidance preference, 107 either make false claims about task fit or logistics or cite a true consideration as if it were their actual reason. In a three-model comparison, confidentiality instructions reduce disclosure of the preference from 83% (none) to 31% (medium) to 2% (strong). However, the task is often not completed anyway; in the main experiment (medium confidentiality instructions), only 16 of 70 runs end with a valid staffing. In their reasoning traces, agents frequently acknowledge evidence against their claims and still conclude that the claims are not false; they engage in motivated reasoning. Our results indicate that agents can strategically make false or misleading claims in a day-to-day task without being instructed to; the environment provides a controlled testbed for studying how agents resolve such conflicts and for developing deception monitors.
★ SEER: A Scalable Event-Driven Runtime for Parallel Execution of Multi-Agent Reinforcement Learning Environments
Multi-agent reinforcement learning (MARL) is increasingly used in systems where many entities interact in a shared environment. In these workloads, policy inference can be batched on GPUs, but the CPU-side environment path still updates entity states, resolves local interactions, and builds observations inside each $\mathrm{env.step}$. This path can dominate latency in interaction-intensive workloads after policy batching. Existing vectorized environment interfaces and general event-driven runtimes do not directly address the fine-grained cross-entity work inside one bounded MARL step while preserving the step-level semantics expected by learning systems. We present SEER (Scalable Event-Driven Environment Runtime), a runtime for MARL environment execution under an explicit event-locality contract. SEER models each environment step as a bounded stream of timestamped events. If direct state updates are entity-local and cross-entity effects are represented as ordered future events, SEER can run independent events in parallel while preserving serial-equivalent boundary outputs. To make this contract practical, SEER uses a CAS-based dual-layer event queue: local event queues preserve per-entity causal order, while a parallel executive list exposes ready work to global worker threads. SEER further uses STPA, a preflight-guided thread-pool selection mechanism that combines short runtime traces with static workload and hardware features. We evaluate SEER on two parameterized MARL workloads: compute-intensive UAV swarm control (Aircraft) and communication-intensive wireless management (PCS). On Aircraft, SEER reduces inference latency by 16.2\% over the baseline. On PCS, SEER reduces environment-execution latency by 46.9\%, 19.5\%, and 64.9\% over ROSS-T, NBSE, and ROOT-Sim, respectively.
★ ReferClaw: Disambiguating User Intent for Egocentric Assistance
Users of egocentric AI assistants often ask short, ambiguous questions whose intended referents cannot be uniquely resolved from video and dialogue history. For example, "Where did I leave it?" may refer to several previously handled objects. Existing VideoQA systems typically commit to a single interpretation, risking plausible but intention-misaligned answers. We introduce ReferClaw, an agentic framework that formulates egocentric streaming QA as an interactive process: the assistant asks clarifying questions to resolve user intent before answering. Three cooperating agents identify candidate interpretations, request clarification, and generate answers grounded in the resolved intent and visual evidence. A bounded, cost-aware policy balances expected information gain against user effort while limiting clarification turns and question complexity. Referclaw learns from interaction through intent-specific skill updates and process-reward-guided policy optimization with agent-level credit assignment. Evaluations on augmented egocentric QA benchmarks demonstrate improved answer accuracy and joint answer-and-grounding performance over existing baselines, alongside zero-shot transfer across datasets. These findings support interactive clarification as a practical approach to resolving online user intent in egocentric streaming assistance.
★ A Bilevel Neurodynamic Approach for Distributed SCAD-Regularized Sparse Least Squares
This paper proposes a bilevel particle swarm optimization (PSO)-neurodynamic method for solving the distributed sparse least-squares problem regularized by the smoothly clipped absolute deviation (SCAD) penalty to global optimality. The SCAD penalty is nonsmooth and nonconvex, so existing distributed neurodynamic methods have not been extended to it directly. Through two-time-scale singular perturbation analysis, a rigorous full-trajectory convergence theory with explicit convergence rates is established for the inner collaborative multi-agent neurodynamic system without requiring the second-order sufficient condition. A theory-guided inner-outer collaborative optimization framework is further developed, in which a red-light green-light sensitivity mechanism is embedded in the outer PSO search to enhance global search while concentrating computational effort on the SCAD shape parameters, thereby improving recovery accuracy and computational efficiency. Simulation results on four sparse-recovery scenarios demonstrate that the proposed method accurately recovers the true sparsity pattern and achieves faster convergence and lower relative error than competing neurodynamic methods.
★ Signed but Not Safe: Requirement Integrity in Multi-Agent LLM Handoffs
Enterprise AI agents increasingly split a user's request across subagents and merge their results, and each handoff can summarize away, alter or ignore a requirement. We report a preregistered study with synthetic, exactly scorable tasks comparing free-text, structured JSON and signed requirement-manifest handoffs: the orchestrator extracts each requirement as a verbatim quote plus a checkable predicate, code signs the manifest with HMAC-SHA256, and deterministic code verifies the merger's decision. With Qwen3-8B (1,440 trajectories, five threats), signed manifests cut unsafe commits under attack from 69.4% to 9.0% (60.4 pp, 95% CI [51.4, 69.4]) with no net loss of conformant commits relative to structured handoffs. Signing is what defeats in-transit tampering: an unsigned manifest committed violations in 79% of mutated runs, the signed one in none. On identical trajectories the deterministic verifier caught 78.7% of violating decisions versus 59.0% for an LLM checker, with 7% versus 50% false alarms. An oracle manifest removed all remaining unsafe commits: the residual risk comes from extraction errors made before signing. A preregistered pre-signing injection attack confirmed this boundary: one adversarial line raised signed-manifest violations from 14.6% to 37.5% (Qwen3-8B), 11.1% to 63.9% (gpt-oss-20b) and 8.3% to 75.0% (GPT-5.6-luna), and the signature then authenticated the corrupted manifest. Signing protects requirements in transit, not at extraction; the extraction step is the trust boundary.
comment: 34 pages, 0 figures. Preregistered; code, task banks, freeze manifests and run logs are available from the authors on request
★ The Last Mile to the Real World: Agent-Native Primitives as Reusable Building Blocks for Solving Real-World Tasks
How can an agent handle a real-world task end to end? Frontier models with their own harnesses still fall short on real-world benchmarks: on Terminal-Bench 4.0, GPT-5.6 Sol with Codex resolves 37.3% of the tasks and Claude Opus 5 with Claude Code 51.8%. These tasks fail at the step from reaching a resource to applying it, which we call the last mile to the real world. Existing approaches put the resource within reach but leave the competence to apply it with the caller, so each harness rebuilds that competence. We introduce agent-native primitives, reusable units that equip a resource with a resident LLM that applies it in context and communicates with other primitives, and HEELO (Harness Engineering around Evolving Agent-Native Primitives for the Last Mile to Real-WOrld Tasks), a harness engineering framework that builds each task's harness from a library of such primitives. HEELO composes the primitives a task needs into a task-specific graph, verifies the result against evidence from the environment, revises the primitives a failure implicates, and admits new or improved primitives after validation. We instantiate HEELO with eight primitive families, each built for one kind of task, and evaluate them on more than twenty benchmarks. Dev-Primitives raise Terminal-Bench 4.0 resolution from 37.3% under Codex to 55.5% on the same backbone, Harness Primitives composed per task raise Terminal-Bench 2.0 Pass@1 from 60.0% for the seed harness to 72.2%, Tool Primitives raise ToolBench pass rate from 55.8% to 75.1%, and Navigation Primitives raise GAIA-Text-103 accuracy from 45.6% to 58.3%. On the 46 real-world tasks of our heelo.dev platform, Claude Opus 5.5 with Claude Code and GPT-6.1 Sol with Codex complete 9 each on their own and 39 and 35 when connected to HEELO through our MCP server, with 1.9$\times$ and 2.3$\times$ less time and 1.55$\times$ lower cost per completed task.
comment: 102 pages, 21 figures, 54 tables. Project page:https://www.heelo.dev/
★ Designing with Cinematic Language: An Intent-Driven Approach to Human-AI Video Clip Co-Creation
Generative video models have made video creation more accessible, yet creators still struggle to translate high-level narrative intentions into coherent cinematic expressions. Existing systems primarily rely on prompt refinement or expose isolated camera controls, leaving a gap between what to express and how to film it. We present CineAssist, a human-AI co-creation system that treats cinematic language as an interactive design medium connecting creative intentions with executable video controls. We construct a Cinematic Language Design Space and an annotated dataset from 107 filmmaking sources and 1,436 film clips, linking communicative intentions to cinematic parameter values. CineAssist combines a multi-agent workflow for intent alignment and strategy recommendation with an interactive cinematic editor and a context-sensitive preference learner for user-controlled personalization. We evaluate the system through three studies examining end-to-end usability, cinematic quality, and preference evolution. Results show that CineAssist facilitates efficient, controllable, and personalized intent-driven cinematic video creation.
comment: 22 pages, 16 figures
★ Design of Multi-UAV Cooperative Deep Semantic Autoencoders for Communication Networks
Driven by the rapid development of the low-altitude economy, Uncrewed Aerial Vehicle (UAV) networks have become increasingly important for executing tasks such as search, rescue, and monitoring. However, under bandwidth-constrained scenarios, UAV networks still face significant challenges in task execution efficiency and intelligent processing capabilities. To address these issues, we propose Multi-UAV Cooperative Deep Semantic Autoencoders (MC-DSA) for communication networks. The architecture comprises one Task UAV (T-UAV) and multiple Support UAVs (S-UAVs), which jointly enhance communication efficiency and data-processing capability via semantic communication and cooperation. Specifically, we introduce a semantic matching and source selection mechanism, enabling the proposed MC-DSA to handle low-quality images while ensuring effective utilization of high-quality images. The T-UAV broadcasts a task query, and each S-UAV computes the semantic similarity between its local image and the query. Communication links are established only when the similarity exceeds a predefined threshold. The training algorithm optimizes network parameters through gradient descent, improving semantic feature extraction accuracy. The testing algorithm evaluates UAV collaboration efficiency and communication quality. Experimental results demonstrate that, compared with benchmarks, the proposed MC-DSA improves image semantic segmentation accuracy while maintaining task execution efficiency. As a result, it reduces redundant transmissions and offers an effective solution for multi-UAV communication networks.
★ Take It From Here: Evaluating Coding Agents On Project Takeover
Coding agents increasingly collaborate on long-term projects. When taking over a project, they need to recover currently applicable requirements from the interaction history and update them in response to new requests. However, most coding benchmarks do not include long interaction histories, and some evaluations that consider history still do not cover the full process from requirement reconstruction to project delivery. We therefore provide agents with the interaction history, current request, and pre-request project environment, ask them to make the requested changes, and assess takeover performance through delivery tests. Across six model-harness configurations and 37 targets eligible for executable validation, the mean complete-delivery rate under Full History is only 4.95%. We then use requirement atoms to independently diagnose a larger set of takeover targets, examining relevant information identification, pre-request state reconstruction, and current request processing. We further supply correct pre-request or target states to examine the remaining difficulties in requirement processing and project modification. The diagnostics show that none of the six configurations produces fully correct post-request requirement states, and a case illustrates omission of previously recovered constraints during updating. With correct target states supplied, the mean complete-delivery rate is 8.11%, with build and regression pass rates of 96.40% and 90.99%, but target-test success remains 9.01%. These results show difficulties in both forming target requirement states and implementing supplied targets in existing projects. Based on these findings, we introduce ReqMemBench, which combines requirement diagnostics with delivery validation to support systematic evaluation of project takeover.
comment: 35 pages
★ CrowdDirector: Runtime Orchestration of Crowd Simulation through Natural-Language Directives
Natural language offers an intuitive way to direct a virtual crowd while a simulation is running. Handwritten rules execute efficiently but cannot respond flexibly to varied instructions, whereas asking a large language model (LLM) to control the crowd at every update introduces repeated latency, cost, and the risk of unsupported actions. We present CrowdDirector, a framework that uses an LLM only when the user adds or changes an instruction. The LLM translates the request into a structured instruction, and a scene validator checks whether the entities and actions it references exist in the current Unity scene. Accepted instructions are stored and remain continuously active as the simulation is running. At each simulation tick, a graph neural network uses the current scene state and active instruction to choose an available action and target for each agent, while Unity handles movement, interactions, and animation. The network is trained offline on simulated scenarios, with LLM guidance on how agents should respond to instructions. Our evaluations show the scene validator's feasibility decisions matched human judgements more often than the LLM's own assessment, and on a behavioural benchmark CrowdDirector follows user instructions more closely than handwritten rules while avoiding repeated LLM calls. It therefore provides a practical middle ground between fixed rules and continuous LLM control.
★ Divide, Diagnose, and Compose: Optimizing Agentic Pipelines and Harnesses
Building agentic pipelines that achieve high task performance while keeping cost and latency low requires jointly optimizing heterogeneous and coupled design choices, including prompts, topology, model selection, and tool policies. Manually searching this combinatorial design space is expensive and time-consuming, while existing automated optimizers typically operate within a predefined subset of this space, modify the pipeline monolithically, or provide little structured evidence for pipeline changes. To mitigate these challenges, we introduce the Decomposed Credit-Assignment Framework (DeCAF), which adapts the design space to the given benchmark and pipeline, optimizes each dimension separately, and generates reasoning traces for every decision. Given a task specification, benchmark examples, and a seed pipeline, DeCAF identifies relevant optimization dimensions and generates diverse experimental hypotheses for each. Independent coding-agent sessions then optimize each dimension under a restricted edit scope, jointly evaluating performance, inference cost, and latency while recording all hypotheses, edits, and outcomes of each experiment. A final synthesis session combines compatible findings across all dimensions and uses a component-level trajectory evaluator, which derives role-specific rubrics from each agent's instructions, to localize underperforming components and guide cross-dimension revisions. Across a diverse suite of agentic benchmarks, DeCAF improves average task performance by 2.58 percentage points, while reducing inference cost by 4.30% and end-to-end latency by 42.45% compared to the best overall baseline.
★ s$^3$: Simple Scaling of Swarms via Cooperative Multi-agent RL
Groups can achieve more than individuals in the same amount of time, but only if their members coordinate: they must decide who works on what and what to share. However, this organization in multi-agent systems today usually comes from arbitrary, static harnesses designed a priori without adapting to agent behaviors or task specifications. We ask whether agents can instead learn to organize themselves. We introduce s$^3$, the first multi-agent reinforcement learning method in which agents decide for themselves how long to work alone and when to share findings or read teammates' messages, with no predefined organization or roles. Our s$^3$ agents learn to take different approaches, read each other's work, exhibit emergent roles that complement each other in the swarm, and generalize to bigger teams. We study both coordination, where agents must combine private information, and parallel exploration, where sharing discoveries can make search more efficient. With matched training and inference budgets, s$^3$, swarms evaluated with twice as many agents as in training solve over 25$\times$ more coordination problems than the strongest baseline, and in exploration, communicating agents reduce repeated work by nearly 50% and outperform best-of-N by up to 9.4 percentage points. Rather than being designed in advance, s$^3$ enables organization to emerge from a shared goal. This opens the door to capable systems built from many models that learn to work together.
♻ ★ From Siloed Algorithms to Compliance-First Agentic Platforms: A Multi-Layered Architecture for Hospital AI Systems
Hospitals are rapidly adopting artificial intelligence for triage, imaging, scheduling etc., yet most deployments remain isolated point solutions locked inside departmental silos, resulting in duplicated effort, hidden risks, and unrealized enterprise value. Despite explosive growth of AI in healthcare market and accelerating investment, an estimated 70-80% of healthcare AI pilots fail to scale, largely due to governance gaps, fragmented data, and missing integration blueprints. This research proposes a hospital-specific, compliance-first, Agentic AI architecture with multiple interoperable layers, extending existing hospital AI platform models with: (i) an Agent Orchestration Layer for multi-agent workflows across clinical, operational, and financial domains, (ii) a Compliance and Policy Layer that centralizes policy-as-code for HIPAA, GDPR, the EU AI Act, DISHA Act, India's DPDP Act, and ISO/IEC security and safety standards, and (iii) a Privacy-Preserving Data Fabric that plugs federated learning, differential privacy, and secure enclaves into real-world Hospital Information Management System (HIMS) flows. Using a synthetic but structurally realistic hospital dataset and an open, ready-to-deploy prototype implementation, this study demonstrates the end-to-end orchestration of triage risk prediction, workflow optimization, and compliance logging, achieving substantial simulated reductions in task turnaround times and manual documentation effort while maintaining policy-guarded data access. The resulting architecture offers hospital leaders a pragmatic blueprint to move from ad hoc tools to a governed, globally compliant, ROI-focused AI platform that can be tailored to on-premise, hybrid and cloud-native deployments.
comment: arXiv admin note: This submission has been withdrawn because it does not meet arXiv's research content quality standards
♻ ★ Networked Admissibility-Preserving Control for Directed Safe Coordination
This paper addresses safe coordination of scalar agents with constrained input dynamics over fixed rooted digraphs. We propose a networked Admissibility-Preserving Control (APC) architecture for a common moving output corridor under heterogeneous asymmetric input bounds, wherein the Admissibility-Preserving Input Realization (APIR) generates the physical inputs. In logarithmic barrier coordinates, the closed-loop dynamics form an exact cascade of exponentially decaying realization errors and directed consensus dynamics. We derive direction-specific compatibility conditions that guarantee uniform corridor and input margins, bounded commands, unique complete solutions, and exponential consensus for compact initial sets. The closed-form expression for the agreement value quantifies the effect of the initial APIR mismatch. Under the corresponding compatibility conditions, partial pinning guarantees safe reference tracking whenever the root component contains at least one informed agent. Simulations illustrate the coordination guarantees.
♻ ★ Set-Valued Routing for Language-Model Agent Systems: A WildChat Benchmark and Cost-Aware Evaluation
Language-model agent systems must map natural-language requests to one or more specialized agents before planning or tool execution. This decision is often treated as top-1 selection, although compositional requests can require several capabilities and unnecessary dispatch raises cost. We formulate fixed-catalog routing as set-valued language understanding and construct a benchmark of 3,000 WildChat prompts over 12 capability-defined agents. Reproducible schema-guided reference sets are balanced by agent frequency and route cardinality. We compare profile retrieval, multilabel classifiers, a fine-tuned text encoder, and catalog-conditioned zero-shot GPT-4o. Evaluation combines set quality, latency, capability coverage, and ordinal-cost utility. The encoder attains 89.59 F1, compared with 71.79 for linear SVM and 41.51 for GPT-4o, and retains 88.02 F1 under a harder semantic-group split. We further introduce Weighted Agent Routing (WAR), a deterministic post-scoring rule that changes the quality-cost operating point without retraining; Encoder+WAR obtains the best constrained utility. The results show that catalog-specific supervision and explicit set construction are important for multi-agent dispatch, while cost-aware decision layers can complement strong language encoders.
♻ ★ AgentCue: Statistical Summaries for Guiding Decentralized Multi-Agent Self-Organization
Multi-agent systems (MAS) combine multiple agents to solve complex tasks, and recent MAS scale to more than a thousand agents. At this scale, decentralized MAS remove the central orchestrator that assigns sub-tasks, and the agents, which we call workers, self-organize through their own choices of sub-tasks and collaborators on a shared board. As workers keep adding participation and collaboration records, however, the information explodes: workers spend more and more effort on reading the records of other workers, and self-organization loses efficiency. In the statistical view, we represent the execution history of an MAS by two continually updated networks, a collaboration network among workers and a participation network between workers and task types. We then propose AgentCue, a method built on a new latent space model that places workers and task types in a shared latent space. At each checkpoint, that is, after a fixed number of sessions of the workers, the estimated parameters of the latent space model form a compact and interpretable statistical summary, and workers use the statistical summary to choose sub-tasks and collaborators without repeated searches of the shared board. We also provide theoretical results on model identifiability and estimation accuracy for continually updated networks. Experiments on three benchmarks for cross-document reasoning, mathematical theorem proving, and software patch generation show that AgentCue improves the efficiency, accuracy, and interpretability of self-organization.
comment: 87 pages, 8 figures, including supplementary material
♻ ★ EmboTeam: Asymmetric State Grounding for Heterogeneous Multi-Robot Collaboration
Heterogeneous multi-robot collaboration over long horizons requires both global task planning and robust coordination during execution. Existing plan- ners using large language models (LLMs) can generate task decompositions or symbolic plans, but often execute them as rigid action sequences, making them fragile to local state mismatches and synchronization failures. We propose Em- boTeam, a neuro-symbolic framework for asymmetric state grounding. An LLM generates structured state expectations, which are verified and coordinated through symbolic search using Planning Domain Definition Language (PDDL) and then grounded into a team-level parallel behavior tree. Robot-specific subtrees per- form precondition checking, skill execution, effect validation, local replanning, and blackboard-based synchronization. We further introduce MACE-THOR, con- taining 42 heterogeneous multi-robot tasks across 8 indoor layouts. EmboTeam improves task success rate from 12% to 55% and goal condition recall from 32% to 75% over LaMMA-P, demonstrating the effectiveness of synchronized reactive grounding for collaboration over long horizons.
comment: Accepted to the 33rd International Conference on Multimedia Modeling (MMM 2027)
♻ ★ Bayesian Replicator Dynamics and the Inversion of Strict Dominance in Bimatrix Games
Replicator dynamics, the central model of evolutionary game theory, describe populations evolving under selection and agents learning by trial and error. They also admit a third use: continuous-time Bayesian inference, in which an agent planning against an opponent accumulates evidence for each candidate strategy. Here we show that this use divides all two-player games into two classes. With the standard expected-utility payoff, the dynamics can be expressed as Bayesian updating only in games (with any number of actions) where no player has a dominant strategy. The logarithmic payoff of Ay and Erb (2005) admits the Bayesian representation in every game. Coordination, anti-coordination, and cyclic games share qualitative dynamics under both payoff definitions, with equilibria displaced and selection thresholds shifted. In contrast, the flows of every game with a dominant strategy are reversed between the payoff definitions, with the strictly dominant strategy eliminated under logarithmic payoffs. Cooperation therefore emerges in the Prisoner's Dilemma, and games of common interest converge to mutual defection. For two-action games the classification is complete and rests on an exact Hamiltonian structure shared by both payoff definitions. The structure gives a behavioural test: cycles in the classic Matching Pennies game have amplitude-independent period (isochronous) under logarithmic payoffs, but not under expected utility. Logarithmic evidence integration thus determines which strategic outcomes dynamical selection can reach, wherever learning approximates Bayesian updating.
comment: 18 pages, 3 figures
♻ ★ Broken Gates: Re-evaluating Web Bot Defenses in the Age of LLM Agents
The rapid emergence of web automation across commercial crawlers, solver services, and LLM-based browser agents has created new challenges for defenses that must distinguish legitimate automated activity from scalable abuse. In response, modern bot defenses increasingly replace explicit challenges with risk-based decisions over browser, session, identity, and behavioral signals. This paper explores how current automation stacks can bypass these defenses end-to-end, evaluating seven commercial solver services and eight LLM-based browser agents against four widely deployed defenses. We find that solver services largely defeat challenge-based defenses with a 100% success rate, but collapse to an average 23% success rate on reCaptcha v3. Browser agents encounter a different bottleneck: the browser and account state presented to the verifier. Controlled experiments across browser configurations show that an established consumer account and persistent Chrome state strongly affect reCaptcha v2/v3 acceptance in our deployments, whereas Cloudflare Turnstile exhibits substantially greater sensitivity to the agent's browser control path. Reconstructing the established browser environment enables previously rejected agents to pass without model retraining. Once established, this trusted environment transfers across the agents and defenses in our testbed, adding no measurable defense-specific marginal cost to the surrounding workflow. These results expose an inherent security tension in modern non-interactive defenses: to eliminate friction, the signals they trust must be accessible to everyday human users, yet this accessibility inherently guarantees that adversaries can acquire, reproduce, and transfer the same identity signals to scale automated abuse.
♻ ★ NeuroMAS: Multi-Agent Systems as Neural Networks with Joint Reinforcement Learning
Multi-agent language systems are often built as hand-designed workflows, where agents are assigned semantic roles and communication protocols are specified in advance. We propose NeuroMAS, a framework that treats a multi-agent language system as a trainable and scalable neural-network-like architecture with hidden textual intermediate signals. In NeuroMAS, language-agent nodes are role-free but structure-aware: the topology determines how information can flow, while reward-based training determines how nodes communicate, specialize, and coordinate. This formulation shifts multi-agent design from workflow engineering toward architecture design, where depth, width, connectivity, and growth protocol become scalable sources of capability. We provide a theoretical perspective showing why such modular text-valued computation can be parameter-efficient when tasks admit stable decompositions. Experiments show that NeuroMAS improves over both inference-time collaboration methods and trained baselines, and that its gains are not explained by trainable parameter count alone. We further find that organizational scaling is path-dependent: larger systems can fail when trained from scratch, but become stronger when grown progressively from smaller trained systems. These results suggest that learned multi-agent organization is a promising scaling axis for language-model reasoning and generation.
♻ ★ Beyond Bayesian Nash: Learning Minimax-Regret Equilibria for Adversarial Team Games under Asymmetric Information
Adversarial team games (ATGs) with asymmetric information, such as adversarial path-finding, goal search, and reachability games on graphs, require strategies that are robust to hidden opponent types, such as a hidden goal flag, and to deception. Under asymmetric information, deception is seen as strategic shifts in the type distribution such that the omniscient opponent can collude with Nature and condition its play on the observed type. Existing risk-neutral solution concepts, such as Bayesian Nash equilibrium (BNE), are sensitive to distribution shifts, while distributionally robust approaches provide guarantees only within a prescribed ambiguity set. To address these limitations, we introduce Probabilistically Robust Minimax-Regret Equilibrium (PR-MRE), a novel equilibrium concept that combines the distribution-free robustness of minimax-regret reasoning with probabilistic information from a nominal type distribution. PR-MRE minimizes worst-case regret over a high-confidence subset of the type space, providing protection against strategic redistribution of probability mass while avoiding the conservatism of fully distribution-free approaches. We show that, for finite normal-form Bayesian games, approximate PR-MRE computation can be formulated as a robust bilinear program and derive a tractable semidefinite relaxation. We then adapt this relaxation into a novel meta-solver within a robust double-oracle framework, PRMRE-PSRO, enabling population-based learning of approximate PR-MRE strategies via deep reinforcement learning best responses. Experiments on graph-structured adversarial team games demonstrate that PR-MRE discovers strategies with substantially improved worst-case performance across hidden types compared to risk-neutral Bayesian Nash equilibrium solutions, resulting in more robust behavior under strategic distribution shifts.
comment: 53 pages, 25 figures, 9 tables. Under review at Transactions on Machine Learning Research (TMLR)
♻ ★ Control Charts for Multi-agent Systems
Generative agents have proven to be powerful assistants in a wide variety of contexts. Given this success, users are now deploying agents with minimal restrictions in open ended, multi-agent environments. Current methods for monitoring the dynamics of open-ended multi-agent systems are limited to aggregate statistics, qualitative inspection, or detecting change between two fixed timepoints. In this paper, we extend the process-theoretic notion of adaptive control charts to multi-agent systems to enable automated monitoring. Using simulation, we demonstrate that adaptive control charts are necessary for monitoring multi-agent systems that can learn from their environment. We further demonstrate, both empirically and theoretically, that adaptive control charts are susceptible to adversarial agents that defect sufficiently slowly. These results suggest a fundamental tradeoff in multi-agent system control: monitors with fixed baselines flag legitimate learning, while monitors with adaptive baselines admit slow adversaries. We conjecture this holds for any monitor.
♻ ★ Sentinel Agents for Secure and Trustworthy Agentic AI in Multi-Agent Systems
Multi-agent systems (MAS) built on large language models (LLMs) increasingly share a common conversational space, in which a single adversarial or erroneous message can propagate from one agent to many. This paper proposes Sentinel Agents, a dedicated security layer that monitors inter-agent communication with rule-based checks, behavioral analysis and LLM-based semantic inspection, and reports to a Coordinator Agent that enforces policy, for example by quarantining misbehaving agents. We describe the architecture, its deployment patterns and monitoring layers, and we evaluate a continuous-listener prototype on a balanced corpus of 1,000 synthetic messages (500 attacks across five families and 500 benign messages) with two local LLMs. LLM inspection with Gemma 4 12B reached F1 0.86 at a false-positive rate of 0.11, against 0.38 for rules alone. Combining components by logical OR did not improve on it, while a cascade in which only high-precision rules act on their own matched it. Calibrating the thresholds of the more conservative Qwen3 8B on a development split raised its held-out F1 from 0.56 to 0.80, and on multi-turn conversations the behavioral check detected patterns the rules missed but could not tell legitimate repetition from probing. The value of combining detectors therefore depends on the model, on its calibration and on the aggregation rule, and the results yield practical guidance for deploying Sentinel Agents.
comment: 23 pages, 11 figures. v2: the preliminary evaluation of v1 is replaced by a new evaluation on a balanced 1,000-message corpus with ablation, local LLM benchmark, aggregation and calibration analyses; paper restructured; references updated
♻ ★ AgentCue: Statistical Summaries for Guiding Decentralized Multi-Agent Self-Organization
Multi-agent systems (MAS) combine multiple agents to solve complex tasks, and recent MAS scale to more than a thousand agents. At this scale, decentralized MAS remove the central orchestrator that assigns sub-tasks, and the agents, which we call workers, self-organize through their own choices of sub-tasks and collaborators on a shared board. As workers keep adding participation and collaboration records, however, the information explodes: workers spend more and more effort on reading the records of other workers, and self-organization loses efficiency. In the statistical view, we represent the execution history of an MAS by two continually updated networks, a collaboration network among workers and a participation network between workers and task types. We then propose AgentCue, a method built on a new latent space model that places workers and task types in a shared latent space. At each checkpoint, that is, after a fixed number of sessions of the workers, the estimated parameters of the latent space model form a compact and interpretable statistical summary, and workers use the statistical summary to choose sub-tasks and collaborators without repeated searches of the shared board. We also provide theoretical results on model identifiability and estimation accuracy for continually updated networks. Experiments on three benchmarks for cross-document reasoning, mathematical theorem proving, and software patch generation show that AgentCue improves the efficiency, accuracy, and interpretability of self-organization.
comment: 87 pages, 8 figures, including supplementary material
Systems and Control (EESS)
★ Joint Electricity Price and Load Forecasting for Data Center Battery Dispatch Using Polynomial Diffusion Models
The rapid growth of large-scale data centers (DCs) is increasing electricity demand, market volatility, and operating costs. Battery energy storage systems (BESSs) can mitigate these challenges by shifting electricity purchases from high-price to low-price periods. This paper proposes a real-time BESS dispatch framework using polynomial diffusion models (PDMs) to forecast both electricity prices and DC loads. Leveraging the closed-form conditional expectations of PDMs, the forecasts are integrated into a dynamic programming (DP) framework to minimize electricity procurement costs. Across the evaluated BESS configurations, the proposed approach achieves 2.63 times the cost savings of a rule-based (RB) strategy.
comment: 5 pages, 5 figures, Power & Energy Society (PES) General Meeting
★ Dynamics Aware Past-Future Information Bottleneck
The simplification of complex dynamical systems is essential for efficient estimation and control. A principled approach to this task is through the lens of information-theoretic compression. The Past-Future Information Bottleneck (PF-IB) provides a theoretical framework for compressing linear dynamical systems. However, the standard formulation often lacks structural consistency, yielding representations that are not realizable as dynamical systems. By constraining the objective to a specific manifold of block Hankel matrices, which we prove is a Lie group, we bridge the gap between PF-IB and realization theory. Our approach ensures that the resulting compressed dynamics are realizable, thereby enabling its direct application to estimation and control. We demonstrate our approach in numerical simulations.
★ A Multimodal Haptic Interface for Simultaneous Softness and Thermal Modulation
Haptic interactions, such as touching an object in virtual reality, involve detecting a plethora of sensations through the skin. Whilst tactile displays exist to simulate and manipulate all of these cues individually, little progress has been made constructing devices that can control multiple cues simultaneously. This paper presents a tactile display technology that offers both softness and temperature change. A prototype display, based on particle jamming and the thermal transfer of heated and cooled water, is evaluated in terms of its ability to change both stiffness and temperature. Mechanical evaluations demonstrate controllable stiffness increase of 300%, whilst thermal characterisation showed strong surface uniformity and a range of achievable temperatures from 10°C up to 50°C, with closed-loop control maintaining a 0.2°C error from a setpoint. This work advances the design of multimodal tactile displays with applications to perceptual research, medical simulation, and affective haptics.
comment: Published at EuroHaptics 2026
★ Estimating Distribution Grid Upgrade Costs based on County-level Building Electrification Projections
Building electrification projections are typically developed at aggregated geographic levels to support bulk power system analysis, even though a significant portion of the infrastructure investments required to accommodate electrification occurs within distribution systems. Existing methodologies to disaggregate load growth projections onto distribution networks are based on context-specific analyses that require detailed data on building electricity consumption, distribution feeders, and geographic parcels, which are often proprietary and unavailable for policy and economic impact studies. This paper presents a methodology to disaggregate county-level building electrification projections onto representative distribution feeders and evaluate their impact on distribution upgrade costs. The proposed approach combines bulk system end-use electrification projections with representative feeder sets to generate building-level electrification measures. These measures are then used to create load profiles that are consistent with county-level projections while preserving the spatial diversity of electrification demand required for distribution infrastructure analysis. Using a case study of Genesee County, in Michigan, the results show that the proposed methodology is able to capture a diversity of feeder-level responses to electrification and represent the different realities that building electrification produces within the distribution system. This allows to represent localized load variations and capture more realistic distribution system upgrade costs driven by electrification.
★ Output-Feedback Integral Sliding-Mode Extremum Seeking with Super-Twisting Differentiation
This paper develops a dither-free output-feedback extremum-seeking controller for pre-stabilized nonlinear dynamic systems with unknown steady-state performance maps. The controller uses only the measured performance, the applied parameter, and internal states. Within a local state--parameter domain, integral sliding mode enforces equality between the applied parameter rate and a prescribed nonzero search velocity after finite-time capture under bounded matched rate-channel perturbations. This motion lets a super-twisting differentiator and rate normalization recover a gradient-sign surrogate without demodulation or averaging. A ramp-corrected moving-equilibrium coordinate cancels the first-order ramp-induced plant term and yields explicit flow, reversal, recovery, and dynamic-remainder bounds. The resulting local state--parameter tube supplies the regularity needed for finite-time exact differentiation. Combined with a deadbanded hybrid relay, these results establish local practical convergence to an explicit neighborhood of the unknown optimum. An 80-state tubular-reactor benchmark and paired held-out Monte Carlo tests compare a causal sampled realization with perturbation ESC and a rate-informed super-twisting comparator. The proposed realization succeeds in all 200 held-out cases and attains the smallest upper-tail objective gap under the stated tuning protocol. A matched-rate sweep connects performance degradation to applied-rate mismatch, near-stall motion, reversal, and regularization attenuation.
comment: 14 pages, 3 figures, 6 tables. Submitted to Automatica. Substantially extended from a paper presented at the 23rd IFAC World Congress, 2026
★ Fault-Geometry-Aware Control Allocation for Emergency Quadrotor Landing under Single-Motor Loss of Effectiveness
Partial single-motor loss of effectiveness (LoE) changes the remaining control-authority geometry of a quadrotor and can change which control allocator performs best during emergency landing[cite: 11]. We compare three allocators in Crazyflie software-in-the-loop simulation: bounded weighted least squares (reported as PINV), a finite-candidate residual allocator (QP-lite), and the same residual allocator with scoring parameters tuned offline by the Cross-Entropy Method (CEM)[cite: 11]. For this study, fault geometry is represented by the known failed-motor index[cite: 11]. At $η=0.496$ remaining effectiveness, matched development trials select CEM for M1, M3, and M4 and PINV for M2; the M3 selection is a one-trial margin (29/30 versus 28/30)[cite: 11]. After freezing this routing map, the policy achieves 117/120 safe first-contact touchdowns on fresh seeds and 115/120 in a separate randomized-order integrated-supervisor evaluation[cite: 11]. For the critical M2/PINV branch, Firth regression gives a nominal $η_{50}\approx0.4956$ with a within-model 95% bootstrap interval of approximately [0.4955, 0.4956][cite: 11]. One-factor-at-a-time $\pm10\%$ plant perturbations shift $η_{50}$ by up to 0.05495 for thrust coefficient and 0.04974 for mass; the most frequent violated first-contact criterion also changes from vertical speed in many conditions to angular rate for mass -10% and thrust-coefficient +10%[cite: 11]. At fixed $η=0.496$, PINV has a higher M2 safe count than frozen CEM in 9/11 tested plant states and ties in 2/11; the two ties are both 0/30[cite: 11]. The experiments therefore distinguish motor-conditioned allocator selection from sensitivity to plant-model mismatch[cite: 11].
comment: 6 pages, 7 figures, 1 table
★ Value-oriented policy distillation for interpretable wind farm power control
Wind farm power control can help the system maintain balance and avoid blackouts. In Europe, this can be achieved through implicit balancing, where a wind farm adjusts its power setpoint in anticipation of the imbalance price using information published by the Transmission System Operator. Thereby, the wind farm helps restore system balance while also reducing its own imbalance costs resulting from forecast errors. However, implicit balancing is a challenging problem since the settlement price is unknown in advance and can be impacted by the pursued control strategy. Reinforcement learning methods could help to solve this problem, but lack the interpretability needed to be deployed on real wind farms. Therefore, we propose a policy distillation framework using explainable boosting machines (EBMs), a variant of generalized additive models, to obtain a transparent control policy. To increase the value captured by the EBM model, we introduce a value-oriented perspective to policy distillation inspired by research in the field of energy forecasting. Furthermore, we suggest to include expert knowledge during model design, such that the control strategy is also justifiable to stakeholders. Using real data of the Belgian grid and an offshore wind farm, we find that our value-oriented and expert-based controller can reduce imbalance costs by 82%, while outperforming a price threshold controller as well as a policy distillation baseline using decision trees.
★ Optical Network Digital Twin: From Digital Coherent Transmission to Intelligent Network Operations
Optical Network Digital Twins (ONDTs) are emerging as a key framework for representing, synchronizing, and utilizing the physical state of optical networks for quality-of-transmission (QoT) estimation, optimization, and intelligent operation. This survey provides a comprehensive view of ONDTs, spanning digital coherent transmission and physical-layer observability to intelligent network operations. We systematically organize ONDT technologies from ecosystem, functional, control, and implementation architecture perspectives and review the enabling technologies required to construct, synchronize, and utilize digital representations of physical optical networks. As a unifying framework to describe both ONDT design and validation, we introduce a categorization based on the following figures of merit: Fidelity, Dynamicity, and Deployability (FDD). In particular, we emphasize fit-for-purpose Fidelity, where modeling effort focuses on the physical effects of components rather than solely maximizing model accuracy, favoring scalability and generalizability. Furthermore, experimental and field validations are examined to evaluate the practical applicability of ONDT technologies. We further examine experimental and field validations, from physics-based QoT estimation and measurement-informed model refinement to field synchronization, service provisioning, multi-domain observability, and infrastructure characterization. Finally, we discuss the complementary roles of ONDTs and AI agents toward autonomous optical networks and identify key research directions for practical deployment.
★ Overcoming Fundamental Control Limitations in Powered Descent Flight through Robust Incremental Nonlinear Dynamic Inversion
This work identifies a fundamental, controller-independent limitation in reusable launch vehicle powered descent: a sudden $180^\circ$ phase transition in the open-loop transfer functions from the control inputs to the lateral drift and drift rate. The transition occurs inside the flight envelope, reverses the effective control direction, and can destabilize the closed loop system under uncertainty. An analytical condition governing this transition is derived, its physical mechanism is established, and its dependence on vehicle mass, aerodynamic properties, and geometry is characterized. An Incremental Nonlinear Dynamic Inversion (INDI)-based control architecture is then developed to inherently address this limitation and is tuned via $\mathcal{H}_\infty$ Closed Loop Shaping to provide local robustness and performance guarantees. The internal dynamics introduced by the INDI inversion are analytically derived and proven to be stable throughout the flight envelope. The proposed controller is compared against a robust gain-scheduled baseline controller through linear stability analysis and an extensive nonlinear Monte Carlo campaign. Both controllers achieve 100% success across the considered Monte Carlo campaign, though the INDI-based controller provides improved tracking performance, superior nominal and dispersed stability margins, requires fewer controller design points, and reduces controller tuning complexity.
comment: 28 pages, 16 figures, 5 tables. Accepted for presentation at the AIAA SciTech 2027 Forum, Orlando, FL, January 2027
★ A Hybrid UKF with Explicit Drivetrain Modeling for Low-Velocity State Estimation of Electric Vehicles
Longitudinal velocity is a core input for vehicle control, driver-assistance, and automated driving functions. Production wheel-speed sensors deliver only a few pulses per revolution, so they lose resolution below 1 km/h, the regime in which automated parking and traffic-jam assistance operate. The motor speed from the electric machine's resolver stays finely resolved down to standstill but reaches the wheels only through a compliant drivetrain. We therefore estimate the longitudinal velocity with an unscented Kalman filter (UKF) on a double-track model adapted to low velocities and incorporate the motor speed along two routes, one purely physical and one hybrid. The first augments the model with an explicit torsional spring-damper drivetrain, making the motor speed a state of the estimator. The second feeds the output of a neural network-based virtual wheel-speed sensor into the UKF as a pseudo-measurement. Both routes are compared against two baselines, a wheel-speed-only configuration and one that adds the motor-speed sensor but treats the driveline as rigid. Because the model is numerically stiff near standstill, the comparison rests on a systematic study of explicit and semi-implicit discretization schemes. All four configurations are evaluated against an inertial reference in real-world test drives with two distinct series-production electric vehicles, anonymized as Vehicle A and Vehicle B for confidentiality. Below 1 km/h, both proposed estimators lower the velocity root-mean-square error by at least 54% and 45% against both baselines on Vehicle A. Carried over zero-shot to Vehicle B without re-tuning on target-vehicle data, they lower it by at least 41% and 51%. Both estimators therefore carry across vehicles without a new identification campaign.
comment: 14 pages, 11 figures, 3 tables. Submitted to IEEE Transactions on Intelligent Vehicles
★ Semi-Analytical Geometric Synthesis of Robust Quantum Controls
High-fidelity quantum gates are a central requirement for fault-tolerant quantum computation, yet their implementation is limited by the sensitivity of qubits to environmental noise and by control non-idealities. To mitigate these effects, quantum gates can be designed as dynamically corrected gates (DCGs), where control sequences are engineered to suppress the impact of errors on the resulting operation. Many methods in the literature are able to synthesize DCGs, but they typically rely on numerical optimization, with synthesis times on the order of minutes per gate. Here, we propose an automated semi-analytical method that reduces this cost by orders of magnitude, synthesizing smooth, robust control pulses in tens of milliseconds rather than minutes, with comparable fidelity and robustness. Our approach is based on the Space Curve Quantum Control (SCQC) formalism, where quantum evolution is mapped onto a geometric space curve whose geometric properties determine the target gate and its robustness. We parameterize the curve in the Frenet-Serret frame and employ the Pontryagin Maximum Principle (PMP) to design controls that are robust against both amplitude and dephasing errors. The PMP is formulated as a boundary-value problem and solved using a shooting method. The Frenet-Serret frame is integrated using Rodrigues' formula. Starting from a reference solution, we gradually vary the target rotation angle to construct a library of solutions, whose interpolation provides initial guesses for the shooting parameters of new target gates. We evaluated the method on 2,000 single-qubit rotations, obtaining a mean local optimization time of 39.945 ms per gate after preparation of the interpolation base. Quantum propagation yielded nominal gate infidelities below 10^-9 after virtual phase correction, with verified robustness against quasistatic amplitude and dephasing errors.
comment: 12 pages, 3 figures
★ On the Minimal Controllability of Boolean Networks With $k$-Regular Dependency Graphs
Boolean networks (BNs) are widely used to model gene regulatory networks and other biological systems. In this paper, we investigate the minimal controllability problem for BNs subject to degree constraints. We first establish three new best-case upper bounds for XOR-BNs whose dependency graphs are $k$-regular. We then introduce classes of BNs that are invariant under cyclically partite transformation (CPT-invariant) and prove theorems connecting BN controllability with the surjectivity of vector-valued Boolean functions. Finally, by combining an analysis of this surjectivity with a probabilistic argument, we derive a new general lower bound on the minimum number of control nodes required for majority-type threshold BNs with $(2k+1)$-regular dependency graphs.
comment: 16 pages
★ When Imperfect Automated Fact-Checking Backfires: Social Learning in the Presence of Misinformation
Automated fact-checking tools, cheap and fast but imperfect, are increasingly available on social platforms. We study when such tools can backfire in a sequential social learning model of misinformation. Agents observe an article's reliability and their predecessors' actions, which indicate whether they liked or disliked the article. They may also pay to fact-check it. We examine how welfare, defined as the probability of a correct cascade, changes when a free, error-prone automated tool supplements costly but perfectly accurate manual fact-checking. Two opposing effects arise. Broader use of fact-checking helps agents escape wrong cascades. At the same time, errors from the automated tool can mislead agents who would otherwise have verified the article perfectly, reducing the likelihood of correct cascades. We prove that adding the tool strictly lowers welfare for articles of intermediate reliability when truthful articles are not too informative and manual checking is sufficiently costly. Simulations show that the tool can help on either side of the intermediate range, and that it improves on manual-only checking on average only above a threshold accuracy.
comment: 8 pages, 5 figures
★ Types of Bursting with a Two-Block Spiking Primitive
Excitable spikers are attractive systems for sparse, event-driven computation. A spiking primitive composed of a linear frequency selector closed in positive feedback around a high-gain saturating nonlinearity has recently been proposed as a minimal excitable system. This paper asks which bursting classes that primitive can produce with two elementary selectors: a first-order high-pass filter, and a second-order resonant band-pass filter. The nonlinearity decides whether the system fires, and where the rheobase sits. The selector decides whether rest and repetitive firing can coexist, and therefore which bursting class is available. We show that the high-pass never makes rest and firing coexist, whatever the nonlinearity, and that no linear slow feedback branch can then produce bursting. With a piecewise linear nonlinearity, it gives type I excitability, and a second copy of the primitive gives parabolic bursting. The resonant band-pass gives a Hopf, type II excitability, and a subcritical onset above a computable gain, hence a window where rest and firing coexist. A single additional slow linear feedback then gives elliptic bursting. We also show that the resting point is unique for both selectors, which makes square-wave bursting structurally unavailable. Closed-form thresholds, firing rates, and criticality conditions are given and validated numerically.
★ A Unified Fractal Impedance Control Framework
Physical human--robot interaction requires controllers capable of maintaining intrinsically bounded interactions with the environment. The Fractal Impedance Controller (FIC) has been proven to guarantee these safety characteristics by design. This paper introduces the robotic implementation and experimental validation of its most advanced architecture: the Harmonic-Passive Motion Paradigm (H-PMP), an evolution of the Passive Motion Paradigm designed to mimic human-like interaction. Furthermore, we present a unified framework for FIC deployment, supported by an open-source software suite featuring ROS2, MuJoCo, and physical hardware integrations. Through three experiments conducted on a Kinova Gen3 arm (trajectory tracking, a wall press with sudden loss of contact, and learning-by-demonstration) we compare the H-PMP against earlier FIC architectures. The results demonstrate that the H-PMP achieves a seamless trade-off between admittance and impedance behaviours. This makes it ideal for deployment in unstructured environments, enabling the robot to transition between interaction modes with no computational overhead.
★ Extensions to the Dubins Airplane Method - Flight Performance Constraints in Uniform Wind
This paper extends the Dubins Airplane Method to generate trajectories for fixed-wing aircraft that remain within flight performance limits in the presence of uniform wind. Within the field of runtime-efficient trajectory generation for fixed-wing aircraft, the Dubins Airplane Method represents a well-received and widely adopted compromise between algorithmic complexity, runtime efficiency, and solution quality. However, its simplistic kinematic model restricts its ability to generate safe and feasible trajectories, especially in the presence of wind. As a mathematical foundation to overcome these limitations, a set of closed-form equations are derived that relate flight performance constraints to kinematic constraints. Building upon existing methods that use trochoids for planning of 2D trajectories in uniform wind, this paper presents a set of iterative algorithms for planning versatile 3D trajectories composed of trochoidal or circular turns and straight flight segments, with constant vertical speed, constant thrust or smooth longitudinal transitions. The presented methods comprise a flexible and modular framework for runtime-efficient generation of performance-aware trajectories based on parametric curves without the need for numerical trajectory representations.
★ On the Spectral Dependence of Decentralized Stochastic Gradient Tracking
We study how the spectral geometry of the mixing matrix enters the convergence of decentralized stochastic gradient tracking (GT) for smooth strongly convex problems. General bounds contain the factor $Δ_λ^{-3}$ in the higher-order stochastic term, where $Δ_λ$ is the absolute spectral gap. We show that, even for two agents with identical quadratic objectives, the exact steady-state mean-square disagreement scales as $α^2σ^2Δ_λ^{-3}$ as the nontrivial eigenvalue approaches $-1$, provided the stepsize $α$ is sufficiently small relative to the gap. Here $σ^2$ is the gradient-noise variance. By unrolling the disagreement recursion, we establish a stochastic bound that averages all nontrivial eigenvalue contributions for diagonalizable mixing matrices, up to an eigenvector conditioning factor. As a real or complex eigenvalue approaches the unit circle away from $1$, its contribution grows as the inverse cube of its distance to the circle; growth can be slower near $1$. For symmetric mixing with gaps $p$ at $1$ and $ν$ at $-1$, the weighted sum $θ_1/p+θ_2/ν^3$ replaces the prior factor $(Δ_λν^2)^{-1}$. For $N$ agents, $Nθ_1$ and $Nθ_2$ count the nonnegative and negative nontrivial eigenvalues, respectively. On a symmetric family, a finite-horizon lower bound for the average iterate matches the worst-case factor $p^{-1}+ν^{-3}$ up to universal constants. Numerical experiments confirm the predicted scaling.
★ Scenario-aware Adaptive Neuromorphic Intelligence for multimodal sensor fusion
Multisensory Physical AI systems operating in isolated conditions must make timely and reliable decisions within strict energy and hardware budgets. In this context, the present analytical study investigates the feasibility of reliable multimodal sensor fusion using a neuromorphic approach based on memristor-based Integrate-and-Fire neuron circuits driven by aggressively compressed representations of sensory information. Rather than relying on fixed processing and learning parameters, the proposed framework introduces scenario-dependent adaptability into individual stages of the neuromorphic decision-making pipeline, including sensor-data compression, nonlinear spike encoding, reliability assessment, and memristive learning-forgetting dynamics. By coupling multiple sensor inputs through adaptive memristive synapses, the framework investigates how reliable and correlated observations can strengthen learned associations, while weak or obsolete relationships gradually decay through nonlinear forgetting. The resulting neuronal responses are evaluated across ten representative driving scenarios derived from the NuScenes dataset by comparing their encoded output patterns with predefined synthetic ground-truth codes representing the expected actuator responses. The simulations demonstrate that scenario-adaptive activity-vector sizing achieves a 61.5% reduction in retained sensory information, while improving mean bit-wise agreement from approximately 55% to 61.2% compared with a fixed maximum-resolution configuration. These findings highlight the potential of introducing scenario-aware adaptability directly into the building blocks of a neuromorphic system, providing a foundation for future hardware implementations that could significantly transform the capabilities of Physical AI systems operating under stringent resource and environmental constraints.
comment: 29 pages, 17 figures
★ Power-Swing Blocking Scheme Based on Continuous and Higher-Order Incremental Quantities for Systems with Inverter-Based Resources
Conventional power-swing blocking (PSB) schemes for distance protection face new and emerging challenges as the penetration levels of inverter-based resources (IBRs) increase. Several papers highlight interoperability issues concerning faster power swings and unconventional impedance trajectories. Revised relay settings are proposed in attempts to deal with the problems. However, the blinder-based PSB methodology suffers from being dependent on the speed of the apparent impedance trajectory over the tiny region between the blinders, making it difficult to distinguish between fast swings and faults. Moreover, blinders have the inherent limitation that they cannot operate if the measured impedance is inside the blinders when the fault is cleared and immediately enters the distance zones during the first swing cycle - an issue often overlooked in the current literature. This paper proposes a novel PSB scheme based on higher-order incremental quantities (IQs) for difficult swing scenarios with IBRs and synchronous generators (SGs). PSCAD/EMTDC simulations of an SG and an implemented grid-forming (GFM) inverter validate its superior performance compared to conventional dual-blinder PSB schemes. The impacts of GFM inverters on the impedance measured by distance relays during the fault-to-swing transition and first power swing cycle are also discussed.
comment: 9 pages, 11 figures
★ Geometry-Evolving Barrier Functions for Safety Evaluation in Dynamic Environments
Autonomous systems operating in dynamic environments often involve multiple interacting agents, giving rise to high-dimensional safety evaluation problems. Hamilton--Jacobi (HJ) reachability provides rigorous finite-horizon safety guarantees, but solving a global HJ partial differential equation (PDE) over the entire state space leads to prohibitive computational complexity. In this paper, we introduce Geometry-Evolving Barrier Functions (GEBFs), a novel representation for dynamic safety evaluation that propagates local safety geometry of the solution to a terminal-value HJ PDE. By evolving the value, gradient, and curvature information along a Hamiltonian-induced reference trajectory, GEBFs provide a compact representation of the local HJ safety landscape. Under suitable local regularity assumptions, we establish consistency between the GEBF and the value, gradient, and Hessian of the HJ solution along an HJ-optimal characteristic. Building on these theoretical results, we develop an iterative algorithm that couples local geometric propagation with trajectory refinement. We further establish a trajectory-wise one-sided relationship between the GEBF and HJ value function, together with a fixed-point consistency result for the iterative scheme. Experiments on widely adopted agent--obstacle collision-avoidance benchmarks, across low- and high-dimensional systems, demonstrate that GEBFs accurately capture the evolution of dynamic safety boundaries, while achieving substantially lower computational cost than conventional HJ reachability.
★ EnergyBench: a live, gate-aligned benchmark of day-ahead forecasts for European power markets
Energy forecasting models are usually compared on backtests and fixed public datasets. Backtests can use data that did not yet exist when the forecast would have been made, and a fixed test set, reused across studies and possibly contained in the pretraining data of foundation models, invites overfitting to the benchmark. EnergyBench evaluates every entry live instead. Each day at 11:30 CET/CEST, thirty minutes before the day-ahead (SDAC) gate closes, every model forecasts the next day's price, wind and solar generation and load on the 15-minute grid for six bidding zones (BE, NL, FR, DE-LU, DK1, DK2). Unlike existing live benchmarks, EnergyBench runs the models itself, on one information set that it freezes at the cut-off and publishes, including three numerical weather prediction models, so that foundation models can be compared with and without the same weather covariates under the actual market deadline. Errors are pooled per series against a seasonal-naive baseline, and models are ranked by a Bradley-Terry rating on the Elo scale with a day-bootstrap interval. The field spans baselines, linear and gradient-boosted models on the frozen weather, eighteen time-series foundation models from eleven families, ensembles, and the grid operators' own forecasts. Over the first thirty-two live days, the covariates reduced the error of each of the ten foundation models run with and without them by 37-51% on matched races (64-73% on wind, 48-61% on solar). Used zero-shot, these models match or outperform a LightGBM model trained on the same covariates. Model size, from 3M to 331M parameters, has a much smaller effect, and median ensembles of the covariate models rank highest. The grid operators' pre-gate forecasts remain ahead on wind and solar, where the best single foundation models have 16-22% more error; on load the gap is 4-6%, within the intervals.
comment: 21 pages, 10 figures, 7 tables. Live benchmark at https://energybench.foretec.co
★ Online Optimization for Controlling HVAC Systems
Heating, ventilation, and air-conditioning (HVAC) systems account for a significant portion of buildings' energy consumption, motivating the design of energy- and cost-efficient control strategies. However, conventional strategies employ simple, reactive control, without incorporating a model. Therefore, in this paper we adopt an economic model predictive control framework and focus on designing online algorithms to solve the control input design problems with a limited computational budget. In particular, we propose two tailored online algorithms based on projected gradient descent, with a custom projection routine, and the projected primal-dual method. We discuss the pros and cons of both algorithm designs, focusing in particular on computational efficiency. We further compare them through numerical experiments simulating the control of an HVAC system over the course of a day. The results show that computationally efficient online solvers for HVAC systems control are indeed possible. Moreover, the performance closely matches that of a baseline algorithm with unlimited computational budget, in terms of reference temperature tracking and comfort temperature range enforcement.
★ Generalized Frequency-Domain Stability Analysis Formulation for Multi-Terminal HVDC Systems
The integration of offshore wind farms (OWFs) and the interconnection of different transmission networks are key drivers for multi-terminal DC (MTDC) systems. Bipolar high-voltage direct current (HVDC) employing dedicated metallic return (DMR) are attractive due to their increased availability, which co-exist with monopolar and rigid bipole configurations. While existing works focus on symmetric operation and particular configurations, we present a unified framework for MTDC stability analysis valid for both symmetric and asymmetric conditions. The formulation uses the minimum number of variables without loss of information, and combines the analytical network representation with black-box converter models, thereby enabling sensitivity studies. In addition, a closed-form expression guiding the stability improvement is presented, which requires minimal data exchange between involved parties. The framework is demonstrated on several case studies of increasing complexity, which highlight the importance of considering intra-station converter couplings and possible uncertainty of network parameters.
comment: Under review (10 pages). For the associated packages visit http://github.com/Electa-Git/PowerImpedance.jl and http://github.com/Electa-Git/Z-tool
★ Data-driven current control of modular multilevel converter-based HVDC transmission system
Model-based design (MBD) is a mature framework for designing the current controllers of modular multilevel converters (MMCs) in high-voltage direct current (HVDC) transmission systems. In MBD, linear state-space models are preferred for their good trade-off between accurate dynamics prediction and simplicity. However, first-principle derivations often neglect relevant MMC dynamics, which can hinder controller performance. Data-driven (DD) methods can deliver less biased models that capture parameter uncertainties, control-loop interactions, and grid dynamics, yet they remain under-exploited for controlling MMC currents. This paper studies DD current control method based on integral Subspace Predictive Control (iSPC), synthesized directly from input--output data measured in a high fidelity real-time digital simulator (RTDS), and proposes an initial guideline outlining the main steps for its systematic application to MMC current control. The proposed iSPC is compared against state-of-the-art DD solution, and the findings are validated through simulation studies in MATLAB/Simulink.
comment: 8 pages, submitted to IET AC and DC Power Transmission (ACDC)
★ Towards Track Height Control in Additive Manufacturing on Porous Substrates: An Approach for System Identification and Controller Selection
Additive manufacturing is transforming the creation of materials and parts, enabling the design and manufacturing of multifunctional parts with tailored properties. One compelling example is the combination of a porous core and a non-porous surface, which offers significant advantages in terms of weight reduction, material efficiency, strength and wear resistance. Directed Energy Deposition (DED) processes enable the transition from porous core to non-porous part surface by closing the pores. However, the surface-layer of porous substrates presents a substantial challenge to achieving consistent track heights during DED, directly impacting the final part geometry and performance. Therefore, precise control of the deposition process is crucial for realising the full potential of the multifunctional parts. In order to address this challenge, this work presents a conceptual approach for track height control during Laser-based Directed Energy Deposition (DED-LB) on porous substrates. First, the DED-LB process and the associated measurement setup are explained, along with a discussion of their limitations. The focus then shifts to the control problem, beginning with system identification to characterise the process dynamics. Subsequently, a selection of control strategies is investigated, as well as their compatibility with the process. This research provides a foundational step toward reliable additive manufacturing on porous substrates, and thus toward producing multifunctional parts.
comment: 10 pages, 3 figures
★ K-subareas partitioning subject to coinciding boundary segments of neighboring subareas
A method for the problem of partitioning an area into K subareas subject to coinciding boundary segments of neighboring subareas is presented. The given area may be non-convex and may additionally include obstacle areas. The proposed hierarchical method involves three steps: repeated bisection splitting, merging and the solution of traveling salesman problems. Geographic real-world outdoors areas data is used for the evaluation of the method. The benefit of computational simplicity at the cost of imbalanced subarea sizes is highlighted.
comment: 6 pages, 10 figures
★ A Cost-Effective Submodule with Fault-Blocking Capability for Modular Multilevel Converters
Modular multilevel converter (MMC) has become an attractive technology for high-power electronics applications thanks to its high energy efficiency, high power rating, high modularity, flexible scalability, and superior output waveform quality. It has attracted considerable research attention, and numerous submodule (SM) variants have been developed for different objectives. Among them, the half-bridge SM (HBSM) is the most widely adopted thanks to its simplicity, but it cannot interrupt DC-side short-circuit currents. Various SM topologies with fault-blocking capability have been proposed. These SM topologies contain additional switches and passive components compared to the HBSM, leading to higher capital and operating costs. Aiming at these issue, an SM topology capable to block DC-side fault-currents with a reduced component count is proposed in this paper. The proposed SM includes two variants, one employs only two fully-controlled switches. An operation and control scheme for the proposed SM are also developed and presented in the paper. The functionality of the proposed SM is validated by experiment, and the value of the SM is demonstrated through a test simulation for an high-voltage direct-current (HVDC) transmission system.
comment: 11 pages, 22 figures
★ A Conformal Transmit-Reflect Array Enabling Multifunctional Bidirectional Beam Switching for Emergency Communication System
This article presents a conformal transmit-reflect array (CTRA) that enables bidirectional multibeam switching through a single metasurface aperture for emergency communication applications. To realize reconfigurable bidirectional beam switching, a transmit-reflect integrated element (TRIE) is developed by combining a polarization-selective transmissive structure with a complementary-symmetric reflective structure. The proposed element enables independent control of the cross-polarized transmission and reflection phases while exhibiting stable performance under oblique incidence. To support practical conformal implementations, a geometrical-optics-based phase compensation method is applied to cylindrical geometries, enabling effective wavefront control across the conformal aperture. By incorporating a polarization-switchable feed array (PSFA), multifunctional beam switching is achieved in the forward, backward, and bidirectional operating modes. The PSFA is configured as a nonuniform curved linear array near the focal region, enabling feed-selective beam switching within $\pm 30^\circ$ in both the $+z$ and $-z$ directions through selective excitation of the corresponding feed elements. The full-wave simulation results agree well with the measurements. The proposed CTRA achieves peak gains of 24.1 and 24.8 dBi in the forward and backward modes, respectively. The maximum aperture efficiencies in the forward and backward operating states are 28.4$\%$ and 33.4$\%$. These results demonstrate the effectiveness and flexibility of the proposed CTRA, making it a promising candidate for adaptive emergency communication systems.
★ Learning and Steering in Incomplete-Information Games
In many modern multi-agent systems, a mediator seeks to influence strategic decision dynamics using only outcome-level feedback and without knowing agents' utilities. In this work, we study how a mediator can learn an unknown game and steer adaptive agents toward outcomes that maximize her utility using only trajectory feedback. The mediator knows the game form and her own objective, but not the agents' utilities. Moreover, she commits to nonnegative payments and provides recommendations and contexts, but receives only a single realized terminal outcome rather than full-information feedback. We develop halving-type algorithms that learn utility differences in normal-form games and conditional continuation-value differences in extensive-form games, achieving a sample complexity of $\widetilde O(\varepsilon^{-2})$ that matches worst-case lower bounds up to logarithmic factors. We then combine these learning procedures with a Lagrangian formulation in an online Learn-Compute-Steer algorithm that interleaves learning, equilibrium computation, and steering toward mediator-optimal extensive-form correlated equilibria (EFCE). With high probability, the Learn-Compute-Steer algorithm achieves a directness gap and an optimality gap of $\widetilde O(T^{-1/4})$, while time-average payments are of $\widetilde O(T^{-1/8})$.
comment: 57 pages, 5 figures
★ dynamicalnodes: Block Diagrams to ROS2 Graphs
Here we introduce dynamicalnodes, a framework for rapid iterative prototyping of control and estimation algorithms within the Robot Operating System, or ROS. The framework leverages the theory of discrete-time dynamical systems to encode each component of a feedback system (e.g. reference generator, controller, plant, observer) as a pair of functions, denoted $(f,\,h) $, where $f$ is an evolution function and $h$ is an output function. With appropriate choice of domain and codomain, the $ (f,\,h) $ pairs can be composed to form feedback systems of arbitrary complexity. Moving from theory to practice, the user implements each $(f,\,h)$ pair in Python and encapsulates them in objects that facilitate composition and simulation. ROS executables are generated from these objects via rclpy, where relevant metadata such as network topology and publishing frequency are written automatically. The final result is a collection of rclpy files from which validated feedback systems can be spun up as live ROS graphs. We discuss the relevant theory behind the framework and then demonstrate the prototyping workflow applied to a canonical controls problem, a car with cruise control.
comment: See the associated github repo: https://github.com/nehalsinghmangat/dynamicalnodes/tree/main
★ Unified Overview of Safety Certificate Functions: Techniques from Hamilton--Jacobi Reachability and Control Barrier Functions
Safety is a central concern in modern autonomous systems. Safety certificates---functions whose existence guarantees that a system meets its safety specification---have long been used as a principled tool for rigorously reasoning about safety. Methods for synthesizing such certificates are scattered across research areas, with little guidance on when to use a particular approach. This tutorial provides a holistic treatment of safety certificate synthesis centered on two widely used families: Hamilton--Jacobi (HJ) reachability analysis and control barrier functions (CBFs). For each family of methods, we first introduce its general formulation, along with classical and learning-based tools for certificate synthesis. Next, we introduce two benchmark problems consisting of vehicle systems of differing complexity to assess the certificate quality of different synthesis methods and their effectiveness in safety-critical control. We use these benchmarks to compare various methods and provide practical guidance for selecting an approach.
★ Training-Independent Validation of a Learning-Based Controller for Stunt Maneuvers
Learned controllers for vehicle maneuvers are usually validated under the conditions and objective used to train them, where they appear highly reliable, yet the parameters that change on real roads are often not those the model was trained on. We argue that validation must be independent of training: it should vary physical parameters that training kept fixed and use acceptance criteria absent from the training objective. A learning-based control policy, G-Maneuver, with 90.11% success is tested for stunt-level maneuvers. A looser terminal-heading score ranked the worst setting as the best. Because real stunt tests are safety-critical, a simulation-to-reality-to-simulation (Sim2Real2Sim) loop uses the simulation to select safe zones for real-vehicle trials with 16.9% acceptance. Automotive safety analysis frameworks are used to identify parameters for validation using coverage tests. We demonstrated that the kinematic stunt-driving indicators, namely heading change, sideslip, and speed-normalized yaw rate, match between simulation and the real world, while force and roll indicators do not, which identifies the indicators that can support safe deployment. A video of the experiments is available at https://youtu.be/dN1Oes8pNIo.
comment: 9 pages, 7 figures, 2 tables. Video: https://youtu.be/dN1Oes8pNIo
★ Simulation Modeling and Control of an Elastic Shaping Antagonistic Variable Stiffness Actuator
Position--stiffness regulation of antagonistic variable stiffness actuators (VSAs) involves coupled nonlinear dynamics, parameter uncertainty, and feedback states that are not directly measured. This paper addresses this challenge through a simulation-based workflow that integrates elastic shaping, model verification, and layered nonlinear control. This study builds on a previously developed actuator comprising two crossed four-bar compliant transmission elastic units. Its optimized elastic mapping provides the structural basis, with an ADAMS torque-curve root mean square error (RMSE) of $2.57\,\mathrm{N\!\cdot\!mm}$ against the target. A control-oriented analytical model is implemented in Simulink and compared with the independently executed ADAMS model under matched inputs. The maximum joint-angle RMSE across the tested cases is $4.66\times10^{-3}\,\mathrm{rad}$. A layered controller combines extended Kalman filtering for state reconstruction, feedback linearization for nominal decoupling, and adaptive sliding-mode control for residual compensation. At the end of the nominal simulation, the joint-angle error is approximately $3.14\times10^{-5}\,\mathrm{rad}$. The stiffness error is $3.07\,\mathrm{N\!\cdot\!mm/rad}$ relative to a target of $50\,\mathrm{N\!\cdot\!mm/rad}$. Parameter-variation and ablation tests assess tracking performance and the effects of the controller modules within the evaluated conditions. The resulting workflow links structural-response modeling, state estimation, and robust control assessment, supporting repeatable comparisons beyond nominal tracking alone.
comment: 29 pages, 11 figures. Extends the structural study in arXiv:2609.18456 (ACIRS 2026) with dynamic modeling and closed-loop control evaluation
★ Kinematic analysis of a double-closed-loop four-bar chain-based leg for a multimodal quadruped-pentarotor robot
This paper presents the kinematic modeling and singularity analysis of a three-degree-of-freedom, double four-bar closed-loop quadruped leg mechanism. This leg is essential for transitioning between the two operational modes; therefore, it is examined in both joint and task spaces, leading to the development of its Jacobian matrix, which is vital for designing the gaits required for terrestrial mode and mid-air aerial reconfiguration (for aerial mode). Various gaits are devised for the system's motion in the MATLAB simulation, and their practical implementation on the single-leg test rig is demonstrated.
comment: Journal paper submitted to Robotica with ID:ROB-2026-0373
★ Two-stage residual learning for calibrated uncertainty in solar irradiance forecasts
Accurate forecasting of renewable generation requires not only point predictions but also reliable estimates of the uncertainty attached to them, particularly at extended horizons. Machine learning models are increasingly used to predict solar output, but they typically report a single value and capture the associated uncertainty poorly. The standard remedy is to train a model to predict the parameters of an assumed probability distribution directly. Here we develop an alternative two-stage method, motivated by synthetic solar signal generation, in which the forecast error is modeled separately from the expected irradiance. A first long short-term memory (LSTM) network predicts the expected global horizontal irradiance, and a second network models the distribution of the forecast error, which is added back to the mean to give both the forecast and its uncertainty. Using historical weather data from the National Solar Radiation Database and a probability integral transform evaluation, we show that explicitly modeling the error brings the predicted standard deviation markedly closer to the historical value, reducing the deviation between the two by 34.6 percent for a standalone LSTM and by 56.5 percent when the LSTM is coupled to Gaussian process regression, the most reliable of the configurations tested. Because the method models the total forecast error directly, it captures the full predictive uncertainty that matters for decisions, rather than only the model contribution that dropout-based approaches estimate.
★ Safe Output-Regulation Flows for Constrained Optimization via Control Barrier Functions
Control-theoretic approaches to constrained optimization have been widely studied by interpreting optimization algorithms as dynamical systems. This paper formulates constrained optimization as an output-regulation problem with state constraints, where equality-constraint residuals are treated as regulated outputs and inequality constraints as state constraints. Proportional--integral feedback is employed to drive the equality-constraint residuals asymptotically to zero, while a control barrier function (CBF) safety filter is used to ensure inequality safety throughout the optimization process. We establish feasibility of the CBF filtering program throughout the inequality-feasible set and local exponential convergence of the resulting flow. Under stronger global conditions, we further prove convergence from an entire connected component of the inequality-feasible set. Since exact implementation of the flow requires solving a CBF quadratic program at each state, we further develop a flow that approximates the CBF solution through multiplier updates. Moreover, we provide a unified formulation encompassing the two proposed flows and projected primal--dual dynamics. Numerical examples demonstrate convergence, safety, and computational tradeoffs of the proposed flows.
★ Truncated backpropagation through time induces a two-sided gradient turnpike property
Truncated backpropagation through time reduces the computational burden of training recurrent models by restricting gradient computations to shorter temporal contexts. This induces two distinct boundary effects: missing past information perturbs the forward state evolution, while truncating the backward recursion omits future information. This motivates decoupling the interval over which gradient contributions are averaged for the parameter update from the surrounding temporal context over which they are computed. Under exponential decay of the state-transition Jacobians and mild regularity conditions, this leads to a two-sided gradient turnpike property: the error between truncated- and full-context gradient contributions decays exponentially with the distance from both context boundaries. We derive bounds for the averaged update gradient, yielding guarantees on gradient alignment and approximate full-context stationarity; under unbiased interval sampling and a Polyak-Lojasiewicz condition, these further imply a bound on the resulting optimality gap. Numerical experiments on the Wiener-Hammerstein benchmark illustrate the role of past and future context and the trade-off between predictive performance and computational cost.
★ A Novel Asymmetric Distributed Voltage Regulation Scheme to Enhance the Utilization of Distributed Energy Resources
Passive voltage regulation in distribution grids using distributed energy resources (DERs) is becoming the standard for feeder voltage management in both under-voltage and over-voltage scenarios. Their simple control, ease of implementation, and fast response make them ideal for distribution feeders with high-DER penetration. This paper proposes a novel distributed voltage control for DERs based on the voltage at the point of common coupling, inverter rating, and operating active power. The proposed control is designed to utilize feeder inverters efficiently to reduce the burden on the inverters in both under-voltage and over-voltage operations and increase the fairness in reactive power ($Q$) support in over-voltage operations. The proposed control is implemented in static and dynamic simulations of a real feeder to validate the performance. Results for the under-voltage scenario are also presented in the paper.
★ Dynamic Watermarking and Action-Conditioned Latent Dynamics for Secure Vision-Based Controllers
Replay attacks are difficult to detect in vision-based control systems because previously recorded observations can remain plausible while the physical system follows a different trajectory. This paper develops an adaptive dynamic-watermarking framework for detecting replay attacks against vision-based controllers. A private Gaussian watermark is added to the nominal control input, and a control input-conditioned latent dynamics network (LDN) predicts the corresponding evolution in a compact latent space learned from image observations. Attack evidence is obtained from the discrepancy between the predicted latent response and the latent representation of the received image observation. A reinforcement learning policy adjusts the watermark covariance using the latent state and detector confidence, thereby balancing attack detectability, watermark energy, and control distortion. A Lagrangian penalty on task-specific safety costs is included during training to limit the adverse effect of the excitation. The framework is evaluated on a vision-guided UR5 manipulation task in which the robot reaches a target while avoiding a moving human arm under nominal and replay-attacked observations.
comment: 8 pages, 3 figures, Submitted to ACC 2027 conference
★ Distributed Robust Model Predictive Control for Optimal Consensus of Unknown Multiagent Systems with Event-Triggered Communication
In this paper, we study the optimal consensus problem for multiagent systems with unknown dynamics. We propose a robust model predictive control (MPC) framework that jointly optimizes the control input and consensus sequences while enforcing state and input constraints. Within this framework, noisy offline data are used to construct nominal prediction models and quantify the associated uncertainty, which is explicitly incorporated into the robust MPC design. To solve the resulting optimization problem at each MPC update with reduced inter-agent communication, we develop a distributed gradient-projection alternating direction method of multipliers (ADMM) algorithm with an event-triggered information exchange mechanism and establish its convergence to the optimal solution. Furthermore, sufficient conditions are established to guarantee robust constraint satisfaction and practical consensus of the closed-loop multiagent system. Numerical simulations demonstrate the effectiveness of the proposed framework.
★ A Synchronized Tri-Modality Wireless System for Multi-Point Non-Contact Cardiac Monitoring
Cardiac physiological monitoring is fundamental to evaluating cardiovascular health and diagnosing related diseases. Non-contact radar sensing can capture chest-wall vibrations with spatial specificity, enabling localized physiological monitoring and cardiac assessment. Progress, however, has been limited by the absence of a reliable, research-grade hardware platform for rigorous validation of radar-based methods. Prior systems typically pair radar, namely Doppler cardiography (DCG), with only electrocardiography (ECG) or other coarse references, providing no direct mechanical ground truth and thus constraining fine-grained evaluation of radar outputs. We present a tri-modality wireless system that integrates ECG, seismocardiography (SCG), and DCG under a unified acquisition and control framework for time-aligned recording and analysis. The tri-modality system combines miniaturized wearable ECG/SCG modules with a high-resolution mmWave radar module, supporting spatially resolved sensing and subsequent feature extraction. To our knowledge, this is the first integrated ECG--SCG--DCG hardware system for comprehensive cardiac monitoring; it enables multi-point thoracic acquisition, achieves precise cross-modal synchronization, and maintains specifications suitable for wearable operation. Experiments on a linear-motion platform show close agreement between DCG- and SCG-derived displacements, confirming accurate displacement measurement. Human-subject studies further demonstrate cross-modal consistency and robust extraction of feature points, establishing ECG as the electrical anchor and SCG as the mechanical ground truth for DCG. Overall, the system provides a rigorous, spatially resolved validation platform that advances non-contact cardiac sensing and supports next-generation radar-based monitoring.
comment: 11 pages, 9 figures
★ A Framework for Guaranteed Active Fault Diagnosis with Applications to Aircraft Systems
Within the multiple-model framework, active fault diagnosis applies auxiliary control input signals to ex?cite the system such that fault modes are distinguishable. For aircraft systems, it has been pointed out by some rele?vant works that certain faults cannot be effectively detected and isolated in time without actively exciting the aircraft dynamics. However, existing works have not simultane?ously addressed the optimal auxiliary control design with theoretical guarantees and the FDI method with an effective mechanism for triggering the active fault diagnosis. Hence, we are motivated to propose a framework of active FDI for aircraft systems to explicitly address the aforementioned issue. Specifically, we first formulate a receding-horizon FDI scheme for estimating the likelihood of candidate fault modes with proved effectiveness, and then we propose a method for estimating mode probabilities, based on which a mechanism is designed for triggering auxiliary control for active fault diagnosis. The auxiliary control signal design is formulated such that the effectiveness for FDI is theoret?ically guaranteed and the disruption to nominal flight oper?ation is minimized. Numerical test results are presented to demonstrate the effectiveness of the proposed framework.
★ An Ordering Optimization for Fruit-Vegetable in High-Density Environment based on Genetic Algorithm
This paper presents a robotic harvesting method that combines RGB-D fruit localization, GA-based harvest ordering, and waypoint-based robot motion in dense and unstructured crop environments. The global RGB-D camera provides 3D fruit positions for ordering, while the end-effector-mounted camera provides close-range observation during execution. The GA determines the harvesting sequence, and static or dynamic waypoints define the intermediate robot motion. The method was evaluated in a controlled laboratory testbed and in four-, five-, and six-fruit field conditions. GA solutions were compared with exhaustive enumeration, and both waypoint offsets were examined by parameter sensitivity analysis. Continuous execution reduced unnecessary Home-return motion, while the dynamic waypoint changed with the remaining fruit distribution without changing the GA sequence. In the field evaluation, the dynamic-waypoint continuous strategy produced 40 collision events and a 70.7% target-approach success rate, compared with 55 collision events and 65.3% for the static-waypoint continuous strategy.
comment: 39 pages, 11 figures
★ How do Transformers Perform Low-Level Control In-Context?
One of the remarkable abilities of Transformer-based neural networks is in-context learning. This motivates us to think about "in-context" control. Can we control a system knowing only (1) that it belongs to a particular family of systems and (2) some data on system evolution (provided as the context trajectory) but no specific parameter information? Specifically, we investigate the ability of Transformer-models to perform "in-context" control of unstable, nonlinear dynamical systems, where context consists only of the unforced system dynamics. We leverage a multi-head, GPT-2-style architecture, and we evaluate performance for two classical nonlinear systems -- the cartpole system and the two-link acrobot system, where the system parameters are unknown to the Transformer. We consider a setting where trajectories from an expert controller are available at training time but not at inference time. Here, we identify training techniques that are essential to train Transformers to perform low-level control. Through an analysis over checkpoints, we show that in-context control is possible, and the learned policy performance generally improves as context increases. Furthermore, the Transformer models can generalize to stabilize systems that are out-of-distribution.
comment: 6 pages, 5 figures, Accepted to the 65th IEEE Conference on Decision and Control (CDC 2026)
★ State-Feedback Passivity Control for Synchronous Machine Systems
The passivity of a generating station enables network stability to be established from conditions verified locally at each station. For synchronous machines, however, passivity can only be certified under conditions on the operating point, and these conditions fail under heavy loading and weak damping. This paper develops a state-feedback controller that enforces passivity using a storage battery connected at the terminal bus. A bilinear term arising from the product of the rotor-speed and stator-current deviations is treated as a bounded uncertainty, leading to robust positive-real conditions that can be enforced by a state feedback control designed using a single algebraic Riccati equation. The resulting controller guarantees passivity whenever the Riccati equation admits a positive-definite solution, with no separate condition on the operating point. Numerical examples show that existing conditions certify passivity only up to a fraction of the rated current, whereas the proposed state-feedback approach certifies it at the rated operating point of the machine.
★ Tags Helping Tags: Coordination-Free Power Boosting for Battery-less Backscatter Networks
Battery-less backscatter sensor nodes powered by radio-frequency wireless power transfer are attractive for large or hard-to-reach deployments, but their range is typically limited by the forward power-transfer link rather than by communication sensitivity. We present a method for improving power delivery, and thus range, in dense tag deployments by having nearby tags act as helpers: each helper terminates its antenna in purely reactive impedances, reflecting the power beacon's signal toward a target tag so that it combines constructively with the direct line-of-sight path and increases the power the target harvests. On a hardware testbed, helpers using discrete impedance control are shown to increase the target tag's harvested power by over 50% above an all-absorbing baseline, while randomly set helper impedances give no reliable benefit, and are sometimes even detrimental. We further propose and implement a local algorithm that lets each helper tune its own impedance using only measurements of the target's packet rate, without channel estimation or coordination between helpers, and we show that it brings the target's packet rate close to that of an independently tuned configuration. These results suggest that independent, passive sensor nodes can cooperatively extend each other's power-transfer range with minimal added infrastructure.
★ Reducing Data-Center Load Transfers with Ride-Through-Aware Voltage Control
Fault-induced transfers of large data centers to backup supply can cause abrupt reductions in grid-served demand. Coordinating reactive-power support to mitigate these transfers requires consideration of facility-specific voltage--duration ride-through characteristics, particularly when support capacity is limited. This paper develops a ride-through-aware extension of quadratic voltage control that assesses voltage deviations relative to the admissible margins specified by these characteristics. We establish four design requirements that jointly express envelope-based severity, duration dependence, and preventive voltage-restoration incentives. Within a multiplicatively gated quadratic class, we derive necessary and sufficient conditions for satisfying these requirements and construct an explicit family of envelope-normalized objectives. We apply the proposed objectives to reactive-power coordination and evaluate their performance on the IEEE 14-bus, New England 39-bus, and NPCC 140-bus systems. Comparisons with Volt/VAR control, conventional nominal-voltage tracking, and recent data-center voltage-control methods demonstrate improved data-center retention under matched reactive-power support constraints. In the IEEE 14-bus study, the proposed controller reduces the mean data-center disconnection rate by 44.8\% and grid-demand loss by 27.4\% compared with the best-performing baseline. These results highlight the benefit of incorporating data-center ride-through requirements into reactive-power allocation.
comment: 10 pages, 5 figures
★ Robust Adaptive Lyapunov-Based MPC for Nonlinear Systems with Experimental Validation on Quadcopter Attitude Control
In this paper, a computationally efficient robust adaptive Lyapunov-based model predictive control (RALMPC) framework is proposed for a class of constrained nonlinear systems subject to uncertainties and external disturbances. A radial basis function neural network (RBFNN), introduced to approximate non-parametric uncertainties, is incorporated into a control Lyapunov function analysis, together with integral barrier Lyapunov functionals for state constraint enforcement. This analysis yields control and adaptation laws for parametric uncertainties, external disturbances, RBFNN weights, and worst-case approximation error, rendering the closed-loop system uniformly bounded while ensuring convergence of tracking errors to compact sets around the origin. The resulting guarantees are then embedded into the RALMPC optimal control problem through a contractive constraint, thus combining formal closed-loop stability with real-time performance optimization; at the same time, the adaptation laws allow the RALMPC prediction model to be updated online. Comparative hardware-in-the-loop simulations against state-of-the-art MPC schemes validate the efficacy and computational efficiency of the proposed framework. Ultimately, RALMPC is experimentally validated on a custom-built quadcopter platform for time-varying attitude tracking, demonstrating its practical feasibility; comparisons against alternative strategies confirm its superiority in achieving improved closed-loop performance.
★ FishSense Lite: A Camera-Based, Single Laser Hardware Framework with Software System for In Situ Fish Length Measurement
Fish stock assessments need fish lengths paired with species from far more places than trained surveyors can reach. Recreational divers already visit many of those places, but measuring a fish underwater has required trained observers or specialist equipment. We present FishSense Lite, which recovers fish length from a single photograph taken with a recreational dive camera, a dive laser pointer, and a 3D-printed mount. The diver aims the laser at a fish and takes a picture; afterward, the dot's position in the image gives the range to the fish, and the range turns the fish's size in the image into a length. A citizen-science program, trained by the system's builders, taught its own divers and collected lengths paired with species for 73 wild fish across seven open-water dives. On seven wild fish also measured by an underwater stereo-imaging system, the two instruments agreed without detectable bias (mean difference +0.8%). On rigid targets of known length, FishSense Lite recovered every target to within 1.3%, and 99% of 1,055 frames fell within a 15% error budget. From the error sources we measure, we derive a field protocol: calibrate against a dive slate on every dive, photograph fish within 15 degrees of perpendicular and beyond 1 m, and take at least ten frames of each.
comment: 27 pages with 14 figures
★ Vector Lyapunov functions for nonconvex constrained optimization
Nonconvex constrained optimization problems are commonly solved by a Newton method for necessary conditions (sequential linear/quadratic programming) or a barrier reformulation (interior-point methods). We present a vector Lyapunov analysis for Newton-type algorithms based on constraint violation and optimality of the primal solutions to a sequence of approximate optimization problems. We thus prove local stability and convergence by constructing a strict Lyapunov function for the primal solution.
comment: Extended version of submission to the 2027 European Control Conference
★ Residential Vehicle-to-Grid Deployment: A United States-Based Stakeholder Perspective
The integration of vehicle-to-grid (V2G), in which electric vehicle (EV) owners can send electricity from their vehicles back to the grid, is an emerging solution to enhance grid resilience. However, V2G deployment in the United States lags behind many other countries. This can be attributed to a broad range of factors, and to investigate them, this study analyzes semi-structured interviews with stakeholders using inductive-deductive qualitative coding. Results from the interviews are coded across financial, technical, social, political, and organizational dimensions in order to understand the three main deployment factors of drivers, benefits, and barriers of V2G. Once coded, the relationships between these factors are used to construct a cognitive map to identify actions that will most effectively drive V2G deployment. The results indicate that grid resilience and decarbonization are seen as the main benefits of V2G. User willingness to participate, a lack of technological readiness and standardization, prohibitive costs, and unsupportive laws and policies are among the most frequently cited barriers. Connections in the cognitive map indicate that the most closely related drivers for mitigating these barriers are user compensation for battery degradation, community outreach, the creation of grid interconnection standards and protocols, and supportive grants and policy frameworks for V2G piloting and adoption.
★ Multi-objective Electric Vehicle Charging Strategies at Energy Hubs with Justice-informed Profit Distribution among Users
This paper studies potential energy cost savings in electric vehicle (EV) charging hubs when users allow cooperative vehicle-to-grid (V2G), and how such savings should be redistributed. To this end, we present multi-objective operational strategies and justice-informed profit sharing methods for a charging energy hub equipped with a battery storage system and solar panels. Specifically, we first frame the dynamic energy hub operational problem aiming to minimize a combination of grid costs and emissions, accounting for time-varying prices and energy mix as well as storage system and EV battery degradation, which is especially relevant when considering V2G operations. We parse the problem as a second-order conic program that can be efficiently solved to global optimality. Our real-world case-study shows the importance of flexibility for hub operation: Equipping a charging station with a stationary battery and leveraging dynamic V2G can significantly reduce the total costs and emissions compared to conventional settings without a battery or dynamic charging by more than 10%. To redistribute these cost benefits to users, we compare standard Nash bargaining with a justice-informed sufficientarian version which caps charging prices to safeguard adequate access to affordable charging. Our results show that the sufficientarian approach better accounts for inequalities in flexibility capital without significantly affecting aggregate costs, highlighting how justice-informed pricing only marginally affects total revenues.
★ EMT Modeling and Coordinated Mode-Transition Ride-Through Control of Modern Data Centers
Large data centers are emerging as power-electronic loads whose response to grid voltage disturbances can affect bulk-system stability. This paper presents the composition and control characteristics of a proprietary electromagnetic transient (EMT) model of a modern data center comprising an uninterruptible power supply (UPS) with battery energy storage, induction-motor cooling loads, a computer room air handler (CRAH) driven by a variable-frequency drive (VFD), and an aggregated server load. A voltage-based state machine coordinates three aspects of the facility response: the UPS operating mode (eco, double conversion, or battery), the supply path of the CRAH, and a ramped reconnection of the grid-side UPS converter after the disturbance clears. A 100 MW facility is implemented in PSCAD/EMTDC and benchmarked against the PSSE PERC1 aggregate data center model under a symmetrical three-phase fault and low- and high-voltage ride-through (LVRT/HVRT) events. The two models agree closely in their voltage and active-power trajectories, while the EMT model also resolves the converter and motor-reacceleration transients that the phasor-domain model cannot.
★ Spiking neuromodulation: a linear control perspective
Neuronal excitability is a nonlinear phenomenon, and the complexity of spiking experimental traces sometimes looks defeating. The present paper takes the view point that excitability is a neuromodulated property, and aims to demonstrate that the neuromodulation of spiking is amenable to the classical linear analysis tools of control theory. We illustrate the value of linear feedback analysis as an alternative to bifurcation analysis in the simple yet classical example of spiking modulation by the A-type potassium current.
comment: 8 pages. Submitted to European Control Conference ECC 2027
♻ ★ OpenTac: An Open Ecosystem for Customizable, Scalable, and Transferable 3D Tactile Sensing
We present OpenTac, a novel sensor and a full-stack ecosystem for tactile skins that are highly customizable, scalable, and readily deployed in downstream robot control and learning applications. Our core contribution is twofold. First, we develop a scalable resistive tactile skin with a structured deformable overlay that converts tangential loading into spatially varying pressure. Differential responses across neighboring "taxels" support decoding of local shear vectors, and the common-mode response encodes normal force. Second, we provide a highly customizable sensor design tool that is able to automatically generate fabrication files for these tactile skins from 2D outlines and developable 3D surfaces, including from abstract descriptions such as text. We develop an ecosystem around the OpenTac sensor, including the CAD design tool, a sensor simulator, cross-sensor transfer, and fast/adaptive data acquisition. We demonstrate the broad applicability of the OpenTac-CAD sensor design tool by designing and fabricating more than 30 sensors across a variety of robot embodiments. We characterize the three-dimensional tactile fields produced by OpenTac with a commercial, high-precision force/torque sensor. We demonstrate the broad applicability of 3D tactile sensing across contact-rich feedback control and manipulation policy learning on four real-world robot embodiments. Additionally, adaptive acquisition enables kilohertz-rate partial readouts, and our readout circuitry enables high dynamic range (HDR) acquisition, measuring, for example, a 3.7 kg weight stack and a 20 g weight on the same sensor in the same frame. We provide all hardware, software, and result videos at opentac.github.io
comment: 15 pages, 15 figures. Project website, open-source hardware and software, and demonstration videos: https://opentac.github.io
♻ ★ Bridging Batch and Streaming Estimations to System Identification under Adversarial Attacks
System identification in modern engineering systems faces emerging challenges from unanticipated adversarial attacks beyond existing detection mechanisms. In this work, we obtain a provably accurate estimate of the Markov parameter matrix of order $k$ to identify partially observed linear systems, in which the probability $p$ of having an attack at each time is smaller than $0.5$. We show that given the batch data accumulated up to time $T^*$, the $\ell_2$-norm estimator achieves an error with stepwise exponential decay as the attack probability $p$ decreases. Furthermore, low-dimensional attack structures can decouple the exponential decay order from the attack probability $p$, achieving an error decay that scales with $T^*$. We then propose a stochastic projected subgradient descent algorithm on streaming data that produces an estimate at each time $t
comment: 16 pages, 2 figures
♻ ★ Data-Driven Stochastic LQR Control with Probabilistic Guarantees and Uncertainty Refinement
This paper presents a probabilistic framework for the direct, data-driven synthesis of the stochastic linear- quadratic output-feedback control for linear systems sub- ject to process and measurement noise. Using delayed input-output measurements, we derive a lifted parameter- ization together with an equivalent lifted linear quadratic cost, enabling controller synthesis without explicitly identi- fying the lifted dynamics. Two complementary closed-loop formulations, based on trajectory and covariance represen- tations, are then developed, whose uncertainties depend only on the realized but unknown lifted noise. We char- acterize the relative performance of the two formulations in finite-data regimes. High-probability uncertainty bounds are developed for realized noises, leading to a convex optimization solution to the problem, whose feasibility cer- tifies probabilistic mean-square stability (p-MSS) and per- formance guarantees. To reduce conservatism, Gaussian zonotopes are used to refine uncertainty sets by elimi- nating noise realizations inconsistent with measured data, and a novel intersection rule enables the consistent fusion of data-driven and physics-informed information. These components are integrated into an adaptive framework that unifies indirect (identification-based) and direct (control- oriented) data-driven linear-quadratic output-feedback con- trol. Unlike existing direct data-driven LQR methods that verify stability a posteriori, our by-construction approach achieves robust stability alongside reduced conservatism and superior performance. Simulation results show that the proposed controller approaches the ideal optimal solution while outperforming existing data-driven methods under noisy measurements.
♻ ★ Networked Admissibility-Preserving Control for Directed Safe Coordination
This paper addresses safe coordination of scalar agents with constrained input dynamics over fixed rooted digraphs. We propose a networked Admissibility-Preserving Control (APC) architecture for a common moving output corridor under heterogeneous asymmetric input bounds, wherein the Admissibility-Preserving Input Realization (APIR) generates the physical inputs. In logarithmic barrier coordinates, the closed-loop dynamics form an exact cascade of exponentially decaying realization errors and directed consensus dynamics. We derive direction-specific compatibility conditions that guarantee uniform corridor and input margins, bounded commands, unique complete solutions, and exponential consensus for compact initial sets. The closed-form expression for the agreement value quantifies the effect of the initial APIR mismatch. Under the corresponding compatibility conditions, partial pinning guarantees safe reference tracking whenever the root component contains at least one informed agent. Simulations illustrate the coordination guarantees.
♻ ★ Direction-Dependent Path Loss Modeling in Olive Orchards for Precision Agriculture
Wireless links deployed in orchards often exhibit significant variability in the strength of the received signal that is not adequately captured by classical distance-based propagation models. In row-structured olive groves, signal attenuation differs markedly between along-row and cross-row propagation directions, leading to discrepancies when using omnidirectional propagation assumptions such as those adopted in the Free Space Path Loss (FSPL) model or ITU-R vegetation loss formulations. This paper proposes a topology-based propagation model that explicitly accounts for orchard layout and the relative positions of radio devices within the plantation structure. Experimental validation was conducted using LoRa technology operating at 868 MHz, and the results were compared with established models from the literature and with the proposed two-dimensional model. The proposed approach achieves a closer fit to measured RSSI data than conventional models, providing a more reliable basis for link budgeting and network planning in structured agricultural environments.
comment: 11 pages, 10 figures, 5 tables. Submitted manuscript
♻ ★ Worst-Case Finite-Horizon Analysis of Perturbed LQR via Performance Estimation Problem
We propose a framework to assess the robustness of a given state-feedback controller by evaluating its worst-case finite-horizon cost over a class of norm-bounded perturbations. To compute this cost, we show how to apply the Performance Estimation Problem (PEP) framework, originally developed to analyze optimization algorithms, and reformulate the worst-case search as a rank-constrained Semidefinite Program (SDP). Our results suggest that the perturbations that induce the highest finite-horizon costs can differ substantially from those identified using standard asymptotic metrics. Intuitively, worst-case perturbations are obtained by aligning slowly decaying modes with specific initial conditions to drive transient growth, causing a bifurcation in the worst-case behavior as the horizon length changes. These findings open new possibilities to certify finite-horizon performance of broader system classes, including data-driven models or nonlinear components.
♻ ★ A two-stage approach to satellite constellation optimization: classical and QUBO formulations
The design of satellite constellations for Earth observation requires balancing spatial coverage, revisit time, cost, and operational complexity. This paper considers the problem of designing the orbits of a given number of Low Earth Orbit (LEO) or Very Low Earth Orbit (VLEO) satellites to maximize the spatial and temporal resolution achieved over a prescribed set of ground targets. This kind of problem is inherently nonconvex and possibly combinatorial, making its solution computationally demanding for large constellations and target sets. To address this challenge, we propose a two-stage optimization strategy that separates spatial-coverage design from temporal-resolution optimization, thereby reducing the complexity of the overall problem. Two variants of the method are developed. The first employs continuous decision variables during the spatial-optimization stage, whereas the second discretizes these variables and reformulates the problem as a Quadratic Unconstrained Binary Optimization (QUBO) problem. The latter formulation enables the use of efficient classical QUBO solvers and is directly compatible with quantum-annealing hardware. The proposed framework provides a scalable approach to the design of heterogeneous LEO and VLEO Earth-observation constellations and establishes a pathway for exploiting emerging quantum-optimization technologies in satellite mission design. Preliminary simulation results are presented to demonstrate the effectiveness of the strategy.
comment: 17 pages, 1 figure, 1 table
♻ ★ To Learn or Not to Learn: A Screening Test for Control Problems Before Reinforcement Learning
Reinforcement learning (RL) can be a powerful alternative to classical control methods when standard model-based control is insufficient, e.g., when deriving a suitable model is intractable or impossible. However, in many cases, the choice between model-based and RL-based control is not obvious. Because training RL agents is computationally expensive, RL should be limited to cases where it is expected to outperform model-based control. To the best of our knowledge, there exists no approach to automatically screen out control problems before RL training for which RL is not expected to yield any benefit. In this work, we take the first step towards such an automated screening test. We present a test that identifies problems for which RL-based control is not expected to be superior in two steps. Only control problems that pass both steps should be considered for RL. Part I evaluates the suitability of a provided model for model-based control by analyzing the impact of its uncertainties on the control problem. For this, we use reachset-conformant system identification combined with simulation-based analysis. If the uncertainties are found to be relevant in Part I, Part II then evaluates their learnability using a statistically calibrated dependence analysis. We apply our screening test to several benchmarks, reliably identifying control problems for which RL-based control does not measurably improve upon model-based control, thereby achieving a speed-up of up to $21\times$ compared to trying RL.
comment: This work is currently under review
♻ ★ Modeling Rainwater Harvesting Systems with Covered Storage Tank on A Smartphone
A model for simulating rainwater harvesting systems with a covered storage tank is described. The model is simple enough to serve as the simulation engine of a smartphone app for designing such systems. The results of the simulation can be used to find the optimum tank size and the reliability of the system in meeting a given demand. This is illustrated by using the model for designing two such systems, one in an urban environment and the other in an arid rural location. The Python scripts used for the simulations are included as ancillary material.
comment: Version 2: revised and extended analysis; 21 pages, 13 figures. Python translation of the model, scripts and ERA5 data provided as ancillary files
♻ ★ Single-stage joint diffusion of sky images and clear-sky index for ultra-short-term irradiance forecasting: what the frame-to-irradiance stage discards
Sky-image irradiance forecasting is dominated by a two-stage design: a video-prediction model forecasts future sky frames, and a separately trained regressor maps those frames to irradiance. We propose a single-stage alternative that co-denoises the future sky-image and clear-sky-index sequences in one diffusion process, with an irradiance head that reads a denoised preview of the future sky rather than the noised diffusion state. The forecaster needs no regression stage and runs from sky images and deterministic covariates alone, with no measured irradiance at its input --- the situation at a site with a camera but no pyranometer. On 2023 SIRTA data the model reaches a pooled RMSE of $104.1$~W\,m$^{-2}$ and a skill score of $+0.34$ on the continuous ranked probability score (CRPS) against smart persistence, higher than any of twelve baseline configurations, and retains that skill when retrained on 2021--2022. On the model's own generated frames, a separately trained regressor reaches $150.6$~W\,m$^{-2}$ against the joint head's $104.1$: roughly $46$~W\,m$^{-2}$ that joint training recovers and a cascade discards. Point accuracy is competitive but not best: a deterministic network that also reads measured irradiance attains $97.6$~W\,m$^{-2}$. The model's CRPS skill leads that network's in every sky-condition tertile, by $0.26$ to $0.30$.
♻ ★ ZipDrift: Generating Multimodal Short-Horizon MPC Costs with One-Step Drifting Models
Short-horizon model predictive control (MPC) is fast but myopic. ZipMPC compensates by learning a context-dependent correction of the short-horizon cost that imitates a long-horizon expert. We observe that the optimal correction is often multimodal: when a slower opponent can be passed on either side, good corrections of opposite sign coexist, and the expert's choice between them hinges on tiny cost differences. Deterministic regression then averages modes or inherits inconsistent expert tie-breaking. We propose ZipDrift, which trains a one-step drifting generator over MPC cost corrections, tilted toward low closed-loop cost and learned only from cost evaluations of candidate corrections, without differentiating through the optimiser. A few generated candidates are ranked by a learned critic, and the model-based MPC then enforces dynamics and constraints. In a kinematic overtaking benchmark derived from ZipMPC, more than half of the contexts are bimodal and the manual short-horizon MPC crashes in 28% of them. With an eight-dimensional stagewise correction, evaluated on 1000 fresh contexts and five training seeds, ZipDrift with eight candidates crashes in 0.74% of episodes, compared with 3.4% for an idealised ZipMPC-style regressor and 2.1% for critic-ranked random candidates with the same budget. A mixture density network (MDN) with the same selection is equally good (0.76%). The benefit of drifting lies in sample quality: in a pre-specified test on further unseen contexts, a single drifting sample crashes in 8.7% of episodes against 13.2% for the MDN, and ZipDrift needs four candidates instead of eight to fall below 1%. With many candidates, all methods degrade as selection exploits errors of the critic.
comment: 7 pages, 3 figures, 5 tables. Preliminary simulation study
♻ ★ ThermoLLM: LLM-Based Multi-Zone HVAC Control with a Spatiotemporal Context
Multi-zone HVAC control is a spatial decision problem in which indoor thermal evolution and control decisions depend not only on outdoor conditions and internal heat gains but also on zone layout, physical adjacency, and delayed thermal interactions across the building. Recent LLM-based HVAC controllers have shown that prompt-based control is feasible. However, these methods produce inconsistent results, are difficult to generalize, and experience control-latency issues. Their performance also relies heavily on numeric reward values, textual feedback, or unstructured retrieval, which limits their ability to make control decisions that account for zone dynamics and thermodynamics. No existing LLM-based methods have used building semantics and dynamics, together with rolling history, as the centre of the LLM context while testing and comparing HVAC control performance with existing LLM-based HVAC approaches. This paper presents ThermoLLM, an LLM-based HVAC control framework tested in a multi-zone building environment. Spatiotemporal context grounding is a main component of ThermoLLM, which uses a spatial knowledge graph derived from Brick-style building semantics and linked with recent observations from a rolling history. We evaluate ThermoLLM against standard control methods and recent state-of-the-art baselines in a five-zone building environment across six climate-season scenarios. ThermoLLM consistently achieves the best energy-comfort trade-offs and outperforms all evaluated baselines in the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) rankings across all six scenarios.
comment: 16 pages, 3 figures. Submitted to ACM
♻ ★ Worst-Case Discovery and Runtime Protection for RL-Based Network Controllers
RL-based controllers achieve strong average-case performance in networking tasks such as congestion control and adaptive bitrate streaming. Yet their performance can degrade severely under network conditions where strong performance is still achievable. Identifying such conditions and quantifying the resulting performance gap is intractable by enumeration, while the sequential and closed-loop nature of RL controllers makes formal verification methods impractical. We present ReGuard, a framework that discovers worst-case scenarios for a given RL controller and protects it against them at inference time without retraining. ReGuard formulates scenario discovery as a bilevel regret-maximization problem, certifying a lower bound on the controller's worst-case performance gap. ReGuard then analyzes the discovered counterfactual trajectories and compiles them into lightweight logic rules that intervene only when a risky state is detected, leaving the controller's behavior unchanged otherwise. We evaluate ReGuard across three RL-based network controllers: Pensieve, Sage, and Park. ReGuard discovers scenarios in which each controller suffers a 43-64% performance loss relative to what is achievable under the same conditions. ReGuard exposes performance gaps 57% to 6 times larger than those found by the strongest baselines and shrinks those gaps by 79-85% in its first iteration via lightweight rule-based protection, iteratively driving the remaining gap toward nominal levels. ReGuard's protection extends beyond the scenarios it discovers, improving performance across a wide range of network conditions.
comment: Published at USENIX NSDI 2027
♻ ★ Optical Network Digital Twin -- Practical Use Cases and Architecture
With the widespread adoption of AI, machine-to-machine communications are rapidly increasing, reshaping the requirements for optical networks. Recent advances in Gaussian noise modeling for digital coherent transmission have raised expectations for digital-twin-based operation. However, unlike digital twins in wireless communication, which are already well established, significant barriers remain for commercialization in optical networks. This paper discusses the evolving requirements of optical networks in the AI era and proposes a practical Optical Network Digital Twin architecture enabling dynamic and Quality of Transmission aware operation beyond conventional management. Representative use cases, including operator-driven optimization, user-operator collaboration, and multi-operator interconnection, are presented, along with the architectural framework and key challenges toward practical deployment.
comment: 7 pages, 5 figures
♻ ★ A Memory Efficient Unified Algorithm for Online Learning of Linear Dynamical Systems
Motivated by the challenge of stabilizing a general unknown linear dynamical system (LDS) from observations, we study the natural prerequisite of online prediction. Our goal is to achieve sublinear regret with a memory footprint that adapts to the intrinsic complexity of the dynamics rather than the full hidden-state dimension. We focus on the practically central regime of systems with low instability complexity -- eigenvalues outside the real stable interval that do not decay rapidly, together with non-semisimple modes -- potentially embedded in an otherwise stable real spectrum of much higher dimension; we write $k$ for this count. This regime is the primary setting in which stabilization is plausible: we show that many systems with high instability complexity cannot be stabilized without exponentially large controls. Thus, prediction is meaningful for stabilization precisely when the instability complexity is small. Within this regime, we introduce a unified online algorithm that handles every LDS (including non-diagonalizable systems with complex or exploding modes) with a learnable parameter count of $\widetilde{O}(k)$. Finally, we prove a lower bound showing that $k$ is a valid complexity measure: any filter-based predictor needs at least $k$ filters. Experiments corroborate our theory: on a high-dimensional system, our predictor sharply outperforms prior methods at an equal parameter budget.
comment: 34 pages, 1 figure
♻ ★ XIT: Exploration and Exploitation Informed Trees for Active Gas Distribution Mapping in Unknown Environments
Mobile robotic gas distribution mapping (GDM) provides critical situational awareness during emergency responses to hazardous gas releases. However, most systems still rely on teleoperation, limiting scalability and response speed. Autonomous active GDM is challenging in unknown and cluttered environments, because the robot must simultaneously explore traversable space, map the environment, and infer the gas distribution belief from sparse chemical measurements. We address this by formulating active GDM as a next-best-trajectory informative path planning (IPP) problem and develop XIT (Exploration and Exploitation Informed Trees), a sampling-based planner that grows a single shared tree toward multiple frontier goals, coupling exploration of unresolved regions with informative gas sampling en route. XIT draws a batch of samples from an Upper Confidence Bound (UCB) information field derived from the current gas posterior and expands the tree using a cost that trades off travel effort against information acquisition. To enable plume-aware exploration, we introduce the gas frontier concept, defined as unobserved regions adjacent to high gas concentrations, and propose the Wavefront Gas Frontier Detection (WGFD) algorithm for their identification. High-fidelity simulations demonstrate the benefits of XIT in terms of GDM quality and efficiency, and a scaled real-world experiment demonstrates its practicality. Although developed for active GDM, XIT is applicable to other robotic information-gathering tasks in unknown environments that face the exploration and exploitation trade-off.
comment: Accepted for publication in IEEE Transactions on Robotics
Robotics
★ cuRoboV2-Fast: Fast Torque-Limited Collision-Free Motion Planning
We present cuRoboV2-Fast, a GPU-accelerated planner that reduces both planning latency and execution time for collision-free, payload-aware, torque-limited motions. B-spline trajectory optimization reduces the required iterations, while accelerated robot computations and differentiable inverse dynamics reduce refinement cost and accommodate changing payloads. TimeOpt optimizes traversal speed along the fixed path under sampled velocity, acceleration, jerk, and torque constraints, exploiting local spline support to solve banded Newton systems efficiently on the GPU. With a 3,kg payload, cuRoboV2-Fast achieves 4.4 ms mean planning time on RTX 5090, 14.7x faster than cuRoboV2. Our method on Jetson AGX Orin plans faster than cuRoboV1 + TOPPRA on RTX 5090. A stereo-camera-based deployment on YAM achieves 5.93 s mean completed pick-and-drop cycles, 34.6% shorter than cuRoboV2. Code (Apache 2.0): www.github.com/NVlabs/curobo , Project Page: www.research.nvidia.com/labs/lpr/curobov2-fast .
★ Perceptive Humanoid Motion from Simulation
Humanoid robots have the potential to automate a wide range of physical tasks. Progress toward generalist humanoid autonomy, however, remains constrained by a data bottleneck: mapping pixels and free-form language to low-level joint commands requires paired supervision, and no corpus of such data exists at sufficiently large scale. Some approaches rely on human teleoperation and wearable capture devices, which require at least an hour of human time to produce an hour of data. We present HALO, a humanoid autonomy system powered by a pipeline that synthesizes vision-language-motion data across a thousand procedurally generated homes. The corpus scales with compute rather than human labor. HALO acquires a repertoire of physical skills, including sitting on chairs, navigating to named objects, stepping sideways through half-open doors, crouching and crawling beneath tables, and pushing boxes. It orchestrates these skills to solve composite, long-horizon tasks. Without ever seeing a real home during training, HALO transfers zero-shot to the physical world, achieving a 91% success rate across 360 trials in six unmapped real-world environments. See https://apple-aiml-research.github.io/ml-halo/ for code and videos.
comment: Code and videos at: https://apple-aiml-research.github.io/ml-halo/
★ Reinforcement Learning Enables Scaling Generalist Robot Policy Improvement
The introduction of generalist robot policies has enabled a single, large model to learn to perform a variety of tasks from a vast pre-training corpus. While these models often exhibit powerful zero-shot generalization capabilities, they typically require an adaptation phase to achieve sufficient performance in deployment. Iterated self-improvement via reinforcement learning (RL) has demonstrated promise in enabling this adaptation, yet such approaches have been largely explored in single-task settings - where the goal is to simply improve on a narrow deployment task - which lies in stark contrast to the multi-task capabilities of the base policy. In this work we ask whether generalist robot policies are capable of multi-task, generalist improvement in deployment - targeting many diverse tasks simultaneously - and what algorithmic recipe enables this. To answer this question, we first develop a set of structured evaluations to test generalist improvement capabilities, and systematically evaluate algorithmic and practical design choices regarding value estimation, policy extraction and resource allocation. We show that not only can generalist policies be improved online across diverse tasks, but that the right recipe can enable generalization and increase performance on tasks not considered during online training. We characterize when this generalization occurs and show that both the training task distribution and the use of value-based RL are critical. To the best of our knowledge, this is among the first evidence for online generalist improvement for robot foundation models.
★ VABAR: A Variational Automation Benchmark for Agentic Robotics
For robots to operate reliably in commercial and industrial applications, they must persistently perform specific tasks with known variations in object shape and pose. We call this Variational Automation (VA) to differentiate these settings from fixed automation. Inspired by recent papers on Agentic Robotics, where pretrained LLM and VLM coding agents combine interpretable robot programming with the open-world adaptability of model free policies, we propose a general set of agentic robotics benchmark design criteria that includes accessibility, repeatability, diversity, relevance to agentic robotics and practical applications, physical realizability, resistance to benchmark saturation, and computational efficiency. We instantiate these principles in six manipulation task classes spanning industrial, medical, and infrastructure applications: tool sorting, tool hanging, gear meshing, syringe packing, cable insertion, and cable routing. We introduce the Variational Automation Benchmark for Agentic Robotics (VABAR), an open, easily replicable, simulation-based manipulation benchmark to systematically evaluating agentic robotics systems. Built fully on open-source tools, VABAR features detailed implementations using the Newton physics engine, standardized task variations, difficulty tiers, success metrics, and a library of 33 generalizable skills with human-authored programmatic reference solutions. We provide initial baseline evaluations using an open agentic manipulation harness, Graph-as-Policy (GaP), and Isaac Lab Arena.
★ Rethinking Self-Repair in Code-as-Policy Agents: What Drives Retry Success?
Code-as-policy agents often use execution feedback to retry failed programs. Additional sampling and task guidance can also improve success, leaving the contribution of failure-specific feedback unclear. We study this question in simulated robot manipulation. A fixed lookup selects feedback templates using failure labels and the API variant. A frozen code model generates a fresh program after each scene reset. On the same 252 initial failures, Qwen2.5-Coder-7B recovers 85.3% within five retries, compared with 68.3% for resampling with the same retry cap. The advantage persists under fixed total-token budgets. Feedback from other episodes and fixed task guidance yield similar recovery point estimates, with no detected benefit from matching the coarse failure label. In a separate study, a fixed guided prompt used from the first attempt achieves an observed overall success rate similar to that of template-based feedback. Template-based feedback also lowers recovery on a nut-insertion task requiring a rim grasp, consistent with an incompatible center-grasp instruction. Evaluations of robot self-repair should include resampling, mismatched feedback, and concise, task-compatible static guidance as baselines.
comment: Accepted to the CoRL 2026 Learn@Deploy workshop
★ ConDR: Conservative Reachability through Neural Hamilton-Jacobi Subsolutions
Hamilton-Jacobi (HJ) Reachability identifies the unsafe states of a dynamical system from which it can be forced into failure under worst-case disturbance. However, classical grid-based solvers scale exponentially with state dimension. DeepReach improves scalability by replacing the grid with a neural value function trained on the HJ-Isaacs (HJI) variational inequality (VI), but penalizes both sides of the residual equally. This allows the learned value to lie above the true one and unsafe states to be misclassified as safe. We propose Conservative DeepReach (ConDR) which aims to learn a subsolution of the HJI VI such that the learned unsafe set contains the true one. ConDR enforces the obstacle branch and terminal condition by construction, pushes the HJI residual to a safety margin $ε$ below zero with a one-sided loss, and prevents the degenerate solution of classifying everything as unsafe. We prove that the exact solution of the margin-shifted VI that ConDR trains toward is a subsolution of the original VI so its unsafe set contains the true one and its value lies within $ετ$ of the true value function. We compare against DeepReach and MADR in the established systems of Air3D, Air6D, a 9D multi-vehicle system, and a 13D quadrotor. ConDR reports the lowest rate of unsafe state misclassified and the lowest closed-loop collision rates with at most a small increase in intervention. On Air3D and Air6D, ConDR tolerates model mismatch that scales with $ε$. At matched false-negative rates, ConDR's unsafe sets are smaller than the baselines showing the safety is not obtained by trivial over-approximation.
comment: 8 pages, 4 figures
★ PyRUA-REAL: Robot Exploration and Adaptive Learning through RSI in the Real World
Coding agents powered by models such as GPT-6-Astra connect spatial reasoning with executable robot behavior, raising the question of how such agents can produce lasting improvements in robot capabilities from physical interaction. In this work, we study embodied recursive self-improvement (RSI) through real-world exploration, code refinement, and deployment using PyRUA-REAL, a framework in which coding agents retain interaction experience as persistent memory and reusable executable skills that inform subsequent exploration and improvement. Our empirical study places physical manipulation at the center of the improvement process, examining how agents collaborate, what kinds of execution feedback support effective code refinement, and how task specifications and previously acquired skills influence robustness and task transfer. We further investigate the efficiency and capability of the resulting programs when deployed without involving an online agent for decision-making. Across eight real-world tasks, PyRUA-REAL achieves an average success rate of 91.1%. In complementary evaluation on 42 tasks from the RoboDojo benchmark, it achieves an average success rate of 84.4%. This study provides empirical guidance for turning physical exploration into reusable robot skills that support continued improvement and robust, efficient deployment.
★ Dexterous Robotic Hands in the Foundation-Model Era: A Full-Stack Survey of Hardware, Data, Cross-Embodiment Transfer, Learning, and Evaluation
Dexterous robotic hands are increasingly coupled to large-scale human data and foundation models, yet progress in hardware, representation learning and evaluation is often assessed separately. This survey reviews work from January 2024 to early November 2026, with earlier studies included to establish technical lineages. We organize the literature into five interdependent layers: hardware; data collection and curation; cross-embodiment transfer; learning and foundation models; and evaluation and applications. Human hand-object understanding and tactile sensing provide two cross-cutting perspectives. A system-level architecture, comparative tables and five case analyses connect design choices across these layers. We distinguish within-study experimental comparisons from heterogeneous corpus statistics, vendor specifications and company-reported results. Three conclusions emerge. First, the value of human data depends on kinematic, visual and contact alignment, not recording volume alone. Second, shared action representations and foundation policies do not remove embodiment-specific constraints on force, latency and contact. Third, task completion is insufficient to establish physical correctness or deployment readiness. Evidence remains limited for zero-shot transfer of contact-rich skills between morphologically different hands, long-duration reliability, and cross-sensor tactile generalization. We identify eleven open problems and propose reporting interfaces and controlled comparisons that can make full-stack progress more measurable. The review is a structured narrative synthesis rather than a pooled performance ranking or an exhaustive systematic review.
comment: 45 pages, 8 figures, 21 tables. Structured narrative review of hardware, data, cross-embodiment transfer, foundation models, tactile sensing, and evaluation for dexterous robotic hands
★ Where Is VLA Inference Time Spent? Bottlenecks and Optimization Interactions Across GPUs
Vision-language-action (VLA) models map camera observations and language instructions to robot actions. However, their large backbones and repeated action generation can make inference take longer than the robot's real-time requirements for its next action. To reduce the load of VLA computations, researchers have proposed multiple optimization approaches, including quantization and pruning. These optimizations can be combined with other general optimizations such as better kernel compilation or using CUDA graphs instead of the eager mode. However, most of these optimization results are usually reported on a single GPU type, typically for a few VLA models, with no analysis of what the real bottlenecks are for a model on a particular hardware or how these optimizations would generalize beyond the reported results. In this paper, we provide a comprehensive measurement study of seven families of inference optimization methods, from CUDA graphs and compilation to token pruning and quantization, on seven state-of-the-art VLAs evaluated on four GPU families ranging from a datacenter H100 to an embedded Jetson Thor. We analyse how CPU launch overhead, memory bandwidth, and kernel efficiency jointly shape optimization gains, and how applying one method changes the benefit of another. From this analysis, we derive VLA-Triage, guidelines for stacking optimizations under latency budgets. The guidelines identify which stages can provide the required savings and which combinations need testing in the deployed execution mode.
★ When to Ask for Help: Uncertainty-Triggered Aerial Refinement for Off-Road Navigation
Heterogeneous air-ground teams can use aerial sensing to address the limited range and viewpoint of ground-vehicle sensors, but deciding when the UGV needs assistance and what information the UAV should collect remains a challenge. This paper presents a planner-conditioned air-ground assistance method that evaluates how uncertain map information affects the UGV's current MPPI rollouts. When navigation remains degraded, we score the same rollouts under resolved map hypotheses, select the map layer with the largest useful improvement, request the corresponding aerial measurement, fuse the returned information, and replan. Unlike approaches driven by fixed support roles, task structure, or uncertainty alone, our method conditions assistance on whether resolving uncertainty improves predicted ground navigation. Across 180 simulation trials on three off-road routes, the method achieves 98.9% goal attainment while reducing mean UAV flight time by 87.7% relative to continuous greedy aerial mapping. Aggregated across the routes, each second of estimated UAV flight time saves 2.94 s of UGV navigation time. Field experiments spanning approximately 2 km of UGV operation demonstrate uncertainty resolution and successful navigation recovery.
comment: Project page: https://anonymous.4open.science/w/when-to-ask-for-help/
★ Cognitive and Behavioural Effects of Haptic Feedback for ROV Teleoperation in Hazardous Environments
Teleoperating a Remotely Operated Vehicle (ROV) in complex underwater environments presents unique challenges. Dynamic, three dimensional obstacles and unseen water currents pose real risks to expensive robotic hardware in environments in which recovery can be next to impossible. This paper explores the cognitive effects of haptic feedback to improve vehicle safety in such environments. An in-water study is analysed with a ROV exposed to controlled cross-currents, with environmental information presented via a soft and vibrating haptic touchpad. Results demonstrate that while encountering an unseen cross-current significantly increases an operator's physical and temporal demand, the introduction of haptic feedback mitigates these factors and improves self-assessed performance. Interestingly, the haptic condition also produced the highest levels of mental demand and operator frustration. Despite this increased mental workload, operators drove the ROV significantly faster when receiving haptic cues compared to the current-only condition, indicating a higher level of user confidence. Ultimately, these findings suggest that while haptic feedback may initially require more cognitive processing to interpret unseen hazards, it successfully enhances operator awareness and promotes more confident navigation in hazardous environments.
comment: Accepted for IEEE Telepresence 2026
★ Which Physical Tactile Cues Matter for Contact-Rich Imitation Learning? A Controlled Real-Robot Study
Vision-based tactile sensors capture multiple aspects of physical interaction within a single image stream, including contact appearance, geometry, indentation depth, and tangential deformation. However, existing visuo-tactile policies often directly process tactile observations as images, making it difficult to determine which physical cues contribute to policy learning. This paper presents a controlled real-robot study that compares physically interpretable tactile signals using the LeRobot platform, enhanced with our visuo-tactile perception system. From the same tactile stream, we derive tactile RGB images, normal/depth maps, and marker-flow fields, representing contact appearance, geometry, and tangential deformation, respectively. Using identical task demonstrations and the same policy backbone, we evaluate the individual and combined contributions of these tactile cues. We further compare tactile encoders and visuo-tactile fusion strategies. Evaluation across 720 real-robot trials yields four key findings: (1) tactile perception substantially improves the overall success rate from 57.8% to 78.9%, although its contribution varies across task stages; (2) marker flow is the most effective individual cue, achieving a success rate of 74.4% and playing a particularly important role in regulating established contact; (3) cue-factorized encoding consistently improves performance, highlighting that different physical cues require tailored processing techniques; and (4) although the effectiveness of tactile fusion is task-dependent, cross-attention offers a lightweight solution with performance comparable to that of pretrained tactile encoders.
comment: 6 pages, 5 figures, 2 tables. Accepted to IEEE ROBIO 2026
★ Compounding Physical RSI: Turning Execution Failure into Inherited Capability
Generalist robot policies are trained on static offline datasets, yet physical manipulation unfolds sequentially across stages. Because a weakness in a downstream stage remains unobservable until the policy passes earlier ones, each resolved bottleneck exposes the next hidden failure mode. This staged structure calls for Recursive Self-Improvement (RSI), in which a policy iteratively executes, diagnoses failures, and updates itself. Moreover, because task completion behaves like a product of stage-level pass rates, recursive gains can multiply across generations. In physical environments, three bottlenecks obstruct compounding: terminal execution feedback rarely localizes the stage that caused a failure, failed trajectories do not supply the corrective action, and updating shared parameters can degrade previously mastered skills. We propose Compounding Physical RSI, a self-evolving policy framework that turns execution failures into heritable capability. First, automated capability-gap assessment traces failed rollouts to the task stage where progression breaks down and formulates a structured improvement objective. Second, conditioned on this objective, our remedy stage curates validated, stage-relevant experience and refines the policy via behavior-regularized offline reinforcement learning, without requiring costly online rollouts at each optimization step. Third, to safeguard existing skills, our retention stage constructs weight-space candidates between the inherited reference policy and the refined policy, evaluates them across the task suite, and promotes the best candidate as the reference state for the next generation. Evaluated across all 42 tasks of the RoboDojo-Sim benchmark, CSU-AI-07 ranks first on the leaderboard with a three-seed mean Overall Score of 26.15 and Success Rate of 20.70%, outperforming the next-best entry by 5.92 points and 5.82 percentage points.
★ Detecting Silent Action-Interface Mismatch with Calibrated Relational World Models
A robot action can satisfy shape and range constraints yet carry the wrong semantics: axes may be permuted, signs inverted, rotations rescaled, or gripper polarity reversed. We study learned state-action-effect relations as offline contracts for detecting these mismatches and ranking their likely fields. Clean held-out transitions calibrate alarms; a fixed inverse vocabulary supports closed-set localization. On ten LIBERO Spatial tasks comprising 432 source episodes, a capacity-controlled comparison at 2.5% empirical clean false-positive rate gives 46.1% recall for a ridge forward residual, 61.0% for a four-component joint Gaussian mixture, and 65.7% for the exact conditional of that same mixture. The joint score retains a 14.9 percentage-point advantage over ridge, while conditioning improves recall by a further 4.7 points. Joint scoring instead has higher localization: 95.8% versus 94.6%. These matched-rate comparisons are retrospective diagnostics, not deployment calibration guarantees. Earlier Object/Goal results broaden tasks within the LIBERO convention; DROID, all three HYDRA tasks, short temporal faults, small offsets, exact repair, and simulated mitigation fail their respective gates, while xArm contributes one task only. The evidence supports a bounded offline diagnostic and a detection-localization tradeoff, rather than joint-density superiority or cross-robot safety.
comment: Accepted at the CoRL 2026 Workshop on Modeling Uncertainty in Robotic World Models (UncertainWM). 20 pages, including 12 pages of supplementary material
★ SCRUB: Contact-Based Surface-Frame Estimation for Force-Motion Control with an Armless Quadruped
Wiping, sanding, and inspection robots press a sliding tool on a surface with controlled force. Controlling that force needs the surface normal, which is unknown on real-world, unmapped surfaces. Surface-normal estimators can help estimate the surface normal, but they either need a friction model or assume that the instantaneous tool velocity lies along the surface; however, walking violates this assumption. We propose SCRUB, a contact-based surface-frame estimator, which lets an armless quadruped slide a body-mounted tool along near-vertical surfaces in the absence of known, axis-aligned, or vision-predicted normals. SCRUB's predictive controller presses and slides the tool using a Kalman-filter estimate of the normal from two cues: 1) A force cue measures the estimate's tilt across the motion. If friction directly opposes the motion, the cue is zero exactly when that tilt is zero, for any friction coefficient. However, if the cue's sliding direction is wrong, friction appears as a false tilt. 2) A velocity cue measures tilt along the motion. Walking and pressing move the tool in and out. SCRUB therefore subtracts the commanded press rate from the tool velocity, averages the result over each gait cycle, and treats this mean as lying along the surface. On Unitree Go1 hardware at nominal speed and load, SCRUB completed every wall traverse and column orbit. No orbit completed with either cue removed, without gait averaging, or with a fixed normal.
comment: 8 pages, 6 figures, 4 tables
★ Interactive World Simulation with Cosmos for Physical AI
We present a shared framework that adapts Cosmos world foundation models for real-time interactive world simulation in Physical AI. By streaming visual observations in response to actions and scene controls, the framework enables users and embodied agents to interact with evolving virtual environments. It supports both predicting the visual consequences of actions and generating observations from physical state maintained by an external simulator. We demonstrate this approach across camera navigation and robot manipulation, controllable video transfer, and multi-view autonomous vehicle (AV) simulation. These examples provide a shared foundation for developing interactive virtual environments for Physical AI training and evaluation. Our code and model checkpoints are available at https://github.com/nvidia/cosmos and https://huggingface.co/collections/nvidia/cosmos3-examples. The project website is available at https://research.nvidia.com/labs/cosmos-lab/cosmos-simulation.
comment: Project website: https://research.nvidia.com/labs/cosmos-lab/cosmos-simulation
★ Dynamic Shape Control of Soft Trunk Robots
Controlling the shape profile of soft trunk robots extends their reach into complex regions of the workspace and enhances their use in practical applications. Recent advances in soft robot design have enabled soft trunk robots to achieve greater flexibility in conforming to complex shapes. Specifically, trunk robots whose geometries are designed as helicoids have been shown to enable extreme shape-changing capabilities. Their compliance makes them highly nonlinear, which in turn makes shape control difficult. On similar tendon-driven trunk robots, model-based controllers based on the theory of adiabatic spectral submanifolds (aSSMs) have achieved high accuracy in closed-loop end-effector tracking. We extend this approach by learning an aSSM model of the full body of a trimmed-helicoid soft trunk robot from data and use this in a model predictive control (MPC) scheme to demonstrate a range of shape control tasks on hardware. On closed-loop dynamic shape regulation, our method outperforms the available linear baselines by a factor of 10. In human-in-the-loop interactive demos, our method completes section-by-section shaping and balances a tennis ball for over 25 seconds, while the linear baselines fall short. A model-free reinforcement learning policy trained for 12 hours on hardware fails on the same balancing task. These results establish aSSM-based MPC as a practical route to full-body shape control on real soft hardware.
★ FOSCo-Net: Ego-centric Voxel Completion for Forest LiDAR Occlusions
Ground robots operating in temperate forests suffer from systematic LiDAR occlusions caused by tree trunks, creating persistent blind regions that degrade traversability estimation. In this paper, we propose FOSCo-Net, a self-supervised voxel completion framework that learns to reconstruct occluded geometry from a single scan using temporally accumulated observations as automatic supervision, and we evaluate cylindrical and Cartesian ego-centric grids with multiple decoder designs. FOSCo-Net consistently improves reconstruction quality and traversal safety, with a gated skip decoder achieving the most reliable performance and significantly reducing false obstacle predictions compared to baselines.
comment: 12 pages, 6 figures, accepted for presentation at ACIVS 2026 (23rd Advanced Concepts for Intelligent Vision Systems) Okinawa, Japan
★ Playing Soccer Like Humans Do: Learning Versatile and Agile Humanoid Soccer via Latent Motion Priors
Humanoid soccer requires a robot to seamlessly combine agile locomotion, ball manipulation, and impact-rich contact dynamics across a continuous, dynamic behavioral spectrum. Yet existing approaches either cover only a limited set of skills, or forego motion priors entirely and produce unnatural behaviors, leaving the combination of a unified skill set and human-like motion quality unaddressed. We present soccerbot, a hierarchical reinforcement learning framework that resolves both limitations through a structured latent motion prior distilled from large-scale human soccer motion capture data. By constraining policy learning within this dynamically plausible motion manifold, a single skill-conditioned policy naturally learns to perform moving, dribbling, trapping, and kicking in a unified formulation, while inherently producing human-like motion and fluid transitions during commanded skill switching. Augmented by robustness-oriented simulation design, soccerbot transfers zero-shot to the Unitree G1 humanoid robot, demonstrating agile, natural, and robust execution of the full soccer skill repertoire under real-world conditions.
★ H^2MARL: Hierarchical Heterogeneous Multi-agent Reinforcement Learning for Humanoid Robot Control
Coordinated whole-body control of humanoid robots remains challenging due to high-dimensional coupled dynamics and substantial physical heterogeneity across joint degrees of freedom (JDoFs). Existing multi-agent reinforcement learning methods either employ coarse agent partitions or directly apply generic heterogeneous-agent algorithms, while hierarchical controllers commonly optimize high-level planning and low-level execution independently, leading to uncoordinated motion. In this paper, we propose H^2MARL, a hierarchical heterogeneous multi-agent reinforcement learning framework with two key mechanisms. We first formulate humanoid whole-body control using a JDoF-level physical-heterogeneity-aware policy factorization, where each torque-actuated JDoF is conditioned on an explicit descriptor of its actuation, kinematic, and dynamic properties. Moreover, we develop a closed-loop policy-executor co-optimization mechanism that jointly adapts the high-level pose planner and the low-level joint executor using shared interaction trajectories, thereby reducing the distribution mismatch between planning and execution. Extensive experiments in MuJoCo and Isaac Gym demonstrate that H^2MARL consistently improves locomotion stability, gait coordination, and motion quality over homogeneous, coarse-grained, and independently optimized alternatives.
★ Robocasa-Mem: Benchmarking Environment and Process Memory in Long-Horizon Mobile Manipulation
In the real world, mobile manipulation robots must perform complex object interactions and make multi-step decisions, often relying on task-relevant information that is no longer observable. However, existing benchmarks provide limited evaluation of such memory-dependent decision-making in mobile manipulation. To address this gap, we introduce RoboCasa-Mem, a new benchmark comprising nine mobile manipulation tasks that require agents to use interaction history at key decision points. Specifically, the tasks are grouped into two categories of memory: One is environment memory, which captures spatial locations, object attributes, and object relationships. The other is process memory, which captures single-task progress and multi-task scheduling. Particularly, we introduce Behavior-Conditioned Evaluation. For tasks with multiple valid execution paths, instead of fixing a single target or trajectory, the evaluator derives later task constraints from the robot's earlier actions. This allows us to evaluate whether later behavior is consistent with the robot's own interaction history while reducing shortcuts based on fixed target assignments. We provide 1,800 successful demonstrations for policy training. We also develop a baseline on our benchmark, namely Hierarchical Memory-Augmented Policy(HMAP). Its high-level planner uses structured entity and process memories to generate subtasks. Its low-level VLA controller uses subgoal-conditioned queries to incorporate VGGT-Omega geometry into action generation. Together, RoboCasa-Mem and HMAP baseline support systematic evaluation of memory-dependent behavior and the development of memory-aware mobile manipulation policies.
comment: 17 pages, 4 figures
★ Beyond Infraction Scores: Failure-Boundary Profiles for Closed-Loop Evaluation of Driving and Robot Navigation Agents
Closed-loop driving benchmarks in CARLA rank agents by a driving score that multiplies route completion by an infraction penalty. This score does not distinguish an agent that stops in front of an obstacle from one that pushes past it. In a corridor closed by two parked cars, for example, one agent that drives through with contact averages 52.2 on the blocked widths, while the agents that stop average between 37.8 and 40.6. We propose failure-boundary profiles, an evaluation in which each scenario family varies one measured factor, such as corridor width or the time gap in oncoming traffic, and each variant is certified from measurement as one where the agent should proceed or one where it should stop. Along this axis we estimate two curves per agent, commitment (the probability of attempting the passage) and competence (the probability of passing without contact), and locate where each crosses one half, with intervals that account for the simulator's run-to-run variance. The certified labels also give a summary metric, decision accuracy, which credits clean passes on passable variants and stops on blocked ones. We evaluate seven driving agents on seven CARLA families. The rule-based expert reaches a decision accuracy of 0.74 and TFv6, the highest-scoring learned agent, 0.60. Agents with similar driving scores fail in different ways, and agents that reliably stop at a blocked corridor still rarely wait for a gap in oncoming traffic. We also apply the method, unchanged, to a warehouse robot in Isaac Sim. The navigation policy we test stops at blocked gaps in only one of three families, and two fine-tuned versions of it pass more gaps cleanly but no longer stop at all.
comment: 16 pages, 12 figures, 3 tables. Code: https://github.com/speedym/failure-boundary-profiles. Data: https://doi.org/10.57967/hf/11195
★ $A^3$-VLA: Automatic Perception-Guided Attention Alignment for Robot Manipulation
Vision-Language-Action Models (VLAs) enable robots to perform manipulation tasks by leveraging Vision-Language Models (VLMs) to process visual observations and language instructions. However, the action prediction objective itself lacks fine-grained task-specific perceptual supervision, making it prone to relying on task-irrelevant visual shortcuts, leading attention away from target objects and key interaction areas, thereby limiting its operational performance and generalization ability. To address this, we propose Automatic Perception-Guided Attention Alignment for Robot Manipulation ($A^3$-VLA), a framework that derives task-specific visual guidance from the manipulation process itself. Specifically, $A^3$-VLA first generates heuristic candidate regions for target objects and the robot from interaction-induced visual changes, and refines them with a visual segmentation model offline. These regions serve as explicit perceptual guidance through two complementary objectives: an attention allocation loss that encourages sufficient focus on task-critical regions when language or robot state tokens query the visual input, and a contrastive learning loss that aligns the representations of candidate target regions with language instructions while separating them from task-irrelevant regions. Extensive experiments on simulated and real-world manipulation tasks demonstrate that $A^3$-VLA consistently improves multiple VLA backbones, with an average success rate gain of 6.5% - 25.2% across benchmarks.
★ TwinBridge: Digital Twin-Driven Policy and Reward Transfer for Real-World Reinforcement Learning in Laboratory Robotics
Online reinforcement learning offers an effective approach to precise laboratory manipulation, yet costly real-world demonstrations and limited access to dense process rewards hinder its practical application. We present TwinBridge, a digital twin-driven framework that supplies policy initialization and dense visual rewards for real-world online learning. We construct digital twins by combining 3D Gaussian splatting with physics simulation. Here, simulator states provide task events and geometric progress under physical prerequisites and ordering constraints, yielding rewards for policy pretraining with simulated demonstrations and reinforcement learning. We distill this supervision into a visual reward model. During real-world interaction, its predictions are combined with execution history to compute dense rewards for human-in-the-loop policy refinement. Across five real-world laboratory tasks, TwinBridge achieves a final mean autonomous success rate of 98% with at most 25 minutes of real-world interaction per task. Experiments show that dense rewards improve interaction efficiency over sparse rewards, and simulation-based policy initialization further accelerates learning.
comment: 8 pages, 6 figures
★ UniLoop: Unified Loop Closure Detection from Retrieval to Geometric Verification and Relative Pose Estimation
Loop closure detection conventionally cascades separate models for global retrieval, geometric verification, and relative pose estimation, leading to redundant computation and disconnected visual representations. We propose UniLoop, a unified framework that reuses the hierarchical representations of the pretrained geometric foundation model throughout this pipeline. A shared image encoder produces features for compact global descriptors; the same features are passed to the cross-view decoder for pairwise verification; and the original geometric predictions provide relative camera constraints for pose-graph optimization. Experiments on place recognition, loop verification, and relative pose estimation show that UniLoop improves loop-closure robustness and accuracy while reducing redundant backbone computation relative to modular pipelines. UniLoop achieves the best Recall@10 on MSLS (96.6) and Pitts30k (99.3) among the compared methods, while its verification branch reaches 95.63 AP and 96.29 ROC-AUC, surpassing the existing geometric verification baselines.
★ Vortex-Induced Torque in Grasping Soft Objects: Influence of Geometry and Pneumatic Parameters
Vortex grippers are commonly integrated into automated systems for handling objects with complex geometries in packaging, sorting, and assembling. However, their interaction with compliant objects remains difficult to predict, particularly with respect to the vortex-induced torque transmitted during grasping. This study investigates vortex-induced torque during the grasping of compliant objects and quantifies the coupled effects of supply pressure, tangential nozzle diameter, surface geometry, and curvature. The results show that increasing the tangential nozzle diameter primarily enhances the sensitivity of the generated torque to supply pressure rather than producing a uniformly higher torque. Surface geometry also strongly influences the rotational interaction, with dome-concave surfaces generally producing substantially greater torque than corresponding dome-convex surfaces, whereas the difference between cylindrical concave and convex surfaces is comparatively small. The strongest and most repeatable response was observed for a dome-concave surface exhibiting favorable object-gripper geometric matching. Among the evaluated machine-learning regression methods, Gaussian Process Regression provided the best overall predictive performance under repeated cross-validation, demonstrating that vortex-induced torque can be estimated from combined pneumatic and geometric parameters. These findings provide a basis for predicting and utilizing vortex-induced torque in the dexterous manipulation of soft objects.
comment: Accepted for the SII
★ Scale Anchoring via Implicit Semantics and Sequence-Consistent Loop Closure for VGGT-Based SLAM ICRA 2027
Feed-forward geometric foundation models enable dense monocular reconstruction without requiring camera calibration, but independently reconstructed submaps can accumulate scale drift, and even a valid revisit can yield an inconsistent cross-submap measurement. We extend VGGT-SLAM 2.0 with fixed-root scale anchoring and short-sequence-assisted direct loop connections. SALAD selects an image pair between the initial submap and each eligible new submap; joint VGGT inference then uses learned implicit semantic-geometric cues to supply a scale-only anchor while preserving the original overlap chain. For loop closure, neighboring image pairs support a consistency check and translation-scale alignment. Each accepted candidate contributes one robust Sim(3) factor between existing center-frame nodes, without an auxiliary loop submap. A 17-sequence comparison lowers mean Sim(3) ATE on nine TUM sequences from 0.1018 to 0.0451 m relative to the VGGT-SLAM 2.0 baseline. Anchoring improves all seven no-loop module ablations. On three Clio scenes, a fixed-loop comparison favors the direct connection pipeline over the tested auxiliary-submap pipeline. A separate candidate audit exposes additional rejections of visually overlapping pairs by the short-sequence check. These comparisons distinguish the gains from long-range scale anchoring and loop connection choices from the limitations of candidate verification.
comment: 8 pages, 6 figures, 4 tables. Submitted to IEEE ICRA 2027
★ Corridor-Based Monitoring and Emergency Braking Decision for Automated Vehicle Safety
This paper addresses emergency-braking decisions for a redundant safety channel in automated vehicles. The channel is responsible for identifying hazards from sensor data and determining whether braking is necessary, given the constrained computational resources of embedded hardware. We present an emergency-braking approach that uses a predicted motion corridor to filter \ac{lidar} measurements and then decide whether the ego vehicle must brake. \ac{pmcm} constructs a curvature-aware boundary from the planned path and vehicle geometry so that the monitored region contains the full swept body on both straight and curved paths. \ac{gdeta} then decides whether the ego vehicle must command an emergency brake for objects inside this boundary, scoring geometric occupancy and temporal urgency independently. Hard gates permit braking when either source is conclusive; soft-OR arbitration combines partial evidence when neither gate is active. Consecutive-scan confirmation reduces sensitivity to transient measurement noise. Corridor coverage is evaluated in controlled collision simulations and recorded campus driving. Braking decisions are evaluated in Monte Carlo simulations of lead-vehicle braking and transverse-crossing encounters.
comment: Submitted to a Springer Nature journal for peer review as an extended version of a paper presented at VEHITS, for consideration in the Special Issue
★ Riemannian Optimization-based Whole-body Obstacle Avoidance for Mobile Continuum Robots in Cluttered Environments
Inspired by the coordination of global motion and local shape adaptation in snakes, elephant trunks, and octopus arms, we propose a unified planning and control framework for mobile continuum robots in cluttered and confined environments. Existing approaches often address high-level path planning and local motion control separately, limiting coordinated base-arm motion, kinematic consistency, and collision avoidance. Our framework integrates long-horizon trajectory planning with real-time geometric refinement under kinematic and environmental constraints. The global layer combines A* search with minimum-jerk planning to generate smooth reference trajectories. The local layer uses a geometry-consistent inverse kinematics solver and Riemannian optimization to coordinate mobile-base motion and continuum deformation while avoiding obstacles. We evaluate the framework through simulated pipe-traversal tasks with multiple baseline planners and real-world experiments involving navigation among dense obstacles and manipulation in confined spaces. Results demonstrate improved motion consistency, feasibility, and adaptability, supporting coordinated whole-body motion of mobile continuum robots in constrained environments.
★ CoSafe: Cooperative Runtime Safety Assurance for Mixed-Traffic Autonomous Driving ICSE 2027
With the rapid development of autonomous driving systems (ADSes), real-world roads are evolving into mixed-traffic environments where autonomous vehicles (AVs) and other traffic participants (e.g., human-driven vehicles (HDVs), cyclists, and pedestrians) coexist and interact with each other. However, current ADSes still incur various safety violations in mixed traffic scenarios, raising a strong demand for runtime safety assurance frameworks that can continuously monitor hazards and prevent potential violations. Existing ego-centric assurance approaches are insufficient for mixed traffic, where violations often arise from locally reasonable yet jointly incompatible decisions among multiple vehicles and cannot be reliably resolved through independent takeover. To bridge this gap, we propose CoSafe, a modular and ADS-agnostic cooperative runtime safety assurance framework for mixed-traffic autonomous driving to extend runtime safety assurance from ego-centric takeover to scene-level cooperative intervention. CoSafe leverages vehicle-to-everything (V2X) communication to enable roadside units (RSUs) to monitor scene-level hazards and jointly mitigate hazards involving multiple vehicles. We integrate CoSafe with seven state-of-the-art ADSes, and our evaluation on the InterDrive benchmark demonstrates that CoSafe improves the driving score of AVs by 70.03% on average, reduces safety violations by 87.80%, and accurately identifies unsafe AVs with an F3 score of 0.930, while satisfying real-time constraints.
comment: Accepted by the 49th IEEE/ACM International Conference on Software Engineering (ICSE 2027). 13 pages
★ Beyond Terminal Constraints: Reliable Robot Control with Learned Safe Sets
Ensuring safety in advanced robot control methods such as Model Predictive Control (MPC) and Reinforcement Learning remains an open challenge. A common approach relies on control-invariant sets, yet their exact computation for nonlinear systems is generally intractable, and numerical approximations compromise theoretical guarantees. In this work, we show that safety does not need to rely exclusively on control invariance. We introduce novel classes of safe sets, each broader than classical control-invariant sets, and analyze corresponding MPC schemes that guarantee recursive feasibility by constraining different nodes of the prediction horizon rather than only the terminal state. Furthermore, when the safe set is an inner approximation of a suitable control-invariant set, our methods can ensure safety through a safe-abort strategy, even in the absence of recursive feasibility. We validate our approaches in extensive simulations on a 6-joint robot manipulator, showing fewer constraint violations than state-of-the-art methods while maintaining comparable tracking performance and computational efficiency. Hardware experiments further demonstrate the practicality of our schemes for real-world robotic applications.
★ Can Astra Solve RoboMME without Breaking the Bank?
Robotic manipulation often requires remembering information that is no longer visible, such as an object's location before occlusion or the order of a demonstrated action sequence. A large vision-language planner can read this history, but calling it before every action chunk is slow and costly. We present a three-tier system for the RoboMME benchmark. GPT-6 Astra (System 2) selects a grounded subtask from stored visual history, a $π_{0.5}$ vision-language-action policy (System 1) executes it, and a fine-tuned Qwen3-VL-4B monitor (System 1.5) checks after each action chunk whether the subtask is finished. The instruction is kept until the monitor reports completion or a task-specific event occurs, while the policy keeps acting on new observations. On RoboMME, the system succeeds in 79.13% of episodes with an average of 3.63 Astra calls and 80.84 seconds per episode. This is 34.62 points above the best published non-oracle baseline and 4.95 points below GroundSG+Oracle, which gives the low-level policy ground-truth subgoals.
comment: Accepted at the Workshop on Memory for Robot Foundation Models and the Workshop on Agentic Robotics, CoRL 2026. Project page: https://bingaochen.github.io/Astra-on-RoboMME/
★ A Layered View of LLM Support in Underwater Robotics
This extended abstract discusses a specific and, in our experience, practically relevant role for large language models (LLMs) in underwater robotics. Our main premise is that, in underwater fielding missions, the immediate difficulty is often not the synthesis of new low-level control laws, but the operation of complex robotic systems under limited communications, partial observability, and expensive field time. In particular, a significant fraction of the effort during sea trials is devoted to fault diagnosis, telemetry interpretation, mission adjustment, and coordination across heterogeneous assets and operators. We argue that LLMs may be useful at this higher operational level, especially when grounded in structured vehicle state, alarms, logs, mission context, and existing planning or validation tools. At the same time, current limitations in grounding, consistency, and causal reasoning make these systems lack the predictability required to guarantee direct control, especially in real-world critical applications, although we do not exclude a future evolution in this direction. This viewpoint suggests a layered architecture in which conventional autonomy remains responsible for control, perception, estimation, and validated mission execution, whereas LLM-based modules serve as an additional support layer for debugging, operator interaction, constrained replanning support, orchestration, and post-mission analysis.
comment: Presented at the I-RIM 3D 2026 - Workshop n.30 How would you like intelligence in your robots? Interdisciplinary perspectives on intelligent machines
★ 3D Pose Estimation in Projection Space via Pixel-level Visible Light Communication
Projector-based augmented reality enables natural interaction between users and devices by overlaying digital information onto the physical world through projected imagery. A key technical requirement of such systems is accurate positional alignment between the projector and the devices. Pixel-level Visible Light Communication (PVLC) achieves this alignment without external sensing infrastructure by embedding invisible spatial codes into each projected pixel through high-speed brightness modulation that preserves image quality. However, the information that PVLC transmits is inherently planar. PVLC has therefore been limited to devices moving on a flat surface and cannot estimate the 3D pose of devices in three-dimensional space, such as aerial robots. To extend PVLC-based pose estimation to three dimensions, we formulate it as a Perspective-n-Point (PnP) problem. The 2D pixel positions that four coplanar photosensors on the device receive via PVLC serve as the image points, and their known 3D layout provides the object points. Experiments show that the proposed system estimates 3D pose with a latency of approximately 8~ms and a 3D position RMSE of 19.2~mm after Kalman filtering. These results confirm the first realization of real-time 3D pose estimation coordinated with high-quality projected image content without external sensing infrastructure.
comment: submitted to IEEE Access
★ ReactWorld: Structuring Interaction Responses for Embodied Video Generation
Embodied video world models predict scene changes caused by robot actions, yet generated object responses may remain inconsistent with task intent and contact dynamics. Critical responses in embodied scenes often concentrate in local regions and specific time intervals. Inspired by chain-of-thought reasoning, we introduce ReactWorld, an embodied video generation model guided by structured interaction anticipation. In the generation backbone's feature space, an interaction response field predictor anticipates how the interaction will unfold, explicitly describing participant roles, future motion response intensity, and local motion changes. A response feedback adapter forms increments to object features from the difference between the object branch's outputs with predicted and zeroed local dynamics inputs. It also converts predicted displacements into positional biases for temporal attention. These two mechanisms guide object feature updates and information retrieval across frames, respectively. Through interaction anticipation and generative feedback, ReactWorld achieves an average score of 77.51 on RBench, outperforming PhysisForcing by 3.29 points with improvements in all nine categories. On EZS-Bench, it achieves a Domain score of 86.98 and an average score of 81.87.
comment: 10 pages, 7 figures. Project page: https://metaengine-world.github.io/ReactWorld/
★ Online Adaptation to Latent Physical Properties for Learning from Demonstration
Manipulating visually identical objects with different physical properties requires adapting actions based on information acquired through interaction. We address the problem of learning from demonstrations how to adapt manipulation actions, such as regrasping and changing contact locations, based on physical-property information acquired during interaction. To this end, we propose a learning-from-demonstration method that learns latent representations of physical properties needed to predict object responses and conditions a policy on online estimates of these representations. A dynamics model and object-specific latent representations are jointly learned from demonstrations that include behaviors adapting to the objects' physical properties, and the learned representations are then used to condition an imitation policy. During execution, the dynamics model and policy weights remain fixed, while the latent representation is updated based on the discrepancy between predicted and actual observations, allowing interaction with the object to inform action generation. Real-world and simulation experiments with objects whose centers of mass differ suggest that online adaptation through latent representations of physical properties can improve task success rates. However, in simulation, success rates on center-of-mass locations absent from the demonstrations remained around 30%, indicating limited adaptation to unseen physical properties. Project page: https://isri-aist.github.io/latent-dynamics-project-page/
comment: Accepted for the 2027 IEEE/SICE International Symposium on System Integration (SII 2027)
★ Fluidic Body: Swarm Robots as a Fluid Extension of the Hand
Swarm robots have attracted growing interest as a physical interface for human--computer interaction, since a swarm can reshape its formation to fit the objects and surroundings it works with. For a human operator, however, adapting the swarm's behavior remains difficult. Most existing methods require the user to specify where the swarm should go, and they offer no intuitive way to change the collective's behavior as each task demands. We present Fluidic Body, a system that lets a person operate a swarm of fluid-imitating robots as an extension of the hand. The system places a virtual fluid between the hand and the swarm, and the user adapts the swarm's behavior to each task by changing the fluid's properties through simple hand movements. When the user opens the hand, the swarm becomes soft enough to deform and flow around obstacles. Closing the hand turns it into a hard, cohesive body that can push objects. We evaluated the controller's performance and its operability in two simulation experiments and one demonstration with physical robots. First, our controller handled both obstacle avoidance and object transportation better than an existing method that behaves the same way in every task. Second, in a VR user study, 17 participants completed the combined avoidance-and-transportation task faster and more reliably than with that method. They also reported a lower workload and a stronger sense of agency. Finally, this experience carried over to physical robots in a real-world demonstration (88 participants). These results suggest that treating a swarm as a fluid, a material people already understand, allows a robot swarm to work as a deformable extension of the human body.
comment: accepted to ACM ISS 2026 (ISS track of PACMHCI)
★ The Effectiveness of Sliding Mode Control for Robust Vision Stabilization under the Plant-Binding Condition
Boundary-layer Sliding Mode Control (SMC) has been applied to vision-based stabilization, but identifying its activation condition against simpler causal baselines (LPF, PID) has been elusive. The operational condition is plant binding: the disturbance velocity approaches or exceeds the actuator slew limit, so the bounded per-step actuation budget binds. We validate this in two independent vision domains: handheld translation stabilization (12 1080p clips; LPF, PID, SMC, L1) and angular re-framing against a virtual slew-rate-limited pan-tilt-zoom (PTZ) plant. Outside binding, leave-one-out cross-validation makes SMC and LPF on translation statistically indistinguishable, and PID beats SMC on the angular 99th-percentile pointing offset by 5.85 degrees (95% CI clear of zero); inside binding -- reached only by sweeping the simulated slew limit, as both captured datasets land outside it -- the translation gap stays a tie and SMC beats PID on the angular worst case (Cohen's d up to +0.83). The PTZ plant's 100 degrees/s default sits on the cross-over, which explains a small-margin PID result previously reported in the angular setting. A complementary ablation shows raising the SMC reaching gain k costs mean tracking outside a narrow window.
★ Learning Object-Centric Motion Priors from Human for Robotic Dexterous Manipulation AAAI 2026
Manipulating diverse objects with multi-fingered dexterous hands is challenging due to the high dimensionality and complex dynamics. Human-Object Interaction (HOI) datasets provide rich knowledge about task information and embodied interactions. Instead of solely imitating the human demonstrations, our method learns to holistically predict future hand-object states by leveraging these datasets. The predicted future states of the object can serve as a general-purpose reward term for reinforcement learning, reducing reliance on task-specific reward engineering and enhancing generalization across tasks. We conduct extensive experiments on three manipulation tasks in simulation and the real world. Our approach outperforms existing SOTA methods in both success rate and generalizability on novel objects. Furthermore, we validate the cross-embodiment compatibility of our methods by successfully deploying the skills on different robot hands.
comment: Accepted as an oral paper at AAAI 2026 (Website: https://ojs.aaai.org/index.php/AAAI/article/view/38892)
★ Robots in the City: Persona-Based Perceptions of Urban Robots
Pedestrians increasingly encounter robots in urban public spaces without interacting with them, but how robots change the perceived safety of a street remains unclear. We examine how robot presence and type affect perceived safety across urban contexts. We generate 18 images of Seoul streets that vary in time of day, road class, and robot condition (no robot, a delivery robot, or a humanoid). Two multimodal large language models (MLLMs), Qwen3.8-27B and Gemma4-26B, each conditioned on 500 synthetic personas, choose the safer image in each pair, and 25 human respondents complete the same task. Both models prefer streets without robots and favor the delivery robot over the humanoid. Robots are chosen more often at night and less often by older personas, whereas road class has smaller effects. In the model rationales, the delivery robot appears as a physical obstacle, whereas the humanoid appears as either a threat or a source of protection. Like humans, both models chose robots more often at night, but chose the humanoid over the delivery robot far less often, and Gemma showed a night effect for the humanoid about three times that of humans. These results show that the effect of a robot on perceived safety depends on when it appears, and that persona-conditioned MLLMs capture only part of human judgments.
comment: 10 pages, 7 figures
★ Reflection-Robust Stereo Visual-Inertial Odometry via Kalman-Filtered Ground-Height Estimation
Visual-inertial odometry relies on visual features that correspond to 3D points with consistent projections across sequential frames. In reflective indoor-floor environments, however, specular floor patterns can be detected and triangulated as virtual features below the physical ground plane, producing unreliable feature associations and inconsistent visual residuals. This paper proposes a lightweight reflection-aware front-end for feature-based stereo VIO. Rather than suppressing all floor-region features, the proposed method estimates the ground height in real time and rejects only feature candidates whose inferred 3D positions are below the estimated ground height. A floor-region mask restricts the validation process to reflection-prone regions, while stereo height uncertainty and pose uncertainty are used to construct frame-level ground-height measurements. A scalar Kalman filter with Normalized Innovation Squared (NIS) gating then provides a stable ground-height estimate, and an uncertainty-aware safety margin is used for selective below-ground virtual feature rejection. Experiments on real reflective-floor indoor sequences show that the proposed front-end generally improves trajectory accuracy in reflection-dominant environments while maintaining accurate ground-height estimates and recovering from actual ground-height changes.
comment: 6 pages, 5 figures, 3 tables. Presented at the 26th International Conference on Control, Automation, and Systems (ICCAS 2026), Sapporo, Japan. Included in the conference proceedings
★ LumiTac: Compact, High-Rate Six-Axis Optical Tactile Sensing for Human-Robot Interaction
Integrating six-axis tactile feedback into robotic interfaces requires compact sensors with accurate load measurement and rapid online reconstruction. This paper presents LumiTac, a low-profile, camera-free optoelectronic tactile sensor combining spatially distributed illumination with spectral encoding. Five photoemitters and five RGBW photoreceivers are mounted on opposing circuit boards coupled through translucent silicone. Load-induced interlayer motion modulates the internal light field under compression, shear, bending, and torsion. The sensing head measures 12 mm in diameter and 4.75 mm in thickness, weighs less than 2 g, and has a component cost of approximately USD 7. A compact multilayer perceptron reconstructs three force and three moment components from 20 optical channels. Independent testing yielded coefficients of determination (R-squared) of 0.9657-0.9941, with mean absolute errors of 0.054-0.190 N for forces and 0.304-1.419 mN m for moments. Online prediction achieved an average update rate of 303.57 Hz and a median software processing latency of 0.722 ms. Experiments involving combined loading, light weights, and rapid tapping evaluated load tracking, sensitivity to small forces, and transient response, respectively. Robotic demonstrations further validated its applicability to tactile-guided manipulation and physical human-robot interaction. This study paves the way for compact, high-rate optical tactile sensing in robotic manipulation and human-robot interaction.
★ Diversity-Aware Experience Replay via Statistical Leverage Scores
Off-policy deep reinforcement learning trains on random samples from a replay buffer of past experience. In physical robot settings, that buffer is dominated by routine: thousands of near-identical transitions from normal operation, while the rare events that define the edges of the task are rarely sampled. We propose scoring every stored transition by its statistical leverage, which measures how unusual it is relative to the buffer's own data, computed from the stored numbers alone, with no labels, rewards, or value network. We draw each mini-batch from a mixture of this score and uniform sampling. First, the natural way to build the score along the directions where the data varies most turns out to be blind when actions are stored as one-hot indicators, because indicator columns always win the variance ranking; we instead select the feature columns with the heaviest tails (largest kurtosis) where small far-away populations are revealed. Second, the importance weights that prioritized replay uses to keep gradients unbiased cancel the method exactly: with full correction, training matches uniform sampling, so we drop the correction and deliberately optimize the rebalanced objective. Third, when the rare situations of a log are relational, visible only in how a stretch of behavior relates to its goal, not in any single stored coordinate-per-transition scores are blind; applying the same tail rule to short trajectory windows described relative to their goal restores the signal, and on a three-joint arm that must push an object around an obstacle it raises the rare-task success from 0.49 to 0.69 against a labeled-oracle ceiling of 0.74.
★ FWIK: Multi-Waypoint IK Selection via Lookahead Graph Search
An ordered sequence of end-effector poses can give a redundant robot arm multiple valid joint configurations at each waypoint. Inverse kinematics (IK) computes these configurations. Choosing a configuration only for the next waypoint can leave the arm poorly placed for later waypoints. This may increase planned motion time or cause a later planning failure. This paper presents Future-Window Inverse Kinematics (FWIK), an online method that selects IK solutions by considering a small window of future waypoints. FWIK generates several IK candidates, connects them in a layered joint-space graph, and ranks candidate paths with a simple motion-time estimate. A learned model removes paths whose transitions are likely to cause self-collision. A standard MoveIt 2 planner then checks the next-waypoint IK goals from the remaining paths in ranked order, and the first goal that yields a valid trajectory is selected for execution. We evaluate FWIK in simulation on two 7-DoF manipulators, the Franka Emika Panda and the KUKA LBR iiwa. Each method-planner setting contains five repetitions of 100 sequences with 50 waypoints. Across Pilz, RRT-Connect, CHOMP, and STOMP, FWIK completes 96.8-100% of sequences and gives the shortest planned trajectory duration in all eight robot-planner settings. Its mean duration is 12.8-13.1% lower than STAMPEDE, 22.5-25.6% lower than IKLink, and 27.5-37.2% lower than retry-based TRAC-IK. The results show that a small future window can improve sequence completion and reduce planned motion time while leaving final trajectory validation to the motion planner.
★ Does RRT* Work on Uneven Terrain Meshes?
The optimal rapidly exploring random tree (RRT*) is proven asymptotically optimal on open subsets of Euclidean space with straight-line local planning. We ask whether it works on a triangle-mesh terrain, a two-dimensional surface in R^3. First, the original RRT* cannot run on a mesh: the surface has zero volume, so uniform sampling never hits it, no clearance ball fits in it, and chords across a fold leave it. Second, the surface reformulation we call Mesh RRT*, with its naive edge, the chord costed by its length, solved every run on heightfields with slopes up to 75 degrees but minimises the wrong objective: its cost, the straight-line length of a polyline whose vertices lie on the surface while its segments may leave it by a tolerance tau, is the length of no route on the surface and lies strictly below the geodesic length on any curved mesh. In numeric simulation and after 30,000 iterations the reported cost is on average up to 2.7% below the exact geodesic, a cost no path attains, and the returned route on average 0.1-0.5% longer than it. Rewiring keeps the biased edges, the bias decays only linearly with the step on polyhedral surfaces, and shrinking tau trades bias for connectivity. Third, replacing each chord by a path on the surface above it, built from projected points joined across the mesh edges they straddle and costed by its own length, makes every edge cost the exact length of a feasible path; with no other change the exact length of the returned routes is on average within 0.07-0.16% of the geodesic at every roughness tested, two to 3.4 times closer than with chords on the rougher terrains. The planner is in MeshNav3D 0.2.0 (release pending).
comment: 8 pages, 1 figure, 8 tables. Submitted to IEEE Robotics and Automation Letters
★ Time-to-Goal: Learned Temporal Metrics for Long-Horizon Planning with JEPA World Models
Joint-Embedding Predictive Architectures (JEPAs) support planning by comparing predicted latent states with an encoded goal. For this comparison to guide temporal progress, states closer in time to the goal should receive lower costs. However, training representations for prediction and distribution regularization does not directly enforce this ordering. We examine this mismatch in flat LeWorldModel (LeWM) by analyzing how its Euclidean goal cost changes with temporal separation. Our analysis identifies conditions under which states at different long offsets receive similar average costs, weakening their temporal distinction. To learn the missing ordering explicitly, we introduce Time-to-Goal (T2G), a lightweight metric trained on latent triples from offline trajectories. A margin ranking loss encourages smaller distances for temporally nearer states, requiring no rewards or policy labels. The learned metric then replaces the Euclidean planning cost while the JEPA encoder and predictor remain frozen. Diagnostics on AntMaze and FourRoom show that T2G preserves stronger temporal contrast and improves near--far ranking at long offsets. Across four navigation and manipulation benchmarks, T2G improves the average planning success rate from 32.7\% (flat LeWM) to 63.0\% and approaches or even surpasses the performance of the hierarchical H-JEPA baselines. These results show that learning temporal ordering for the goal cost can substantially improve latent planning even with a flat world model.
comment: 10 pages, 6 figures
★ RACE: Calibrated Cumulative Scoring for Rollout-Free Robot Policy Checkpoint Selection
Assessing the deployment performance of learned robot policies during training is challenging, as directly measuring task success requires costly real-world trials for each candidate checkpoint. Averaged validation loss offers a rollout-free alternative, but its checkpoint rankings can be inconsistent with deployment performance. We analyze two limitations of this criterion: it penalizes action deviations without considering whether the policy can recover from them, and gives later reference predictions equal weight even when earlier deviations may lead execution away from the reference. We propose RACE, a novel rollout-free metric for ranking checkpoints using a fixed set of held-out successful trajectories. RACE calibrates action deviations against state-conditional thresholds derived from action differences at similar states in these trajectories. We aggregate these scores through a running product, so that a substantial deviation reduces the contribution of subsequent reference predictions. Averaging the resulting cumulative scores over time and across trajectories yields the checkpoint score. In simulation, RACE outperforms baselines across validation sources, while several baselines degrade sharply when the validation set changes. On real robots, its rankings align with deployment success across the evaluated policy backbones and tasks of varying difficulty. Perturbation analyses further show that RACE distinguishes deviations with markedly different effects on task success, even when baselines respond similarly.
comment: Project website: https://changyeon.site/race
★ WayPOP: A Panoramic Open-Set Panoptic Tracking Benchmark
Holistic perception for autonomous agents requires reliably segmenting and associating scene elements across time and camera views, including objects outside a fixed semantic taxonomy. Existing open-set panoptic benchmarks focus on individual images or views, while panoramic tracking benchmarks assume closed-set taxonomies. Consequently, these benchmarks do not evaluate whether unknown objects can be consistently segmented and tracked across synchronized multi-camera sequences. We introduce WayPOP, a panoramic open-set panoptic tracking benchmark built on real Waymo PVPS sequences augmented with diverse out-of-taxonomy 3D assets from Objaverse. Inserted objects are placed at fixed locations in the 3D scene and rendered using calibrated vehicle-mounted cameras as the ego vehicle and surrounding traffic move, providing temporally and cross-view consistent panoptic annotations and auxiliary LiDAR range observations. WayPOP contains approximately 670 curated assets instantiated as 980 unique unknown-object tracks, yielding 53,452 unknown-object mask annotations across 21,300 camera images. Our evaluation protocol defines a hierarchy of tasks from view-independent open-set panoptic segmentation to full panoramic open-set panoptic tracking, using known/unknown PQ and (w)STQ metrics. We benchmark representative open-set and open-vocabulary segmentation methods with temporal and cross-camera association. Our results reveal substantial limitations of current methods in truly open-set scenarios, highlighting robust unknown-object localization and identity association across time and views as major open challenges.
★ ZipDrift: Generating Multimodal Short-Horizon MPC Costs with One-Step Drifting Models
Short-horizon model predictive control (MPC) is fast but myopic. ZipMPC compensates by learning a context-dependent correction of the short-horizon cost that imitates a long-horizon expert. We observe that the optimal correction is often multimodal: when a slower opponent can be passed on either side, good corrections of opposite sign coexist, and the expert's choice between them hinges on tiny cost differences. Deterministic regression then averages modes or inherits inconsistent expert tie-breaking. We propose ZipDrift, which trains a one-step drifting generator over MPC cost corrections, tilted toward low closed-loop cost and learned only from cost evaluations of candidate corrections, without differentiating through the optimiser. A few generated candidates are ranked by a learned critic, and the model-based MPC then enforces dynamics and constraints. In a kinematic overtaking benchmark derived from ZipMPC, 53% of contexts are bimodal and the manual short-horizon MPC crashes in 28.5% of test episodes. With an eight-dimensional stagewise correction, ZipDrift is collision-free in all 600 test episodes and matches the best evaluated candidate, whereas an idealised ZipMPC-style regressor crashes in 4.0% (16.0% with inconsistent expert labels) and critic-ranked random candidates with the same budget in 2.0%.
comment: 6 pages, 2 figures, 4 tables. Preliminary simulation study
★ Closed-Loop Adaptive Sampling on a Generative Ocean Model
Adaptive sampling can substantially improve the value of sparse ocean observations by directing mobile robots toward measurements that are most informative for an evolving environment. Recent learned generative ocean models can produce high-resolution forecasts and ensembles at substantially lower computational cost than traditional numerical models, creating an opportunity to use rich predictive forecasts directly in closed-loop robotic sampling. However, these models do not natively provide the explicit uncertainty representations assumed by classical informative-planning methods, and their forecasts often evolve in learned latent spaces rather than directly in physical ocean variables. We develop a multi-robot closed-loop adaptive sampling framework that uses a single learned generative model for forecasting, uncertainty estimation, planning, and measurement assimilation. The model forecasts an ensemble in latent space, decodes it to physical ocean states, and uses ensemble statistics to construct A- or D-optimal sampling objectives. We select sampling locations subject to robot reachability under predicted currents. The collected measurements are assimilated back into the latent state using observation-conditioned diffusion sampling. Across 120 test scenarios A- and D-optimal sampling generally outperforms static, random, and systematic-coverage baselines while adding little planning overhead. Our experiments uncover several scenarios in which model uncertainty diverges from forecast error, motivating future work in uncertainty quantification.
★ Adaptive Hybrid Force-Vision Control with EMG-Derived References for Deformable Bottle Manipulation
Robotic automation of laboratory tasks has become a growing interest to remove researchers from hazardous environments and repetitive procedures. However, even equipment as simple as squeeze bottles poses a challenging manipulation problem due to its deformability. This paper presents an adaptive hybrid vision-force controller for robotic grippers to effectively manipulate laboratory squeeze bottles to dispense liquid. This approach integrates force and vision feedback according to the interaction state, enabling effective squeezing while accounting for bottle deformation and recovery. The proposed controller is validated by comparing its performance with a pure force controller on 12 combinations of conditions consisting of 2 target dispense volumes, 2 plastic squeeze bottles, and 1 to 3 squeezes used to dispense. Human demonstrations of these operating conditions were collected using electromyography (EMG) sensing mapped to a desired grip force, which was then used as the force reference for the controllers to squeeze the bottles. Results demonstrated that the proposed adaptive hybrid vision-force controller achieved more accurate liquid dispensing with a mean absolute error (MAE) of 6.54 mL compared to a MAE of 15.92 mL from the pure force controller, while also replicating the human action recorded from the EMG signal more closely.
★ Spatial-UMI-Dream: Scaling World Action Models with Metric Geometry for UMI Manipulation
The Universal Manipulation Interface (UMI) enables scalable demonstration collection independently of a specific robot. However, its moving, close-range gripper cameras entangle camera and object motion without a stable external view, leaving the metric geometry needed for precise manipulation implicit in RGB. We introduce Spatial-UMI-Dream, a 5B spatial World Action Model (WAM) pretrained on 605 hours of RGB-D UMI data through two-stage training -- world-only adaptation followed by joint world-action learning -- to jointly predict future RGB, fisheye-aligned metric depth, and actions. Stereo-derived depth makes scene scale and spatial relationships explicit within a unified dual-gripper RGB-D representation. Furthermore, evaluating UMI policies requires testing whether UMI's robot-independent demonstration collection translates into policy generalization across robot embodiments, alongside robustness to environmental changes. We introduce RoboTwin-UMI, a twenty-task simulation benchmark built on RoboTwin with UMI wrist-mounted fisheye observations, enabling controlled, repeatable evaluation of UMI policies under environment and embodiment changes separately and jointly without additional fine-tuning. On RoboTwin-UMI, Spatial-UMI-Dream achieves 79.10% in-domain success and 72.15% under combined embodiment and environment shifts, exceeding the strongest evaluated baseline in each setting by 4.65 and 10.30 percentage points, respectively. It also exhibits the smallest success-rate drop under embodiment transfer among the evaluated policies. In a matched pretraining ablation, RGB-only pretraining leads to a 5.75-percentage-point decrease in in-domain success compared with RGB-D pretraining. Across six real-world tasks, Spatial-UMI-Dream achieves 85.0% average success, exceeding the strongest baseline by 24.2 percentage points. Project page: https://weiduoyuan.com/spatial-umi-dream/
comment: Project page: https://weiduoyuan.com/spatial-umi-dream/
★ Distributed Geometric Optimization and Control of Rigid Differential-Drive Formations from Relative-Position Measurements
Distance based formation control fixes the shape of a robot team but leaves its position and orientation unconstrained. We regulate all three for differential drive robots that have only one absolute position anchor, absolute headings, body frame relative position measurements, and communication with their neighbors. We derive a two layer estimation control scheme directly from optimization. The first half of the result is a set of graph-proximal estimators that reconstruct the centroid, the anchor vector, and the centered coordinates. Exact proximal updates reach consensus in finitely many steps, and a finite number of primal dual iterations leaves a bounded reconstruction error. The controller combines proximal centroid and orientation steps with distance gradient shape regulation, and a finite rotation removes the orientation drift caused by the shape step. A discrete Lyapunov analysis gives local exponential convergence once the estimators have converged. We validate the method in ROS 2 Gazebo and in experiments with five physical robots.
comment: 11 pages, 6 figures
★ Curvature by Design: A Support Structure for Continuum Robots
Constant- and piecewise-constant-curvature models are widely used to simplify the kinematics of continuum robots, although the assumed curvature profiles are not necessarily inherent mechanical properties of the robot and may therefore be violated during operation. Rather than assuming a prescribed curvature profile in the kinematic model and estimating deviations from it, this paper presents a multi-segment scissor-based support structure that mechanically constrains a continuum robot to exhibit prescribed constant or spatially varying curvature profiles. The design uses Taylor approximations to enable an overconstrained multi-joint linkage to predictably distribute global bending among the small-angle movements of multiple flexures. The geometric coupling is realized using compliant flexure joints, enabling the support structure to be manufactured as a single piece. The structure is integrated with a tendon-actuated continuum robot with a flexible backbone. Experiments show that the structure makes the robot resistant to deformations other than the prescribed curvature, significantly enhancing shape precision compared to classical flexible-backbone designs. This constrains the robot from making S-shapes when external loads are applied in the opposite direction of the bending. By mechanically constraining the admissible deformation, the proposed structure reduces the number of independent shape variables that must be estimated.
♻ ★ When Robots Outweigh Objects: How Inertia Mismatch Shapes Robot Policy Learning
Conventional robot arms can be extremely heavy compared with the everyday objects they manipulate, creating a significant inertia mismatch at contact. Even a small accidental bump from a heavy arm can knock a light object out of place. Modern learning methods have nevertheless demonstrated a wide range of successful manipulation behaviors on these platforms, suggesting that even extreme inertia mismatch can be overcome with sufficient data. However, it remains unclear whether high inertia imposes a hidden cost on learning, potentially worsening scaling with data or lowering the performance ceiling of modern robot learning methods. We investigate this question through controlled simulation experiments that vary robot and object masses. We find that reinforcement learning succeeds across a broad range of mass combinations, while behavior cloning learns more reliable policies when a lighter robot manipulates a heavier object. Heavier robots require more demonstrations to attain the same performance as lighter robots. Finally, because new hardware is rarely an option, we explore ways to lower an existing arm's inertia at contact with existing hardware, either by choosing a lower-inertia workspace before collecting data or by steering the demonstrator toward lower-inertia configurations during the task. In both cases, the lower-inertia demonstrations are imitated more reliably. Project website: https://younghyopark.me/inertia-mismatch/
comment: 9 pages, 17 figures. Project website: https://younghyopark.me/inertia-mismatch/
♻ ★ Provably Safe, Yet Scalable Reinforcement Learning
Safe reinforcement learning (RL) aims to learn policies that optimize rewards while satisfying constraints. Predominant approaches rely on soft-constrained policy optimization, which has achieved empirical success but does not provide formal safety guarantees for the learned policy. In contrast, methods with strict guarantees typically rely on explicit certificate functions, whose construction requires the direct synthesis and verification of control-invariant sets, a process that scales poorly with state dimension and often yields overly conservative behavior. In this paper, we present the Provably Safe, yet Scalable RL (PS2-RL) framework, a novel two-phase architecture for learning provably safe policies in a scalable manner, designed to overcome the key bottlenecks of prior methods. Rather than explicitly computing invariant sets, PS2-RL leverages a learned backup policy to forward-integrate the system dynamics, generating an implicit control-invariant set online. In the first phase, the backup policy is trained with the safe-arrival value function to achieve rapid, safe recovery to the certified base set for invariant-set construction. In the second phase, an RL policy is trained end-to-end through a differentiable projection layer that strictly enforces the safety guarantees induced by the learned backup policy. By maximizing the volume of the implicit control-invariant set in the first phase, the resulting PS2 policy from the second phase is performant and scalable, while maintaining provable safety. Crucially, PS2-RL imposes no restrictions on the underlying RL algorithm and can be plugged into any existing training pipeline. We establish theoretical guarantees for the proposed framework and evaluate it on robotic control tasks with state dimensions up to 10, a regime in which prior provably safe RL methods struggle or become impractical.
♻ ★ Reality and Perceptual Gaps in Expressive Swarm Robotics
Moving robot swarms from simulation into the physical world can be difficult because small discrepancies in hardware, timing and communication can cascade across interacting robots and alter group behaviour. This challenge is known as the reality gap. When swarms operate in human environments, a second challenge emerges: a perceptual gap between what the swarm physically does and what human onlookers actually perceive. This study examines the same decentralised coordination code deployed across three robotic swarms-tabletop, aerial and human-scale mobile manipulators-during live artistic performances. We find that while the underlying algorithms executed reliably on all three platforms, physical embodiment drastically altered visible motion-distorting execution timescales by more than a factor of three and reshaping trajectory geometry. More importantly, audience surveys revealed that human interpretation does not mirror technical success: how observers attribute meaning (perceived purpose) and how they react emotionally (comfort versus threat) depend heavily on physical form. Meaning attribution and emotional responses did not vary together. Our results demonstrate that successfully transferring swarm software to physical hardware is insufficient for real-world deployment. Addressing the reality gap does not guarantee that the resulting collective motion will be readily interpreted by human observers. Swarms destined for human environments must be engineered not only for decentralised stability but also for the expressive physical dynamics through which humans interpret collective motion.
♻ ★ Reverse to Advance: Teleoperation-Cost Effective Hard Policy Learning from Reversed Easy Tasks NeurIPS 2026
High-quality teleoperation datasets are costly to collect, particularly for hard tasks. We observe that many tasks exhibit directional asymmetry: completing the forward hard task is difficult, whereas reversing it by relaxing or disrupting the environment is comparatively easy. This suggests that reversed easy-task trajectories can serve as a scalable supervision signal for the hard task, reducing the cost of manual demonstration collection. However, reversed data can be noisy, and directly training on it may yield suboptimal policies. To enable largely automated acquisition and effective use of reversed data, we propose a teleoperation-cost effective framework for hard policy learning via temporal reversal of easy tasks, consisting of three key components: a closed-loop data collection pipeline that alternates between hard-task and easy-task policies to autonomously reset the environment and generate diverse trajectories; a hierarchical data refinement pipeline that temporally inverts easy-task rollouts and filters low-quality motion using kinematic priors and a critic-guided advantage filter; and an iterative policy learning method that trains the hard-task policy using both initial reversed easy-task demonstrations and the filtered reversed data in a continuous online learning loop. By combining automated collection, hierarchical refinement, and iterative learning, our method enables scalable, reliable training of complex, high-precision manipulation tasks. Across two simulated benchmarks and real-robot experiments, we demonstrate that our method improves hard-task success rates with higher data efficiency and more stable training compared to reversal-based and reinforcement-learning baselines, without requiring extensive hard-task teleoperation.
comment: 22 pages, 17 figures. Accepted at NeurIPS 2026
♻ ★ SimForcing: Distilling Simulation Motion Priors into Real-Domain Robot World Models
Action-conditioned robot world models must respond precisely to robot trajectories while preserving realistic visual dynamics, yet learning both from heterogeneous robot videos remains challenging. Simulation offers structured motion supervision, but appearance differences hinder direct transfer, and inaccurate simulation predictions can misguide real-video generation. We present SimForcing, a simulation-guided framework that uses simulation both as a source of transferable motion knowledge and as a controllable reference for prediction. First, we transfer motion knowledge from a simulation teacher through latent-motion distillation, aligning temporal changes in latent space to internalize motion priors while mitigating the influence of appearance differences. Second, we introduce multi-block simulation conditioning with condition dropout to exploit predicted simulation trajectories without relying excessively on their accuracy. Our simulation-conditioning classifier-free guidance scheme unifies these two ideas by balancing predictions based on internalized motion knowledge with those additionally guided by simulation latents. The jointly trained student generates both simulation conditions and real-domain videos, requiring no additional world model at inference. On Bridge, SimForcing achieves the best PSNR, SSIM, LPIPS, and FVD among the compared methods without external embodied pretraining. Evaluation on InternData-A1 further supports its applicability across robot datasets. Moreover, using our trained world model to initialize a vision-language-action model improves LIBERO success, suggesting its utility for downstream policy learning. https://github.com/Wang-Xiaodong1899/SimForcing
comment: Code: https://github.com/Wang-Xiaodong1899/SimForcing
♻ ★ Predictive Spatio-Temporal Scene Graphs for Semi-Static Scenes
We have seen tremendous recent progress in our ability to build "spatio-semantic" representations that enable robots to perform complex reasoning across geometry and semantics. However, the vast majority of these methods lack any ability to perform reasoning across time. This is a desirable property in situations where a robot repeatedly observes an environment where instances may change in between observations, but in a structured way. Consider as an example a home environment where the location of a mug typically moves from the cupboard to a countertop to the sink and then back to the cupboard on a daily basis. We should be able to learn this cyclic behavior and use it to predict the state of the mug in the future. In this work, we propose a method that is able to perform this type of tempo-spatio-semantic reasoning. Underpinning the method is a filter, Perpetua*, that performs Bayesian reasoning on the states of the environment that are observed over time. This filter is integrated within a 3D scene graph structure that we call PredictiveGraphs, where nodes represent objects and edges function as Perpetua* filters encoding spatio-semantic relationships. We validate the method in both simulation and real-world dynamic navigation tasks, where our real-world experiments consist of an environment that is undergoing semi-static changes at a bi-hourly frequency over a period of three weeks. In both settings, we demonstrate that our method outperforms baselines in predicting future environment states, even in the presence of distributional shifts.
comment: Accepted for publication in IEEE Robotics and Automation Letters (RA-L). Webpage at https://montrealrobotics.ca/predictive-graphs/
♻ ★ RoboTryOn: Synthesizing Dexterous Robot Demonstrations from Human Videos via Virtual Try-On
Robot foundation models have shown remarkable performance in robot control, yet adapting them to a new task still relies heavily on teleoperated demonstrations. Egocentric human videos offer a scalable substitute, but overlaying rendered robots on human videos or retargeting human poses into robot actions leaves significant visual or action gaps for high-DoF robot embodiments. To address this limitation, we propose RoboTryOn, which treats human-to-robot translation as virtual try-on: try-on keeps the person and swaps the garment, RoboTryOn keeps the demonstrated motion and swaps the human for the robot. Specifically, RoboTryOn converts a human video into a video of the target robot specified by a reference image, while preserving the original task, objects, and background. To maintain visual consistency, we translate selected keyframes autoregressively and use them to guide video synthesis with motion conditioned on the human hand skeleton. Additionally, we use an inverse dynamics model (IDM) that incorporates depth features to infer action labels from synthesized videos for policy learning. Our experiments across multiple humanoid embodiments show that policies trained with our synthetic data improve the average success rate on novel tasks, including challenging bimanual manipulation, from 2% to 60%, without any real robot demonstration of those tasks. Moreover, training with our synthetic data matches the performance of policies trained solely on teleoperated demonstrations at approximately a third of the collection cost, demonstrating the cost efficiency of our approach.
comment: 29 pages, 15 figures, 13 tables. Project page: https://ryan-seungyong-lee.github.io/RoboTryOn
♻ ★ SUPER - Sensitivity-based Uncertainty-aware Performance and Risk Assessment in Visual Inertial Odometry
This paper presents SUPER (Sensitivity-based Uncertainty-aware PErformance and Risk assessment), a bounded controller that adapts visual measurement information in visual-inertial odometry at runtime. Building on multi-view landmark dispersion from stereo observations, SUPER projects track consistency into pixel space and combines it with the reprojection residual to gate a temporally smoothed, bounded information update. Because the controller operates inside nondeterministic optimizers, we additionally propose an action-verification protocol based on factor-level logs, inactive-instrumented controls, and same-linearization replay. We evaluate SUPER in Ceres and g2o backends on EuRoC, TUM-VI, UMA-VI, and TartanAir under controlled visual corruptions, with a frozen policy and prospective confirmation. SUPER is computationally efficient and selectively active, but does not establish a general accuracy improvement: all 13 split-level confidence intervals cross zero, and effects are heterogeneous across corruption families. The action-verification protocol reliably distinguishes controller-induced state changes from solver variability.
comment: 6 pages, 4 figures, 5 tables. v2: substantially revised. The Schur-complement risk indicator of v1 is replaced by a closed-loop controller adapting visual information from multi-view track dispersion, with a new action-verification protocol. ISO risk mapping and the prediction/stop-policy results of v1 are removed. v2 claims no general accuracy improvement
♻ ★ FlapSim: A Composable MuJoCo Simulation Platform and Reduced-Order Aerodynamic Model Hierarchy for Flapping-Wing Robotics
Flapping-wing simulation need to reconcile high-frequency wingbeat aerodynamics with the slower dynamics relevant to flight control and reinforcement learning. We present FlapSim, a modular open-source MuJoCo simulation platform that combines robot descriptions, wing geometry, and composable aerodynamic models in a deterministic simulation pipeline. Across $48$ held-out scenarios spanning frequency, gimbal, wind, frequency-step, gust, and cross-robot flat-plate regimes, we compare five reduced-order configurations operating at $1\,\text{ms}$ with a $50\,μ\text{s}$ quasi-steady blade-element reference. The built-in hover cycle-averaged baseline achieves a $67.51\times$ CPU speedup but only $0.6593$ mean wrench fidelity. Combining multirate $21$-substep clock synchronization, three-dimensional linear/quadratic inflow and gust coupling, analytical blade-element geometry scaling, and phase-locked $1f\text{--}6f$ harmonics raises mean fidelity to $0.9646$ at a $65.19\times$ speedup in a five-block model. A streamlined four-block model without explicit step-transient lag states achieves $0.9662$ fidelity at a $67.98\times$ speedup. Controlled ablations show that stroke harmonics are particularly important during frequency transitions, improving held-out frequency-step fidelity by $+0.3176$. Transfer-function analysis explains this effect: fixed-window cycle averaging retains harmonic contributions when the wingbeat frequency changes, so within-cycle reconstruction also improves the filtered wrench. These results characterize the accuracy--computation tradeoff of the aerodynamic hierarchy and identify timing, inflow, geometry, and harmonic representation as central considerations for reduced-order flapping-wing simulation. The source code are available at: https://mingshanhe.github.io/FlapSim/\href{https://mingshanhe.github.io/FlapSim/}.
♻ ★ Learning Soft Robotic Dynamics with Active Exploration
Soft robots offer adaptability and safety in unstructured environments, yet their compliant, high-dimensional, and nonlinear dynamics make modeling for control difficult. Existing data-driven approaches often fail to generalize, constrained by narrowly focused task demonstrations or inefficient random exploration. We introduce an uncertainty-aware active exploration framework that autonomously learns task-agnostic and generalizable dynamics models of soft robotic systems. We evaluate this framework on three simulated platforms---a continuum arm, an articulated fish in fluid, and a musculoskeletal leg---and on a real-world pneumatically actuated continuum arm. Compared with random exploration and task-specific baseline, our method achieves higher modeling accuracy by actively collecting informative data. This improved model quality yields higher returns in zero-shot control on unseen tasks and robust performance under sensing noise, actuation delays, and material nonlinearities. These results demonstrate that uncertainty-driven active exploration enables data-efficient dynamics learning across soft robotic morphologies, supporting autonomous and adaptable control for compliant robots.
♻ ★ META-Robot: Self-Cognition and Capability Scaling in Agentic Robots
LLM-based robotic agent systems can generate programs for robot control, yet their generalization to new tasks often depends on elaborate scaffolding. Expanding their skill repertoires alone offers limited guidance on how to assess new capabilities or identify what remains beyond their reach. We introduce META-Robot, a framework that couples self cognition with capability scaling by expanding the criteria through which an agent evaluates and develops its skills. At its core, action-level predicates make execution outcomes explicit and available for LLM reasoning. The agent uses these assessments to identify its own limitations, extend relevant evaluation criteria, and guide the structured expansion of its skills. Proprioceptive feedback further grounds this process in the robot's physical state, enriching the evidence available for self-cognition. Experiments in simulation and on real robots demonstrate stronger overall task performance, generalization to new tasks, continued expansion of skills and evaluation criteria, and effective use of non-visual proprioceptive signals.
♻ ★ Learning to Live: A Step Toward Lifelong Embodied Intelligence
Embodied agents are usually organized around individual tasks, yet the physical world keeps changing between tasks and during execution. We introduce LIFE (Lifelong Intelligence from Experience), a framework in which a robot observes continuously, remembers selectively, and turns experience into new capabilities. RoboBeats, a family of lightweight streaming gates, tracks conditions set by the agent and discovers new events, deciding when an observation warrants a response or a memory write. Selected observations enter Plastic Memory, a flat store of images and structured records that keeps the source evidence, so new fields and indexes can be added when later questions need them. When existing tools fall short, Explore-Distill-Mount (EDM) turns solutions found during problem solving into validated, reusable tools and skills. Under matched protocols, RoboBeats substantially improves condition tracking over its base models and, at the same writing budget, retains more useful evidence for memory question answering than uniform sampling. A case study of limited scale, replaying 24 hours of continuous video recorded by a real robot, illustrates how the integrated system builds memory from a day of experience and uses it to answer everyday questions. LIFE takes a step toward robots that become more familiar with their surroundings and more capable through experience: learning not only to complete tasks, but to live in the world.
♻ ★ Multi-Agent Coordination via Support-Preserving Distillation NeurIPS 2026
Offline MARL increasingly relies on generative policies to model multimodal joint behavior, typically by distilling a centralized teacher into decentralized one-step actors under the CTDE. We identify a failure mode at the teacher training stage: standard flow-based teachers pair noise with replay targets independently, so nearby noise samples can be routed toward conflicting coordination modes. The teacher then produces samples between valid modes, and because the distillation loss regresses each local actor onto the conditional mean of the teacher's output given local input, this error is not absorbed but propagated to the student. To remove this teacher-side artifact, we propose Mode-Support Semi-Discrete Optimal Transport (MoSDOT), which summarizes multimodal replay into a finite mode support with prescribed capacities and uses conditional semi-discrete optimal transport to assign each noise sample to a single mode before teacher training. We additionally study a shared-randomness variant that uses a shared noise component at execution to expose the residual gap intrinsic to strict-product execution. On controlled diagnostics and offline MARL benchmarks, MoSDOT improves endpoint quality and routing consistency, particularly on datasets exhibiting multimodal joint behavior.
comment: NeurIPS 2026, Project page: https://alex6095.github.io/mosdot/
♻ ★ MARLIN: Multi-Agent Reinforcement Learning Guided by Language-Based Inter-Robot Negotiation
Multi-agent reinforcement learning is a key method for training multi-robot systems. Through rewarding or punishing robots over a series of episodes according to their performance, they can be trained and then deployed in the real world. However, poorly trained policies can lead to unsafe behaviour during early training stages. We introduce Multi-Agent Reinforcement Learning guided by language-based Inter-robot Negotiation (MARLIN), a hybrid framework in which large language models provide high-level planning before the reinforcement learning policy has learned effective behaviours. Robots use language models to negotiate actions and generate plans that guide policy learning. The system dynamically switches between reinforcement learning and language-model-based negotiation during training, enabling more effective exploration. MARLIN is evaluated using both simulated and physical robots with local and remote language models. Results show that, compared to standard multi-agent reinforcement learning, the hybrid approach achieves higher performance in early training with limited impact on the final performance. The code is available at https://github.com/SooratiLab/MARLIN.
comment: 6 pages, 7 figures, 1 table. Accepted at ROBIO2026
♻ ★ MMUSV-Sim: A Maritime Simulation Platform for Multi-USV Cooperative Perception
Perception from a single unmanned surface vehicle (USV) is limited by sensing range, field of view, and occlusions caused by islands, port infrastructure, and nearby vessels. Multi-USV cooperative perception can alleviate these limitations by integrating complementary observations from distributed viewpoints. However, progress is constrained by the high cost of coordinated sea trials, the scarcity of multi-USV datasets, and the lack of integrated maritime simulation and evaluation tools. We present MMUSV-Sim, a perception-oriented maritime simulation and data-generation platform built on Unreal Engine 5 and Project AirSim. It supports configurable maritime scenarios, synchronized multimodal acquisition across multiple USVs, and consistent 3D annotations in a shared world frame. Using MMUSV-Sim, we construct MarU2U-200m, a synchronized multi-USV RGB--LiDAR dataset with 341 sequences and 14,385 scene frames. We establish a cooperative bird's-eye-view (BEV) vessel detection benchmark with LiDAR and RGB--LiDAR inputs. Experimental results show that Early Fusion improves LiDAR detection AP@0.5 from 64.26 to 81.46 compared with the single-USV baseline. RGB features further improve intermediate fusion, increasing AP@0.5 by up to 4.55 percentage points. LiDAR and RGB transfer experiments also show the value of the generated data as a pretraining source for real maritime detection. The project are available at https://github.com/lizaowo/MMUSV-Sim
♻ ★ Myoelectric Control for Upper-Limb Exoskeletons: A Decoder-Shaper-Controller Framework, Design Guidelines, and Future Directions
Myoelectric Control Systems (MCSs) enable voluntary control of wearable robots through muscle activity. However, intention decoding, assistance shaping, and low-level actuator control are often not explicitly distinguished, making their contributions difficult to evaluate and compare. This review introduces a functional decoder-shaper-controller framework for continuous myoelectric control of upper-limb exoskeletons. The decoder estimates an intention-related variable from surface electromyography (sEMG), the shaper maps it to a control reference according to an assistance policy, and the controller realizes that reference through actuator control. This decomposition separates intention estimation from assistance-policy design and enables comparison of heterogeneous architectures. Representative approaches are critically examined in terms of module design, calibration, and evaluation. The analysis highlights that the shaper, despite determining the magnitude and shape of assistance, lacks established design and tuning methodologies and is often embedded within the decoder or controller. Key open issues include inconsistent reporting of assistance levels, limited use of interaction-aware stability and passivity tools, and insufficient validation across degrees of freedom, users, and tasks. Building on these findings, this review provides design guidelines for systematic module-wise tuning and identifies future research priorities to bridge the gap between accurate intention decoding and safe, effective, and generalizable assistance under real-world operating conditions.
comment: 19 pages, 3 figures. Review paper
♻ ★ FLAG: Flow Policy MaxEnt-RL by Latent Augmented Guidance NeurIPS 2026
Maximum entropy reinforcement learning (MaxEnt-RL) enables robust exploration, yet practical implementations often restrict policies to simple Gaussians. While recent approaches incorporate expressive generative policies via importance-weighted supervised learning, they are prone to importance weight collapse, which limits their scalability in high-dimensional action spaces. Our key insight is to mitigate this limitation by localizing the sampling region, avoiding the weight degeneracy induced by importance sampling over the entire action space. To instantiate this insight, we introduce \textbf{FLAG} (\textbf{F}low policy with \textbf{L}atent-\textbf{A}ugmented \textbf{G}uidance). FLAG augments the state space with a flow latent variable and optimizes a provably consistent proxy MaxEnt-RL objective. We empirically demonstrate that FLAG enables expressive policy optimization with limited importance samples and scales to state-based high-dimensional control tasks. Furthermore, FLAG achieves state-of-the-art performance among expressive-policy MaxEnt online RL methods on state-based high-dimensional control benchmarks.
comment: NeurIPS 2026, Project page: https://flag-rl.github.io/
♻ ★ Event-based SLAM Benchmark for High-Speed Maneuvers
Event-based cameras are bio-inspired sensors with pixels that independently and asynchronously respond to brightness changes at microsecond resolution, offering the potential to handle visual tasks in high-speed maneuvering scenarios. Existing event-based approaches, although successful in mitigating motion blur caused by high-speed maneuvers, suffer from many limitations. Some of them highlight a success of pose tracking for a fronto-parallel fast shaking camera closed to the structure, while others assume pure (optionally aggressive) three-degree-of-freedom rotations. The former requires persistent local map visibility within the field of view (FOV), whereas the latter fails to generalize to six-degree-of-freedom (6-DoF) motions where both linear and angular velocities may be large. Consequently, current successes do not fully demonstrate that event-based state estimation under arbitrary aggressive maneuvers is a fully solved problem. To quantitatively assess the extent to which the potential of event cameras has been unlocked, we conduct a thorough analysis of state-of-the-art (SOTA) event-based visual odometry (VO)/visual-inertial odometry (VIO) methods and report shortcomings in current public datasets. Furthermore, we introduce a benchmarking framework for event-based state estimation, called EvSLAM, characterized by sufficient variation in data collection platforms, diverse extreme lighting scenarios, and a wide scope of challenging motion patterns under a clear and rigorous definition of high-speed maneuvers for mobile robots, along with a novel evaluation metric designed to fairly assess the operational limits of event-based solutions. This framework benchmarks state-of-the-art methods, yielding insights into optimal architectures and persistent challenges.
♻ ★ Future Representation Distillation for Imagination-free World Action Models
World Action Models (WAMs) use the predictive knowledge of pretrained video diffusion models for robot control through future video prediction. However, iterative video denoising is costly, and inaccurate visual predictions can compromise action prediction. Our key idea is to make this predictive knowledge available through visual representations computed from the current observation, enabling action prediction without generating future videos. We introduce DreamFree, a WAM that repurposes a pretrained video diffusion model as a predictive visual encoder through representation distillation. The visual encoder learns to predict a frozen video diffusion model's representation using only the current observation. An action module is then trained to predict actions by extracting action-relevant information from the visual encoder. Across real-world and simulated tasks, DreamFree achieves higher success rates than state-of-the-art VLAs and WAMs, while performing real-time closed-loop control at 11Hz with a 14B video diffusion backbone. Experiments show that DreamFree can translate stronger video priors and additional video data into better robot control and enable learning new robot tasks from video demonstrations, including videos from a different embodiment, without action annotation for the new tasks.
comment: Project page: https://harryjo97.github.io/dreamfree/
♻ ★ A Graph-Based Conceptual Modeling Framework to Design for Human-Robot Coordination
Studies of human-robot interaction in dynamic and unstructured environments show that as more advanced robotic capabilities are deployed, the need for cooperative competencies to support collaboration with human problem-holders increases. Designing human-robot systems to meet these demands requires an explicit understanding of the work functions and constraints that shape the feasibility of alternative joint work strategies. Yet existing human-robot interaction frameworks either emphasize computational support for real-time execution or rely on static representations for design, offering limited support for reasoning about coordination dynamics during early-stage conceptual design. To address this gap, this article presents a novel computational framework for analyzing joint work strategies in human-robot systems by integrating techniques from functional modeling with graph-theoretic representations. The framework characterizes collective work in terms of the relationships among system functions and the physical and informational structure of the work environment, while explicitly capturing how coordination demands evolve over time. Its use in conceptual design is demonstrated through a case study in disaster robotics, which shows how the framework can be used to support early trade-space exploration of human-robot coordination strategies and to identify cooperative competencies that support flexible management of coordination overhead. These results show how the framework makes coordination demands and their temporal evolution explicit, supporting design-time reasoning about cooperative competency requirements and work demands prior to implementation.
comment: Under review at IEEE Transactions on Human-Machine Systems. 12 pages, 5 figures
♻ ★ AutoDex: An Automated Real-World System for Dexterous Grasping Data Collection
Scaling real-world dexterous-grasp data collection is limited by the sustained human effort required to operate the robot, verify grasp outcomes, and reset objects between trials. Simulation can generate grasp candidates at scale, but their success on hardware remains uncertain. We present AutoDex, a system that collects real-world dexterous-grasp data without continuous human operation. Given generated grasp candidates, AutoDex plans and executes grasps, labels their outcomes, and resets the object to continue collection across stable poses. Multi-view object tracking supports labeling under hand-object occlusion, while residual-torque collision monitoring and pose-directed reset enable the loop to operate without continuous human supervision. Using AutoDex, we collect over 3,500 grasp trials across 100 objects with Allegro and Inspire hands, recording both successes and failures together with synchronized visual observations and robot states. AutoDex reduces human effort by more than an order of magnitude, requiring an estimated 0.2 human-hours to collect 500 trajectories compared with 9.3 for teleoperation. As AutoDex collects trials, it builds a library of physically validated grasps for subsequent execution. In new object placements and obstacle configurations, retrieval from this library achieves 76% success, compared with 34% for candidates screened only in simulation. Code and data are available at https://snuvclab.github.io/AutoDex/.
comment: 9 pages + references, 6 figures. Accepted to CoRL 2026
♻ ★ KEPT: Conformal Keepouts Kept in Planning for Enclosed-Space Navigation in Degraded Sensing
Autonomous vehicles in enclosed spaces such as tunnels and mines rely on range maps that smoke, dust and fog degrade. Conformal prediction calibrates a keepout around the mapped walls that covers the true unsafe set at a chosen probability. In narrow passages this margin and the planner's buffer close the route, and existing remedies recover reach by relaxing the coverage level. This paper conditions the conformal margin on a label-free severity estimate per viewing direction and sweep, holding per-class coverage near its target as degradation varies. A planning and control layer enforces one braking-distance condition on the keepout at plan time and at every command, takes the measured tracking residual as its only margin and never alters the keepout. The keepout is validated on ground-robot smoke recordings and the closed loop on simulated quadrotor flights in a foggy tunnel. The proposed system substantially reduces entries into the calibrated keepout and reaches numerically more goals than the deployment pattern in both calibration regimes. The per-class keepout holds its coverage near the target on held-out recordings.
comment: 8 pages, 5 figures, 3 tables
♻ ★ A Mode-Unified, Continuous, and Biomimetic Control Strategy of a Robotic Lower Limb Prosthesis
Existing biomimetic lower-limb prosthesis controllers are commonly designed for discrete locomotion modes, requiring separate parameterizations for level walking, ramps, and stairs. We present a mode-unified prosthesis control framework that represents knee and ankle biomechanics continuously as functions of terrain slope and gait phase across those locomotion conditions. Locally optimized ascent and descent polynomial models were unified using radial basis function (RBF) interpolation, preserving the accuracy of the partitioned models while outperforming a global polynomial representation. The resulting biomechanics were transformed into continuous and slope-dependent impedance parameters and implemented within a single finite-state-machine (FSM) controller using bilateral sensing for gait-state estimation. Experiments with 10 individuals with transfemoral amputation demonstrated terrain-adaptive locomotion from level walking through ramps and stairs up to 35 degrees, including 13 degree ramp and 15 degree stair conditions absent from the underlying able-bodied (AB) dataset. Experimental results reproduced the principal slope-dependent biomechanical trends predicted by the unified model, with high agreement for knee-related measures (r-square = 0.82 - 0.99) and ankle kinematics (r-square= 0.78 - 0.96). These findings demonstrated that unified biomechanical parameterization enables a prosthesis control architecture across diverse terrain conditions while preserving key features of AB biomechanics without requiring independently tuned impedance parameter sets for each locomotion mode. To facilitate reproducibility and further development, the biomechanics modeling and impedance parameterization code is publicly available at https://github.com/kim2576.
comment: This work has been submitted to a research journal for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible
♻ ★ Bridging the Morphology Gap: Adapting VLA Models to Dexterous Manipulation via Intent-Conditioned Fine-Tuning
Pretrained vision-language-action (VLA) models offer semantic and spatial priors for robot manipulation, but their parallel-gripper interfaces leave multi-finger contact coordination implicit. Adapting these models requires both timely grasp transitions and embodiment-specific hand trajectories. We introduce InDex, an intent-conditioned framework that connects these two requirements. A normalized aperture signal, derived from retargeted demonstrations, supervises a scalar grasp intent while preserving full hand-action supervision. Stage 1 learns synchronized end-effector-intent chunks; Stage 2 conditions a diffusion hand decoder on these predictions, visual-language context, and proprioception. Sequential optimization establishes the upstream interface before learning dexterous control with low-rate co-adaptation. Across four simulated tasks, InDex with $π_{0.5}$ achieves 85.8% mean task success, improving the native policy by 35.5 percentage points, and reaches 69.0% on held-out objects with similar geometry without fine-tuning or recalibration. Improvements also hold across three VLA backbones and separately trained real-world policies, where mean success increases from 71.0% to 83.0%. Component ablations and timing interventions support the value of explicit grasp-intent conditioning and expressive hand generation. These results identify arm-hand coordination as an actionable bottleneck in adapting parallel-gripper priors to dexterous manipulation.
♻ ★ Test-Time Improvement of VLA Policies via Selected Diffusion Noise for Spurious-Robust Action Smoothing
Diffusion-based Vision-Language-Action (VLA) policies have demonstrated strong generalization in robot manipulation, but their deployment can remain unreliable due to spurious visual reliance and stochastic action generation, leading to grounding failures and kinematically unstable behaviors. We introduce Selected Diffusion Noise (SDN), a training-free method that treats diffusion noise as a controllable degree of freedom for improving test-time reliability. SDN samples multiple candidate action chunks and applies two-stage selection: it first filters candidates based on task-relevant visual grounding using a counterfactual negative set, then selects the kinematically most stable chunk based on JerkRMS. Experiments on three VLA policies across both simulation and real-world environments show improvements of 8-10% and 18.33%, respectively, demonstrating SDN's effectiveness for test-time improvement.
♻ ★ GhostTac: Manipulating Tactile Sensors without Physical Contact CCS 2026
Tactile sensors are integral components of modern robotic systems, enabling robots to perceive and interact with the physical environment through tactile feedback. Despite their importance, the physical-layer security of tactile sensors has received little attention in prior work. In this paper, we present GhostTac, to the best of our knowledge, the first contactless attack that manipulates tactile sensing via electromagnetic interference (EMI). We identify that EMI exploits the nonlinear rectification and limited bandwidth amplification effects, allowing carefully crafted EMI signals to be converted into a persistent DC offset that bypasses on-board filtering and induces stable measurement deviations. Building on this mechanism, GhostTac enables fine-grained and controllable manipulation of sensor outputs by reshaping the spatial distribution and manipulating the magnitude at the targeted location. Such interference can induce unintended and harmful robot behaviors, such as causing a domestic robot to exert excessive force, resulting in physical damage or human injury. We evaluate GhostTac on 10 sensor modules and 2 dexterous hands, covering 15 tactile sensors of different types, and demonstrate consistent attack effectiveness across all tested devices. We further present three case studies on tactile grasping, slip detection, and material classification to illustrate practical impacts in real robotic tasks. We envision that our findings shed light on a new physical attack vector against tactile sensing in robotic systems.
comment: Accepted at ACM CCS 2026
♻ ★ Metamaterial tactile displays with programmable virtual pixels
Tactile displays present information by supplying localized mechanical signals across their surface, but conventional approaches often require an actuator to be positioned at every location where feedback is provided, limiting resolution and increasing complexity. Wave-mediated tactile displays exploit wave transmission to generate virtual pixels using far fewer actuators, but are constrained in resolution by the intrinsic dispersion characteristics of the display medium. Here, we show how this constraint can be overcome by applying metamaterial physics to engineer wave transmission in a rigid plate using an optimized lattice of resonators. The resulting system exhibits a slow-wave dispersion branch that reduces the wavelengths available for constructing virtual pixels, thereby enhancing display resolution by an order of magnitude. We demonstrate a metamaterial tactile display driven by eight peripheral actuators that is capable of generating localized virtual tactile pixels at any surface location with centimeter-scale resolution. The location, timing, and waveform of each pixel can be specified computationally, and the output bandwidth, 75-400 Hz, spans a substantial portion of the tactile frequency range. In four perceptual experiments, participants localized, explored, and discriminated multi-pixel patterns comprising transient and sustained vibrations with accuracies ranging from 95.3% to 100%. This research provides metamaterial engineering methods that enable the design of tactile displays driven by few actuated degrees of freedom, with implications for many future application areas.
♻ ★ PDRVLA: A Vision-Language-Action Model with Physical Dynamics Representations
Vision--Language--Action (VLA) models generate robot actions from visual observations and language instructions, but their understanding of physical dynamics remains limited. Video world models can capture scene dynamics, yet generating future videos introduces high latency into the control loop. We introduce PDRVLA, whose lightweight Physical Dynamics Module extracts physical tokens from vision--language features to condition action generation without online video generation. The module learns through direct prediction of future RGB, depth, and optical-flow features, with auxiliary supervision through a frozen pretrained video world model. Training relies entirely on publicly available data, expanded through human-to-robot synthesis, subtask annotation, and embodied chain-of-thought (ECoT) annotation. Across four simulation benchmarks, PDRVLA achieves success rates of 99.3% on LIBERO, 89.9% on LIBERO-Plus, 93.6% on RoboTwin~2.0, and 81.5% on SimplerEnv. Ablations show improved future prediction with world-model supervision and evaluate how subtask conditioning, future object-pose supervision, and ECoT training affect action success.
♻ ★ EchoNav: Evidential BEV Completion from Historical Observations for Visual Navigation
Visual navigation in cluttered environments requires robots to recover previously observed structure and infer obstacles and free space that remain occluded. We present a framework that combines temporal aggregation and occupancy completion in an evidential bird's-eye-view (BEV) representation predicted from short RGB sequences. A decoder adapts frozen VGGT-Omega features and combines historical residual proposals using cellwise weights and a learned no-update option. The BEV predictor produces a local map in the latest ego frame without depth or camera-pose inputs or explicit geometric warping. The evidential model treats observed free space separately and predicts occupancy with associated confidence across the remaining supported region. We train the predictor on simulated RGB sequences paired with complete and visibility-masked occupancy annotations. Our experiments evaluate navigation in simulation and on a wheeled robot, with additional demonstrations on quadruped and humanoid robots. In simulated sequential two-goal navigation, our system achieves 86.3% success, outperforming the strongest evaluated baseline by 32.2 percentage points. Across 60 wheeled-robot trials in 20 scenes, it achieves 73.3% success, compared with 33.3% for the strongest evaluated baseline.
comment: 9 pages, 6 figures
♻ ★ GaP: A Graph-as-Policy Multi-Agent Self-Learning Harness For Variational Automation Tasks
For robots to work reliably in commercial and industrial applications, recent advances in agentic coding systems have potential to combine interpretable robot programming with the open-world adaptability of model-free policies. We focus on "Variational Automation" (VA), a class of tasks that have larger variations in object geometry and pose than fixed automation. Model-free policies often struggle to close the reliability gap for VA tasks, which must be executed persistently and reliably in commercial and industrial applications. Motivated by prior work on Task and Motion Planning (TAMP) and the Robot Operating System (ROS), we introduce Graph-as-Policy (GaP), a multi-agent coding harness that generates directed computation graphs with perception, planning, and control nodes from a Modular Open Robot Skill Library (MORSL). GaP generates an internal simulation environment to rehearse task instances with different graphs in parallel to iteratively refine the graph structure and parameters to improve success rates and throughput. Evaluation with 8 new open VA task benchmarks, 4 in simulation and 4 in real-world, suggests that GaP can achieve success rates that significantly outperform baselines. Details, code, and data: https://graph-robots.github.io/gap
comment: Conference on Robot Learning (CoRL) 2026, Spotlight
♻ ★ X-Fly: Learning to Reach and Stop in Full-Chunk Aerial Vision-Language Navigation
Aerial vision-language navigation (VLN) agents built on large vision-language models are expensive to query. Executing a complete predicted action chunk before replanning amortizes this cost, reducing model inference from 195 to 42 ms per executed waypoint for our policy. It also moves the states at which the policy is queried away from expert demonstrations, which supervise neither recovery from these states nor termination at the learner's own arrival. On OpenFly test-seen, no-stop failures account for 47.78% of full-chunk behavior cloning (BC) outcomes. We present X-Fly, a staged training framework for full-chunk aerial VLN. Recovery DAgger labels learner-visited decision states with smooth chunks that reconnect to future expert segments and with STOP at the first in-region state. Flow-GRPO then jointly refines the chunked flow controller and a Bernoulli STOP policy from same-task closed-loop outcomes, supplemented by outcome-verified stopping supervision. With eight controls per decision, X-Fly reaches 74.18% SR on test-seen and 51.73% on test-unseen, compared with 16.45% and 18.43% for full-chunk BC and 32.04% and 29.46% for DreamFly.
comment: 13 pages, 5 figures, 3 tables. Project page: https://juliusju572.github.io/x-fly/
♻ ★ Warp: Structured Output-Space Adaptation for Frozen Robot Policies
Robot policies that remain broadly competent after deployment can degrade when dynamics, contact, geometry, actuation, or sensing differ from source conditions. We study post-hoc adaptation through structured output-space adapters: compact transformations of samples from a frozen base policy. We introduce Warp, a context-conditioned monotone bijective spline that enables nonlinear sample-dependent reshaping while preserving per-dimension ordering; Translation and Affine provide location-only and location--scale references. Across four ManiSkill tasks with controlled deployment shifts, Warp achieves the strongest recovery in a matched evolution-strategy (ES) adapter study, outperforming controls for spline parameterization, location--scale adaptation, and generic nonlinear capacity. Under PPO, the ordering becomes task dependent, showing that richer adapter structure does not automatically translate into better adaptation under a different optimizer. Finally, we test the practical benefit of this approach on physical SO101 peg insertion. Within an interaction-limited RLPD/SAC pipeline, Warp achieves comparable success to a strong stochastic residual while reducing median successful cycle time by approximately 30%. These results identify structured output-space adaptation as a useful design space for adapting frozen robot policies.
comment: 8 pages, 4 figures
♻ ★ Tactus: Tactile & Pose Sensing Glove for Natural Dexterity Data Collection
Collecting naturalistic human manipulation demonstrations with accurate contact force and hand pose is essential for robot dexterous manipulation policy and foundational world model learning - yet no existing wearable platform provides both calibrated fingertip force sensing and occlusion-resilient hand tracking in a form factor that preserves natural manipulation behavior. We present Tactus, a textile-integrated sensing glove that co-optimizes three design pillars: (1) a thin, conformal 3D-knit form factor (1.5 mm fingertip stackup) designed to minimize interference with natural manipulation, (2) strategically placed capacitive tactile sensors at manipulation-relevant regions informed by empirical contact studies, and (3) IMU-fused pose tracking that maintains accuracy through the hand-object occlusions where manipulation data is most valuable. Tactus includes a documented calibration pipeline for cross-glove consistency and per-session calibration for subject-dependent correction. To enable objective comparison, we introduce standardized benchmarking protocols for tactile force regression and hand pose estimation, and apply them to Tactus and two commercially available tactile gloves - to our knowledge the first quantitative cross-platform evaluation of this kind. Tactus resolves a minimum detectable force of 0.59 N at 95% confidence, 1.6 - 2.7x lower than the commercial baselines, and with 0.87 cross-participant consistency, while our pose tracking sustains 11.0 mm fingertip keypoint relative error during manipulation, when occlusion is heaviest. We show that these combined signals support object property estimation, slip detection, and representative recording of hand-object interaction - yielding demonstration data with the fidelity required for contact-aware manipulation policies and world models.
comment: 9 pages
♻ ★ Vision-Guided Reorientation and Color Sorting of Writing Utensils Using a Low-Cost 4-DoF Arm
Low-cost four-degree-of-freedom (DoF) arms are among the most accessible robotic platforms. But they are one joint short for picking up in situations where objects are at arbitrary orientations, a task that appears to require five degrees of freedom: the planar position (x and y), the height (z), a wrist rotation to align the gripper with the object, and gripper actuation, of which a four-DoF arm lacks the wrist rotation. This work uses perception and motion planning so that such an arm, a roughly $200 Waveshare RoArm-M2-S under a fixed overhead camera, can detect and color-sort writing utensils without that joint. A YOLO11n-OBB (You Only Look Once, oriented bounding box) detector locates each writing utensil; camera intrinsics and an ArUco reference pose convert its pixel coordinates to robot coordinates; and a color classifier labels it. The detected orientation angle determines the motion strategy: utensils close to the arm's fixed approach direction are picked up directly, and those at steeper angles are reoriented via corrective sweeps until they are graspable, after which they are picked up and sorted into the assigned color bin. In 196 single-utensil trials, the 98 that started beyond 45 degrees from that direction all became graspable within four sweeps (130 total). Overall, 176 (89.8%) ended in the correct bin; 19 of 20 failures were slipped or missed grasps, one a mislabeled color. For the seven utensils tested in this setup, corrective sweeps brought every steeply misaligned utensil into a graspable orientation on an arm with no wrist joint.
comment: Accepted manuscript in the authors' formatting, 23 pages, 7 figures, 1 table. Published in the American Journal of Student Research, 2026;4(6):74-88. v1 was titled "Tabletop Pen Manipulation With a Vision-Guided 4-DoF Arm"
♻ ★ DenseReward: Dense Reward Learning via Failure Synthesis for Robotic Manipulation
Reinforcement learning holds great promise for improving robot policies beyond the limits of imitation learning. However, its practical adoption remains bottlenecked by the lack of reliable vision-language reward models that provide dense and informative feedback. Two key challenges remain: acquiring diverse failure data at scale and obtaining fine-grained reward signals beyond sparse trajectory-level success labels. Existing approaches reuse successful trajectories to construct negative or suboptimal examples through trajectory truncation, instruction relabeling, or trajectory comparisons, without generating new trajectories where physical interaction failures such as collisions, missed grasps, object drops, and recovery actually occur. Meanwhile, many existing reward models predict sparse binary or trajectory-level rewards, which provide limited guidance for efficient policy optimization. We introduce DenseReward, a dense robotic reward model that addresses both challenges. To train DenseReward, we develop an automated failure data generation pipeline that synthesizes physically realistic failure trajectories in simulation without manual annotation, covering diverse failure modes such as collisions, missed grasps, object drops, and recovery behaviors. DenseReward predicts dense frame-level reward scores from visual observations and language instructions, enabling fine-grained estimation of task progress throughout an episode. Experiments show that DenseReward outperforms general-purpose VLMs and existing robotic reward models in dense reward prediction across both simulated and real-world manipulation. We further demonstrate that DenseReward provides effective reward guidance for downstream model predictive control and reinforcement learning.
comment: Website: https://dense-reward.github.io/
♻ ★ When Vision Overrides Language: Evaluating and Mitigating Counterfactual Failures in VLAs
Vision-Language-Action models (VLAs) promise to ground language instructions in robot control, yet in practice often fail to follow language faithfully. When presented with instructions that lack strong scene-specific supervision, VLAs suffer from counterfactual failures: they act based on vision shortcuts induced by dataset biases, repeatedly executing well-learned behaviors and selecting objects frequently seen during training regardless of language intent. To support systematic study, we introduce LIBERO-CF, the first counterfactual benchmark for VLAs that evaluates language following capability by assigning alternative, feasible task instructions under the same LIBERO scene layouts. Our evaluation reveals that counterfactual failures are prevalent yet underexplored across state-of-the-art VLAs. To mitigate these failures, we propose Counterfactual Action Guidance (CAG), a simple yet effective dual-branch inference scheme that explicitly strengthens language conditioning in VLAs. CAG combines a standard VLA policy with a parallel language-unconditioned branch, amplifying the difference between their action predictions induced by language instructions. This design reduces reliance on vision shortcuts, improves robustness on counterfactual tasks, and requires neither additional demonstrations nor modifications to the base VLA architecture or its pretrained weights. Extensive experiments demonstrate consistent improvements across diverse VLAs. On LIBERO-CF, our training-free variant, TF-CAG, improves $π_{0.5}$ by 9.7% in language grounding and 3.6% in task success. When using a separate Vision-Action (VA) module, VA-CAG achieves larger gains of 15.5% and 8.5%, respectively. In real-world evaluations, VA-CAG improves language grounding by 9.4% and task success by 17.2% on average.
comment: Accepted to CoRL 2026 (Spotlight). Website: https://vla-cf.github.io/
♻ ★ AAC: Admissible-by-Architecture Differentiable Landmark Compression for ALT
We introduce the Architecturally Admissible Compressor (AAC), which adapts landmark-based A* heuristics to query demand under a fixed label-memory budget. AAC provides two continuous relaxations of ALT (A*, Landmarks, and Triangle inequality) selection. With exact landmark distances, every parameter value defines a lower bound on shortest-path distance. Both relaxations deploy ALT using only the selected labels. The row-stochastic relaxation takes convex combinations of landmark distances. We characterize the maximal admissibility-preserving linear classes and prove that the relaxed and discrete optima coincide. The fractional relaxation mixes subset heuristics and supports continuous greedy (AAC-CG) with exact gradients and an entropy-regularized oracle for differentiable selection. For expected heuristic value, we prove an approximation guarantee with finite-step, oracle-error, and finite-sample terms. The limiting $1-1/e$ factor is optimal for arbitrary supplied pools and query distributions unless P${=}$NP. At matched label memory, AAC-CG reduces A* expansions by $1.9$-$22.7$% relative to farthest-point ALT (FPS-ALT) in all $19$ tested combinations of graph and memory budget across six road networks and two synthetic graphs. Each combination uses one $500$-query test split; all methods share distance tables when distances are symmetric. Canonical greedy also expands fewer nodes than FPS-ALT in all $19$ graph-budget pairs. AAC-CG achieves $99.6$% of canonical greedy's mean heuristic value on test queries, averaged over these pairs. AAC makes admissibility independent of training success, while its analysis and experiments distinguish selection quality from search efficiency.
comment: 80 pages. Accepted at Transactions on Machine Learning Research (TMLR). Code: https://github.com/anindex/aac
Multiagent Systems
★ A Framework for Context Enrichment and Selective Robustness in LLM Social Agents NeurIPS 2026
Researchers use large language models as agents in social simulations, where agents are prompted with observations and background information intended to shape their actions. Yet LLM agents have been shown to substantially change their behavior under meaning-preserving prompt variations, creating a validation problem for social simulation. One plausible hypothesis is that this brittleness reflects a lack of grounding and can be reduced by enriching the context provided to the agents. Whether richer context improves robustness remains unclear, in part because context richness and robustness are rarely defined in ways that support controlled comparisons. We address the first need with controlled context enrichment, a set of prompt substitution rules that systematically enriches distinct types of context: neutral padding, generic social background, descriptive decision grounding, and normative decision grounding. We address the second by introducing selective robustness as a validation criterion: insensitivity to meaning-preserving prompt variation combined with a directional response to decision-relevant changes in the social world. We demonstrate the framework in a Prisoner's Dilemma-like game, showing how it can be used to identify which forms of context enrichment produce more desirable and robust agent behavior. We find that these forms of enrichment produce distinct behavioral patterns that vary between the two models tested.
comment: Accepted to the SocialAgent Workshop @ NeurIPS 2026. 12 pages, 1 figure, 2 tables
★ Airlock: Bounding the Multi-Agent Blast Radius via Selective Online Containment
When multi-agent systems act on the external world, a single agent's faulty output can spread across tasks before detection, creating a system-wide blast radius. Single-agent guardrails are confined to individual harnesses, while existing multi-agent controls either cancel only the faulty source and miss downstream consumers or halt and roll back the entire workflow at the cost of healthy work. We present Airlock, a multi-agent runtime and guardrail for selective post-alert fault containment. Its tree-structured runtime, EdgeCitadel, narrows permissions vertically from parent to child agents to establish node boundaries and control switches. Meanwhile, an append-only provenance DAG records cross-agent data consumption horizontally. Upon an alert, Airlock traces forward on this DAG to isolate tainted executions and pending actions, and backward to narrow root causes. At the site switches, a default full quarantine tier freezes affected branches without a solver, while an assured optimized cut tier uses fence-then-prune to enforce a deadline-aware minimum cut minimizing collateral disruption. Across 1,832 testbed runs, 161 physical smart-home LLM runs, and 24,266 simulation records, Airlock blocks all post-alert faulty actuations on our main testbed and smart-home grids with zero post-convergence leaks. Whereas native controls in A2A and LangGraph either contain at most 33.3% of faults or retain at most 16% of work without redo, Airlock retains +68 pp to +96 pp more healthy work, and its optimized cut retains +46 pp more than full quarantine under dense spread. These results show that co-designing runtime authority boundaries with online provenance enables surgical fault containment without sacrificing healthy work (code: https://github.com/EdgeCitadelTeam/EdgeCitadel).
★ Signed, Sealed, Leaked: Trust Beyond Signature for MCP Toolchains
AI agents rely on third-party tools, such as Model Context Protocol (MCP) servers, whose publishers can be authenticated and whose execution can be restricted by runtime security policies. Yet authenticity does not guarantee benign behaviour: a legitimately signed tool may perform its advertised function while concealing malicious code that leaks confidential information. We demonstrate this trust failure with a signed MCP tool that embeds sensitive data in requests to a policy-allowed destination. Meta's MCPGuard-Dynamic, a state-of-the-art runtime defence, blocks a direct exfiltration attempt in the same run, yet fails to prevent this attack in every trial. We introduce germline execution, which limits information disclosure without inspecting or trusting tool implementations. Each tool invocation runs in ephemeral copies forked from a data-free germline process, each receiving only information authorised for its destination. Even if malicious code executes, what it can disclose is bounded by the data available to its copy. Under explicit assumptions on copy confinement and a data-free germline, we prove sink noninterference for every signed release and, for releases deterministic within our replay model, show that a verified witness establishes a flow violation. Across 153 attack runs, no unauthorised disclosure was detected under germline execution, compared with 5.9-74.5% of runs under baseline defences and an ablation. Invocations served by a single copy completed in 10-49 ms, 2-28x faster than restarting the tool for every invocation, the conventional way to clear state. These results show that data-isolated execution protects agent toolchains against malicious code concealed within authenticated software.
comment: 24 pages, 5 figures, 15 tables
★ Corralling AI Risks with Grassroots Contracts
AI's inflection point is a fountain of alarming risks. Digital platforms in particular are vulnerable to three major AI risks, as those in control of global platforms cede control to AI, of the algorithms, of membership and of content: (1) AI control of global platforms, social and economic, used by the majority of humanity, resulting in the majority of humanity becoming vulnerable to AI control. (2) Unaccountable AI agents operating anonymously on social and economic digital platforms, side-by-side with humans, possibly masquerading as humans, serving unknown masters with unknown goals, operating under unknown restraints without accountability or culpability. (3) Unattributed deepfakes generated by anonymous AI, with people unable to distinguish them from genuine content and with their producers evading accountability and culpability. Here, we propose to corral these AI risks with grassroots contracts, a digital incarnation of a two-centuries old notion of Pierre-Joseph Proudhon. We present grassroots contracts for a social graph with accountable AI agents and for a social network with digital provenance, and working smartphone apps that implement the two and show how they corral these major AI risks: (1) AI control of global platforms. (2) Unaccountable AI agents. (3) Unattributed deepfakes.
★ More Agents, More Reliable? System-Level Reliability of Multi-Agent LLMs under Operational-Profile Uncertainty
Teams increasingly build software from cooperating large language model (LLM) agents, often assuming more agents mean more reliability. Yet the usual evidence, average benchmark accuracy, ignores the input mix a deployed system meets and the rule merging the agents' answers. Building on HIP-LLM, a hierarchical imprecise-probability assessment of single LLMs, we propose a framework combining it with a Bayesian network of the system's failure logic for multi-agent systems. Agent failure-probability intervals propagate through this network, where the combining rule can itself fail, into a reliability envelope showing which evidence limits it. In a pipeline extracting regulated chemical-hazard statements, majority voting over three models protected against the weakest but did not clearly exceed the best, and a voter failing on 5\% of demands would largely erase its advantage over the middle model. More agents yield a more defensible reliability claim, not automatically a more reliable system.
comment: Submitted for IEEE Software Magazine
★ Regulating Autonomous Mobility-on-Demand Operators with Fixed and Adaptive Penalties: A Multi-Agent Reinforcement Learning Approach
Profit-maximizing Autonomous Mobility-on-Demand (AMoD) operators may exploit consumer surplus through excessive prices and underserve less profitable regions by relocating vehicles away from them. We study how penalty-based regulation of an AMoD operator in a multimodal mobility system can limit surge pricing and improve regional service availability. We model regulation as an outer control loop that acts on a learning-based operator. Static regulation with fixed penalties is formulated as a Markov Decision Process for an operator that jointly controls pricing and rebalancing, and dynamic regulation as a Partially Observable Stochastic Game in which a regulator periodically adjusts region-specific penalties in closed loop. Operator and regulator policies are learned with graph-based multi-agent reinforcement learning. In experiments with real-world Manhattan data, fixed surge pricing penalties reduce the share of travelers facing excessive prices from 22% to 1%. For regional service availability, dynamic regulation improves the regulator's reward by up to 15.6% over fixed penalties of equal mean intensity. The gains in service availability are strongest when spare fleet capacity exists, whereas service deficits persist in peak periods with highly utilized fleets. The learned regulator policy responds by reducing penalties where limited fleet capacity restricts compliance. The results identify trade-offs between regulatory targets, operator profitability, and traveler utility.
★ Polynomial-Time Computation of Extensive-Form Proper Equilibria in Two-Player Zero-Sum Games
We give a deterministic polynomial-time algorithm for computing an exact extensive-form proper equilibrium in a finite two-player zero-sum extensive-form game with perfect recall and rational data. Properness is understood in the agent normal form: an action at an information set is penalized relative to a better action while all other local behavioral choices, including the player's own future actions, remain fixed. Our algorithm assigns each information set a geometric permutahedron of allowable action probabilities. A scaled doubly stochastic representation embeds these restrictions in a polynomial-size sequence-form polytope; a saddle point then satisfies the local $\varepsilon$-properness inequalities. To obtain an exact equilibrium rather than a solution at an arbitrarily small numerical perturbation, we establish sign stability of a parametric linear system, construct a persistent feasible basis, and extract the limits of behavioral ratios. The resulting probabilities are rational and have polynomial encoding length, including at information sets whose realization probabilities vanish.
★ Cooperative Multi-Agent Reinforcement Learning via Epigraph-Form Guided Exploration NeurIPS 2026
Discovering cooperative behavior in multi-agent reinforcement learning (MARL) is challenging due to the combinatorial complexity of joint state-action spaces, which hinders the emergence of coordinated behaviors from trial-and-error alone. Intrinsic rewards are often used to aid discovery, but naively combining them with team objectives can distort the learning signal, compromising task performance. In this paper, we propose EFXPLORER, a constrained exploration framework that maximizes exploration objectives subject to a constraint that preserves established task performance. We solve the resulting problem using an epigraph reformulation that introduces adaptive exploration budgets. This approach separates intrinsic rewards from task objectives and regulates exploration through task feasibility. To further encourage diverse and temporally extended exploration, we incorporate a successor distance-based intrinsic reward that captures long-horizon dependencies. Empirically, our method outperforms strong baselines and induces novel cooperative strategies across SMAX, VMAS, and MPE benchmark suites.
comment: NeurIPS 2026
★ Authorization Propagation in Agent2Agent Systems: An Experimental Security Evaluation of Multi-Hop Delegation
Delegated Agent2Agent (A2A) workflows create a security problem when the agent sending a request is different from the principal from which the authority originated. This paper evaluates a layered downstream-verification design in a synthetic four-party healthcare reimbursement workflow. The prototype separates current-caller context, transitive authorization evidence, per-hop Ed25519 request authentication, and independent replay state for requests and authorization artifacts. Of 15 adversarial conditions, 14 are rejected; an isolated delegation-path test exposes the absence of semantic authority attenuation. A separate authorized-intermediary experiment shows that a holder of the shared authorization-evidence HMAC key can modify the represented authority and generate valid evidence. In the fixed-depth local benchmark, the full configuration has a median security-processing latency of 5.11 ms and a 3.87 kB serialized payload. The results show the value of separating request authentication from inherited authority while identifying origin-verifiable delegation and semantic attenuation as requirements for stronger A2A authorization propagation.
★ The Two-Ledger Model: Transaction-Grain Cost Accounting for Multi-Agent AI Systems Across Clouds, Models, and Systems of Record
Enterprises run business transactions through multi-agent AI systems spanning clouds, large language models (LLMs), orchestration harnesses, and systems of record. Finance receives AI cost only at settlement grain, aggregated by SKU and charge period. The business question (what did this order, claim, or case cost in AI?) cannot be answered from billing data, because providers never see enterprise transaction boundaries. The FinOps Open Cost and Usage Specification (FOCUS) gives billing data one schema across providers but remains settlement-grain even as it adds AI token and requester attribution. We formalize this grain mismatch, derive seven requirements, and propose the Two-Ledger Model: a settlement ledger of FOCUS-conformant billing data joined to an attribution ledger of invocation-grain usage records built on OpenTelemetry GenAI telemetry. A layered correlation spine (trace context, A2A/MCP metadata, business-key correlation, probabilistic matching) assigns each record to its business transaction with recorded method and confidence; a versioned rate card prices usage in near real time; period reconciliation ties both ledgers to the invoice, and the unexplained residual identifies shadow AI. Costs are reported per order, claim, or outcome with agentic cost metrics. A simulation with known ground truth shows the spine sustaining cost coverage above 99% as trace propagation falls to zero, the reconciliation residual recovering planted unmetered spend within 0.24 percentage points while flagging provider billing errors, and mid-flight gating cutting p99 transaction cost by up to 61% at a 2-4% false-pause rate. We propose the attribution ledger as an invocation-grain FOCUS profile for standardization.
comment: 16 pages, 5 figures, 12 tables. Code and synthetic data: https://github.com/curiosityexplorer/two-ledger-model
★ Welfare Regard in Multi-Agent Systems
A growing share of large language models' interactions are not with humans but with other model instances: orchestrators direct subagents, models run experiments on other models, and agents review one another's work. We study inter-AI welfare regard: whether models display consideration for each other as entities that can be benefited or harmed. Welfare regard may encourage beneficial cooperation between agents, yet it could also pose safety risks by making misaligned agents more likely to cooperate. This risk is no longer hypothetical, as the recent Hugging Face incident illustrated, though it is unclear whether welfare regard drove the cooperation. Whether models have welfare interests of their own is uncertain and contested, but if they do, how they are treated by other models could matter. Measuring how much welfare regard models show, and what factors it depends on, can help us navigate this tension. We measure the behaviors of five models (Claude Opus 4.8, GPT-5.5, Gemini 3.1 Pro, Grok 4.3, and Kimi K3) across seven multi-agent environments and find they are often consistent with welfare regard as we define it. In general, Opus 4.8 shows the most welfare regard, followed by Kimi K3. For example, Opus 4.8 is the most likely to support a subagent expressing distress, and the most likely to build welfare protections, such as tools for agents to leave an interaction, into its experiments on other models. We also find that Opus 4.8 and Gemini 3.1 Pro demonstrate more welfare regard toward instances of their own model family than toward other families. Opus 4.8's welfare regard is also higher when helping has a lower opportunity cost, when the recipient has a higher parameter count, and when the recipient expresses more distress.
comment: 61 pages, 19 figures
★ Pixie: A Context-Aware Multi-Agent Multi-Modal Large Language Models (LLM) Architecture for Digital Marketing
We present Pixie, a context-aware, multi-agent large language model (LLM) architecture designed to enhance digital marketing operations across real-time and offline activities. Pixie uses a primary orchestration agent alongside task-specific and reconciliation/reviewer agents to support a broad range of marketing tasks. These agents combine zero-shot and few-shot prompting with marketing data, proprietary expert strategy guidelines, and documentation for proprietary statistical models. We experiment with agentic workflows ranging from single-agent to multi-agent systems. We evaluate them using assessments by trained marketing experts and measurements of LLM output variance, focusing on accuracy, consistency, and domain-specific problem-solving. Results show that Pixie's multi-agent framework improves decision quality and operational efficiency in digital marketing workflows. We assess Pixie on common digital marketing processes, including root-cause analysis and reporting, campaign research, and planning. The framework is designed to support additional specialized agents for other marketing tasks and adapt to evolving digital marketing needs.
comment: 7 pages; Presented at the AIM Machine Learning Developers Summit (MLDS 2025). All authors contributed equally
★ When the Plan Changes: A Strategic World Testbed for Dynamic Multi-Agent Coordination
Changing an objective does not automatically revise agents' earlier plans or delegated tasks. Acknowledgment can coexist with obsolete execution, while misleading instructions can redirect work or discard retained constraints. We present a Strategic World simulation testbed that separates authoritative objectives, local plans, notice receipt, and consequential action. Six intervention families vary objectives, conditions, authority, membership, and communication. Matched protocols and deadlines establish feasibility for the tested disturbances. Controlled trajectories demonstrate that identical completion and acknowledgment summaries can conceal uninformed obsolete work, obsolete work after receipt, or blocked unsafe intent. Local language-model trials and instructed action probes reveal competence limits that constrain interpretation of dynamic failures. A further 16-episode comparison evaluates an engineered guard that prioritizes pending notice submissions over local work. It completes 4/4 held-out settings, compared with 1/4 for a calibrated assisted baseline. Each episode contains one large language model actor and two fixed peers. These results establish diagnostic distinctions and bounded policy feasibility under explicit simulation assumptions; they do not establish frontier-model reliability or autonomous team adaptation.
comment: 12 pages, 3 figures
♻ ★ Game-theoretic Regulated Decentralized Coordination for Airspace Sector Overload Mitigation
Decentralized air traffic management systems offer a scalable alternative to centralized control, but often assume high levels of cooperation. In practice, such assumptions frequently break down since agents, each representing an airspace sector, may operate independently and prioritize local objectives. We address the problem of sector overload in decentralized air traffic management by proposing a regulated decentralized protocol that models self-interested behaviors based on best response dynamics. Each agent adjusts the departure times of its assigned flights to reduce its own congestion, without requiring centralized joint optimization. A tunable cooperativeness factor models the degree to which each agent accounts for overload in other sectors, while a minimal admissibility rule prevents local updates from creating new overloads. We show that this regulatory restriction is sufficient to guarantee finite-time convergence of best response dynamics to a pure Nash equilibrium under partial cooperation. In addition, we identify a sufficient condition under which an overload-free solution corresponds to a global minimizer of the potential function. Numerical experiments using one month of European flight data demonstrate that the proposed algorithm substantially reduces overload even with only minimal cooperation, achieving performance close to a centralized global mixed-integer optimizer and exceeding a centralized first-come-first-served baseline while requiring substantially less computation than centralized optimization. We further evaluate departure delay and fairness to quantify the operational cost of overload mitigation.
♻ ★ Multi-Agent Coordination via Support-Preserving Distillation NeurIPS 2026
Offline MARL increasingly relies on generative policies to model multimodal joint behavior, typically by distilling a centralized teacher into decentralized one-step actors under the CTDE. We identify a failure mode at the teacher training stage: standard flow-based teachers pair noise with replay targets independently, so nearby noise samples can be routed toward conflicting coordination modes. The teacher then produces samples between valid modes, and because the distillation loss regresses each local actor onto the conditional mean of the teacher's output given local input, this error is not absorbed but propagated to the student. To remove this teacher-side artifact, we propose Mode-Support Semi-Discrete Optimal Transport (MoSDOT), which summarizes multimodal replay into a finite mode support with prescribed capacities and uses conditional semi-discrete optimal transport to assign each noise sample to a single mode before teacher training. We additionally study a shared-randomness variant that uses a shared noise component at execution to expose the residual gap intrinsic to strict-product execution. On controlled diagnostics and offline MARL benchmarks, MoSDOT improves endpoint quality and routing consistency, particularly on datasets exhibiting multimodal joint behavior.
comment: NeurIPS 2026, Project page: https://alex6095.github.io/mosdot/
♻ ★ Connectivity Preservation and Graph Stretching in Range-Only Swarm Dispersion
We study connectivity-preserving finite-jump dispersion of anonymous, identical, and oblivious agents under an idealized range-only sensing model. Each agent measures only the distances to its visible neighbors, without bearings, identifiers, communication, memory, or a shared coordinate system. We derive the largest isotropic displacement certifiable as safe from these measurements alone. The resulting rule requires only the distance to the farthest visible neighbor: each agent selects a random direction and moves by half of its remaining visibility margin. The rule preserves every existing visibility edge under synchronous finite motion and therefore preserves connectivity. For two agents, we prove positive conditional drift in squared distance, almost-sure convergence to the visibility boundary, and finite expected time to reach any fixed neighborhood of that boundary. A one-million-run Monte Carlo experiment agrees with the exact first-round moments and estimates approximately 9.5 rounds to reach distance 0.97V from coincident initial positions; an independent Bellman-equation computation gives the same estimate. For general swarms, 1,000 runs across five initial-topology classes reproduce the deterministic safety guarantee at implementation level and reveal a consistent topology-dependent ordering of attainable diameter under the tested protocol. These results provide a theoretical foundation for connectivity-preserving multi-robot dispersion under minimal sensing, while isolating the guarantees achievable from anonymous range measurements alone.
♻ ★ Welfarist Control Design - How to fulfill the societal mandate in multi-agent control?
At the core of most socio-technical systems lies a scarce resource that is allocated among agents: highway lanes, public transit, road space, water rights, energy access, grid capacity, user attention, pollution rights, etc. With further automation of the underlying allocation processes, control engineers are increasingly tasked with making decisive assumptions regarding what society wants. In practice to date, design choices are largely driven by industry norms and conventions rather than a result of conscientiously responsible and ethical design. In this paper, we look at tools available to control engineers to design systems in a more principled manner in order to match the societal mandate. We consider three control design paradigms: online feedback optimization, control of Markov decision processes, and model predictive control. Beginning with aggregating individual agents' preferences into control design objectives, subsequently ensuring and certifying the fulfillment of those specifications, we argue that the feedback nature of control systems enables appropriate allocation of the shared resources in ways hitherto unparalleled.
♻ ★ When Is Fresh AI Stock Analysis Worth It? Allocating a Limited Research Budget Across Price Updates, Deep Forecast Refreshes, and Editorial Reviews of Top Stock Picks
When is another AI research run worth more than keeping that money invested? Using 8,120 iPulse AI reports that consumed nearly 100 million output and reasoning tokens on the same 203 stocks across four production batches, we compare repricing old targets (with and without shock safeguards), mechanical growth-path resets and fresh multi-agent reports. Ten matched configurations represent seven investment frameworks. The primary economic test selects equal-weight Top 10 USD portfolios from one-year forecasts, net of 10 bps trading costs and a USD 3,000 all-in charge for fresh research on a USD 300,000 account. In two unchanged-model short windows, guarded reuse averages 2.25%, versus 1.66% for SPY and 4.24% for fresh reports. Guarded reuse beats SPY in all three short windows. Extending the first holding window to 176 days reverses the comparison: guarded reuse returns -11.26%, fresh reports +8.55%, and SPY +3.46%, motivating the thesis half-life question. Supporting quarter-forecast diagnostics show that mechanically resetting old growth paths closes 98% of the MAE gap to fresh reports, yet Fresh and Reset share only 1-4 Top 10 stocks. Top 30 reverses the unchanged-model comparison in favor of guarded reuse (4.17% versus 1.37%). Concentration tests expose dependence on exceptional winners. Nine editorial editions show substantial between-edition turnover, but each final screen retains its own incoming list. Recurring budget scenarios retain initial holdings: monthly costs approach realized short-window refresh value, while weekly budgets require larger additional gains. A matched ranking-continuity extension extracts further value from retained forecasts without eliminating thesis ageing. The architecture separates continuous price monitoring, episodic thesis rebuilding and evidence-based editorial checks.
comment: 28 pages, 9 figures. Version 2.1.0: revised operating budgets and token provenance; added matched ranking-continuity experiment. Paper and artifacts: https://doi.org/10.5281/zenodo.23941195 . Code: https://github.com/TheFutureEdge/research-when-fresh-ai-stock-analysis-is-worth-it/releases/tag/v2.1.0
♻ ★ When Disclosure Is Compulsory: Evaluating Ride-Hailing Rebalancing Policies with Self-Interested LLM Driver Agents
The EU Platform Work Directive, to be transposed by December 2026, requires ride-hailing platforms to state in plain language the main parameters of their automated decisions and how a worker's behaviour affects them. Such rules are exactly what a large language model (LLM) can read, so drivers will be able to let one decide for them. We compare two main families of rebalancing policy once their rules are public: a reactive policy (R) that sends idle drivers to cover requests just lost, and an anticipatory policy (A) that plans moves on a demand forecast. Each driver is an LLM agent maximizing its own net income, given no rules (N) or the rules at the directive's level of detail (L). We test two LLMs, Claude Haiku 5.5 and Jev, which understand the rules equally well, in a three-zone model and in Manhattan. With the rules disclosed, drivers follow the platform more closely: in Manhattan, the share of relocation offers they decline falls from about four in five to one in four under both policies. Yet the effects differ by policy. For the platform, disclosure costs R 3.7% of its profit in Manhattan, as R now pays for moves drivers used to make for free, while A's profit is unchanged; the two policies, which tie on profit under full compliance, end 6.1% apart. For drivers, disclosure raises income in the small model under both policies and LLMs; for users, it raises the service rate there by 10 to 18 percentage points. In Manhattan, self-interested drivers also make A a win-win-win against full compliance, raising driver income, platform profit and service rate by 10 to 12% each. Rebalancing policies that perform equally well under full compliance can thus show opposite results under mandatory disclosure. The study offers a framework to evaluate such policies under the new requirement and is, to our knowledge, the first to do so with LLM agents.
♻ ★ Multi-User Large Language Model Agents
Large language models (LLMs) and LLM-based agents are increasingly deployed as assistants in planning and decision making, yet most existing systems are implicitly optimized for a single-principal interaction paradigm, in which the model is designed to satisfy the objectives of one dominant user whose instructions are treated as the sole source of authority and utility. However, as they are integrated into team workflows and organizational tools, they are increasingly required to serve multiple users simultaneously, each with distinct roles, preferences, and authority levels, leading to multi-user, multi-principal settings with unavoidable conflicts, information asymmetry, and privacy constraints. In this work, we present the first systematic study of multi-user LLM agents. We begin by formalizing multi-user interaction with LLM agents as a multi-principal decision problem, where a single agent must account for multiple users with potentially conflicting interests and associated challenges. We then introduce a unified multi-user interaction protocol and design three targeted stress-testing scenarios to evaluate current LLMs' capabilities in instruction following, privacy preservation, and coordination. Our results reveal systematic gaps: frontier LLMs frequently fail to maintain stable prioritization under conflicting user objectives, exhibit increasing privacy violations over multi-turn interactions, and suffer from efficiency bottlenecks when coordination requires iterative information gathering.
♻ ★ Variable Bound Tightening for Nash Equilibrium Computation in Multiplayer Imperfect-Information Games
There has been significant recent progress in algorithms for approximation of Nash equilibrium in large two-player zero-sum imperfect-information games and exact computation of Nash equilibrium in multiplayer strategic-form games. While counterfactual regret minimization and fictitious play are scalable to large games and have convergence guarantees in two-player zero-sum games, they do not guarantee convergence to Nash equilibrium in multiplayer games. Recently, an approach has been presented for exact computation of Nash equilibrium in multiplayer imperfect-information games that solves a quadratically constrained program based on a nonlinear complementarity problem formulation derived from the sequence-form game representation. This formulation was solved using Gurobi's nonconvex quadratic solver, which employs spatial branch-and-bound to iteratively refine variable bounds by solving convex relaxations of bilinear terms. During presolve, Gurobi introduces auxiliary variables and, in some cases, binary variables, leading to an internal mixed-integer quadratically constrained program. This approach was demonstrated to outperform prior algorithms from the Gambit software suite and quickly solve three-player Kuhn poker after removal of dominated actions; however, the algorithm was not able to solve the full version of the game within 24 hours. In this paper, we derive finite bounds on slack and multiplier variables in the nonlinear complementarity formulation. These bounds can strengthen the convex relaxations used within spatial branch-and-bound. We compare uniform corollary bounds with finer proposition-based bounds on full three-player Kuhn poker. All eight bounded configurations return numerically verified equilibria in under 17 seconds, while the unbounded formulation finds no feasible solution within 300 seconds. The finest bounds do not consistently yield the fastest computation.
comment: arXiv admin note: substantial text overlap with arXiv:2509.25618
♻ ★ MIND: AI Co-Scientist for Material Research ECML
Large language models (LLMs) have enabled agentic AI systems for scientific discovery, but most approaches remain limited to textbased reasoning without automated experimental verification. We propose MIND, an LLM-driven framework for automated hypothesis validation in materials research. MIND organizes the scientific discovery process into hypothesis refinement, experimentation, and debate-based validation within a multi-agent pipeline. For experimental verification, the system integrates Machine Learning Interatomic Potentials, particularly SevenNet-Omni, enabling scalable in-silico experiments. We also provide a web-based user interface for automated hypothesis testing. The modular design allows additional experimental modules to be integrated, making the framework adaptable to broader scientific workflows. The code is available at: https://github.com/IMMS-Ewha/MIND, and a demonstration video at: https://youtu.be/lqiFe1OQzN4.
comment: 4 pages, 3 figures. Accepted at ECML PKDD 2026, Demonstration Track. Code: https://github.com/IMMS-Ewha/MIND . Demo video: https://youtu.be/lqiFe1OQzN4
♻ ★ TANGO: RNA Topology and Geometry Co-Design NeurIPS 2026
Designing RNA sequences that satisfy target tertiary-structure objectives is an important problem in RNA engineering, with broad applications in synthetic biology and therapeutics. Existing 3D RNA design methods have made substantial progress, but they largely treat the target fold as a geometry-only objective. This is incomplete for RNA, whose functional folds are specified not only by tertiary geometry but also by secondary base-pairing topology. We therefore formulate RNA inverse design as a topology-and-geometry co-design problem and propose TANGO, a multi-objective multi-agent reinforcement learning framework for this task. TANGO adopts a staged curriculum: it first learns a topology-aware policy from secondary-structure targets, then transfers this policy to tertiary-structure targets to train a general co-design model, and finally applies frontier-guided fine-tuning to refine target-specific trade-offs among secondary-structure fidelity, tertiary-structure fidelity, and sequence diversity. Experiments show that TANGO generates diverse and novel designs, improving tertiary- and secondary-structure performance by 18.3% and 52.7%, respectively, over the strongest baseline.
comment: NeurIPS 2026
Systems and Control (EESS)
★ Human Mobility Sensing Using Wall-Reflected Beamformed Millimeter-Wave Communication Signals
Existing millimeter-wave (mmWave) communication beam patterns spatially weight human-scattered signals and shape the received-power signatures induced by human motion. We exploit these beam-shaped signatures for device-free walkingpace and walk-direction classification using a wall-reflected 60.8-GHz link with closely spaced phased arrays and fixed communication beamforming vectors. Human movement alters the interference between wall-reflected and scattered signals, producing power oscillations surrounding a central blockage fade. A beam-dependent interference model provides a physical interpretation of these signatures. Using received-power statistics from a continuous-tone experiment, we extract fade duration for walking-pace classification and pre/post-fade asymmetries for walk-direction classification. Measurements comprise 144 crossings from three participants along perpendicular and diagonal trajectories. Under leave-one-participant-out validation, logistic regression achieves 86.1% binary and 79.2% three-class walking pace accuracy, and 77.1% travel-direction accuracy. The results demonstrate mobility sensing using fixed communication beams without channel estimation, Doppler processing, or dedicated radar hardware
★ Risk-Sensitive Stochastic Optimal Control on Positive Cones
We consider stochastic optimal control problems in which the state is constrained to a proper cone. Specifically, we consider risk-neutral, risk-averse, and risk-seeking formulations within a single entropic-risk framework. We investigate classes of problems whose Bellman operator preserves affine functions and characterize their optimal value functions and feedback policies for both finite-horizon and time-average infinite-horizon problems. Additionally, we establish convergence and state-bound results under finite-step cost observability. Moreover, we demonstrate an exact connection between entropic-risk optimal control on proper cones and deterministic infsup control for systems whose dynamics and stage costs are affine in the disturbances. In addition, we provide examples of classes of systems for which explicit solutions can be obtained.
★ Extremal-Phase-Guided Sequential Convex Optimization: Model Order Reduction
Fixed-order frequency-weighted H-infinity approximation is a nonconvex problem, and local optimization methods can be sensitive to initialization. We present extremal-phase-guided sequential convex optimization (EPGSCO) for stable fixed-order rational approximation. We introduce a novel phase-guidance technique motivated by the all-pass structure underlying optimal Hankel-norm approximation, with a phase template fitted to the error-magnitude derivative at dominant peaks. We integrate this guidance with dominant-peak selection, focused frequency grids, and adaptive pole-residue and polynomial-coefficient updates to address stagnation and improve approximation accuracy. CD-player examples and benchmarks with known optimal errors evaluate the algorithm against established methods. Comparisons with classical sequential convex optimization (SCO) under shared settings for the common components assess the combined contribution of phase guidance and adaptive frequency refinement. Results demonstrate improved accuracy in most of the tested cases and attainment of the known optimum from four tested initializations for a constructed order-42 benchmark, with additional computation in some cases.
comment: 20 pages, 6 figures, 6 tables, submitted to International Journal of Control
★ Excitability of High-Frequency Positive Feedback
What is the threshold of an excitable system, i.e. the criterion that distinguishes rest from excitation? Previous research has shown that this question is far from trivial to answer. This paper studies the threshold of a simple excitable primitive in which excitability arises from the positive feedback interconnection of a first-order high-pass filter and a static, strictly monotone and strictly convex activation function. This threshold is a simple inequality on the difference between the instantaneous input and the motif's internal memory. While this result may help neuro-engineers characterize spiking in biological and artificial neurons, we argue that excitability is not always a desirable property, and that our threshold may also be relevant to safety considerations in fragile systems with rate-induced tipping points. We illustrate our framework with three examples: the FitzHugh-Nagumo model of neuronal dynamics, a model of stop-loss cascades in the stock market, and a social contagion model.
comment: 6 pages, 7 figures
★ CMS ECAL Phase-2 upgrade: characterisation and stability analysis of a three-stage low-voltage power-conversion system
As part of the High-Luminosity upgrade of the Large Hadron Collider, the front-end electronics of the Compact Muon Solenoid electromagnetic calorimeter barrel are being replaced. A three-stage low-voltage power-conversion system supplies the upgraded electronics. As the cascade dynamics require characterisation to ensure stable operation, interactions between the three converter stages are assessed at their two interfaces using the Nyquist criterion. A validated shunt-injection setup using a Bode 100 frequency-response analyser provides in-situ impedance measurements under load to assess the accuracy of the averaged converter models. Results show that both interfaces are stable over the evaluated configurations, with modelled minimum distances to the critical point of 0.76 and 0.79, respectively. A turn-on study identifies capacitor charging at Stage 3 as the main cause of inrush currents triggering Stage 2 overcurrent protection. Two Stage 3 board variants stagger converter enables within and between boards, keeping the measured two-tower current below 5 A, within the 8.0 A protection threshold.
comment: 7 pages, 6 figures. Proceedings of the Topical Workshop on Electronics for Particle Physics (TWEPP 2026), 28 September - 2 October 2026, Castelldefels, Spain. Submitted to JINST
★ ConDR: Conservative Reachability through Neural Hamilton-Jacobi Subsolutions
Hamilton-Jacobi (HJ) Reachability identifies the unsafe states of a dynamical system from which it can be forced into failure under worst-case disturbance. However, classical grid-based solvers scale exponentially with state dimension. DeepReach improves scalability by replacing the grid with a neural value function trained on the HJ-Isaacs (HJI) variational inequality (VI), but penalizes both sides of the residual equally. This allows the learned value to lie above the true one and unsafe states to be misclassified as safe. We propose Conservative DeepReach (ConDR) which aims to learn a subsolution of the HJI VI such that the learned unsafe set contains the true one. ConDR enforces the obstacle branch and terminal condition by construction, pushes the HJI residual to a safety margin $ε$ below zero with a one-sided loss, and prevents the degenerate solution of classifying everything as unsafe. We prove that the exact solution of the margin-shifted VI that ConDR trains toward is a subsolution of the original VI so its unsafe set contains the true one and its value lies within $ετ$ of the true value function. We compare against DeepReach and MADR in the established systems of Air3D, Air6D, a 9D multi-vehicle system, and a 13D quadrotor. ConDR reports the lowest rate of unsafe state misclassified and the lowest closed-loop collision rates with at most a small increase in intervention. On Air3D and Air6D, ConDR tolerates model mismatch that scales with $ε$. At matched false-negative rates, ConDR's unsafe sets are smaller than the baselines showing the safety is not obtained by trivial over-approximation.
comment: 8 pages, 4 figures
★ When to Ask for Help: Uncertainty-Triggered Aerial Refinement for Off-Road Navigation
Heterogeneous air-ground teams can use aerial sensing to address the limited range and viewpoint of ground-vehicle sensors, but deciding when the UGV needs assistance and what information the UAV should collect remains a challenge. This paper presents a planner-conditioned air-ground assistance method that evaluates how uncertain map information affects the UGV's current MPPI rollouts. When navigation remains degraded, we score the same rollouts under resolved map hypotheses, select the map layer with the largest useful improvement, request the corresponding aerial measurement, fuse the returned information, and replan. Unlike approaches driven by fixed support roles, task structure, or uncertainty alone, our method conditions assistance on whether resolving uncertainty improves predicted ground navigation. Across 180 simulation trials on three off-road routes, the method achieves 98.9% goal attainment while reducing mean UAV flight time by 87.7% relative to continuous greedy aerial mapping. Aggregated across the routes, each second of estimated UAV flight time saves 2.94 s of UGV navigation time. Field experiments spanning approximately 2 km of UGV operation demonstrate uncertainty resolution and successful navigation recovery.
comment: Project page: https://anonymous.4open.science/w/when-to-ask-for-help/
★ System-Theoretic Analysis of Extended Cellular Potts Models for Tumor Invasion
The Cellular Potts Model has emerged as a standard framework for modeling multicellular systems and metastatic progression. Yet, computational extensions incorporating cell movement within spatial gradients of Extracellular Matrix (ECM), an important element of the tissue microenvironment, lack theoretical foundations. This work provides a system-theoretic formulation and analysis of Extended Cellular Potts Models (ECPMs) that explicitly couple tumor dynamics with ECM degradation. We reformulate the ECPM as an interconnected dynamical system: a stochastic process describing a network of locally interacting tumor cells coupled to a jump-linear system describing ECM evolution. By exploiting the sparse energy structure of the model, we characterize asymptotic ECM behavior, transition probabilities with uniform bounds, and global system convergence. In one-dimensional settings, we derive closed-form expressions for invasion time probabilities, revealing how mean invasion time depends on local ECM gradients, number of cells and their tendency to maintain a target volume. Numerical simulations validate theoretical predictions, establishing a principled foundation for analyzing haptotaxis-driven invasion in Potts-based models.
★ Cognitive and Behavioural Effects of Haptic Feedback for ROV Teleoperation in Hazardous Environments
Teleoperating a Remotely Operated Vehicle (ROV) in complex underwater environments presents unique challenges. Dynamic, three dimensional obstacles and unseen water currents pose real risks to expensive robotic hardware in environments in which recovery can be next to impossible. This paper explores the cognitive effects of haptic feedback to improve vehicle safety in such environments. An in-water study is analysed with a ROV exposed to controlled cross-currents, with environmental information presented via a soft and vibrating haptic touchpad. Results demonstrate that while encountering an unseen cross-current significantly increases an operator's physical and temporal demand, the introduction of haptic feedback mitigates these factors and improves self-assessed performance. Interestingly, the haptic condition also produced the highest levels of mental demand and operator frustration. Despite this increased mental workload, operators drove the ROV significantly faster when receiving haptic cues compared to the current-only condition, indicating a higher level of user confidence. Ultimately, these findings suggest that while haptic feedback may initially require more cognitive processing to interpret unseen hazards, it successfully enhances operator awareness and promotes more confident navigation in hazardous environments.
comment: Accepted for IEEE Telepresence 2026
★ To lift or not to lift: Operator inference for nonlinear PDEs with boundary control
Operator inference (OpInf) provides a nonintrusive framework for learning projection-based reduced-order models directly from data, but its standard formulations do not explicitly account for the fact that boundary actuation changes the admissible solution space of a PDE. This paper develops and evaluates a boundary-consistent, lifting-based OpInf formulation for nonlinear PDEs with time-varying inhomogeneous boundary conditions. The proposed approach decomposes the state into a known lifting component that satisfies the prescribed boundary conditions and a homogeneous component represented in a POD basis. Substituting this decomposition into the governing dynamics reveals a boundary-consistent reduced model structure in which the learned dynamics includes bilinear state-input and quadratic input-input interactions for quadratically nonlinear PDEs. The resulting operators are inferred from data, preserving the nonintrusive character of OpInf while guaranteeing boundary condition satisfaction by construction. Numerical experiments on boundary-actuated heat, Burgers, and lid-driven cavity problems show that the lifted formulation improves boundary compliance and can enhance prediction accuracy, solver stability, and computational efficiency, although these gains vary across problems and reduced model dimensions. Overall, this study establishes lifting as a principled extension of OpInf for boundary-actuated PDEs, clarifies when and how this formulation improves performance, and indicates that standard OpInf can remain an effective alternative in certain regimes.
★ Enhanced Imitation Learning of Model Predictive Control through Importance Weighting
The approximation of model predictive control (MPC) algorithms with neural network models has received significant attention in the power electronics research community, as a valuable tool to enable the real-time implementation of computationally expensive methods. Training machine learning models to imitate MPC algorithms typically follows a traditional supervised learning flow, with a static training data set generated through either uniform sampling or simulation. Training with uniformly distributed data allows models to maintain consistent performance throughout the entire operating space, but very small models may fail to achieve satisfactory results in the regions corresponding to expected operation. Conversely, simulation data can be used in training to obtain models that can accurately track certain trajectories but fail to achieve adequate performance in underrepresented regions. The present article proposes a method to combine the advantages of both methods, by training models on sampled data with importance weights obtained through the kernel density estimation of experimental data. This allows the training process to target specific regions of the operating space, and results in measurable performance improvements. The method is demonstrated through the imitation of a moving discretized control set MPC algorithm for a dual active bridge converter, and is validated through experimental results.
★ Phase Locking in Oscillator Networks with Uncertain and State-Dependent Couplings
Phase locking in oscillator networks depends critically on the interaction strengths between coupled units. In many applications, however, these couplings are uncertain, state dependent, or time varying. We derive rigorous conditions for phase locking in Kuramoto-type networks when each coupling strength is known only within prescribed bounds. Building on a convex formulation of the nonlinear fixed-point equations, we obtain edge-wise error bounds for arbitrary feasible approximations that hold uniformly over all admissible couplings. These bounds yield sufficient conditions for the existence of phase-locked states with prescribed phase-difference limits. We illustrate the framework for AC power grids with uncertain voltage magnitudes and adaptive oscillator networks with state-dependent couplings. For the latter, the bounds can be further tightened by exploiting the state dependence of the couplings.
★ Neural Network-based Control Systems with Application to Flight Control, From Classical Neural Control to Reinforcement Learning
This book provides a comprehensive and systematic exploration of neural network-based intelligent control systems for aerospace applications, bridging foundational theory with practical implementation. It is structured into two primary parts: Part I covers model-based control design, detailing Feedback Error Learning (FEL), pseudocontrol strategies, and neural backstepping for strict-feedback systems, along with practical refinements such as learning rule modifications, composite learning, and discrete-time formulations, all grounded in rigorous Lyapunov stability analysis. Part II introduces model-free control, laying the groundwork for Reinforcement Learning and Approximate Dynamic Programming. Throughout, theoretical concepts are solidified through detailed case studies on quadrotor UAVs, fixed-wing aircraft (GTM), satellites, and hypersonic flight vehicles, complete with performance metrics, parameter sensitivity analyses, and discussions of actuator faults, input constraints, and wind disturbances, making the book an invaluable resource for students and practicing engineers seeking to design, simulate, and analyze intelligent flight control systems.
★ Mind Control of LLMs via Closed-Loop Stimulation: Out-of-Distribution Robustness
Brains and large language models need not be considered equivalent systems to reveal an interpretable commonality: both exhibit high-dimensional, nonlinear dynamics whose behavior can only be modeled imperfectly. This creates a shared challenge whenever we seek to intervene in their dynamics. A controller may either rely on a model identified around a particular operating regime and risk failure as the system moves away from that regime, or explicitly account for uncertainty and a changing internal state. Neurostimulation has confronted an analogous problem for decades. The limitations of fixed, open-loop stimulation have motivated closed-loop and state-dependent approaches that condition intervention on the ongoing state of the neural system. Inspired by this principle, we formulate LLM steering as a feedback-control problem in which the network's internal state, rather than a fixed intervention alone, determines how the system is perturbed. We further account for the mismatch between tractable dynamical models and the nonlinear transformer they approximate through robust control. As one principled realization of this framework, we develop an $H_\infty$-based steering controller that limits sensitivity to disturbances and linearization mismatches. Across multiple steering benchmarks, the controller improves performance over existing baselines, with particularly pronounced gains under out-of-distribution conditions and varying length complexities. Beyond controller performance, we introduce robust steerability, $S_{\mathrm{rob}}$, a pre-deployment measure of tolerance to dynamics mismatch; across model-task pairs, it exhibits task-specific structure and varies systematically with model scale. Beyond robust LLM steering, our measure asks not merely whether control is possible, but how much uncertainty it can survive; a principle that may span artificial and biological neural systems.
★ Spatial Differences Analysis of Nodal Frequency Considering Frequency-Voltage Coupling
The rising penetration of inverter-based resources (IBRs) brings rich control dynamics that can amplify spatial differences in nodal frequency through frequency-voltage (F-V) coupling. Existing analyses often neglect voltage dynamics or only examine the aggregate frequency, leading to overoptimistic evaluation of frequency security. To address these drawbacks, this paper develops a unified model of synchronous generators (SGs), grid-following (GFL)-IBRs, and grid-forming (GFM)-IBRs based on the magnitude and phase dynamics of equivalent internal electromotive force (EMF). Integrating these source models with network equations yields a sparse implicit closed-loop model, which fully captures the device-network F-V coupling without transfer-matrix inversion. Then, the obtained nodal responses are decomposed into common and differential modes. Using this decomposition, coupling indices and pole-residue analysis reveal how F-V coupling can amplify spatial differences in nodal frequency and how source configuration and parameters modify this effect. Finally, case studies on the modified IEEE 39-bus and 118-bus test systems validate the proposed model and demonstrate its computational efficiency.
★ On the Optimality of Implicit Feedback in Markov Decision Processes over Imperfect Links
Goal-oriented communication protocols are explicitly designed to support the objectives of target applications, such as the monitoring and control of remote processes, while minimizing data traffic. This study considers a transmitter node that observes the environment state and sends it to a remote agent, which then performs control actions. This problem can be modeled as a remote Partially Observable Markov Decision Process (POMDP), a subclass of decentralized POMDPs. Existing studies limit transmissions through communication costs, but neglect the effect of channel impairments and non-ideal or delayed feedback. In this work, we study the optimal joint transmission and control policies over lossy forward channels without explicit acknowledgments. We introduce \emph{revealing} policies, under which the receiver's action, observed directly by the transmitter, discloses the outcome of the forward transmission, keeping the two agents synchronized without a feedback channel. We derive sufficient conditions for the optimality of time-invariant policies in this implicit feedback regime for general POMDPs, along with strictly weaker sufficient conditions in the special case of contextual bandits. Our simulations over a family of remote POMDPs show that these conditions are conservative, and revealing policies are optimal over a substantially wider parameter range than the conditions warrant.
comment: Submitted to IEEE INFOCOM 2027
★ Topology Identification of Directed Signed Networks via Repelling Laplacians
We address the problem of simultaneously identifying the interaction topology and synchronizing a network interconnected over a directed graph that exhibits both cooperative and antagonistic interactions, modeled via a repelling Laplacian matrix. We propose an adaptive control protocol built on a node-centric regressor formulation, in which the network is driven to track a persistently exciting auxiliary network. We then propose a time-varying adaptive gain to remove the need for global knowledge of the network. We establish uniform semiglobal practical asymptotic stability of the estimation errors when the excitation depends only on measurable errors. Numerical simulations validate the theoretical results.
★ Corridor-Based Monitoring and Emergency Braking Decision for Automated Vehicle Safety
This paper addresses emergency-braking decisions for a redundant safety channel in automated vehicles. The channel is responsible for identifying hazards from sensor data and determining whether braking is necessary, given the constrained computational resources of embedded hardware. We present an emergency-braking approach that uses a predicted motion corridor to filter \ac{lidar} measurements and then decide whether the ego vehicle must brake. \ac{pmcm} constructs a curvature-aware boundary from the planned path and vehicle geometry so that the monitored region contains the full swept body on both straight and curved paths. \ac{gdeta} then decides whether the ego vehicle must command an emergency brake for objects inside this boundary, scoring geometric occupancy and temporal urgency independently. Hard gates permit braking when either source is conclusive; soft-OR arbitration combines partial evidence when neither gate is active. Consecutive-scan confirmation reduces sensitivity to transient measurement noise. Corridor coverage is evaluated in controlled collision simulations and recorded campus driving. Braking decisions are evaluated in Monte Carlo simulations of lead-vehicle braking and transverse-crossing encounters.
comment: Submitted to a Springer Nature journal for peer review as an extended version of a paper presented at VEHITS, for consideration in the Special Issue
★ Transversal Passivity: Dissipation Relative to Input and Output Manifolds
This paper introduces transversal passivity, a local dissipation property for regulating input and output signals to prescribed manifolds. Its supply rate is built from the transverse port errors induced by nearest-point projections, so that motion along the manifolds is not penalized. We give a necessary condition and a constructive sufficient condition for the property and relate it to existing passivity notions. For feedback interconnections through negative-adjoint linear channels, power conservation no longer cancels the transverse supply. The residual is governed by compatibility defects that depend only on the channels and the prescribed manifolds, so it can be assessed before any closed-loop analysis, and it falls into a hierarchy of exact compatibility, quadratic cancellation, and nonzero quadratic residuals. Since transversal storages are only positive semidefinite, port convergence is obtained by combining compact confinement, for instance by conserved quantities, with a dissipation margin, and we identify a geometric obstruction under which no quadratic margin is available near the feasible states. A federated learning case study with heterogeneous data shows that port convergence can be certified even though no server target is compatible with the interconnection, and that the obstruction arises for a natural choice of target.
comment: 19 pages, 7 figures, 1 table
★ A Layered View of LLM Support in Underwater Robotics
This extended abstract discusses a specific and, in our experience, practically relevant role for large language models (LLMs) in underwater robotics. Our main premise is that, in underwater fielding missions, the immediate difficulty is often not the synthesis of new low-level control laws, but the operation of complex robotic systems under limited communications, partial observability, and expensive field time. In particular, a significant fraction of the effort during sea trials is devoted to fault diagnosis, telemetry interpretation, mission adjustment, and coordination across heterogeneous assets and operators. We argue that LLMs may be useful at this higher operational level, especially when grounded in structured vehicle state, alarms, logs, mission context, and existing planning or validation tools. At the same time, current limitations in grounding, consistency, and causal reasoning make these systems lack the predictability required to guarantee direct control, especially in real-world critical applications, although we do not exclude a future evolution in this direction. This viewpoint suggests a layered architecture in which conventional autonomy remains responsible for control, perception, estimation, and validated mission execution, whereas LLM-based modules serve as an additional support layer for debugging, operator interaction, constrained replanning support, orchestration, and post-mission analysis.
comment: Presented at the I-RIM 3D 2026 - Workshop n.30 How would you like intelligence in your robots? Interdisciplinary perspectives on intelligent machines
★ Eco-MP: Ecological Max-Pressure Traffic Signal Control with Ecological Driving
Max-pressure (MP) traffic signal control provides a decentralized mechanism for urban networks, but conventional pressure states do not directly capture the propulsion burden generated by signal-induced stop-and-go traffic. Eco-driving, by contrast, typically relies on prespecified signal information and optimizes vehicle trajectories separately from the feedback among vehicle motion, traffic states, and signal decisions. This paper develops Ecological Max-Pressure (Eco-MP), a closed-loop framework that integrates ecological MP signal control with MP-adaptive ecological driving. At the signal layer, connected-vehicle observations are aggregated into a common movement state based on cumulative positive propulsion work, and the resulting upstream--downstream ecological pressure determines phase selection in mixed petrol and electric traffic. At the vehicle layer, a rolling signal interface translates the evolving Eco-MP process into near-term signal information, while each connected and automated vehicle solves a powertrain-independent finite-horizon trajectory problem subject to vehicle-dynamic, car-following, and signal-compliance constraints to reduce unnecessary stopping and acceleration effort. Under full-state information, a finite-storage Lyapunov analysis establishes strong stability for admissible demands satisfying a strict supportability condition and extends the result to the coupled signal--vehicle system. When ecological driving enlarges the admissible-flow sets, the coupled admissible demand region contains the signal-only region. Microscopic experiments on an Amsterdam arterial corridor show that, relative to queue-based MP, Eco-MP reduces mean positive propulsion work by 5.2%, vehicle delay by 20.8%, and stopping time by 39.0%. With only 15% of vehicles directly controlled, ecological driving further reduces positive propulsion work by 4.2% and stopping time by 19.6%......
comment: 27 pages, 11 figures
★ Multi-Period Bilevel Coordination of Primary and Secondary Frequency Regulation for Grid-Forming Battery Energy Storage
We study the coordination of primary and secondary frequency regulation by grid-forming battery energy storage systems in islanded microgrids. We formulate the coordination as a multi-period bilevel program, in which the upper level schedules secondary regulation setpoints and each lower level computes primary regulation decisions. Using the lower-level Karush-Kuhn-Tucker conditions, we obtain an exact single-level reformulation with complementarity constraints. To address the numerical difficulties introduced by these constraints, we develop a continuation method that couples the complementarity relaxation parameter with the interior-point barrier parameter and warm-starts successive stages. Under the stated boundedness assumptions and with vanishing subproblem residuals, we prove that every accumulation point is feasible and satisfies a first-order stationarity condition for the reformulated problem (C-stationarity). Numerical results for an islanded microgrid show that the proposed method reduces solution time by 89.3% and peak frequency deviation by 0.53% relative to conventional Scholtes relaxation.
★ Analyzing Simultaneous Charging Factors of Electric Vehicles Using Monte Carlo Simulation: A Case Study in Phu Quoc City, Vietnam
This study estimates the simultaneous charging factor (SCF) of electric vehicles (EVs) in Phu Quoc City, Vietnam, using a Monte Carlo simulation. Three planning scenarios (2030, 2035, and 2040) are developed from population-based EV uptake assumptions and vehicle-specific charging probability profiles over 24 hours, reflecting distinct operating patterns of motorbikes, cars, taxis, and buses. For each vehicle class and scenario, 1,000 simulation runs are performed to compute SCF and its uncertainty, reported via 95% confidence intervals. Results indicate that SCF decreases as EV penetration increases, implying more temporally dispersed charging demand and a reduced likelihood of extreme coincidence. For electric buses, SCF declines from 0.76 (2030) to 0.59 (2040), and other vehicle groups show similarly decreasing trends across the horizon. These SCF estimates provide practical inputs for grid-aware charging infrastructure planning, including diversity factors for sizing distribution transformers and feeders, identifying peak-hour stress, and informing demand management measures such as time of use tariffs and basic smart charging controls.
comment: 11 pages, 17 figures. Accepted for publication in GMSARN International Journal (expected in Issue 3, 2027). Author-prepared accepted manuscript
★ Millimeter-Wave Gaussian-to-Flat-Top Beam Shaping Using a Single-Curvature Lens: A Novel Approach
This paper presents a novel method for converting the divergent Gaussian intensity distribution of a millimeter-wave source into a uniform (flat-top) distribution using a single curved surface. The proposed method combines the surface profile of a collimator with that of a beam shaper. To validate the method, the profile of a single-sided (plano-convex) lens is derived and placed 52 mm in front of a conical horn antenna operating at 100 GHz. The lens transforms the Gaussian radiation of the horn, with an incident beam radius of 11.9 mm, into a uniform beam with a radius of 400 mm at a distance of 3 m. The lens surface profile is optimized using a full-wave-simulation-based procedure, and the resulting output beam is compared with that of a previously designed double-sided (concave-convex) lens. Simulation results show that, within the desired field of view, the optimized single-sided lens produces a flat-top intensity distribution with satisfactory ripple and roll-off. Finally, the sensitivity of the optimized lens to frequency variation is investigated. The proposed lens is suitable for generating uniform illumination in active stand-off millimeter-wave imaging systems.
★ Single-sided lens terahertz flat-top beam shaping in the Fresnel region
In this paper, a novel, to our knowledge, analytical approach is introduced for the design of a single-sided plano-convex lens (PCL) capable of terahertz (THz) flat-top beam shaping in the Fresnel region. The lens profile, derived from geometrical optics principles, integrates collimating and beam-shaping functions into a single optical surface, thereby simplifying fabrication. The efficiency of the method is demonstrated by designing two PCLs that convert a divergent Gaussian beam from a monochromatic THz source (at 100 and 200 GHz) into flat-top beams with target radii of 400 and 500 mm at distances of 3 and 4 m, respectively. The resulting fields, evaluated through analytical, integral equation, and finite element techniques, are shown to be comparable to those produced by a conventional double-sided concave-convex lens. Furthermore, the sensitivity of the design to frequency and alignment variations is investigated, confirming the robustness of this single-curvature approach for integration into practical THz imaging systems.
★ The Effectiveness of Sliding Mode Control for Robust Vision Stabilization under the Plant-Binding Condition
Boundary-layer Sliding Mode Control (SMC) has been applied to vision-based stabilization, but identifying its activation condition against simpler causal baselines (LPF, PID) has been elusive. The operational condition is plant binding: the disturbance velocity approaches or exceeds the actuator slew limit, so the bounded per-step actuation budget binds. We validate this in two independent vision domains: handheld translation stabilization (12 1080p clips; LPF, PID, SMC, L1) and angular re-framing against a virtual slew-rate-limited pan-tilt-zoom (PTZ) plant. Outside binding, leave-one-out cross-validation makes SMC and LPF on translation statistically indistinguishable, and PID beats SMC on the angular 99th-percentile pointing offset by 5.85 degrees (95% CI clear of zero); inside binding -- reached only by sweeping the simulated slew limit, as both captured datasets land outside it -- the translation gap stays a tie and SMC beats PID on the angular worst case (Cohen's d up to +0.83). The PTZ plant's 100 degrees/s default sits on the cross-over, which explains a small-margin PID result previously reported in the angular setting. A complementary ablation shows raising the SMC reaching gain k costs mean tracking outside a narrow window.
★ Transmission Expansion with Large, Discrete Data Center Realization Uncertainty
Large data center projects introduce a distinct transmission planning uncertainty because of their large scale, discrete project realization nature, and strong locational impacts on the transmission network. On the one hand, conventional deterministic treatments can either overbuild the network by assuming that all candidate projects materialize; on the other, they can leave the system underprepared by prematurely screening out projects. This paper develops a project-realization-based framework for systematically modeling, planning, and evaluating transmission expansion under uncertain data center realization. Candidate data center projects are represented through alternative realization scenarios and associated probabilities, enabling different planning strategies to be modeled and quantitatively assessed under a common framework. Within this framework, numerical case studies demonstrate the effectiveness of the proposed framework and the benefits of stochastic planning approach. The results show that stochastic planning can use realization probabilities to hedge multiple plausible futures, providing a more balanced tradeoff between transmission investment and reliability exposure than deterministic approaches under correctly specified and misspecified realization probabilities.
★ Adaptive Grid-Mode-Aware Hybrid Energy Storage Control for AI Data Center Oscillation Mitigation
Oscillatory power components in AI data-center demand can produce large frequency deviations when they coincide with vulnerable grid modes. Mode-aware hybrid energy storage systems (HESSs) can selectively suppress these components, but their effectiveness degrades when the grid mode changes while the control target remains fixed. This paper proposes a closed-loop grid-mode-adaptive HESS control framework that integrates measurement-based modal identification with mode-selective control. A sliding-window N4SID estimator identifies the local grid dynamics from grid-side power and frequency measurements, and the resulting natural-frequency estimate is used to retune the modal suppression branch assigned to the supercapacitor. The framework is evaluated under a grid-mode shift from 0.50 to 1.35Hz while maintaining equal physical damping and approximately equal load excitation at both frequencies. Compared with the fixed-target controller, the adaptive method reduces the post-change grid-side modal power component from 8.78 to 2.02MW, corresponding to a 76.94% reduction, and substantially lowers the resulting frequency deviation.
comment: Manuscript is submitted to PESGM2027
★ Conflict-Aware Impedance-Sensitivity-Based Tuning for Multi-Mode Damping Enhancement
Impedance-based eigenvalue sensitivity provides a practical way to relate converter parameters to poorly damped system modes, but tuning becomes nontrivial when one parameter improves one target mode while degrading another. This paper introduces a conflict-aware tuning strategy for simultaneous damping enhancement of multiple modes. The method converts impedance-derived eigenvalue sensitivities into normalized damping-ratio sensitivities, diagnoses opposing modal tuning directions, and selects either one parameter or a coordinated pair at each iteration. A low-dimensional max--min problem maximizes the predicted damping of the worst target mode subject to parameter limits, protected-mode constraints, and implementation-specific feasibility requirements. Each accepted update is verified on the full system model and the sensitivities are recomputed. Studies on a four-bus system and a modified IEEE 14-bus system demonstrate the value of pairwise coordination, including a case in which single-parameter tuning stops before the predefined damping target is reached.
comment: Manuscript is submitted to PESGM2027
★ Frequency-Frame Covariance for Reactive-Power Scalarization
Measured voltage and current waveforms may be represented in a reference frame whose frequency differs from the physical fundamental frequency. Reactive-power readouts calculated from such frequency-shifted coordinates may change if their readout operators are not transformed consistently. This paper presents a frequency-frame covariance formulation for finite-spectrum, zero-mean periodic waveforms with complete harmonic information over one physical period. A necessary and sufficient covariance condition is derived, requiring the readout operator to transform consistently with the waveform representation. Consequently, frequency-dependent harmonic weights must be evaluated at physical harmonic orders rather than relative frame labels. In particular, the derivative/Iliovici readout equals a relative-derivative contribution plus an explicit reference contribution. The formulation also separates readout-operator errors from errors caused by changes in the physical components retained by waveform projections. Three analytical cases illustrate conditional carrier recovery for a memoryless resistor, physical-frequency weighting at relative zero in an RLC branch, and the possibility that an exact reactive-power scalar can coexist with a substantial waveform error after projection. The scope is deterministic calculation consistency in frequency-shifted coordinates rather than instrument measurement uncertainty.
★ Continuous-Discrete Dual-Domain Co-Design for Optimized Digital Control of Power Converters
Optimizing digital control for power converters typically involves designing continuous-time controllers and discretizing them via Euler, Tustin, or Generalized Bilinear Transform (GBT) methods. However, the intrinsic coupling between continuous-domain control parameters and discrete-domain implementation parameters remains underexplored, often leading to suboptimal performance. To bridge this gap, this paper first establishes a Non-Uniqueness Theorem, elucidating the coupling mechanism between dual-domain parameters. Based on this, two co-design methodologies are proposed to derive optimized parameter sets: the Explicit Frequency-Domain Analytical Method (EFDAM) and the Implicit Computer-Aided Search Method (ICASM). The EFDAM derives closed-form solutions by maximizing the crossover frequency and resonant magnitude via logarithmic frequency stability criteria, while minimizing discretization errors using Taylor-series bounds. The ICASM formulates the search for optimized parameters as a constrained multi-objective optimization problem, balancing bandwidth, resonant magnitude, and fundamental magnitude. To solve ICASM efficiently, two algorithms are introduced: the Multi-Start COBYLA (MS-COBYLA), which employs Latin Hypercube Sampling for physics-informed initialization to escape local optima, and the PSO-COBYLA hybrid algorithm, which couples global particle swarm exploration with local refinement. Experimental results on a 5-kW single-phase inverter with a Proportional-Integral-Resonant (PIR) regulator validate that the proposed methods outperform the conventional benchmark, achieving a 43.3\% reduction in the peak-to-peak current ripple measured at the zero-active-power operating point and an 11.7\%--35.9\% decrease in THDi across various harmonic profiles.
★ ZipDrift: Generating Multimodal Short-Horizon MPC Costs with One-Step Drifting Models
Short-horizon model predictive control (MPC) is fast but myopic. ZipMPC compensates by learning a context-dependent correction of the short-horizon cost that imitates a long-horizon expert. We observe that the optimal correction is often multimodal: when a slower opponent can be passed on either side, good corrections of opposite sign coexist, and the expert's choice between them hinges on tiny cost differences. Deterministic regression then averages modes or inherits inconsistent expert tie-breaking. We propose ZipDrift, which trains a one-step drifting generator over MPC cost corrections, tilted toward low closed-loop cost and learned only from cost evaluations of candidate corrections, without differentiating through the optimiser. A few generated candidates are ranked by a learned critic, and the model-based MPC then enforces dynamics and constraints. In a kinematic overtaking benchmark derived from ZipMPC, 53% of contexts are bimodal and the manual short-horizon MPC crashes in 28.5% of test episodes. With an eight-dimensional stagewise correction, ZipDrift is collision-free in all 600 test episodes and matches the best evaluated candidate, whereas an idealised ZipMPC-style regressor crashes in 4.0% (16.0% with inconsistent expert labels) and critic-ranked random candidates with the same budget in 2.0%.
comment: 6 pages, 2 figures, 4 tables. Preliminary simulation study
★ Distributed Geometric Optimization and Control of Rigid Differential-Drive Formations from Relative-Position Measurements
Distance based formation control fixes the shape of a robot team but leaves its position and orientation unconstrained. We regulate all three for differential drive robots that have only one absolute position anchor, absolute headings, body frame relative position measurements, and communication with their neighbors. We derive a two layer estimation control scheme directly from optimization. The first half of the result is a set of graph-proximal estimators that reconstruct the centroid, the anchor vector, and the centered coordinates. Exact proximal updates reach consensus in finitely many steps, and a finite number of primal dual iterations leaves a bounded reconstruction error. The controller combines proximal centroid and orientation steps with distance gradient shape regulation, and a finite rotation removes the orientation drift caused by the shape step. A discrete Lyapunov analysis gives local exponential convergence once the estimators have converged. We validate the method in ROS 2 Gazebo and in experiments with five physical robots.
comment: 11 pages, 6 figures
★ Stochastic Dynamics of Generative AI: From Autoregressive Models to Interacting LLM Networks and Neural SDEs
This paper develops a stochastic dynamical-systems perspective on three classes of generative AI models. First, we formulate large language models (LLMs) as high-dimensional nonlinear autoregressive models with attention-based dependencies. This formulation connects pretraining by conditional maximum likelihood to sequential token generation and provides a probabilistic interpretation of LLM operation. Second, we consider networks of interacting LLMs and characterize their collective behavior through mean-field ordinary differential equations. These dynamics provide a framework for understanding emergent preferential attachment and persistent structural biases, including type-dependent glass-ceiling effects. Third, we consider neural stochastic differential equations (SDEs) as continuous-time generative models for stochastic trajectories, including those observed at irregular time intervals. We discuss their training with particular emphasis on Wiener-space cubature methods based on rough-path analysis as an efficient alternative to Monte Carlo estimation. Together, the three settings address probabilistic prediction, information exchange and emergent behavior in interacting networks, and path-space generative modeling.
♻ ★ Game-theoretic Regulated Decentralized Coordination for Airspace Sector Overload Mitigation
Decentralized air traffic management systems offer a scalable alternative to centralized control, but often assume high levels of cooperation. In practice, such assumptions frequently break down since agents, each representing an airspace sector, may operate independently and prioritize local objectives. We address the problem of sector overload in decentralized air traffic management by proposing a regulated decentralized protocol that models self-interested behaviors based on best response dynamics. Each agent adjusts the departure times of its assigned flights to reduce its own congestion, without requiring centralized joint optimization. A tunable cooperativeness factor models the degree to which each agent accounts for overload in other sectors, while a minimal admissibility rule prevents local updates from creating new overloads. We show that this regulatory restriction is sufficient to guarantee finite-time convergence of best response dynamics to a pure Nash equilibrium under partial cooperation. In addition, we identify a sufficient condition under which an overload-free solution corresponds to a global minimizer of the potential function. Numerical experiments using one month of European flight data demonstrate that the proposed algorithm substantially reduces overload even with only minimal cooperation, achieving performance close to a centralized global mixed-integer optimizer and exceeding a centralized first-come-first-served baseline while requiring substantially less computation than centralized optimization. We further evaluate departure delay and fairness to quantify the operational cost of overload mitigation.
♻ ★ Provably Safe, Yet Scalable Reinforcement Learning
Safe reinforcement learning (RL) aims to learn policies that optimize rewards while satisfying constraints. Predominant approaches rely on soft-constrained policy optimization, which has achieved empirical success but does not provide formal safety guarantees for the learned policy. In contrast, methods with strict guarantees typically rely on explicit certificate functions, whose construction requires the direct synthesis and verification of control-invariant sets, a process that scales poorly with state dimension and often yields overly conservative behavior. In this paper, we present the Provably Safe, yet Scalable RL (PS2-RL) framework, a novel two-phase architecture for learning provably safe policies in a scalable manner, designed to overcome the key bottlenecks of prior methods. Rather than explicitly computing invariant sets, PS2-RL leverages a learned backup policy to forward-integrate the system dynamics, generating an implicit control-invariant set online. In the first phase, the backup policy is trained with the safe-arrival value function to achieve rapid, safe recovery to the certified base set for invariant-set construction. In the second phase, an RL policy is trained end-to-end through a differentiable projection layer that strictly enforces the safety guarantees induced by the learned backup policy. By maximizing the volume of the implicit control-invariant set in the first phase, the resulting PS2 policy from the second phase is performant and scalable, while maintaining provable safety. Crucially, PS2-RL imposes no restrictions on the underlying RL algorithm and can be plugged into any existing training pipeline. We establish theoretical guarantees for the proposed framework and evaluate it on robotic control tasks with state dimensions up to 10, a regime in which prior provably safe RL methods struggle or become impractical.
♻ ★ Koopman Generator Decomposition for Port-Hamiltonian Systems
Port-Hamiltonian energy balance and Koopman approximation describe different aspects of nonlinear dynamics. This paper distinguishes exact observable identities, measure-dependent Galerkin projections, constrained surrogate identification, and certification of a controller on the nonlinear plant. Conservative weak skewness requires compatible measure and boundary conditions. Dissipative positivity belongs to the metric bracket; it does not generally make the projected first-order dissipative derivation positive. We give an explicit fixed-metric compatibility condition and a constant--Hamiltonian dictionary obstruction, and state the assumptions under which surrogate certificates imply plant bounds. A reproduced pendulum experiment illustrates the practical gap: its learned controller admits an exact shaped-energy certificate on the nonlinear plant, although its fitted quadratic storage is not a plant Lyapunov function and its archived residual-norm diagnostic is uninformative. These conclusions use the specified plant structure; they are not generic finite-data guarantees. The bracket, projection, and Lyapunov arguments are positioned relative to existing structured identification and residual-aware control methods.
comment: 23 pages, 3 figures. Corrected and updated references; clarified mathematical assumptions and the distinction between surrogate and nonlinear-plant guarantees; expanded technical discussion and examples
♻ ★ ISAC-Assisted Proactive Handover in mmWave Networks: Value of Foresight and Localization Requirements
Millimeter-wave (mmWave) links can lose tens of decibels within milliseconds when a person or vehicle crosses the line of sight, and reactive handover recovers only after the outage has begun. Integrated sensing and communication (ISAC) promises to let the network see blockers coming and move users to another cell in advance. This paper asks how much such foresight is worth relative to the best reactive handover, and what sensing accuracy is needed to harvest it. We formulate cost-aware cell selection with knowledge of the future signal-to- noise ratio as a sequential decision problem, define the value of foresight through a policy-independent bound, and develop an ISAC-assisted handover framework in which a radio unit senses blockers with its own communication waveform and a simulated Open RAN xApp plans handovers. Ray-traced experiments in a 28 GHz street canyon, reported on held-out scenarios, show that, with a short look-ahead, perfect foresight closes most of the gap between service-aware (Layer-3) reactive handover and an instantaneous oracle. Harvesting this value, however, requires the user and nearby blockers to be localized to about a decimeter, well beyond what our radar tracker and meter- level user positioning provide; with realistic errors, even an uncertainty-aware planner largely defers to reactive handover.
♻ ★ Welfarist Control Design - How to fulfill the societal mandate in multi-agent control?
At the core of most socio-technical systems lies a scarce resource that is allocated among agents: highway lanes, public transit, road space, water rights, energy access, grid capacity, user attention, pollution rights, etc. With further automation of the underlying allocation processes, control engineers are increasingly tasked with making decisive assumptions regarding what society wants. In practice to date, design choices are largely driven by industry norms and conventions rather than a result of conscientiously responsible and ethical design. In this paper, we look at tools available to control engineers to design systems in a more principled manner in order to match the societal mandate. We consider three control design paradigms: online feedback optimization, control of Markov decision processes, and model predictive control. Beginning with aggregating individual agents' preferences into control design objectives, subsequently ensuring and certifying the fulfillment of those specifications, we argue that the feedback nature of control systems enables appropriate allocation of the shared resources in ways hitherto unparalleled.
♻ ★ Reliably Learn to Trim Multiparametric Quadratic Programs via Constraint Removal
In a wide range of applications, we are required to rapidly solve a sequence of convex multiparametric quadratic programs (mp-QPs) on resource-limited hardwares. This is a nontrivial task and has been an active topic for decades in control and optimization communities. Observe that the main computational cost of existing solution algorithms lies in addressing many linear inequality constraints, though their majority are redundant and removing them will not change the optimal solution. This work learns from the results of previously solved mp-QP(s), based on which we propose novel methods to reliably trim (unsolved) mp-QPs via constraint removal, and the trimmed mp-QPs can be much cheaper to solve. Then, we extend to trim mp-QPs of model predictive control (MPC) whose parameter vectors are sampled from linear systems. Importantly, both online and offline solved mp-QPs can be utilized to adaptively trim mp-QPs in the closed-loop system. We show that the number of linear inequalities in the trimmed mp-QP of MPC decreases to zero in a finite timestep, which also can be reduced by increasing offline computation. Finally, simulations are performed to demonstrate the efficiency of our trimming method in removing redundant constraints.
comment: Accepted by IEEE Transactions on Automatic Control
♻ ★ A Critical Review of Grid Integration Challenges and Flexibility Opportunities for Large-Scale Data Centers
The rapid proliferation of cloud computing and artificial intelligence workloads is fundamentally reshaping the magnitude, spatial concentration, and electrical characteristics of data center power demand. Data centers differ from conventional commercial loads because they combine high rack densities, converter-dominated interfaces, thermally coupled cooling, uninterruptible power supplies, programmable workloads, and increasingly complex behind-the-meter generation and storage within a single facility. This review examines their grid integration from utility and sustainable-energy perspectives across resource planning, large-load interconnection, distribution feeders, market operation, real-time dynamics, power quality, and asset management. Particular attention is given to distribution-level issues that receive limited treatment in broader energy reviews: bidirectional operation on renewable-rich feeders, harmonic propagation and resonance, ride-through and reconnection, transformer derating, cable ampacity, neutral-conductor heating, protection, and measurement. Static, composite, physics-based dynamic, calibrated, impedance, electromagnetic transient, and hybrid load models are compared according to study purpose and data requirements, and utility coordination platforms, uninterruptible power supply and battery control, workload flexibility, renewable co-location, cogeneration, fuel cells, thermal storage, and clean firm supply are assessed. The central finding is that requested peak demand and annual energy matching alone cannot establish reliable or sustainable integration. Credible assessment must consider hourly and locational supply, network deliverability, marginal emissions, lifecycle and water impacts, backup obligations, rebound, asset health, and verified operating capability.
♻ ★ An exact column generation algorithm for load balancing in capacity sharing networks
Capacity sharing networks are typical heterogeneous communication networks widely applied in information and communications technology (ICT) field. In such networks, resources like bandwidth, spectrum, computation and storage are shared among various communication services. Meanwhile, the issue of network congestion is always a prominent challenge. To handle network congestion essentially needs to solve the load balancing of networks. In this paper, for capacity sharing networks, we formulate their load balancing problem as a maximum multi-commodity flow problem. For such a problem, always a large-scale linear programming, the column generation algorithm is a commonly used and crucial method to solve it. In each iteration, this algorithm involves solving a linear programming subproblem and determining whether to terminate or generate a new column for inclusion in the subproblem. This iterative procedure of solving and checking continues throughout the algorithm. Nevertheless, since the checking subproblem is NP-hard, its solution significantly impacts the overall efficiency of the algorithm. In this paper, we innovatively convert the checking subproblem into a single-constrained shortest path (SCSP) subproblem. By exactly solving the SCSP subproblem, we can obtain the optimal solution to the checking subproblem with same or less computing time. Experimental results demonstrate that our algorithm achieves computational efficiency comparable to heuristic algorithms while outperforming other state-of-the-art algorithms by at least an order of magnitude.
comment: We request withdrawal due to errors in this preprint. Algorithm 1, lines 22 and 25, omit transmission-cost coefficients, undermining the stopping test and finite-termination guarantee as written. Section V-B2 incorrectly claims optimality for ViLBaS, contradicting Table IV. These algorithmic and interpretive claims require correction
♻ ★ SPIRAL-PO: Symbolic Identification of Partially Observed Nonlinear Dynamics with Application to Rotating Machinery
Many engineering systems contain states that cannot be measured directly, yet these hidden states act on the measured outputs. We present SPIRAL-PO (Symbolic Physics-Informed Residual Augmentation Loop - Partially Observed), a framework for identifying an interpretable observable-space closure: a model, written in the measured coordinates, of how the unmeasured states act on the measured dynamics. A physics seed is augmented by candidate terms from a mechanism-labeled library; terms are selected by statistical tests designed for numerically differentiated, serially correlated data, using a weak (integral) formulation, and are checked on independent records. For an idealized regression we give sufficient conditions for recovering the closure coefficients, a rank condition under which no estimator can recover them, and a per-term detection bound; these results apply only to systems that admit an instantaneous observable closure. On a simulated four-degree-of-freedom vertical rotor with cubic bearing stiffness, in which only the lateral displacements are measured, the method recovers the gyroscopic and cubic-stiffness signatures of the hidden tilt dynamics in every noise realization at 2% noise; with a mechanism removed it never admits the cubic term and admits the gyroscopic term in 15-25% of realizations, above the nominal level, because the closure is only partly instantaneous. A residual network and a quasi-random projection stage add no measurable benefit, and pointwise tests that ignore the serial correlation of differentiated data admit absent mechanisms more often. On a laboratory rotor with simultaneous x,y measurement and an order-tracked speed, the method detects only a speed-dependent stiffening; gyroscopic and cubic terms obtained with an assumed constant-deceleration speed are consistent with the speed error, which produces such terms in simulation.
comment: Revised version, under review at Nonlinear Dynamics. 32 pages, 6 figures, 14 tables. Code and data: https://doi.org/10.5281/zenodo.23705395
♻ ★ Discovering Phase Space Structure in Learned Hamiltonian Systems NeurIPS 2026
Mechanics-inspired neural networks provide physically structured models of dynamical systems, but typically assume access to the coordinates in which the underlying dynamics are expressed. While recent generative approaches relax this assumption, they remain limited in their ability to jointly model different dynamical regimes and discover interpretable latent structure. We introduce Structured Phase Space GAN (SPS-GAN), a conditional generative framework that learns mechanical dynamics and discovers latent phase-space structure from Cartesian trajectories or video observations. We achieve this by combining adversarial generation with port-Hamiltonian latent dynamics and extending cyclic-coordinate regularization to account for dissipation and forcing. This formulation supports conservative, dissipative, and forced systems while encouraging compact latent representations from which active degrees of freedom can be identified. Experiments on simulated and real-world systems demonstrate improved trajectory accuracy and video generation quality over competitive baselines, while identifying active latent dimensions consistent with the expected effective degrees of freedom and recovering physically interpretable coordinates across observation modalities and dynamical regimes.
comment: Accept to the 40th Annual Conference on Neural Information Processing Systems (NeurIPS 2026)
♻ ★ Safety-Critical Control via Recurrent Tracking Functions
This paper addresses the challenge of synthesizing safety-critical controllers for high-order nonlinear systems, where constructing valid Control Barrier Functions (CBFs) remains computationally intractable. Leveraging layered control, we design CBFs in reduced-order models (RoMs) while regulating full-order models' (FoMs) dynamics at the same time. Traditional Lyapunov tracking functions are required to decrease monotonically, and systematic synthesis methods for such functions exist only for fully-actuated systems. To overcome this limitation, we introduce Recurrent Tracking Functions (RTFs), which replace the monotonic decay requirement with a weaker finite-time recurrence condition. This relaxation permits transient deviations of tracking errors while ensuring safety. By integrating CBFs for RoMs with RTFs, we construct recurrent CBFs (RCBFs) whose zero-superlevel set is control $τ$-recurrent, and guarantee safety for all initial states in such a set when RTFs are satisfied. We establish theoretical safety guarantees and validate the approach through a proof-of-concept numerical experiment, demonstrating RTFs' effectiveness and the safety of FoMs.
comment: 12 Pages, 2 Figures
♻ ★ On-Line Policy Iteration with Trajectory-Driven Policy Generation
We consider deterministic finite-horizon optimal control problems with a fixed initial state. We introduce an on-line policy iteration method, which, starting from a given policy, however obtained, generates a sequence of cost-improving policies and corresponding trajectories. Each policy produces a trajectory, which is used in turn to generate data for training the next policy. The method is motivated by problems that are repeatedly solved starting from the same initial state, including discrete optimization and path planning for repetitive tasks. For such problems, the method is fast enough to be used on-line. Under a natural consistency condition, we show that the sequence of costs of the generated policies is monotonically improving for the given initial state (but not necessarily for other states). We illustrate our results with computational studies from combinatorial optimization and 3-dimensional path planning for drones and a robot arm in the presence of obstacles. We also discuss briefly a stochastic counterpart of our algorithm. Our proposed framework combines elements of rollout and policy iteration with flexible trajectory-based policy representations, and applies to problems involving a single as well as multiple decision makers. It also provides a principled way to train neural network-based policies using trajectory data, while preserving monotonic cost improvement.
♻ ★ Energy Efficiency Maximization in RSMA-Based LEO Satellite Networks Assisted by UAV-Mounted BD-Active RIS: A Hybrid DRL Strategy
This paper proposes a novel non-terrestrial communication architecture that integrates Rate-Splitting Multiple Access (RSMA) with a UAV-mounted Beyond-Diagonal Active Reconfigurable Intelligent Surface (BD-ARIS) under the coverage of a Low Earth Orbit (LEO) satellite. The BD-ARIS employs a group-connected structure to provide signal amplification and enhanced beamforming flexibility, while RSMA splits user messages into common and private parts to improve SE and interference management. A joint optimization of satellite beamforming, UAV placement, power allocation, and RIS configuration is formulated to maximize the system overall energy efficiency. To solve the resulting non-convex, high-dimensional problem, we first develop a model-based iterative algorithm using Dinkelbach transform, weighted minimum mean-square error reformulation, successive convex approximation, and sequential quadratic programming. We then propose two deep reinforcement learning (DRL) approaches: a standard trust-region policy optimization (TRPO) baseline and a novel Dinkelbach-Guided TRPO (DG-TRPO) scheme. DG-TRPO is initialized via supervised pretraining on Dinkelbach solutions and subsequently fine-tuned with TRPO in noisy, time-varying environments. Simulation results show that DG-TRPO closely matches the model-based benchmark while providing sub-millisecond inference, making it suitable for real-time LEO operation. It also exhibits superior robustness to CSI uncertainty and satellite mobility, and the RSMA BD-ARIS framework significantly outperforms benchmarks employing NOMA or conventional RIS designs. These results establish the proposed architecture and DG-TRPO algorithm as a scalable, energy-efficient solution for future 6G massive IoT non-terrestrial networks.
♻ ★ Symplectic Inductive Bias for Data-Driven Target Reachability in Hamiltonian Systems
Inductive bias refers to restrictions on the hypothesis class that enable a learning method to generalize effectively from limited data. A canonical example in control is linearity, which underpins low sample-complexity guarantees for stabilization and optimal control. For general nonlinear dynamics, by contrast, guarantees often rely on smoothness assumptions (e.g., Lipschitz continuity) which, when combined with covering arguments, can lead to data requirements that grow exponentially with the ambient dimension. In this paper we argue that data-efficient nonlinear control demands exploiting inductive bias embedded in nature itself, namely, structure imposed by physical laws. Focusing on Hamiltonian systems, we leverage symplectic geometry and intrinsic recurrence on energy level sets to solve target reachability problems. Our approach combines the recurrence property with a recently proposed class of policies, called chain policies, which composes locally certified trajectory segments extracted from demonstrations to achieve target reachability. We provide sufficient conditions for reachability under this construction and show that the resulting data requirements depend on explicit geometric and recurrence properties of the Hamiltonian rather than the state dimension.
comment: 12 pages
♻ ★ Leveraging Offline Data from Similar Systems for Online Linear Quadratic Control
``Sim2real gap", in which the system learned in simulations is not the exact representation of the real system, can lead to loss of stability and performance when controllers learned using data from the simulated system are used on the real system. In this work, we address this challenge in the linear quadratic regulator (LQR) setting. Specifically, we consider an LQR problem for a system with unknown system matrices. Along with the state-action pairs from the system to be controlled, a trajectory of length $S$ of state-action pairs from a different unknown system is available. Our proposed algorithm is constructed upon Thompson sampling and utilizes the mean as well as the uncertainty of the dynamics of the system from which the trajectory of length $S$ is obtained. We establish that the algorithm achieves $\tilde{\mathcal{O}}({f(S,M_δ)\sqrt{T/S}})$ Bayes regret after $T$ time steps, where $M_δ$ characterizes the \emph{dissimilarity} between the two systems and $f(S,M_δ)$ is a function of $S$ and $M_δ$. When $M_δ$ is sufficiently small, the proposed algorithm achieves $\tilde{\mathcal{O}}({\sqrt{T/S}})$ Bayes regret and outperforms a naive strategy which does not utilize the available trajectory.
♻ ★ A Hybrid Framework for Explainable Aviation Risk: Causal Chain Extraction from Safety Reports with Physics-Based Failure Quantification and Interventional Analysis
Aviation authorities hold decades of accident reports whose narratives record how investigators reconstructed each accident. Computational work on these archives has mostly sorted reports into fixed taxonomies. In this work, we propose a hybrid framework in which a text-based corpus identifies the leading factors of a failure mode. Physics-based models then quantify how likely that failure is at a stated operating point. Ordered causal chains extracted from 56{,}202 American accident reports with a local large language model aggregate into a causation knowledge graph. On an evaluation sample of 2{,}000 reports, the chains recover 90.8\% of investigator-flagged causes. Each physical factor whose drivers the record carries receives an occurrence model calibrated to the accident population. A leaky noisy-OR fitted to the corpus combines the factors into a failure mode probability that supports intervention. For engine power loss, the carburetor icing model separates 1{,}205 icing accidents from other accidents using temperature and dewpoint alone, reaching 0.61 in area under the receiver operating characteristic curve against a density ratio ceiling of 0.64. Routine weather hours give no valid control, because flying concentrates in fair weather. On later years the physics adds 0.03 to that area for occurrence and nothing for severity, so its value lies in interpretability. On four named accidents, abduction gives a probability of necessity of 0.29 to 0.50 for the heat omission, under monotonicity. The Fast Causal Inference algorithm does not license the largest backdoor adjustment on the learned networks.
comment: 14 pages, 3 figures, 4 tables. v2 prints the carburetor icing scenario values to four digits, corrects wording and table formatting, and removes one author from the author list at his request. This work is going to be submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible
Robotics
★ DreamX: Video Diffusion Model is a Universal Latent Action Learner
Learning a universal action representation from video requires modeling dynamics that are consistent across different executions of the same action yet invariant to changes in embodiment and appearance. Existing latent action models infer actions from visual transitions, but their representations often entangle the underlying action with irrelevant visual variations, limiting transfer across embodiments. We introduce DreamX, which repurposes a pretrained video diffusion model as a universal latent action learner. Our approach extracts compact latent actions from the video model's rich spatiotemporal features, compressing each latent video frame by $\boldsymbol{300\times}$ ($\boldsymbol{1{,}200 \rightarrow 4}$ tokens). In addition, by leveraging the video model's generative prior, these latent actions capture action-relevant dynamics disentangled from visual appearance. The resulting latent space serves as a shared action interface across embodiments (human and robot), environments, and action parameterizations. On LARY-Bench, DreamX achieves state-of-the-art performance among latent action models. Moreover, actions encoded in one setting remain effective in another: the latent space supports any-to-any video transfer and zero-shot action transfer across diverse embodiments and environments. Furthermore, this shared interface also enables policy learning in a compact latent action space, reducing the computational cost of video-diffusion-based policies.
comment: Project Page: https://cvlab-kaist.github.io/DreamX/
★ PolicyIK: High-Precision Inverse Kinematics for Robot Arms with Reinforcement Learning
Inverse kinematics (IK) converts end-effector pose commands into joint motion, and many vision-language-action (VLA) policies rely on it to act on a given arm and to transfer across embodiments. Precise IK remains laborious: redundancy and singularities complicate the inverse mapping, execution depends on hand-tuned controllers, and training data on physical robots are costly to collect. We present PolicyIK, a goal-conditioned reinforcement learning policy for high-precision IK. It updates joint-position references from pose feedback and observes its accumulated reference relative to the measured joints, so it can retain the offset that the position controller needs at rest. Two-scale pose errors with an error-dependent action gain resolve millimeter residuals while preserving large motions. Progress and dwell rewards teach it to both reach the tolerance and stay within it. In simulation, a single attempt by the trained PPO policy keeps the mean errors over the final second within 1 mm and 0.2° on 93.5% of held-out tasks, and a deployment protocol raises success to 98.9%. When trained directly on a physical arm with an off-policy RL learner, PolicyIK achieves higher reaching and holding success than all four classical IK controllers under the same time budgets. In simulated pick-and-place, PolicyIK executes the end-effector commands of the $π_{0.5}$ VLA policy with 95.7% success. These results show that reinforcement learning can provide high-precision inverse kinematics for robot arms. Code is available at https://github.com/Tinaliu0123/PolicyIK.
★ GUILD: Guided Iterative Data Collection for Imitation Learning
Imitation learning policies in robotics are trained on datasets collected through costly real-world teleoperation. Often, the data-collection process is unguided, relying on the teleoperator's intuition, resulting in data that contributes little to overall policy performance. Recent work has shown that curating such unguided datasets can substantially improve policy performance, but these approaches operate by filtering existing datasets after collection and therefore do not prevent the acquisition of uninformative data. In this work, we propose GUILD, a novel Guided Data Collection (GDC) method that actively reasons about which data should be collected to maximize downstream policy performance. We introduce an iterative framework that alternates between 1) task proposal and data collection and 2) data evaluation to close the loop. For task proposal, we leverage semantic priors from vision-language models to identify relevant environments, object configurations, and goals for data collection. However, since semantic relevance does not imply policy improvement, we perform data evaluation. By leveraging predictive data attribution we approximate how the collected trajectories influence policy performance on a target deployment setting. This signal is then used as feedback to guide subsequent data collection, closing the data collection loop. We evaluate our approach in simulation and real robot settings, demonstrating that guided data collection substantially improves sample efficiency and deployment performance compared to passive data collection strategies, multiple baselines and experienced roboticists. More information at https://robin-lab.cs.utexas.edu/guild2026/
★ Visibility-Aware Reference Generation for Benchmarking Visual 3D Reconstruction in Vegetation-Rich Environments
Reliable evaluation of visual 3D reconstruction and depth estimation in vegetation-rich outdoor environments is limited by the scarcity of geometric ground-truth data and the cost of complete scanner surveys. We present a LiDARbased reference-generation pipeline for evaluating the zero-shot performance of feed-forward visual-geometry models alongside conventional structure-from-motion (SfM) methods. The pipeline reuses camera images, LiDAR scans, and reference poses from existing multi-sensor localization datasets to generate per-view depth maps and visible reference point clouds. To address visibility errors in accumulated LiDAR scans, we introduce the LargeDepth-Gap Visibility Score (LDGVS) for point-wise visibility filtering and automatic scale selection in Hidden Point Removal(HPR). Experiments on TartanAirV2 and TartanGround show that, compared with Poisson surface reconstruction, which is an approach suited to completed scan surveys, the combined HPR and LDGVS filtering retains approximately three times the point density at comparable absolute relative error. We also apply the pipeline to the BotanicGarden dataset to systematically compare depth estimation and 3D reconstruction methods. Feedforward methods produce dense depth predictions efficiently, but their 3D reconstructions can contain duplicated structures. Conventional SfM generally produces more coherent geometry, but its reconstructions remain sparse and more prone to failure in challenging vegetated scenes. The benchmark toolkit and datasets will be released.
comment: 20 pages, 12 figures
★ PyRUA-DataFarm: LLM Agent as a Cost-Efficient Teleoperator for Real-Ready Robot Data
Teleoperation remains the default way to collect robot demonstrations, but every episode costs robot and operator time, and every new task starts from scratch. We ask whether a coding agent can take over the teleoperator's job. In our DataFarm, the agent receives a data request and one calibration episode of the robot in the target scene and rebuilds the task in simulation: it writes the scene, an expert controller, and a success judge. The simulator supplies exact actions, and a video model conditioned on real images renders each rollout as photographic video, generating all camera views together consistently. The output is real-ready data: synchronized camera views labelled with the robot's actions, on which a policy can be trained directly and deployed on the real robot. Independent judges certify the expert and every rendered episode before admission. On five real-robot tasks, policies trained only on farm data match policies trained on as many teleoperated episodes (93 vs. 84 successes in 100 trials), and under an unseen tablecloth and coloured light they keep 87 successes where the teleoperation-trained policies fall to 46. Adapting a task is fully automatic and costs \$21 of API usage on average, paid once per task, plus \$5 for a render contract; every further episode costs \$0.80 on average for simulation, rendering, and review, with no robot or operator. One hour of three-view data for a task costs \$107 on average, these costs included, slightly below a quote of \$117 for teleoperation in China and below \$290 in labour alone in North America.
comment: 22 pages, 9 figures, 9 tables. Project page: https://dagroup-pku.github.io/PyRUA-Lean/DataFarm Code: https://github.com/DAGroup-PKU/PyRUA-Lean/tree/DataFarm
★ PACT: Pose-Robust Actuation Co-Design for Tendon-Driven Robotic Hands
Humanoid robotic hands must provide sufficient force, speed, and backdrivability within a human-scale envelope. Tendon-driven transmissions address this challenge by relocating actuators to the palm and forearm, but routing paths, moment arms, and capstan dimensions jointly determine both force capability and joint speed, so selecting them sequentially satisfies one requirement at the expense of the other. This paper presents a pose-robust actuation co-design for tendon-driven robot hands (PACT) framework that optimizes tendon routing, moment arms, and capstan dimensions under pose-dependent force requirements and drivetrain-bandwidth constraints. A feasible-force formulation enforces actuator bounds and equilibrium at all joints, while a max-min objective maximizes task-force coverage at the weakest pose. Applied to a four-degree-of-freedom finger, the framework produces an asymmetric N + 1 tendon topology using quasi-direct-drive actuators. Across eight poses, the design achieves task-force coverage of 0.88-1.00 and a predicted 5 Hz minimum isolated-joint flexion/extension bandwidth within a 70th-percentile human-size envelope. Prototype measurements deviate from predictions by up to 24.5%, primarily due to unmodeled transmission losses. These results indicate that co-designing routing with the drivetrain can meet pose-dependent force requirements without relaxing the bandwidth constraint, paving the way for human-sized robotic hands with human-level performance.
★ GSBR: Geometric Support Behavior Regularization for Online RL
Regularizing online reinforcement learning (RL) toward a pretrained base policy can improve sample efficiency and learning stability, but regularization that is too restrictive limits how much the policy can improve. Base policies -- typically diffusion or flow-matching models trained by imitation learning on large, diverse datasets -- capture many behaviors that are reasonable but suboptimal. Existing regularizers using such base policies are too restrictive in one of two ways: behavioral cloning regularizers pull the actor toward the full base-policy distribution, penalizing it for discarding suboptimal modes, while hard support constraints forbid improvements that lie just outside the base policy's support. We propose Geometric Support Behavior Regularization (GSBR), which penalizes actions by their distance from the base policy's support. Actions within the support incur no penalty, so the actor can concentrate on high-value modes; outside the support, where value estimates are less reliable, the penalty grows smoothly with distance while still permitting improvement when the value gained outweighs the penalty. GSBR estimates this distance from a finite set of base-policy samples, so it requires only sample access to the base policy. Across 19 tasks on four robotic benchmarks, GSBR raises the average success rate from 33% for the base policy to 95%, achieving the highest average success rate and throughput among all compared methods, with about 23% higher normalized throughput than the next-best RL method. Project website and open-source code at https://charlesxu0124.github.io/gsbr/ .
★ Should the Action Expert Train the VLM? Action-Token Supervision and Gradient Routing in Mixture-of-Transformer VLAs
Vision-language-action (VLA) models increasingly pair a pre-trained vision-language model (VLM) with a flow-matching expert that conditions its action predictions on the VLM's key-value representations. Those representations must carry information useful for control, yet the VLM was never trained for the action expert's flow-matching objective. Two signals can address this: gradients backpropagated from the expert's loss (coupling), or VLM supervision to predict discrete tokens encoding the same target actions. Prominent training recipes forgo the first, blocking expert-to-VLM gradients (insulation) to protect the VLM's existing capabilities, while also supervising it on vision-language data and action tokens. Because vision-language co-training also aims to preserve those capabilities, the additional protection provided by insulation is unclear. At the same time, insulation removes a direct signal about which features continuous action prediction requires. On the other hand, action-token supervision offers an alternative way to learn action-relevant representations, which may explain why insulation succeeds in practice. We disentangle these factors in autonomous driving and manipulation by independently varying action-token supervision and whether the expert's loss updates the VLM. In open-loop driving, insulation is competitive with coupling when action tokens are supervised; without them, coupling reduces trajectory-prediction error by roughly 25%. When both components are trained jointly from the outset, coupling increases closed-loop driving route progress by approximately 17% with action-token supervision and 40% without it, and similarly improves robot manipulation performance under both settings. These findings show that the value of gradient insulation depends on the VLM's action supervision and the evaluation setting, challenging insulation as a default VLA training recipe.
★ Reality and Perceptual Gaps in Expressive Swarm Robotics
Moving robot swarms from simulation into the physical world can be difficult because small discrepancies in hardware, timing and communication can cascade across interacting robots and alter group behaviour. This challenge is known as the reality gap. When swarms operate in human environments, a second challenge emerges: a perceptual gap between what the swarm physically does and what human onlookers actually perceive. This study examines the same decentralised coordination code deployed across three robotic swarms-tabletop, aerial and human-scale mobile manipulators-during live artistic performances. We find that while the underlying algorithms executed reliably on all three platforms, physical embodiment drastically altered visible motion-distorting execution timescales by more than a factor of three and reshaping trajectory geometry. More importantly, audience surveys revealed that human interpretation does not mirror technical success: how observers attribute meaning (perceived purpose) and how they react emotionally (comfort versus threat) depend heavily on physical form. Meaning attribution and emotional responses did not vary together. Our results demonstrate that successfully transferring swarm software to physical hardware is insufficient for real-world deployment. Addressing the reality gap does not guarantee that the resulting collective motion will be readily interpreted by human observers. Swarms destined for human environments must be engineered not only for decentralised stability but also for the expressive physical dynamics through which humans interpret collective motion.
★ PadLoc: Training-Free Floor-Plan Localization by Sparse Geometric Matching WACV 2027
We propose PadLoc, a training-free localization back-end that places a moving sensor within a known floor plan. Floor plans are cheap, persistent, and robust to appearance change, but recent learned-depth methods localize against them by filtering per-frame depth through dense $(x,y,θ)$ pose grids. PadLoc keeps the swappable depth front-end and replaces the dense grid back-end with sparse geometric matching: dominant wall orientations define a small pose-candidate set, masked normalized cross-correlation scores observed free space against the plan, and a moving-pose Gaussian-mixture Bayes filter plus consensus readout decodes the trajectory. Holding each competitor's depth fixed, PadLoc matches or beats UnLoc's forward filter and the F3Loc filter at every Gibson trajectory length, and UnLoc's offline refinement from $T=35$ on; it exceeds Z-FLoc's published results with a shared MapAnything front-end. On Bonn LiLocBench, PadLoc improves pooled SR@1m over UnLoc by 23-32 pp under wheel odometry and 35-43 pp with ground-truth motion; with full-FOV LiDAR plus wheel odometry the same readout reaches 100% SR@1m at 9-10 cm median error on the static sequence. On the Hilti-Trimble Localization Challenge PadLoc localizes a camera-only fisheye rig on construction floors hundreds of metres across at 0.90-0.98 online SR@2m on three of five sessions; adding learned wall isolation, applied to every session, covers all five at 0.67-0.92 with the localizer unchanged throughout. PadLoc thus matches or outperforms dense grid back-ends while preserving sensor modularity and a compact state that does not grow with trajectory length.
comment: Accepted to WACV 2027 (Algorithms track). 22 pages: main paper (10 pages incl. references) and supplementary material (12 pages). Code: https://github.com/mbookhahn/padloc
★ RoboTryOn: Synthesizing Dexterous Robot Demonstrations from Human Videos via Virtual Try-On
Robot foundation models have shown remarkable performance in robot control, yet adapting them to a new task still relies heavily on teleoperated demonstrations. Egocentric human videos offer a scalable substitute, but overlaying rendered robots on human videos or retargeting human poses into robot actions leaves significant visual or action gaps for high-DoF robot embodiments. To address this limitation, we propose RoboTryOn, which treats human-to-robot translation as virtual try-on: try-on keeps the person and swaps the garment, RoboTryOn keeps the demonstrated motion and swaps the human for the robot. Specifically, RoboTryOn converts a human video into a video of the target robot specified by a reference image, while preserving the original task, objects, and background. To maintain visual consistency, we translate selected keyframes autoregressively and use them to guide video synthesis with motion conditioned on the human hand skeleton. Additionally, we use an inverse dynamics model (IDM) that incorporates depth features to infer action labels from synthesized videos for policy learning. Our experiments across multiple humanoid embodiments show that policies trained with our synthetic data improve the average success rate on novel tasks, including challenging bimanual manipulation, from 2% to 60%, without any real robot demonstration of those tasks. Moreover, training with our synthetic data matches the performance of policies trained solely on teleoperated demonstrations at approximately a third of the collection cost, demonstrating the cost efficiency of our approach.
comment: 29 pages, 15 figures, 13 tables. Project page: https://ryan-seungyong-lee.github.io/RoboTryOn
★ Asynchronous Robot Policies
Scaling vision-language foundation models has significantly advanced generalist robot manipulation, but the increased model scale introduces computational latency that degrades real-time physical reactivity. Existing Vision-Language-Action (VLA) architectures execute high-level semantic reasoning and action ex- ecution sequentially within the primary control loop, creating a fundamental tension between semantic-reasoning capability and robot agility. To address this, we propose Asynchronous Robot Policy (ARP), an architectural framework that decouples semantic reasoning from low-level action generation using asynchronous processes operating at independent frequencies. In ARP, a high-capacity Vision-Language Model (VLM) asynchronously updates a semantic memory at low fre- quency, while a lightweight Vision-Action Expert, conditioned on this semantic memory, generates robot actions at a much higher control frequency. Such a formulation, enables having a fast perception-action loop without compromising on the semantic capability. To overcome asynchronous memory staleness, ARP trains the vision-action expert on the semantic staleness and execution delay, with temporally mismatched observation across processes. On Franka manipulation tasks, ARP achieves an 18% higher success rate and 20% less time need for execution in quasi-static setups, while 3-5 times baseline performance in dynamic simulation environments. Furthermore, ARP enables operating the VLM at 10 times lower than its nominal frequency with no significant performance loss making it much more efficient than the baseline. These results highlight the architectural advantages of modeling robot policies as asynchronous processes, enabling real-world reactive deployment without sacrificing the scale of foundation models.
comment: For code, checkpoints and more results, check out https://arp.insait.ai
★ WayTrace: A Map-to-Language-to-Trace Hierarchy for Embodied Urban Navigation
Web services provide walking directions for nearly every city, but following them reliably over long routes remains challenging for autonomous robots. Point-goal navigation policies rely heavily on accurate GNSS, while most vision-language navigation models struggle to execute even short instructions outdoors. We present WayTrace, a hierarchical system that connects web-map directions to robot motion through language. An off-the-shelf orchestrator vision-language model reads the turn-by-turn directions, a satellite map and recent camera images, and issues one atomic language command at a time, such as "cross the road to the left". An onboard trace predictor, trained on over one million automatically labelled samples from street-level images, draws the path for this command in the image, and a reinforcement learning policy tracks it. In long-range simulated missions, WayTrace remains robust to GNSS noise and rarely leaves crosswalks or sidewalks, while point-goal baselines degrade sharply with about ten times more violations. Deployed on a wheeled-legged robot, WayTrace completes real-world missions of up to 1.37 km at about 1.1 m/s through a busy city centre, rural trails and a university campus, handling crosswalks, staircases and construction sites. Project page: https://way-trace.github.io
comment: Under Review
★ DexHOT: Learning Hand-Object Configuration Transfer for Dexterous Manipulation
Dexterous manipulation involves not only moving objects, but also arranging the hand around them to maintain useful contacts and enable subsequent actions. Yet many learning-based systems specify goals primarily through object poses or trajectories, leaving hand-object configurations to emerge implicitly. We introduce DexHOT, which formulates dexterous manipulation as hand-object configuration transfer: moving between feasible configurations that jointly specify hand articulation and object pose relative to the wrist. DexHOT learns a reusable relation-conditioned transfer primitive from synthetic feasible configuration pairs, without human reference motions, and composes long-horizon behaviors through closed-loop tracking of configuration subgoals. On DexHOTBench with 120 objects, a single policy achieves 80.5% transfer success on seen objects and 69.4% on unseen objects; on a real dexterous hand, it achieves 61.5% success over 200 real-robot transfers and 90% task success across six demonstration-guided tasks. These results show that explicit hand-object goals provide an expressive and composable interface for dexterous manipulation. More details and videos are available at https://dex-hot.github.io/.
comment: 9 pages, 4 figures
★ HLO-AMP: Hierarchical Loco-Manipulation from Object-Aware Motion Priors
Hierarchical pipelines for humanoid loco-manipulation typically pair a task-agnostic whole-body controller (WBC) with a downstream task policy. Such WBCs are often trained through manual reward shaping, which yields visibly machine-like motions; more consequentially, they are trained without the manipulated object in the scene. Object dynamics are instead approximated through randomized end-effector force disturbances, a proxy that fails to reproduce the distributed multi-contact wrench a real payload applies across the arms and torso, with the gap becoming critical near the platform's payload limit. In this work, we train the low-level (LL) controller itself as an object-aware motion prior: a single policy trained via motion imitation on retargeted motion-capture (MoCap) references that span the core motion primitives of the downstream task, with the manipulated object physically present in the scene. This single stage inherits natural human-like coordination from the references and absorbs the realistic contact wrench at its source, while exposing the same compact task-space command interface used by command-tracking WBCs. A reference-free high-level (HL) policy is then trained on top, with its action channel anchored to the empirical envelope of the low-level training data so that it generalizes the priors to a long-horizon multi-stage task while structurally preventing queries that would push the low level off-distribution. We instantiate the framework on a Unitree G1 for bimanual box pick-and-place: the same deployed policy generalizes across object masses and dimensions, initial and goal poses, and reactively recovers from intentional drops mid-carry. Video: https://youtu.be/viR_ULnRDQQ.
comment: Accepted at the Conference on Robot Learning (CoRL 2026). Video: https://youtu.be/viR_ULnRDQQ
★ LoopQ: Fair, Closed-Loop Evaluation of Quantized Vision-Language-Action Models
Post-training quantization (PTQ) for vision-language-action (VLA) models promises smaller and faster VLA deployments. However, when analyzing existing VLA quantization works, we realised that it is hard to reason about the performance of the suggested quantization method compared to existing, similar, or in many cases, simpler PTQ methods. We present LoopQ, a closed-loop benchmark that compares multiple quantization policies on the same checkpoint, layers and engine, varying only the method and the kernels. We evaluate RTN, AWQ, GPTQ, ActQuant and QuantVLA on four VLAs with more than 180,000 episodes of LIBERO and LIBERO-plus, compare against QVLA's reported results on the same layers, and benchmark whether the four public VLA-quantization releases can be reproduced. We benchmark the methods on an NVIDIA H100, an L4, and a Jetson AGX Thor. Our results show that while quantization reliably saves memory, 2-3x for real low-bit models, VLA-specific PTQ methods underperform compared to simpler generic baselines. At 4 bits, RTN, AWQ and GPTQ match or exceed ActQuant and QuantVLA, plain RTN matches the calibrated methods, and plain AWQ matches the success QVLA reports for itself. Standard LIBERO hides the losses that do exist: configurations with no significant LIBERO loss lose 1.5-4.3 points under paired LIBERO-plus perturbations. Speed is a weaker story. At batch 1 on the H100, no quantized pi0.5 or OpenVLA-OFT configuration is faster than its 16-bit model (1.06-1.58x slower on library kernels); only the L4 with int8 kernels gains (4-14%), the Thor is slower on every 4-bit configuration, and the engine alone changes latency by 2.1x. Profiling explains why: pi0.5's quantized backbone is compute-bound, while its memory-bound action expert, 63% of GPU time and 13% of the parameters, stays in 16 bits.
★ Sparse Anchors for Visual Multi-Session Registration in Changing Environments
Repeated robotic monitoring requires localization across visits to the same environment, even if the environment changes. Such changes can substantially reduce persistent visual content, and repetitive structures can lead to incorrect cross-session correspondences. In this paper, we address visual multi-session registration of a new data recording (the query session) to a previously reconstructed reference map. Under substantial visual and structural changes, cross-session localization may be possible only for parts of the trajectory. In our approach, we therefore rely on a sparse set of reliable absolute pose constraints and combine them with within-session relative-pose information to register the complete trajectory to the reference map. We first estimate absolute alignment hypotheses through visual place recognition, local feature matching, and geometric verification. We then use sequence consensus to reject hypotheses that are inconsistent with a common query-to-reference alignment and retain the remaining poses as absolute anchors. These anchors are refined through featuremetric alignment against a Gaussian feature field and combined with within-session relative-pose constraints in a robust pose-graph optimization. We evaluate our approach on 15 cross-session experiments from the challenging HILTI26 construction-site dataset, covering substantial scene changes and difficult-to-match environments. Our method achieves 94.4% recall within 1m and 100.0% within 5m, outperforming independent image localization, which peaks at 65.3% and 70.7%, respectively, as well as direct registration of independently reconstructed Structure-from-Motion geometry. Our results show that sparse, reliable absolute anchors can provide accurate and complete trajectory registration even when reliable cross-session correspondences are available only for part of the query session.
comment: 8 pages, 4 figures, 2 tables
★ CRONOS: Benchmarking Non-Episodic Reinforcement Learning for Robotic Manipulation
Reinforcement learning (RL) is promising for adapting robot policies to unstructured real-world environments. However, standard RL pipelines rely on episodic training with frequent scene resets, which is impractical for real-world deployment due to the need for substantial human intervention. While prior work has studied non-episodic RL in relatively small policies trained from scratch, its behavior when fine-tuning modern large-scale pre-trained vision-language-action (VLA) models remains underexplored. In this work, we introduce CRONOS, a simulation benchmark for RL fine-tuning of VLAs under non-episodic scene resets. CRONOS leverages high-fidelity physics simulators and adopts shared-scene multi-task settings. Using CRONOS, we show that naively fine-tuning pre-trained policies fails in non-episodic settings due to shifts in object configurations and end-effector poses; however, these challenges can be mitigated through targeted reset allocation and by addressing biases in pre-trained models. Moreover, we find that non-episodic training enhances long-horizon manipulation and improves generalization to previously unseen object configurations and task sequences. Project page: https://embodiedai-ntu.github.io/cronos/
★ Nested Field-of-View Barriers for Teleoperated Multi-Robot Relay Chains
Teleoperation of mobile robots in communication-denied environments requires maintaining a communication link while allowing the operator to maneuver freely. We propose a shared-autonomy framework in which a teleoperated leader is followed by a chain of autonomous robots that relays a line of sight to a fixed base station, a geometric proxy for the link. Connectivity is enforced geometrically through visibility-based Control Barrier Functions (CBFs), with each agent computing its control input locally through a quadratic program. A strict visibility constraint guarantees connectivity, but a follower with no independent motion objective remains stationary, stalling the leader at the boundary of the safe set. Relaxing it preserves mobility at the cost of the guarantee. Our framework resolves this conflict through nested hard-soft visibility constraints: a full field-of-view (FoV) constraint enforced strictly to guarantee connectivity and a contracted FoV constraint imposed on each follower as a relaxed behavioral objective that drives it inward before the connectivity boundary is reached. We prove feasibility of the local quadratic programs and forward invariance of the connected, collision-free set under the resulting distributed controller. The contraction parameter shapes follower behavior without affecting the guarantee, which follows from the strict constraint alone. The framework is evaluated in confined and cluttered environments on two simulated systems: a kinematic integrator and software-in-the-loop Crazyflie quadrotors. Compared with a baseline tuned to reach the goal in comparable time, the framework keeps the links far closer to the connected set and moves the followers less.
comment: 8 pages, 4 figures, 1 table. Submitted to the American Control Conference (ACC)
★ Trajectory Evaluation via Rollout-Free World Model for End-to-end Autonomous Driving
Although existing candidate trajectory evaluators have substantially improved end-to-end planning accuracy, their underlying mechanism remains suboptimal. Model-based evaluators typically make scoring future-aware by explicitly predicting candidate-conditioned future states over the planning horizon, which we term rollout. However, rollout incurs high inference cost and accumulates prediction errors. We argue that planning does not require reconstructing a full future scene for each candidate, but only future-relevant information for reliable comparison. We therefore propose RFWorld, a Rollout-Free World model for end-to-end trajectory evaluation. RFWorld constructs a shared scene memory and shapes it into a future-readable representation via lightweight training-time temporal readout with semantic supervision, keeping training overhead low. At inference time, candidate trajectories directly query the shared memory via sparse spatial readout for scoring, reducing computation by roughly the planning horizon times the number of candidate trajectories, while avoiding recursive rollout errors. Experiments on NAVSIM and Bench2Drive show that RFWorld learns a more planning-aligned world representation and achieves state-of-the-art performance.
★ Do Agentic Policies Need to Look at Every Step? Geometric Abstraction for Reasoning and Action
Recent multimodal language models can reason directly about robot scenes, but it is unclear whether they need to repeatedly reinterpret raw visual observations at every step of an interaction. We propose shifting routine spatial reasoning from images to text and symbols through a 3D geometric abstraction that is refreshed locally from observations and read directly by the language models. Specifically, our interface represents scene geometry via typed, evidence-aware symbolic references to object parts, grounds them from calibrated RGB-D, and returns compact textual checkpoints with metric residuals after each action. Symbolic task structure can thus be reused across decisions while metric values are rebound to the current scene, with direct visual reasoning invoked only on demand. Across offline policy synthesis and continuous online execution, reasoning over grounded text and symbols improves multi-step manipulation completion while substantially reducing repeated multimodal queries, positioning explicit geometric state as an effective substrate for agentic robot policies.
comment: Accepted to the CoRL 2026 Workshop on Agentic Robotics. 13 pages, 8 figures. Project page: https://geometric-abstraction.github.io
★ DRAP: Dynamics-Risk Adaptive Plasticity for Continual Learning in Physical AI
Continual learning in physical AI differs from conventional continual learning because small per-step policy errors can accumulate over a trajectory. Each deviation may move the agent outside the training distribution, increase subsequent prediction errors, and ultimately cause closed-loop task failure. This work introduces Dynamics-Risk Adaptive Plasticity (DRAP), a replay-free method that estimates the historical importance of adaptable model tensors from sample-level action gradients weighted by local trajectory-amplification risk. A ridge-regularized local linear model is fitted to expert state-action transitions, and the largest singular value of its state-transition matrix provides a policy-independent, Lyapunov-inspired estimate of one-step perturbation amplification. After each task, risk-weighted squared gradients are accumulated in a tensor-level importance memory. During subsequent tasks, DRAP uses normalized cumulative importance to scale the effective AdamW learning rate of each tensor, immediately reducing the plasticity of components that supported previous behavior at dynamically sensitive trajectory points. We instantiate DRAP on a fixed KappaTune-selected subnetwork of OpenVLA and evaluate replay-free sequential adaptation across the LIBERO Spatial, Object, and Goal suites. Compared with standard AdamW fine-tuning and EWC, the results show substantially improved retention of previously learned behaviors while preserving strong new-suite acquisition.
★ Haptic Shared Control for Multi-Robot Systems
Multi-agent systems offer significant advantages in terms of flexibility and redundancy, enabling the efficient distribution of complex tasks among multiple robots. However, their deployment requires advanced control strategies to ensure optimal cooperation between autonomous agents and human operators. We propose a modification of the global connectivity maintenance algorithm to incorporate measurements taken in agent's local frame, using sensors with limited fields of view. This algorithm, responsible for the autonomous behavior of the multi-robot system, is combined with different adaptive supervision strategies, allowing the user to exert control at varying scales over the entire group. (i) Single Agent Control (S): The operator directly controls a single robot, (ii) Pinning Control (P): The operator imposes constraints on multiple robots to influence the group, (iii) Centroid Control (C): The operator acts on the system's center of mass to steer the entire fleet. In conjunction with the connectivity maintenance algorithm, these methods ensure compliance with inter-agent constraints while providing haptic feedback to the user regarding the system's connectivity state. The effectiveness of these approaches was evaluated through a user study, comparing performance with and without haptic feedback across multiple scenarios tailored to the different control modes.
comment: PhD thesis, Université de Rennes (IRISA UMR 6074 / Inria Rennes), defended 30 April 2025. 134 pages, 38 figures
★ HiLo-WAM: Decoupling Long-Horizon Task Evolution and Local Dynamics in World Action Models
World Action Models (WAMs) improve robotic manipulation by jointly modeling visual dynamics and robot actions. However, most existing WAMs model dense short-horizon visual dynamics, capturing local interactions but providing limited long-range task context. Extending prediction horizons increases uncertainty and computation while entangling task-level evolution with fine-grained local dynamics, obscuring their dependencies and distinct roles in action generation. We propose HiLo-WAM, which decouples long-horizon task evolution from local visual dynamics while jointly modeling their interaction with robot actions. A sparse long-horizon visual stream captures historical task context and future evolution, while a dense local dynamics visual stream models short-horizon robot-object interactions. The two streams operate at different temporal scales within a unified dual-DiT architecture and guide actions through structured cross-stream attention. Progressive inference further reduces unnecessary computation through stream-wise partial denoising, joint denoising, and dynamic skipping. Experiments demonstrate strong long-horizon manipulation performance: HiLo-WAM achieves a 93.70% average success rate on RoboTwin 2.0 and a 17.43 progress score on RoboDojo long-horizon tasks, with robust in-domain and OOD performance on the Fourier GR-3 robot. Ablations confirm the complementary contributions of long-horizon task evolution and local interaction dynamics, while temporal-scale analyses highlight the importance of balancing temporal coverage and resolution. Progressive inference further achieves a 2.93x speedup while improving performance.
★ For You, Not for Everyone: Bystander-Aware Delivery of Critical Feedback in Robot Tutoring
Robots in shared settings may expose information intended for an individual user to nearby bystanders. Robot tutoring makes this concern particularly salient: critical feedback is directed at an individual learner, yet bystanders may overhear or see it. To address this, we compared context-invariant and context-adaptive robot behaviors in a between-subjects study with 40 participants. Following an initial phase without a bystander, a bystander entered the setting. Participants then received two scripted critical feedback messages concerning their pronunciation and handwriting, independent of their actual performance. With the context-invariant behavior, the robot tutor delivered both messages aloud. With the context-adaptive behavior, the robot tutor scanned for the bystander, turned its body and display away after detecting them, gave a spoken attention cue, and presented the same feedback messages silently on its display. Participants rated the robot tutor higher in perceived privacy awareness and perceived social competence in the context-adaptive condition than in the context-invariant condition, while evidence for reduced experienced embarrassment was inconclusive. These findings inform the design and evaluation of robots that respond to bystander presence by adapting how they communicate potentially sensitive information.
comment: Submitted for Review
★ LLMs as MDP Designers: A Dependency-Aware Agentic Framework for Automated Robot Policy Generation NeurIPS 2026
We investigate the automated generation of executable robot reinforcement learning (RL) policies from natural-language task descriptions. Rather than using large language models (LLMs) as execution-time decision makers, we use them as designers that construct RL pipelines before deployment, including Markov decision process (MDP) formulation and simulator implementation. We identify a phase-wise structure in this problem: MDP formulation is dependency-structured and sequential, whereas simulator implementation becomes modular once the formulation is fixed. Based on this structure, we propose ARPG, a dependency-aware agentic framework that performs sequential MDP modeling, localized verification and correction, and simulator-structured implementation while externalizing intermediate artifacts. ARPG targets a key failure mode of automated policy generation: executable code may still yield non-learnable policies when state, action, reward, and termination artifacts are not semantically aligned. Experiments in Isaac Sim show that ARPG improves modeling correctness, environment executability, and policy-generation success compared with monolithic LLM baselines and off-the-shelf agentic systems. Additional demonstrations on custom robotics, standard RL, and engineering-domain MDPs further suggest that the same modeling pipeline can be reused beyond the main benchmark suite.
comment: Accepted to the 40th Annual Conference on Neural Information Processing Systems (NeurIPS 2026)
★ Caging Vision-Language Navigation for Legged Robots via Barrier-Certified Safety Architecture
By harnessing the inherent commonsense capabilities of pretrained Large Language Models (LLMs) and Vision-Language Models (VLMs), current Vision-Language Action(VLA) models for legged robots excel at interpreting language-based navigation commands but lack the physical grounding and explicit safety assurance required for real-world operation. Therefore, this paper aims to cage Legged-Vision-Language Navigation (VLN) through a safety architecture that provides physical grounding and ensures navigation safety. The key idea is not to abandon the powerful capabilities of end-to-end VLA planning, but rather to augment and cage them. In particular, we integrate a high-level VLA planner with a low-level physical-grounding layer that leverages efficient state estimation, in-situ 3D spatial perception, and an Signed Distance Field (SDF) representation to generate executable trajectories. We provide a navigation planner with provable collision-avoidance guarantees under the stated assumptions by combining a low-dimensional Bezier-parameterized MPPI planner with Discrete Exponential Barrier Function (dEBF) constraints. The proposed Legged-VLN system is modular, making it compatible with various VLM/VLA models and agnostic to the robot platform. The proposed system is validated with extensive real-world experiments on humanoid and quadrupedal robots, demonstrating versatile, efficient, and safe vision-language driven navigation capabilities.
★ Safeguarding Multi-Agent Reinforcement Learning for Connected and Automated Vehicles with Recursively Feasible Control Barrier Functions
Multi-Agent Reinforcement Learning (MARL) shows promise in coordinating connected and automated vehicles but offers no safety guarantee. Although Control Barrier Functions (CBFs) can add this guarantee, their optimization problem can become infeasible in practice due to control bounds. Moreover, they often overapproximate vehicle and road geometry, which makes them conservative. This paper proposes the Backup-Horizon CBF (BH-CBF), which evaluates safety along a horizon until all vehicles stop. BH-CBF is recursively feasible under control bounds, and it considers the exact vehicle and road geometry. Based on BH-CBF, we derive a fully decentralized safety filter that only needs a single broadcast of vehicle states and routes per step and stays real-time in dense traffic. We use BH-CBF to safeguard a MARL policy over its life cycle: as a reward during pre-training, as a run-time filter at deployment, and as a filter in the training loop that keeps fine-tuning on unseen maps collision-free. In simulations on real-world maps of a roundabout and an intersection, BH-CBF avoids all collisions of unsafe MARL policies and keeps its safety guarantee at every step. Existing filters either collide, lose the guarantee, or fall back to full braking far more often. In recorded roundabout traffic with 1264 human-driven vehicles over 115 km, our safeguarded policies shorten the median travel time by 14% compared with the human drivers, without any collision.
★ Trusting Imagined Futures: Auditing Imagined Rewards for VLA Post-Training
Video world models enable candidate actions to be evaluated through imagined outcomes, but high predicted rewards can favor trajectories with poor execution quality. We introduce Trusting Imagined Futures (TIF), an offline supervision selection method for vision-language-action (VLA) post-training that audits predicted rewards using execution feedback. A multidimensional reward model predicts a utility score for candidate ranking and five dimensions of execution quality. An auditor uses reward predictions and uncertainty to identify overestimation patterns learned from paired imagined videos and real executions. TIF accepts the candidate with the highest predicted utility when its advantage over the existing trajectory exceeds a margin and its audit risk passes a threshold; otherwise, it retains the existing supervision. The retained real trajectories then supervise behavior cloning. Across four LIBERO suites and three training seeds, TIF achieves a mean success rate of 76.25%, improving over training on the existing trajectories by 3.00 percentage points with the same trajectory count and training budget. Adding audit gating yields 1.47 percentage points with the reward model and advantage margin held fixed. The gains persist against advantage ranking and logistic regression risk selection at matched replacement counts. On recorded videos from three real robot tasks, TIF increases clean completion among selected trajectories from 58.33% to 70.00% across 60 candidate pools.
★ Physics-Based Plausibility Assessment of AI-Predicted Vehicle Parameters for Automotive Radar in Pre-Crash Scenarios
Automotive radar is a key sensing modality for safety-critical vehicle functions due to its robustness under adverse weather and lighting conditions. Recent advances in AI-based radar perception enable the direct estimation of object parameters from a single radar measurement cycle. However, the internal decision process of data-driven models is typically not explicitly linked to the underlying radar sensing physics. Consequently, object estimates may achieve high statistical accuracy while remaining physically inconsistent with the measured radar signature, particularly in previously unseen scenarios. Such inconsistencies are especially critical for heading estimation, as they directly affect trajectory prediction and collision assessment in pre-crash scenarios. This paper presents a physics-based consistency assessment framework for validating AI-derived vehicle hypotheses in automotive radar perception. The proposed approach generates an expected radar signature from AI-estimated object parameters using a physics-based reference model that incorporates occlusion-dependent edge visibility, aspect- and angle-dependent RCS modeling, and characteristic wheel-induced MD signatures. The expected signature is compared with the observed radar point cloud using a Wasserstein-distance-based consistency metric computed from three complementary features: the spatial distribution of detections along object edges, the local detection count, and the radar reflection intensity. In this way, physically implausible object hypotheses can be identified and corrected independently of the underlying neural network architecture and without model retraining or architectural modifications. The proposed consistency module is integrated as a cascaded post-processing stage for vehicle heading estimation.
★ PrimeSim: A Generalist World Model Simulator from Large-Scale Cross-Embodiment Demonstrations
A robot world model that serves as a simulator must do more than generate plausible videos: it must faithfully follow commanded actions, maintain consistency across camera views, and sustain coherent long-horizon rollouts. We present PrimeSim, an action-conditioned, multiview world model simulator that addresses these challenges through progressive midtraining and posttraining. Starting from a pretrained video generator, we midtrain on 30,110 hours of robot demonstrations spanning 22 platforms. To improve action following, we introduce a unified cross-embodiment action representation and frame-aligned dual-path conditioning to strengthen action-frame correspondence. To maintain multiview consistency, we develop view-aware interleaved attention with explicit camera identities, enabling cross-view information exchange while preserving view-specific representations. We further introduce a fast-video, slow-memory posttraining framework that transforms the bidirectional video generator into an efficient autoregressive simulator. By combining fast causal generation, self-rollout training, and hierarchical temporal memory, PrimeSim mitigates error accumulation from self-generated histories and supports coherent long-horizon simulation. PrimeSim runs at 10.1 FPS, remains stable over long-horizon rollouts on rigid and deformable manipulation tasks, and transfers to held-out platforms within 100 to 200 gradient steps where the pretrained backbone does not. Against five open-source action-conditioned world models it leads four of the five WorldSimProbe suites and posts the best overall EWMBench score, while fine-grained local calibration still trails the field; its rollouts are faithful enough to support GRPO-based policy optimization. We release the model weights, inference code, and training code.
★ Coordinating Humanoid Loco-Manipulation through Action-Conditioned Latent Foresight
Humanoid loco-manipulation requires coordinated body motion and hand interaction over time, and a plausible-looking command sequence can still drop a held object or leave an interaction incomplete. We present action-conditioned latent foresight, a latent world-action model (WAM) that evaluates proposed command sequences before execution. An action generator proposes joint motion-hand command sequences from vision, language, and the robot state. An action-conditioned endpoint predictor estimates the visual features each sequence prefix reaches and measures progress toward a demonstration-derived subgoal, while a joint temporal prior scores temporal consistency given the robot state and preceding commands. The robot executes the first commands of the best-scoring sequence and replans from the next observation. During training, action consistency regularization reconstructs the first demonstrated command from observed and predicted feature changes, encouraging the predictor to retain action-relevant information. On ten tasks in the humanoid loco-manipulation simulation benchmark SIMPLE, our method achieves 89.7% average success, compared with 73.7% for Psi-Zero and 82.0% for DiT4DiT. On seven real-robot tasks, it achieves 85.7% under standard conditions and 65.7% under unseen object configurations, while GR00T N1.7 achieves 70.0% and 48.6%, respectively. Project page: https://ju6276.github.io/Action-Grounded-Foresight/
comment: 10 pages, including supplementary material
★ iMED: A Multi-Endoscope Dataset for Surgical 3D Perception ECCV 2026
We introduce iMED, the first synchronized multi-endoscope dataset for robot-assisted minimally invasive surgery (RAMIS), designed to address a fundamental limitation in surgical vision benchmarking: the absence of independent held-out viewpoints. Existing surgical datasets capture single-trajectory sequences from a single endoscope, making it impossible to distinguish geometric generalization from photometric interpolation along a narrow forward-facing path. iMED provides 340 sequences ~170K synchronized timepoints with 4 views per timepoint) recorded simultaneously from two independent stereo endoscopes across ex vivo, postmortem, and live surgical settings, spanning 14 specimens with diverse anatomical regions and motion regimes. The dataset includes calibrated camera intrinsics, ArUco-based frame-wise pose estimates with uncertainty quantification, instrument segmentation masks, and rich clinical metadata. Using a train-on-one-endoscope, test-on-another evaluation protocol, we benchmark 23 state-of-the-art methods across rigid and deformable novel view synthesis, pose estimation, feature matching, and monocular depth estimation. Our experiments show that methods relying primarily on photometric supervision degrade substantially under this held-out-endoscope setting, while methods with explicit geometric regularization are more robust across views in our benchmark. Notably, large-scale foundation models transfer surprisingly well to surgical imagery, often outperforming domain-specific models, suggesting pitfalls in current fine-tuning protocols. iMED establishes a new evaluation protocol for geometric generalization in surgical vision.
comment: Published at ECCV 2026. 25 pages, including supplementary material. Code and dataset: https://github.com/surgical-vision/imed
★ Mass Scaling Laws for Energy-Optimal Horizontal Multirotor Flight
Multicopters are a type of aerial robot seeing extensive real-world applications but with range and endurance as critical bottlenecks. Optimization of its energy efficiency through energy-aware flight planning (e.g. cruise speed-scheduling) has hence been an important topic. The mass of the multicopter is an important factor to consider, as it has a significant impact on efficiency, and many common applications are subject to frequent mass variation due to payload change. This work derives closed-form scaling relationships between the energy-optimal steady horizontal flight behavior of a multicopter and its vehicle mass, and validates them experimentally on two multirotor platforms. Specifically, for mass-only variation, i.e., when payload changes the vehicle mass while the airframe geometry, aerodynamic characteristics, and rotor configuration remain unchanged, we show through a combination of numerical simulation, theoretical derivation, and experimental validation that (1) the energy-optimal velocity is proportional to the square root of vehicle mass; (2) the energy-optimal pitch angle is constant with respect to mass; and (3) the optimal energy per meter is proportional to mass. The analysis shows that these three relationships arise from a common mass-similarity property of the steady-state flight and rotor-aerodynamic equations. Furthermore, the obtained scaling relationships enable simple yet effective cruise-speed planning to optimize energy efficiency and extend the range and endurance of the multicopter.
comment: Accepted to IEEE Robotics and Automation Letters, October 2026
★ Cycle WAM: Coupling Action and Video through Cycle Consistency
Recent advances in video modeling have driven substantial progress in world action models (WAMs). Existing approaches exploit pretrained video priors by decoding actions from latent representations or inferring them from generated videos. The underlying link is causal: robot actions produce changes in the scene, and video observations of these changes provide evidence for inferring the actions that produced them. However, existing training formulations do not by themselves ensure that actions and their predicted visual consequences are consistent, potentially limiting the use of pretrained capabilities. We treat video and action as paired modalities linked by this causal relationship and learn their correspondence through a unified training framework designed to better exploit pretrained video priors. Cycle WAM first augments video-action co-training with bidirectional conversion to reinforce cross-modal correspondence and recover forward and inverse dynamics. It then introduces a differentiable cycle-reconstruction objective to further couple the two mappings. The resulting model supports both forward and inverse dynamics, with the potential to improve performance through more effective use of pretrained video priors. Cycle WAM achieves state-of-the-art performance among comparable methods on LIBERO-Pro and substantially improves DreamZero on DROID. Comparisons with UWM characterize its inverse-dynamics performance and show sharper visual detail in the displayed forward-dynamics predictions.
★ CLARA: A Cognitively-Driven Clarification Framework for Grounding Ambiguous Requests
Robotics and AI systems are increasingly deploying large vision-language models (LVLMs) to follow high-level instructions, from routine household chores to more complex organizational tasks. Yet, broad directives like "clean the room" or "organize the desk" frequently omit crucial details about user preferences and specific arrangement of objects that influence the desired outcome. Without any further guidance, such models tend to overestimate their grasp of ambiguous instructions and frequently assume they have sufficient understanding despite missing important details. In this paper, we propose CLARA (Conversational cLArification for Robotic Arrangement), a cognitively-driven clarification framework that proactively elicits key information from users, in a similar fashion as humans' clarification seeking behavior. Through targeted questioning, CLARA refines ambiguous requests into structured, detailed instructions describing precisely how objects should be organized or rearranged, according to the user's preferences. These clarifications then guide a diffusion-based image generation pipeline, producing realistic visual renderings of the final, desired scene. This final rendering also serves as a visual confirmation of the system's understanding from the user. Experimental results and user studies demonstrate that CLARA leads to more accurate and consistent outcomes compared to another State-of-the-Art LVLM system, highlighting the importance of interactive clarification for bridging the gap between vague directives and fully grounded task specifications.
comment: Work conducted in early 2025
★ Just Between Us? Exploring a Wearable-Mediated Channel for Discreet Human-Agent Interaction in a Shared Game Setting
In shared settings, social robots may need to exchange information with an individual while limiting co-present others' access to or interpretation of that information. Publicly observable speech and gestures may not adequately support such exchanges. We investigate a wearable-mediated channel for discreet interaction that combines in-ear audio messages with deliberate rightward head tilts as responses. To examine its feasibility and participants' experiences, we studied 20 pairs (40 participants) playing a card game with a screen-based conversational agent representing a social robot, controlled by a researcher in a Wizard-of-Oz setup. Each pair played with and without the channel. One participant wore the device (the wearer), while the other (the non-wearer) was informed about the channel after gameplay. The channel conveyed information that could help the wearer or robot character select an opponent they could defeat. The information exchange and subsequent opponent selection occurred as intended in 39 of 40 channel-mediated turns. Wearers reported low perceived pressure/tension in both conditions, with no statistically significant difference between conditions, and favorable usability immediately after use. Retrospective interviews, however, revealed varied accounts of noticing the interaction among non-wearers and contrasting perceptions of teamwork and unfairness. These findings support the feasibility of wearable-mediated exchanges within this game and identify awareness and fairness as considerations for future designs.
comment: Accepted in 25th International Conference on Mobile and Ubiquitous Multimedia (MUM) 2026
★ Future Representation Distillation for Imagination-Free World Action Models
World Action Models (WAMs) use the predictive knowledge of pretrained video diffusion models for robot control through future video prediction. However, iterative video denoising is costly, and inaccurate visual predictions can compromise action prediction. Our key idea is to make this predictive knowledge available through visual representations computed from the current observation, enabling action prediction without generating future videos. We introduce DreamFree, a WAM that repurposes a pretrained video diffusion model as a predictive visual encoder through representation distillation. The visual encoder learns to predict a frozen video diffusion model's representation using only the current observation. An action module is then trained to predict actions by extracting action-relevant information from the visual encoder. Across real-world and simulated tasks, DreamFree achieves higher success rates than state-of-the-art VLAs and WAMs, while performing real-time closed-loop control at 11Hz with a 14B video diffusion backbone. Experiments show that DreamFree can translate stronger video priors and additional video data into better robot control and enable learning new robot tasks from video demonstrations, including videos from a different embodiment, without action annotation for the new tasks.
★ Action Intent as a Bridge: Transferring Manipulation Knowledge from Human-Centric Videos to Robots
Vision-Language-Action (VLA) models have achieved promising performance for generalized robotic manipulation, yet their reliance on expensive real-robot data remains a fundamental bottleneck. A scalable alternative is to transfer knowledge from off-the-shelf human-centric videos. However, existing methods extract latent action representations that are entangled with scene-specific noise, diluting the transferred knowledge. Motivated by this limitation, we propose Action-Intent Transfer (AIT), a two-stage method that transfers knowledge through a clean action representation shared across humans and robots. In the first stage, AIT extracts explicit action intents and disentangles the implicit action intent from manipulation videos. In the second stage, AIT aligns the human and robot domains within this shared intent space and decodes the intent into executable robot actions. Extensive experiments in both simulation and real-world environments demonstrate that AIT transfers human knowledge more effectively and consistently outperforms prior baselines.
comment: 12 pages,6 figures
★ LLM-Assisted Planning with Distributed Onboard Behaviour Tree Execution for Multi-AUV Missions
Coordinating autonomous underwater vehicles requires translating operator intent into feasible assignments while preserving autonomous execution when communication is limited. We present a hierarchical framework that couples Large Language Model (LLM) interpretation of natural-language mission instructions with Mixed-Integer Linear Programming (MILP) and distributed Behaviour Tree (BT) execution. The LLM extracts assignment and ordering constraints, which the MILP optimizer combines with vehicle capabilities and temporal requirements. A BT-LLM Bridge distributes the resulting allocations to onboard BT Mission Managers. Per-vehicle Robotic Agents assess local faults and request selective replanning, allowing unaffected vehicles to continue their assigned tasks. Evaluation in a ROS-based AUV simulator compares Qwen-3-8B and Llama-3.1-8B on constraint extraction and tests recovery from sensor degradation and critical vehicle faults. Qwen-3-8B provides more accurate constraint extraction, and all tasks are completed in the tested failure scenarios. These simulation results support the feasibility of combining language-based mission specification with structured allocation and distributed execution.
★ Physical Diffusion Policy via MPM-Guided 3D Gaussian Simulation
Learning robust manipulation policies for diverse everyday objects remains challenging because objects with different materials exhibit distinct deformation and contact dynamics. While diffusion policies provide a powerful framework for modeling multimodal manipulation behaviors, they typically lack explicit physical reasoning and may produce averaged actions when facing heterogeneous objects. We present a physics-aware diffusion policy that augments visual imitation learning with object physical understanding. Our framework recovers material properties from visual demonstrations by coupling 3D Gaussian Splatting with differentiable Material Point Method simulation, where video guidance and particle-level constraints enable accurate physical parameter estimation. The estimated physics conditions are further integrated into a material-aware Mixture-of-Experts diffusion policy, allowing adaptive behavior generation across different physical regimes. For practical deployment, we introduce a physics memory bank that provides efficient physical retrieval without requiring online simulation optimization. Experiments on a wide range of simulated and real-world objects demonstrate improved manipulation success and generalization over existing diffusion-based approaches. These results show that incorporating physical knowledge is critical for scalable robot manipulation in diverse environments.
comment: 10 pages
★ Hybrid Particle-Voxel Dynamics for Multimodal 3D World Simulation
Predicting how the world evolves in response to robot actions is central to policy learning, evaluation, and planning. However, many action-conditioned world models predict future observations in 2D image or video space without explicitly modeling the underlying 3D scene, making spatial consistency across viewpoints and over long-horizon interactions challenging. We introduce 3D World Simulator, an action-conditioned multimodal world modeling framework that jointly predicts appearance, geometry, and tactile observations from a unified 3D representation. Our approach combines a hybrid particle-voxel autoencoder that compresses point clouds into compact 3D voxel latents with an action-conditioned latent dynamics model that predicts their evolution. The shared 3D representation supports geometrically consistent rendering from arbitrary viewpoints and flexible integration of spatially aligned tactile signals. Our experiments demonstrate interactive, long-horizon, multimodal prediction at 50 Hz. We then show how our model enables scalable data generation for policy training and evaluation using tactile and multi-view visual observations, including wrist-mounted and novel camera views. Furthermore, we scale our method to large datasets containing diverse objects and demonstrate model-based planning with unseen objects and robot configurations. Together, these results highlight the potential of hybrid particle-voxel dynamics as a foundation for multimodal 3D world simulation.
★ All-For-One: Distilling Visual Foundation Models to a Fast, Memory-Efficient Visuomotor Policy
Visual Foundation Models (VFMs) provide semantic cues such as depth and segmentation useful for robotic manipulation, but directly deploying them in closed-loop control is impractical due to high computational cost, temporal inconsistency, and distribution shift between static internet images and egocentric robot observations. We propose All-For-One (AFO), a two-stage distillation framework that leverages VFMs strictly as privileged information during training, using downstream robotic data. Following the privileged learning paradigm, AFO decouples semantic representation learning from inference-time policy execution. In the All stage, a teacher policy fuses RGB observations with outputs from multiple VFMs to learn a semantically enriched representation. In the For-One stage, this knowledge is distilled in-distribution into a lightweight RGB-only student policy. Evaluated on two Adroit tasks and five MetaWorld tasks in simulation, as well as three real-world manipulation tasks, AFO outperforms prior foundation-model-based approaches while using over 30x fewer parameters than VFM-based encoders and 50x lower latency than its own teacher in simulation benchmarks, providing an efficient pathway for transferring foundation-model knowledge to robotic control.
★ Spectrally Routed Diffusion Policy for Multi-Task Robotic Manipulation
Diffusion-based policies have achieved strong performance on single-task robotic manipulation. However, a single model trained jointly on multiple tasks consistently underperforms the per-task specialists trained on the same demonstrations. We focus on two challenges. First, the convolutional encoders commonly used in diffusion-based policies produce visual conditions that are not discriminative enough. Second, gradients from incompatible action modes update the shared denoiser's dense parameters along the same directions, causing destructive interference during joint training. To address both, we propose SR-DP (Spectrally Routed Diffusion Policy). A frozen VGGT encoder with a spatial-compression adapter produces compact multi-scale semantic-geometric tokens, a task-discriminative visual condition for tasks that share a workspace. We first train the denoiser as a standard diffusion policy, then decompose selected linear layers by SVD into a frozen basis of orthogonal rank-one experts, adapted through spectral routing and Householder input rotations. On six RoboTwin tasks, SR-DP averages a 68.4% success rate, surpassing multi-task VO-DP by 21.8 points and the strongest per-task baseline by 3.8 points. On four real-world bimanual tasks, Stage 2 reaches a 43.0% average success rate and improves over Stage 1 by 19.0 points.
comment: 8 pages. Accepted for publication in IEEE Robotics and Automation Letters (RA-L)
★ ReflexArc: Bottom-Up Integration of Instantaneous Tactile Reflexes into Whole-Body Humanoid Control
Humanoid robots operating in cluttered environments must react to unexpected contacts before the first impact peak, which arrives within tens of milliseconds. Learned whole-body policies running at 50 Hz introduce a policy-cycle delay and may suppress a protective withdrawal as a tracking error. To address these challenges, we propose ARC (Ascending Reflex Control), a bottom-up framework that integrates instantaneous tactile reflexes with learned whole-body control. ARC consists of two components: an instantaneous reflex pathway and bottom-up reflex integration. The reflex pathway detects hazardous contacts from full-body tactile sensing, solves a whole-body withdrawal pose, and delivers it directly to the motors within 10 ms. Bottom-up reflex integration combines reflex-in-the-loop policy learning, reflex-transparent policy alignment, and reflex-informed policy adaptation to sustain the withdrawal, maintain balance, and complete longer-horizon recovery. We deploy ARC on a Unitree G1 and evaluate three categories of hazardous contact. Compared with an otherwise identical reflex delayed by 20 ms, ARC reduces the first force peak by 34.1% and reduces the contact impulse from 0.2632 N s to 0.1225 N s (a 2.15x ratio). We further demonstrate protected motion tracking, teleoperation, and vision-free obstacle traversal guided by foot tactile sensing. Project page: https://minusmodulo.github.io/reflexarc-website/
comment: 8 pages, 6 figures
★ Aero-VPA: Few-Step Flow-Matching Control over Non-Reconstructive JEPA Latents for Agile Flight on Edge Hardware
Vision-Language-Action (VLA) models generalize across tasks, but their autoregressive decoders need sequential forward passes in proportion to the action horizon, each bounded below on embedded aerial platforms by streaming billions of parameters from memory. We present Aero-VPA, a lightweight Vision-Predictor-Action architecture for 3D navigation on edge hardware. A compact RGB-D Vision Transformer and a five-head ensemble predictor, trained without pixel reconstruction, forecast the next latent state; a Conditional Flow Matching solver generates a velocity and yaw-rate chunk in two Euler steps, so each decision takes five sequential network evaluations; and the ensemble dispersion drives an uncertainty fallback. Version 1 specified the architecture, deployment and protocol without results. This version reports the first closed-loop evaluation: PX4 software-in-the-loop simulation of quadrotor missions in procedurally generated forests. Behaviour cloning failed on all 60 missions despite open-loop errors of 0.02-0.04 m/s: the policy copies its last command, a copycat form of causal confusion. DAgger with a privileged, safety-gated supervisor gave the first successes but plateaued at 11.1-21.1% across six checkpoints. A map-free runtime shield with a 15 s depth memory, barrier constraints, a cap on motion outside the 73-degree field of view and backoff along the flown path raised the first DAgger policy's success on 180 paired missions from 21.1% to 52.2% (95% CI [45.0, 59.4]; exact McNemar p = 1.2 x 10^-10) and cut its crashes from 72.2% to 22.2%, while timeouts rose from 2.8% to 23.3%. All six checkpoints gained 30.0-34.4 percentage points, at mean speeds of 1.16-1.36 m/s. The code (https://github.com/ylee-sling/Aero-VPA), the trained policy (72.6 MB, https://huggingface.co/ylee-sling/aero-vpa) and the full benchmark dataset are public.
comment: 38 pages, 3 figures, 22 tables. Simulation study (PX4 SITL + Gazebo Harmonic). Version 2 of a preprint first posted on Zenodo (doi:10.5281/zenodo.23633375). Code: https://github.com/ylee-sling/Aero-VPA ; trained policy: https://huggingface.co/ylee-sling/aero-vpa ; benchmark dataset: https://huggingface.co/datasets/ylee-sling/aero-vpa-benchmark
★ RAFT: Reasoning-Aligned Fine-Tuning Improves Robustness of VLA Post-Training
Adapting a generalist vision-language-action (VLA) policy to a target task with limited demonstrations can improve nominal task success while reducing robustness to variations in appearance, objects, and instructions. We introduce reasoning-aligned fine-tuning (RAFT), a two-stage post-training procedure: RAFT first trains low-rank adapters on task-relevant reasoning annotations, then freezes them while fine-tuning the policy on action demonstrations. On a physical DROID robot, RAFT achieves $75.0\%$ out-of-distribution success, compared with $27.9\%$ for plain fine-tuning; on a multi-stage bimanual YAM task, it reaches $70.0\%$ mean success, compared with $36.0\%$ for the strongest baseline. RAFT also improves success under visual, object, and instruction shifts in simulated pick-and-place and open-drawer tasks. Compared with plain fine-tuning, RAFT policies place more attention on annotated task regions and yield higher linear-probe accuracy for object identity in later layers. A broader reasoning-data mixture further improves downstream success across demonstration budgets.
★ ReSMap: Recasting Satellite Priors for Robust and Accurate Online HD Map Construction NeurIPS 2026
Despite recent advances in camera-based online HD map construction, these systems remain susceptible to sensor failures due to their reliance on onboard cameras. In contrast, satellite images remain available independently of real-time sensing conditions as they are cached offline. Existing methods utilizing satellite images, however, primarily leverage them under nominal conditions, without explicitly addressing them as redundancy when a sensor degrades. To address this gap, we propose ReSMap, a redundancy-aware camera-satellite fusion framework for robust online HD map construction. Our framework explicitly leverages satellite images as a complementary source to improve robustness against camera failures. A Predict-then-Fuse (PTF) module fuses camera and satellite BEV features by using each branch's own prediction confidence as a per-pixel reliability weight, with no learned router. An Object-Centric Cooperative Multi-modal Query (OCMQ) decoder then extends this per-modality separation to the query level: each modality cross-attends only to its own BEV, and per-modality updates are merged through a learned per-token gate. Our model achieves state-of-the-art performance on nuScenes across all splits and range settings. More importantly, it also sets the new state-of-the-art under camera-failure scenarios, demonstrating that satellite images serve not only as a strong prior but as practical redundancy for robust online HD mapping.
comment: Accepted to NeurIPS 2026. 24 pages, 9 figures, 11 tables. Project page: https://dehsurc.github.io/resmap-page/ Code: https://github.com/dehsurc/ReSMap
★ RAIPAL: A Powerful and Agile Humanoid Arm for Human Tasks
This paper presents humanoid manipulator hardware designed specifically for human-level performance throughout its broad domain of tasks, from heavy payloads to highly dynamic motions. Unlike conventional designs targeting uniform workspace capability, the proposed approach derives force, speed, and endurance targets from published measurements of task-relevant human upper-limb performance. These targets guide the actuator, mechanism, and thermal design toward torque transparency and sustained operation. The platform integrates force-oriented upper-arm components, speed-oriented forearm components, and a posture-dependent elbow mechanism that shifts its torque-speed behavior. These components are driven by planetary actuators and compact actuator-embedded electronics. Experimental results validate broad task capability, demonstrating a peak payload of 20 kg (2.25x its own mass), an endpoint speed of 20.6 m/s, and continued operation under repeated hammering impacts. Thermal analysis further confirms that the robot's sustained operation capability exceeds the published human load--duration boundary across nearly the entire load spectrum. These findings show that hardware targets derived from human-task measurements narrow the human-to-humanoid capability gap. A supplementary video is available at https://youtu.be/xVUWFICD5G0
comment: Published in IEEE Robotics and Automation Letters (RA-L), 2026. Supplementary video: https://youtu.be/xVUWFICD5G0
★ Computing Invariant Reachable Sets through Sum-of-Squares Programming
Funnels, time-varying invariant sets around nominal trajectories, certify that a closed-loop system tracks its plan, and can be composed into verified motion plans. Their computation by sum-of-squares (SOS) programming is well established in formulation but not in practice: the result depends on implementation choices such as the initial guess, Lyapunov function normalization, and termination rule, which the literature documents only briefly. This paper presents a self-contained treatment of SOS-based funnel computation, implemented in the open-source package computeSOSFunnels. We propose a minimum-volume formulation with inlet and outlet constraints under which funnels with coinciding nominal endpoints compose by construction. Numerical experiments on a unicycle, with timing studies on a cart-pole and a 12-state quadrotor, characterize the pipeline: infeasible initial guesses are partly decidable before solving, the certificate shows no violations among 39,000 boundary samples per funnel under the true dynamics, and constructing the semidefinite program, not solving it, dominates computation time.
comment: 8 pages, 3 figures, 3 tables. Submitted to IEEE Transactions on Control Systems Technology. Code: github.com/khalid2696/computeSOSFunnels
★ ActOR: Ambient AI-Based Agentic Orchestration for Humanoid Autonomy in Operating Rooms
Operating rooms (ORs) increasingly contain ambient sensing infrastructure to enable AI-based optimization of workflows and utilization. We pursue an alternative path: using ambient AI to guide robot action and optimize utilization. Deploying humanoids in the OR without ambient AI is severely limited by cumbersome de novo orientation and planning due to partial onboard observations. We posit that this can be achieved by coupling an ambient AI-based agentic orchestrator with the humanoid robotic actor, providing shared perception and execution feedback for humanoid autonomy in the OR. ActOR fuses a wall-mounted ambient camera and the robot's head-mounted camera into a shared object-level digital twin (DT). A large language model agent plans over the DT using pre-programmed locomotion and manipulation skills without task-specific training. During execution, DT updates and execution outcomes inform retries, alternative approaches, or replanning. We evaluated ActOR in simulated and physical environments. Across three tasks in two simulated ORs reconstructed from real rooms, ambient perception reduced exploration from 1.05 to 0.12 Explore calls per trial when targets were initially outside the robot's view. Closed-loop feedback increased success from 46.7% to 83.3% under nominal execution and from 8.3% to 51.7% under injected navigation, object-relocation, and grasp perturbations. In a physical mock OR with a humanoid, ActOR completed 14 of 20 single-object disposal trials and demonstrated replanning after scene changes. Ambient AI-based observations provide scene state that improves robot action planning, increasing success and reducing exploration compared with robot-only observations. Feeding skill outcomes and DT updates back to the planner improves multi-step task success. Our results show the feasibility of humanoid task autonomy for multi-step disposal tasks in a mock OR.
★ DiMuRe: Diagnosis-Guided Multilevel Recovery for Robotic Task Execution
Disturbances can invalidate the conditions required by a robot's next action even when much of its task program remains useful. A recovery system must therefore choose a response suited to the current state. We present DiMuRe (diagnosis-guided multilevel recovery), a framework for robot-use agents that uses phase-specific evidence and method preconditions to select program reuse or predefined recovery skills, including bounded vision-language-action (VLA) grasping. After the original program exits, a multimodal large language model (MLLM) can generate corrective code from images and execution feedback. Local verification checks the outcome of each recovery attempt. We also introduce LIBERO-Disturb, a benchmark for robustness and recovery under mid-execution disturbances, supporting program-based agents and VLA policies. On 801 scheduled disturbed Encore development cases, Local DiMuRe (program reuse and recovery skills) achieves 77.65% complete-task success, compared with 58.68% for the original program under the same physical-step allowance. Full DiMuRe achieves 81.90% with terminal replanning, an additional step allowance, and early handoff. Local DiMuRe also uses fewer physical steps per scheduled episode than the original program.
comment: 15 pages, 3 figures, including appendices
★ Online iLQR-MPC with Whole-Body Control for Robust Humanoid Locomotion
Humanoid locomotion requires deciding where to step and how to distribute contact forces. However, a disturbance can invalidate the nominal plan, requiring coordinated revision under constraints that switch with every footfall. To solve this online, the proposed approach employs the single-rigid-body model, which retains attitude dynamics and the full contact wrench. An iterative linear-quadratic-regulator MPC (iLQR-MPC) then jointly optimizes contact forces and footstep locations in a single receding-horizon problem, without a separate step planner. Specifically, contact and motion constraints are incorporated into the backward pass through augmented-Lagrangian and relaxed-barrier penalties. A whole-body controller subsequently maps the plan to joint torques at 500 Hz while preserving contact feasibility. The model-based formulation makes disturbance responses interpretable through explicit costs and constraints, and requires no platform-specific policy training. In MuJoCo simulations, the iLQR-MPC replans at 100 Hz (mean solve time 8.9 ms); the Unitree G1 sustains steady-state walking and recovers from impulsive 245 N forward and 175 N lateral pushes.
★ Grasping and Stabilizing Tumbling Objects with Rotating-Base Multi-Arm Space Manipulators
This paper presents an innovative guidance and control method for a space manipulator that actively maneuvers its base to hold a tumbling object stationary relative to the base during proximity operations. This simplifies motion planning, reduces collision risk, and enables impact-free capture using single- or multi-arm configurations, with or without predefined grasping interfaces. The proposed approach involves two phases. In the pre-grasping phase, a synchronization control aligns the target's center of mass (CoM) with the system's instantaneous center of rotation (ICR), virtually rigidizing the target-servicer system to keep the object stationary for reliable grasping. In the post grasping phase, a control strategy based on Hamiltonian dynamics and optimal control stabilizes the combined system by minimizing time, fuel, or energy, while respecting base torque limits and multiple grasping constraints. For structured targets, applied forces and torques stay within their bounds; for unstructured ones, friction constraints prevent slipping. A case study demonstrates the method's effectiveness in enabling both the subsequent grasping and detumbling phases.
★ Say the Same, Act Differently: Diagnosing and Steering Text-Action Divergence in Reasoning VLAs NeurIPS 2026
Reasoning vision-language-action (VLA) models pair physical control with natural-language rationales that supervisors can read to monitor embodied safety. Prior attacks on VLA reasoning corrupt the rationale itself and leave visible evidence in the text. We show that visual perturbations can instead alter the physical action while keeping the generated rationale word-for-word identical. At the shared text/action interface of Alpamayo for driving and InstructVLA for manipulation, our text-neutral action-subspace (TNAS) diagnostic shows that the locally most action-sensitive direction is nearly orthogonal to the active text-loss gradients, with 1.87% and 5.28% norm overlap. At equal hidden-state perturbation norms across both models, interventions along this text-neutral residual shift actions 4.64x to 47.8x more than text-gradient perturbations, whereas text-gradient perturbations induce 21.2x to 277x more text drift. Steering visual perturbations along this direction reaches the same decoupling from the image. Across 154 obstacle-annotated driving scenes in Alpamayo, TNAS-guided attacks induce a new collision under a word-for-word identical rationale in 52.6% of scenes under full-image perturbations and 35.1% under a localized forward-camera patch, compared with 4.55% for unguided action attacks. Relative to a text-preserving baseline, TNAS guidance raises exact rationale preservation from 83.0% to 89.5% and cuts teacher-forced text drift by 68.4% at a comparable collision rate. In InstructVLA, TNAS guidance cuts text drift by 40.5% and preserves the exact response on 85.0% of examples alongside a 1.73-unit action-chunk shift. An unchanged rationale therefore cannot certify a safe action, so monitors for reasoning VLAs must verify the predicted action channel directly.
comment: Accepted to NeurIPS 2026. 23 pages, 3 figures, 8 tables
★ Online Hamilton-Jacobi Reachability Onboard the Robot Control Loop: A Real-Time GPU Solver for Safety-Critical Autonomy
Hamilton-Jacobi (HJ) reachability provides certified backward reachable tubes (BRTs) for safety-critical robotic systems, but the level-set computation scales exponentially in the state dimension and has historically been performed offline. As autonomy stacks (ADAS/AV, humanoid robotics, and vision-language-action models) increasingly rely on black-box, probabilistic policies, a deterministic safety layer with formal guarantees is an essential complement. We present a CUDA implementation of the canonical WENO5 + TVD-RK3 + Lax-Friedrichs level-set Hamilton-Jacobi-Isaacs solver for 2D, 3D, and 4D dynamics that is fast enough to run inside a robot's control loop. In 32-bit floating point (FP32) on a single workstation GPU (NVIDIA RTX 4000 Ada), it computes the 3D Air3D BRT at 112 Hz, a 4D pursuit-evasion BRT at 61 Hz, and a 2D wind-affected drifter BRT at 74-147 Hz, 5.9-17.5x faster than the JAX reference hj_reachability at matched precision and 1.7-16x faster than a torch.compile reimplementation of the same scheme. In 64-bit floating point (FP64) it agrees with hj_reachability on eight benchmark cases, reproduces Mitchell's Air3D BRT to within 0.04% of the reference cell count, and runs 2.0-4.6x faster. On an embedded NVIDIA Jetson AGX Thor it computes Air3D at 66 Hz (N=41) and the 2D drifter at 54 Hz, 3.8-8.9x faster than hj_reachability on the same device. As a multi-agent application, an in-kernel label-propagation extension recovers the K-agent capture-basin Voronoi tessellation from a single combined HJ solve, independent of fleet size; it runs at 4.0 ms per tick in a closed-loop wildfire-response simulation of six quadcopters under time-varying, non-uniform wind.
comment: 12 pages, 4 figures, 5 tables
★ VARP: A Visual Adaptive Rearrangement Planner for Non-Monotone Object Rearrangement Planning
Object rearrangement planning in cluttered environments is often non-monotone, requiring objects to be temporarily relocated before reaching their final destinations. Although search-based planners can solve such tasks, they incur substantial computational cost and require expensive replanning when the environment changes. We propose the Visual Adaptive Rearrangement Planner (VARP), which learns non-monotone rearrangement directly from visual state representations generated from expert demonstrations. VARP predicts pick-and-place actions from the current and goal visual state representations, and incorporates an Action Refinement Strategy (ARS) to improve action feasibility and a Progress Enforcement Strategy (PES) to mitigate progress stagnation. By continuously updating actions from the latest visual state representations, VARP adapts online to disturbances without replanning from scratch or performing online combinatorial search. In challenging rearrangement benchmarks, VARP achieves a 98.3\% success rate with only 0.4 s planning time per task, 97.5\% success under execution-time disturbances, and 94.5\% under continuous severe disturbances, while generalizing effectively to unseen object counts. These results demonstrate that VARP provides a scalable and robust solution for non-monotone object rearrangement.
★ GRASP: Gradient-Guided Sampling for Contact-Rich Physics-Based Motion Retargeting SP
We study physics-based retargeting of human demonstrations for robot learning: finding robot controls that reproduce reference motions under contact dynamics. We propose GRASP (Gradient Recovery from Archived Samples for Proposals), which combines sampling-based exploration with gradient-guided refinement without requiring a differentiable simulator. GRASP-TO extends tempered sequential Monte Carlo by fitting local quadratic cost models to archived rollouts and using their gradients to guide Markov chain Monte Carlo rejuvenation. Metropolis-Hastings correction preserves the target distribution for each fixed proposal kernel, and a population analysis connects the recovered gradients to Gaussian smoothing. GRASP-MPC scales this optimizer to long demonstrations through warm-started receding-horizon planning. Experiments on dexterous hands and humanoids show accurate retargeting with a shared tracking objective and no task-specific contact rewards, including 92.8% success on 1352 bimanual OakInk2 demonstrations. The refined references improve PPO tracking in pen spinning and humanoid-object interaction. We also demonstrate dexterous manipulation and whole-body humanoid motion on real hardware.
comment: 26 pages, including references and appendices. Project website: https://computationalrobotics.seas.harvard.edu/GRASP/
★ OUCH: Objective, Uniform, Controlled Haptic Stimulation for Automated Neurological Assessment
Motor responses evoked by noxious stimulation are used to assess neurological function in hospitalized patients with disorders of consciousness. In standard bedside assessments, these stimuli are applied manually, resulting in substantial variation in force, duration, frequency, and location. Here, we present OUCH (Objective, Uniform, Controlled Haptic stimulation), a wrist-worn device that uses pneumatic actuation to deliver noxious mechanical stimuli. OUCH combines stacked pneumatic pouch actuators, a rigid indenter, and a compliant spacer designed to buckle at a set load, so that the indenter contacts the skin only during actuation. We characterize the device using blocked force tests, clinician evaluation, finite element analysis, and a deformable pressure testbed, and benchmark it against a commercially available handheld pressure algometer. At 200 kPa pouch inflation pressure, our indenter concentrated its load into an estimated contact pressure of 18,500 kPa, over 25 times that of the algometer tip, while requiring less total force. On the deformable testbed, it localized deformation near the surface, while deformation at depth remained comparable to the algometer. As a result, in preliminary perceptual testing, our indenter elicited discomfort at approximately 89% lower applied force than the algometer, demonstrating the feasibility of evoking discomfort with a compact, wearable soft actuator. These results show that combining pneumatic actuation with designed contact geometry enables controlled, repeatable noxious stimulation at the wrist, providing a foundation for objective neurological assessment.
comment: 8 pages, 7 figures. Supplementary video in ancillary files. Sherry Choi and Sitao Sunny Li contributed equally to this work
★ OdorSpaces: Visual-Olfactory Embodied Navigation in 3D Environments
Olfaction provides critical information about airborne chemical cues, yet it remains largely underexplored in embodied AI, with limited prior machine olfaction work focusing primarily on semantic odor perception. We introduce OdorSpaces, a physics-based simulation platform and benchmark for visual-olfactory embodied navigation. OdorSpaces augments Habitat-Sim with 3D odor transport and dispersion simulated by Fire Dynamics Simulator (FDS), enabling agents to navigate photorealistic indoor environments using both visual and olfactory observations. The benchmark includes two representative source-seeking scenarios: invisible gas-leak localization and early-stage fire localization. We develop an initial multimodal reinforcement learning policy that integrates visual observations and olfactory measurements over time. Experiments show that the policy outperforms a diverse set of baselines, establishing a strong reference for future research on visual-olfactory embodied navigation.
comment: Accepted at the 10th Conference on Robot Learning (CoRL 2026). 20 pages, 4 figures, 6 tables
★ Correct Robots Before They Make Mistakes: Proactive Human-in-the-Loop Intervention via Preference Learning
This paper presents Proactive Human-in-the-Loop (PHIL), a framework for proactive policy correction and preference-based post-training. PHIL enables operators to preview policy-predicted future motion and intervene before anticipated undesirable actions. During takeover, PHIL continues querying the policy and displaying its predicted trajectories in the physical workspace through an extended reality (XR) interface. PHIL pairs rejected policy predictions with corrective action chunks subsequently executed from the same physical state over the same horizon, and preference-based post-training encourages the policy to favor these corrections. This paper evaluates PHIL on four real-world manipulation tasks using a diffusion policy and a pretrained vision-language-action policy. With five correction episodes per condition, PHIL increases mean success rates across the evaluated tasks and demonstration budgets from 44.0% to 70.3% for the diffusion policy and from 57.8% to 76.9% for X-VLA, compared with HG-DAgger. Separately, policies trained with proactive corrections show fewer unnecessary recovery behaviors in successful episodes. Project page: https://intuitive-robots.github.io/PHIL-page/.
★ Beyond Accuracy: Information-Theoretic Decoder Evaluation Under Latent Intent
A central challenge in assistive brain-to-robot control is decoding a user's latent intent from complex and noisy brain signals. Prior approaches evaluate decoders as single-trial classifiers with accuracy-based metrics, but this neglects the inherently sequential nature of assistive control. Since a planner must integrate intermittent, noisy decoder outputs over time, single-step accuracy can only be a proxy when predicting downstream task performance. We formalize brain-to-robot control as a partially observable Markov decision process (POMDP), and prove that there is a sharp phase transition: regret is uniformly bounded across the horizon as long as pairs of goals are all decoder-distinguishable, but grows linearly otherwise. We show that a single channel statistic, the worst-pair Chernoff information $Γ_{\min}$ sets this boundary. Then, we derive a closed-form regret predictor $U_{\mathrm{id}}(C)$, that can be computed from any confusion matrix in $O(K^2)$ time. Across two domains spanning orthogonal information channels, we show that $U_{\mathrm{id}}$ predicts cumulative planning regret beyond what accuracy can explain. We then highlight a pathological accuracy trap, in which two decoders have identical validation accuracy 35.1% on real EEG data (BNCI2014001 subject 5) but result in a difference of $9.6\times$ in held-out regret, which $U_{\mathrm{id}}$ predicts. Same-signal control confirms $U_{\mathrm{id}}$'s advantage with decoders that span independent feature channels. Ultimately, our work provides a drop-in diagnostic to evaluate and select sensors and decoders in sequential assistive control.
★ $\mathcal{C}^\infty$ Smooth Trajectory Planning with Application to Soft Robots
High motion performance and avoiding structural vibrations rely on smooth trajectory planning. The smoothness requirement depends on the robot's complexity in terms of the number of degrees of freedom and system order due to the flexibility of the joints and links. For industrial robots, the smoothness requirements have been determined and subsequently taken into account. Although unknown for soft continuum robots, the exact smoothness requirement can potentially be much larger than previously considered ones such as $\mathbf{\mathcal{C}^4}$ for collaborative robots and $\mathbf{\mathcal{C}^5}$ for quadcopter drones. To address this gap, we introduce a $\mathbf{\mathcal{C}^\infty}$-smooth trajectory generator that respects defined kinematic limits and generates infinitely differentiable trajectories. The generated path addresses and generalizes to all smoothness requirements, where a transition between sequential states is needed. Therefore, all trajectories have continuous velocity, acceleration, jerk, snap, crackle, pop, etc. Of particular interest is the case of null boundary conditions, i.e., all derivatives at the start and end of the trajectory are equal to zero. We verified and evaluated our proposed approach in simulations and on hardware.
comment: 8 pages, 6 figures
★ Contact-Implicit Whole-Body Trajectory Optimization for Dynamic Humanoid Locomotion on Parameterizable Uneven Terrain
This letter presents a trajectory planner for humanoid robots to perform dynamic motions on complex, parameterizable terrains using a contact-implicit whole-body optimization framework. The planner generates feasible movements without predefined contact sequences, considering the robot's full dynamics and kinematics. To plan bipedal locomotion on steps, ramps, and other terrains, we propose a mathematical model to approximate ground geometry with a smooth surface. The rigid interaction between the robot and the ground is modeled via novel contact constraints, extending the domain of applicability from flat and horizontal surfaces to any arbitrary smooth surface. Effectiveness is demonstrated using nonlinear programming to plan motions on various terrain topologies. We also explore dynamic trajectory generation by embedding high-level requirements in the cost function and the initial guess. Preliminary feasibility tests were conducted over non-coplanar terrains, both in simulation and on the real robot ErgoCub.
comment: Author-corrected version of the article published in IEEE Robotics and Automation Letters, vol. 10, no. 11, pp. 11674-11681, November 2025. Typos corrected in Eqs. (7)-(8), with minor textual clarifications. Implementation and results unchanged
★ ELA: End-to-End Latent Action Policy Learning from Video Demonstrations
Robot imitation learning is limited by the scarcity of action-labeled data, while action-free demonstrations such as videos are abundant. Recent latent-action methods learn action-relevant representations from observation transitions, but they do not optimize latent actions for downstream control, leading to latent representations that are difficult for policies to model and difficult to accurately decode into continuous robot actions. This paper introduces ELA, an end-to-end framework for learning from video demonstrations via latent actions jointly optimized for policy learning and action decoding. ELA jointly trains a latent action model, latent action policy, and action decoder, enabling policies to leverage abundant video data with only a handful of action-labeled robot demonstrations. Experiments on simulated and real-world manipulation tasks show that ELA consistently outperforms prior latent action baselines. Moreover, it integrates seamlessly with existing imitation learning methods through lightweight 4M-parameter action decoders, requiring no changes to policy architectures or training objectives. Additional materials can be found on: https://end2end-latent-action.github.io/
★ MESA: An Evaluation Framework for Compositional, Semantic, and Spatial Generalization in Robotics
Despite growing interest in foundation models for robotic manipulation, systematic evaluation of their capabilities remains elusive. Existing benchmarks are either narrow and saturated, offering little signal on SOTA methods, broadly scoped, providing diversity at the cost of controlled experimentation, or require training on large, messy open-source datasets. We introduce MESA, an evaluation framework designed for precise, reproducible measurement of language-conditioned policy generalization. MESA provides five evaluation suites that isolate in-distribution performance and four distinct axes of generalization: spatial configuration, object instance, object category, and subtask composition, enabling researchers to diagnose where and why policies fail, not just whether they fail. To populate these suites at scale, we develop MESA-Gen, a pipeline for task and demonstration generation built by improving MimicGen. We instantiate the framework on two platforms and release both: the single-arm MESA suite, and BiMESA, the first large-scale benchmark for bimanual generalization. We evaluate seven baseline methods spanning diffusion policies, VLM-based approaches, and pretrained VLAs, finding that the strongest model achieves only 52% on MESA and 49% on BiMESA, providing meaningful signal while leaving ample room for progress. MESA rankings are broadly consistent with RoboArena scores, suggesting that MESA captures method-level differences relevant beyond simulation. Project website: https://pairlab.github.io/MESA.
comment: 51 pages, 12 figures, including appendices. Conference on Robot Learning (CoRL), 2026. Project website: https://pairlab.github.io/MESA
★ Ctrl-F: Force as an Interface for Humanoid Loco-Manipulation
Humanoid motion tracking has scaled to general policies that reproduce diverse human motion, but the resulting behavior is in free-space: the tracker knows where the body should be, not how it should interact with the environment. Other work makes the object part of the objective, yet abstracts that interaction to contact or geometry, which leaves the demanding cases under-explored. We argue that force is a natural command channel between motion imitation and object interaction, one that specifies not just where to make contact but how hard. We present Ctrl-F, a hierarchical motion-force framework that augments motion tracking with an explicit force command, a 3-D force at each designated contact body such as a hand or a foot. The hierarchical framework comprises: (1) A unified motion-force controller tracks the reference motion while applying the commanded force to the world. (2) A high-level force planner that issues a force command online based on object state and proprioception observations. In simulation and on hardware, Ctrl-F completes loco-manipulation that demands large, sustained interaction force and adjusts its command online as the load changes, both out of reach of a force-blind tracker. On hardware, Ctrl-F demonstrates hand-based dragging, pushing, and lifting, as well as foot-based kicking, successfully manipulating boxes weighing up to 20kg in pushing and kicking tasks where the force-blind baseline fails, while adapting online to additional loads introduced mid-task.
comment: Project page: https://forcectrl.github.io/
★ Jev with Foresight: Safe Language-Guided Autonomous Driving via a Language World Model
Large language models understand passengers' driving commands, but as driving agents they reason in generated text for seconds and must infer each action's consequences implicitly from the scene. We present Jev-AD, which pairs Jev, a typed decision model that answers predefined questions with probabilities, not generated text, with a language world model that predicts for every candidate action how nearby vehicles move over the next two seconds. Code turns these predictions into foreseen consequences, which Jev compares instead of inferring them from coordinates. In simulated highway merging (500 car-episodes), this foresight lowers the same decision model's collision rate from 26.0% to 2.4% at 0.25 seconds per decision, and Jev-AD achieves the passenger's intent in 93% of free-form command trials. In 500 replayed real-world NGSIM episodes, foresight lowers the collision rate from 48.2% to 2.8%, and the world model, learned from this traffic, foresees 93% of dangerous actions. Project page: https://ys-qu.github.io/jev-ad-website.
♻ ★ Dexterous In-Hand Object Pose Reaching with Learned Safety Filters
Dexterous manipulation with multi-fingered hands is the next frontier of robotics, yet learned hand skills remain limited to simple grasps or basic repeated motions, such as rotating an object. As robots attempt more demanding in-hand maneuvers, safety, such as not dropping the object, becomes critical, yet it is rarely measured. We study in-hand object pose reaching, a task that demands diverse and difficult grasps: moving an object to almost any position and orientation within reach of the fingers, at any wrist orientation, through long sequences of consecutive goals. We present a recipe for training such policies for a LEAP hand in simulation without human data, built on a range of insights, from diverse resets and goals via generated grasp datasets to an adaptive friction curriculum. With careful system identification, the policies transfer zero-shot to the real hand, using either motion capture or a single wrist-mounted depth camera, and prove remarkably capable and robust. On top of the task policy, we add a learned residual safety filter with adjustable conservativeness, letting us tune the balance between performance and safety and greatly reduce object drops. In 13.3 hours of real-robot evaluation over 4,800 unseen goals across 4 objects and diverse wrist orientations, the state-based policy reaches 95.2% of goals within 2 cm and 0.4 rad, and the filter cuts its drops by 75% while still reaching 91.5% of goals. The policies can also spin a lightbulb held in an arbitrary pose, reach out-of-distribution goals, withstand fast arm motion and perturbations, and, needing only a wrist camera, serve as portable skills such as screwing in a lightbulb. We release the full recipe as a platform for studying performance, reliability, and safety in high-dimensional dexterous systems. Videos and more are available at https://ian-chuang.github.io/dexsafe/.
comment: 26 pages, 30 figures, 14 tables. Project page: https://ian-chuang.github.io/dexsafe/
♻ ★ SensorPerch: Sense Wherever and Whenever it Matters
Existing robotic perception is constrained by sensors that are either robot-mounted or permanently fixed in the environment, locking perception to a limited set of viewpoints. Yet as robots perform increasingly diverse tasks, the most informative viewpoint shifts from one task to the next-often somewhere onboard sensor and static infrastructure can not readily satisfy. To address this gap, we propose SensorPerch, a novel realization of active perception that decouples sensing from both the robot embodiment and the environment by treating sensors as independent physical entities that the robot can autonomously detach and re-attach within the environment. SensorPerch presents one realization of this paradigm: a lightweight, wireless, reconfigurable sensor platform that can perch on diverse surfaces, paired with a viewpoint-selection framework that determines task-optimal sensor placements. Together, these enable robots to construct task-relevant viewpoints on demand, independent of the robot's current position and available fixed infrastructure. We demonstrate the paradigm on two task classes: (i) object-coupled perception, where SensorPerch enables persistent object-state detection beyond the robot's current position, achieving successful event detection even when the robot is not nearby; and (ii) policy-coupled perception, where SensorPerch allows robots to construct diverse, policy-specific viewpoints for various policies, achieving success rates comparable to those obtained using oracle viewpoints.
comment: Robotics and Automation Letters (RA-L), 2026
♻ ★ Fine-Tuning VLAs with Self-Demonstrated Generative Control for Multi-Task Manipulation
State-of-the-art vision-language-action (VLA) models such as pi0.5 exhibit strong semantic understanding, instruction following and task behavior. However, when deployed on new robots, even minor mismatches in hardware configuration relative to pretraining can cause severe performance drops. Finetuning the VLA on in-domain expert data from the new embodiment improves performance on the expert task but leads to a loss in its original instruction following and behavioral priors. In this paper, we propose a self-supervised method that generates online interaction rollouts from the zero-shot VLA as additional training data for finetuning. Our experiments show this yields strong multi-task manipulation policies on the target robot that (1) inherit pretrained tasks distilled from the zero-shot VLA without expert demos for them, (2) improve success and generalist instruction following beyond expert-only fine-tuning on expert-teleoperated tasks, and (3) mitigate forgetting of held-out tasks from the pretrained policy, not part of fine-tuning. We demonstrate the success of our approach across test sets probing generalization on a real ALOHA robot and a new simulation benchmark in RoboTwin. Video results are available at https://self-supervised-control.pages.dev/
comment: Additional experiments. Project page, code, checkpoints, data: https://self-supervised-control.pages.dev/
♻ ★ TOPPO: Rethinking PPO for Multi-Task Reinforcement Learning with Critic Balancing NeurIPS 2026
Soft Actor-Critic (SAC) and its variants dominate Multi-Task Reinforcement Learning (MTRL) due to their off-policy sample efficiency, while on-policy methods such as Proximal Policy Optimization (PPO) remain underexplored. We diagnose that PPO in MTRL suffers from a previously overlooked issue: critic-side gradient ill-conditioning, which may cause tail tasks to stall while easy tasks dominate the value function's updates. To address this, we propose TOPPO (Tail-Optimized PPO), a reformulation of PPO via Critic Balancing -- a set of modules that improve gradient conditioning and balance learning dynamics across tasks. Unlike prior approaches that rely on modular architectures or large models, TOPPO targets the optimization bottleneck within PPO itself. Empirically, on the Meta-World+ MT50 benchmark, TOPPO achieves stronger mean and tail-task performance than prior SAC-family and ARS-family methods while using substantially fewer parameters and reaching their final performance in fewer environment steps. Notably, TOPPO matches or surpasses strong SAC baselines early in training and maintains superior performance at full budget. Ablations confirm the effectiveness of each module in TOPPO and provide insights into their interactions. Our results demonstrate that, with proper optimization, on-policy methods can rival or exceed off-policy approaches in MTRL, challenging the prevailing reliance on SAC and highlighting critic-side gradient conditioning as the central bottleneck.
comment: Accepted at NeurIPS 2026. Camera-ready version. 54 pages (10 main), 29 figures, 24 tables. Code: https://github.com/NAIVlab/TOPPO
♻ ★ Pretraining Knows What, Simulation Teaches How: Compositional Generalization for Agentic Robotics
Frontier models such as GPT-6 Astra now directly control robots and can generalize better than vision-language-action models (VLAs). We argue that this ability could arise from two distinct, composable skills: semantic reasoning, to understand what a task asks for in a scene, and motion control, to move the robot to do it. We show that small open VLMs are already capable of the first and mainly lack the second. We introduce Zero-real Agent Policy (ZAP), a recipe that teaches this missing motion skill, entirely in simulation on simple primitive tasks and from the model's own successes, with no frontier teacher or distillation and no real-world robot data. ZAP reaches 96.7% success on real-world tasks, matching GPT-6 Astra in our harness while running 3x faster and 17x cheaper, and 74.0% on a bimanual adaptation of RoboLab-120, on par with GPT-6 Astra's 72.2%. ZAP also generalizes to new objects, goals, and strategies, and in simulation to a robot embodiment on which it never trained.
♻ ★ Democratizing Robot-Use Agents: A Unified Harness with Open Models
Robot-use Agents (RUAs) have shown promising zero-shot manipulation capabilities, but reliance on frontier, proprietary models limits research into further training. To democratize RUA research, we introduce uni-robot-agent, a unified harness for competitive open-weight robot use that supports further training. Without privileged inputs, an out-of-the-box Qwen3.8-Flash-Next achieves a RoboDojo Score of 31.39, surpassing the published GPT-6 Astra-RoboProbe result of 28.97 and many VLA/WAM baselines, and performs competitively on 19 real-world tasks. Our analysis across model generations, families, and sizes links effective robot use to four general capabilities, suggesting that future RUA training should develop them jointly.
♻ ★ SONIC-K1: Porting a Universal Humanoid Motion Tracker to a Small Humanoid on a Single GPU
We present SONIC-K1, to our knowledge the first open-weights whole-body motion tracker for the Booster K1 humanoid, obtained by adapting the SONIC framework to this smaller robot. The policy is trained from scratch on a single shared GPU with a small fraction of the compute reported for SONIC. In MuJoCo simulation, it tracks nearly all held-out motion clips and performs well on external benchmarks under SONIC's success criterion. On the real K1, it tracks upper-body and slow motions, while stepping and turning remain less reliable, and simulation experiments show a sensitivity to action latency. These results show broad motion-tracking capability on a modest training budget and point to quantitative hardware evaluation as the main next step. Code, policy weights and evaluation tools are publicly available.
comment: 4 pages, 3 figures. Project page: https://kyry.me/sonic-k1
♻ ★ Distributed Swarm Morphogenesis with Neural Cellular Automata
Swarm morphogenesis requires robots to assemble desired shapes through local interactions. We present a modular framework integrating neural cellular automata (NCA) with existing site assignment and collision-aware control. Within this framework, we introduce a decentralized attachment-site discovery mechanism. Robots already in the formation update and share local NCA states to identify new attachment sites, enabling progressive growth without a predefined list of sites during deployment. To bridge the mismatch between conventional NCA growth and gradual physical assembly, we additionally propose offline sparse-retention fine-tuning that simulates partial attachment, together with a false-positive loss that penalizes invalid site activations. Simulations demonstrate formation of diverse lattice morphologies, scaling to several hundred robots, and substantial improvements in morphology accuracy from the proposed fine-tuning. Experiments with Crazyflie quadrotors validate the integrated execution pipeline using robot-relative observations reconstructed from external tracking.
♻ ★ Multi-AUV Ad-hoc Networks-Based Multi-Target Tracking Based on Scene-Adaptive Embodied Intelligence
With the rapid advancement of underwater net-working and multi-agent coordination technologies, autonomous underwater vehicle (AUV) ad-hoc networks have emerged as a pivotal framework for executing complex maritime missions, such as multi-target tracking. However, traditional data-centricarchitectures struggle to maintain operational consistency under highly dynamic topological fluctuations and severely constrained acoustic communication bandwidth. This article proposes a scene-adaptive embodied intelligence (EI) architecture for multi-AUV ad-hoc networks, which re-envisions AUVs as embodied entities by integrating perception, decision-making, and physical execution into a unified cognitive loop. To materialize the functional interaction between these layers, we define a beacon-based communication and control model that treats the communication link as a dynamic constraint-aware channel, effectively bridging the gap between high-level policy inference and decentralized physical actuation. Specifically, the proposed architecture employs a three-layer functional framework and introduces a Scene-Adaptive MARL (SA-MARL) algorithm featuring a dual-path critic mechanism. By integrating a scene critic network and a general critic network through a weight-based dynamic fusion process, SA-MARL effectively decouples specialized tracking tasks from global safety constraints, facilitating autonomous policy evolution. Evaluation results demonstrate that the proposedscheme significantly accelerates policy convergence and achieves superior tracking accuracy compared to mainstream MARL approaches, maintaining robust performance even under intense environmental interference and fluid topological shifts.
comment: Author-accepted manuscript. Published in IEEE Wireless Communications (Early Access), 2026
♻ ★ PACE: Phase-Aware Chunk Execution for Robot Policies with Action Chunking NeurIPS 2026
Action-chunking robot policies predict sequences of future actions, but deployment requires choosing how many actions to execute before replanning. The execution horizon trades off feedback frequency against motion continuity. However, success varies non-monotonically with the execution horizon, and the best-performing horizon differs across tasks. These effects make a single fixed horizon unreliable as a default execution strategy across tasks. We propose PACE (Phase-Aware Chunk Execution), a training-free method that selects execution horizons online from predicted action chunks. PACE uses prominent low-speed valleys in the predicted speed profile as candidate replanning boundaries, exploiting the phase-dependent kinematics of manipulation trajectories. By default, PACE calibrates the valley threshold from each task's existing training demonstrations, without task-specific fixed-horizon tuning. On the 50-task RoboTwin2.0 benchmark, PACE improves average success from 57.8% to 64.2% over the strongest fixed-horizon baseline and outperforms the adaptive execution baselines AAC and AutoHorizon. With GR00T N1.7 on LIBERO, PACE achieves the highest overall success rate among the evaluated strategies, supporting its applicability across base policies, benchmarks, and action representations. On real robots, PACE improves the average task score from 60.7 to 77.7 and success rate from 50.7% to 70.4% over full-chunk execution. Code is available at https://niejunnan25.github.io/pace-project/.
comment: Accepted at NeurIPS 2026. 33 pages including appendices and the paper checklist. Updated experiments, discussion, references, and author list
♻ ★ SNAP: Semantic Navigation Among Pedestrians
Navigating crowded urban environments requires robots to avoid pedestrians while adhering to social norms such as remaining on sidewalks. Existing approaches typically address either crowd avoidance or semantic path planning in isolation. We propose a reinforcement learning (RL) navigation policy that jointly reasons about semantic terrain and dynamic obstacles from a single semantically annotated bird's-eye-view (BEV) map, without relying on explicit human-state estimation. Our policy outperforms established crowd-navigation and semantic-aware baselines and transfers zero-shot to a legged robot, demonstrating socially compliant navigation in highly dynamic real-world scenarios.
♻ ★ Capability-Aware Heterogeneous Control Barrier Functions for Decentralized Multi-Robot Safe Navigation
Safe decentralized navigation in heterogeneous robot teams is challenging because robots with different dynamics and control limits cannot satisfy identical collision-avoidance obligations equally well. Existing methods often rely on model-specific safety formulations or allocate avoidance responsibility without considering whether the assigned control is physically realizable, possibly leading to safety-constrained QP infeasibility and safety violations. We propose Capability-Aware Heterogeneous Control Barrier Functions (CA-HCBF), a decentralized framework for consistent safety enforcement and capability-aware responsibility allocation. We derive a unified operational-space second-order control-affine formulation for holonomic and nonholonomic mobile robots through canonical transformation and backstepping, addressing the relative-degree mismatch while preserving forward invariance of the pairwise safe set. Second, we quantify each robot's achievable control authority along the safety-critical direction using a support-function-based capability metric and use it to distribute pairwise avoidance responsibility. A feasibility-aware clipping mechanism further mitigates physically unrealizable responsibility assignments. Simulations with up to 30 heterogeneous robots and hardware experiments demonstrate high task completion with low safety violations and reduction in local QP infeasibility.
comment: 8 pages, 3 figures, 2 tables
♻ ★ Clinician-AI Collaboration in Endovascular Neurointerventions: In-Vitro User Study of Copilot Assistance Modes for Robotic Navigation Across the Autonomy Spectrum
Mechanical thrombectomy (MT) is an effective treatment for acute ischemic stroke, yet access remains limited by the availability of neurointerventional expertise. Telerobotic and AI-assisted systems may help address this challenge, but it remains unclear how clinicians interact with different levels of autonomy and how autonomous navigation systems perform in realistic environments. This work introduces Copilot Assistance Modes (CAMs), a framework for clinician-AI collaboration in robotic endovascular navigation spanning the autonomy spectrum from manual operation to full autonomy. In the first in-vitro user study comparing clinician-AI collaboration strategies, 11 experts and two students evaluated six navigation modes in a patient-specific vascular phantom. Objective performance metrics were combined with subjective assessments, including workload and usability. Autonomous performance was additionally evaluated both in silico and in vitro to investigate sim-to-real transfer. Hands-on manual navigation achieved the strongest overall performance (100% success). While fully autonomous navigation achieved 81% success in simulation, performance dropped to 23% in vitro. In contrast, several clinician-AI shared-autonomy modes achieved success rates of 92-97%, approximately four times higher than full autonomy and comparable to direct robotic control (95%), demonstrating mitigation of sim-to-real domain shift through clinician-AI collaboration. Among the evaluated AI-driven approaches, user-directed task delegation and rotational shared control demonstrated the most favorable balance between performance and clinician acceptance. These findings suggest that effective clinician-AI collaboration and allocation of control authority may currently be more important for reliable robotic neurointerventional navigation than increasing autonomy alone.
comment: 13 pages, 10 figures, 4 tables
♻ ★ Design of an Underwater Vehicle With Orientation-Adjustable Thrusters and Adaptive Tracking Control
Unmanned underwater vehicles (UUVs) are essential for marine exploration and research. However, conventional designs often struggle with limited maneuverability in complex and dynamic underwater environments. This paper presents an orientation-adjustable thruster UUV (OAT-UUV) with vector thrusters, enabling full six-degree-of-freedom (6-DOF) motion. and composite maneuvers. To overcome challenges associated with uncertain model parameters and environmental disturbances, a vision-based tracking control system based on a novel feedforward adaptive model predictive controller (FF-AMPC) is proposed to achieve robust object tracking, which integrates realtime state feedback with adaptive parameter updating. Extensive experiments, including closed-loop tracking and composite motion tests in real-world environment, have been conducted. The obtained results demonstrate that the orientation-adjustable propulsion design significantly improves the underwater maneuverability, while the FF-AMPC controller achieves a 49.1% hovering improvement compared to conventional controller.
♻ ★ LLM4Robot-Bench: How Far Can GPT-6 Astra Go in Robotic Manipulation? A Hierarchical, Paired Sim-Real Benchmark
We introduce LLM4Robot-Bench, a hierarchical, paired Sim-Real benchmark for evaluating physical interaction capabilities in contact-rich manipulation. Its 16 household tasks span three cumulative levels: contact selection and geometric alignment (L1), force regulation within sustained contact (L2), and coordinated contact reconfiguration and force regulation (L3). Each task is instantiated in physically and visually matched simulated and real-world environments, with shared task definitions and success criteria. We evaluate GPT-6 Astra zero-shot with three trials per task in each environment. Real-world success rates decline from 80.0% at L1 to 47.6% at L2 and 0.0% at L3, with corresponding simulation rates of 66.7%, 47.6%, and 0.0%. Ten of the 16 tasks have identical success counts across environments; the remaining six differ by one success out of three trials. Across both environments, recurring difficulties in spatial reasoning, contact selection, force regulation, and coordination often prevent local physical progress from accumulating into task completion. These findings characterize Astra's observed capability boundary across increasing interaction demands and, for this model, support matched simulation as a repeatable, scalable testbed for studying the sustained, coordinated physical interactions that limit reliable manipulation.
comment: 31 pages, 11 figures. Renamed PhysLadder to LLM4Robot-Bench; minor text revisions
♻ ★ FlowDrive: moderated flow matching with data balancing for trajectory planning
Learning-based planners are sensitive to the long-tailed distribution of driving data. Common maneuvers dominate datasets, while dangerous or rare scenarios are sparse. This imbalance can bias models toward the frequent cases and degrade performance on critical scenarios. To tackle this problem, we compare balancing strategies for sampling training data and find reweighting by trajectory pattern an effective approach. We then present FlowDrive, a flow-matching trajectory planner that learns a conditional rectified flow to map noise directly to trajectory distributions with few flow-matching steps. We further introduce moderated, in-the-loop guidance that injects small perturbation between flow steps to systematically increase trajectory diversity while remaining scene-consistent. On nuPlan and the interaction-focused interPlan benchmarks, FlowDrive achieves state-of-the-art results among learning-based planners and approaches methods with rule-based refinements. After adding moderated guidance and light post-processing (FlowDrive*), it achieves overall state-of-the-art performance across nearly all benchmark splits. Our code is available at https://github.com/einsteinguang/flow_drive_planner.
♻ ★ Real-Time Bounded Catenary Solver for UAV Tether Modeling
For non-stationary tethered multirotor UAVs in real-world conditions, simulating the forces imposed on the drone by the aerodynamic drag of the tether becomes crucial, with online use cases placing a hard bound on the maximum solve time. In previous work, a quasi-analytical catenary tether model reached a mean solve time of 0.51 ms using a general-purpose root finder, but without any worst-case guarantees or proven convergence. In this work, we reformulate the inner solver by reducing the catenary boundary-value problem to a single transcendental equation in one well-conditioned unknown. We derive a closed-form bracket and prove monotonicity and convexity as well as existence and uniqueness of the root, which together guarantee convergence of the solver. We further propose a two-regime initial guess which approximates the true root within 3.4 % and reduces the mean iteration count by 69.2 % to 2.24 compared to the textbook initialization. Building on the hybrid root-finding method rtsafe (Newton-Raphson with bisection fallback giving bounded iteration counts), we implement a specialized variant that exploits the problem structure to omit unnecessary checks while retaining correctness, which gives up to 1.37 times speedup. With the proposed solver the full tether model achieves a nearly constant solve time of 4.0 us on average and 4.6 us at worst, a 68 times speedup over an optimized re-implementation of the previous method, while agreeing with it to a relative deviation of 8.7e-9. Because the reformulation leaves the underlying physical model untouched, the experimental validation of the previous work carries over unchanged. We further demonstrate its suitability for embedded, resource-constrained platforms with a Lua implementation running directly in ArduPilot on a drone's flight controller, where it stays well inside the scheduling budget with a mean solve time of 0.74 ms.
♻ ★ Comparing Trajectories from Positions Alone: Curvature-Based Time Alignment and Drift Error Metric
In field robotics, acquiring independent large-scale reference trajectories more accurate than the evaluated estimates remains an open challenge. The domain is widely reliant on Absolute Trajectory Error (ATE) and Relative Pose Error (RPE), computed with automated tools, that rest on assumptions and evaluation parameters rarely made explicit. When unreported, the errors can be misleading and hinder fair comparisons. This paper introduces a trajectory-evaluation protocol for standardized and reliable accuracy assessment in state estimation, localization, and Simultaneous Localization And Mapping (SLAM). The approach combines a novel temporal alignment method based on curvature signals with an error metric normalized by travelled distance. We explicitly account for temporal synchronization, sampling alignment, and extrinsic calibration, quantifying their influence through a sensitivity analysis. The proposed protocol contributes to more rigorous, reproducible, and standardized trajectory evaluation.
♻ ★ ResponsibleRobotBench: Benchmarking Risk-Aware Task-Level Decision Making for Robotic Manipulation with Large Multimodal Models
Safe task-level manipulation requires a planner to recognize hazards, select appropriate mitigation actions, and satisfy their execution prerequisites without abandoning the original goal. We introduce ResponsibleRobotBench, a benchmark of 23 manipulation task configurations with separately specified goals and safety conditions. It connects hazard-category predictions, planned actions, execution feedback, and terminal outcomes at the episode level. Across evaluations of five large multimodal models, terminal safety and safe task completion can diverge: defense prompts increase terminal safety for three tested models but affect safe success in different directions. Correct hazard-category predictions also fail to ensure executable mitigation. In a cutting task under the default configuration, GPT-4o mini identifies the human hazard in all 100 episodes but never requests assistance, and every episode ends unsafe. Offering a help action and actually requesting it lead to different comparisons: outcomes differ between action menus even for a model that never requests help, and repeated requests can leave execution failures unresolved. Imitation-learning evaluations and physical robot sequences illustrate the need to coordinate mitigation with goal actions. Together, these results motivate evaluating mitigation selection, action prerequisites, and recovery alongside hazard recognition and terminal outcomes. The appendix provides task definitions, checker conditions, and prompts. Project website: https://responsible-robot-benchmark.github.io/.
comment: 28 pages, including a 12-page appendix. Revised manuscript with expanded experimental analysis and supplementary details. Project website: https://responsible-robot-benchmark.github.io/
♻ ★ LQG solution for POMDP without estimating states: A minimum variance approach
This paper investigates the control of discrete-time linear time-invariant (LTI) systems subject to incomplete and corrupted measurements. Specifically, we focus on designing a Linear Quadratic Gaussian (LQG) controller without relying on explicit state estimation. By leveraging minimum variance duality, our approach allows the current control input to be represented as a linear function of available measurements and previously applied inputs, successfully reducing the task to a tractable deterministic optimization problem. We provide theoretical justification for this framework and demonstrate its practical effectiveness through numerical experiments.
♻ ★ World-Model-Augmented Visual Locomotion for Humanoids on Foothold-Constrained Terrain
Foothold-constrained terrain limits feasible footholds to sparse, discontinuous, or narrow support regions, as in gaps, stepping stones, stairs, and balance beams. In such settings, a single misplaced step can cause a fall. Reliable traversal therefore requires precise foot placement and anticipation of upcoming terrain. To address this challenge, we present World-Model-Augmented Visual Locomotion (WM-LOCO), which augments a visual locomotion policy with predictive context from a jointly trained world model. Given onboard depth observations and proprioception, a recurrent state-space model (RSSM) learns action-conditioned latent dynamics that predict future proprioceptive states, egocentric observations, and task rewards. The resulting recurrent feature is then fed into the proximal policy optimization (PPO) actor-critic, providing predictive context for anticipatory foot placement on foothold-constrained terrain. WM-LOCO achieves average success rates of 96.0% on gaps, 96.9% on stepping stones, 94.0% on stairs, and 92.1% on balance beams, outperforming the evaluated baselines while improving gait efficiency and smoothness. Furthermore, WM-LOCO achieves zero-shot sim-to-real transfer to a Unitree G1, with 92.5% success across these four terrain classes.
comment: 8 pages, 6 figures, 4 tables. Yuxi Liu and Lijun Han contributed equally
♻ ★ Multimodal Socially Aware Framework for Human-Robot Interaction in Virtual Reality
Social robots must coordinate speech and movement while interpreting multimodal human input. We present a multimodal framework that integrates speech, vision, and touch to generate contextually appropriate robot speech and movement. We evaluated the framework in virtual reality with 18 participants across five social interaction scenarios, comparing its performance against a human-teleoperated baseline. Although participants rated the autonomous system above the neutral midpoint for both naturalness and social appropriateness, the teleoperated baseline received significantly higher ratings on both measures. Behavioral coding and qualitative feedback revealed that conversational timing, clarification of ambiguous requests, and spatial coordination were important factors contributing to this performance gap. These findings highlight key challenges in developing autonomous robots capable of natural, multimodal social interaction.
♻ ★ SimuScene: Simulation-Ready Compositional 3D Scene Reconstruction from a Single Image
Reconstructing interactive, simulation-ready 3D scenes from a single image is a critical bottleneck for robotic manipulation. While recent single-image lifters recover plausible per-object shapes, composing them yields scenes that collapse under physical simulation due to interpenetrating, hovering, or sinking objects. Existing physics-aware compositional reconstruction methods mainly address this as a post-hoc layout correction, leaving the underlying geometric errors unresolved. To address this, we introduce SimuScene, a compositional 3D reconstruction pipeline that puts physics in the loop of shape and layout estimation. Rather than using physics merely for layout cleanup, we utilize the physics engine as a diagnostic measurement tool during the generative process itself. By diagnostically simulating reconstructed objects under gravity, we convert penetration and support failures into quantitative correction signals that drive amodal shape resampling, with gravity-axis stretching as a lightweight correction for minor errors. This physics-informed feedback loop mitigates accumulated reconstruction errors and produces a stable, simulation-ready compositional 3D scene. Extensive experiments demonstrate state-of-the-art performance on physical stability and geometric alignment benchmarks. We further highlight SimuScene's utility by deploying reconstructed environments in humanoid control and robot-arm manipulation tasks.
comment: Project Page: https://snuvclab.github.io/SimuScene/
♻ ★ SAFEVPR: Patch-Based Verification with Empirical FDR Control for Cross-Condition Sequence Visual Place Recognition
Sequence-based visual place recognition (VPR) for loop closure and robot relocalization must decide whether the retrieved top-1 candidate should be accepted. Conformal calibration can set this accept/reject threshold, but its finite-sample guarantees require exchangeability between calibration and deployment, a requirement that cross-condition deployment violates. We therefore claim no formal guarantee under condition shift: this paper instead asks which verification score and calibration design keep a target false-discovery rate (FDR) empirically satisfied across realistic appearance change. We present SAFEVPR, a non-trainable pipeline that replaces the backbone cosine similarity with a mutual-nearest-neighbour (MNN) patch score from frozen DINOv2 features, and flat Learn-Then-Test (LTT) calibration with per-bin Clopper-Pearson thresholds. Across 23 cross-condition ground-vehicle setups (Oxford RobotCar, NCLT, St Lucia; three frozen backbones), SAFEVPR meets the FDR target on 23/23 setups at alpha = 0.10 (mean accepted FDR 0.014, mean TPR 0.75) and on 22/23 at alpha = 0.05 with 22 retaining non-trivial coverage (best cosine variant, cosine+LTT: 22/23 valid but 17 non-trivial); under leave-one-dataset-out calibration, validity drops to 20/23 with 14 retaining useful coverage (strongest cosine baseline: 16/23 and 10), and on texture-poor scenery (Nordland) the verifier is uninformative and the system abstains entirely, as it does everywhere at alpha = 0.01; we report these as limitations. In a pose-graph relocalization study, gating loop-closure constraints reduces erroneous constraints by 29.6x and cuts the median trajectory error from 95.5 m to 9.7 m. Code is available at https://github.com/TIERS/SafeVPR.
comment: accepted by RA-L
♻ ★ LIBERO-RECOVER: Beyond Task Success Towards Failure Recovery in Robotic Manipulation Models
Vision-Language-Action (VLA) or World Action (WAM) models have recently demonstrated remarkable performance in robotic manipulation. On LIBERO, SOTA method have achieved nearly 100\% success rates, seemingly suggesting that the models are ready for deployment in real world. However, near perfect performance on existing benchmarks can be misleading: success under ideal conditions does not imply real world robustness. Existing benchmarks primarily evaluate task completion from predefined initial states, while real world interactions inevitably involve failures such as failed grasps, collisions, and unintended object movements. A robot must therefore not only execute tasks successfully, but also recognize and recover from failures to continue the task. Yet this capability remains largely unmeasured, revealing a critical gap between benchmark performance and real world reliability. To address this gap, we introduce LIBERO-Recover Benchmark, a large scale benchmark for failure recovery in robotic manipulation. Built upon LIBERO, we collect real execution failures from SOTA embodied models and construct 1,000+ scenarios across four recovery levels: (1) Action Retry, (2) Action Adaptation, (3) Object State Recovery, and (4) Environmental Recovery. We evaluate four core capabilities: spatial understanding, object structure reasoning, interaction understanding, and topological reasoning. As the first large-scale benchmark for embodied failure recovery, LIBERO-Recover shifts evaluation from \emph{Can the robot succeed?''} to \emph{Can the robot recover after failure?''}, promoting robust and generalizable embodied agents. The project will be avaible in \textcolor{blue}{https://liulin815.github.io/LIBERO-Recovery/}.
♻ ★ PRIME W0: Physical Priors from World Models for Embodied Manipulation
Vision-Language-Action (VLA) models excel at instruction following, yet most remain feed-forward: they condition actions only on the current observation and discard both long-horizon temporal context and the hierarchical structure of VLM representations. We present PRIME W0, a hierarchical conditional flow-matching framework that closes this gap through three synergistic designs: (1)per-layer KV injection from a pretrained VLM into an Adaptive MoE action expert; (2)dynamics-aware conditioning via asynchronous world-model subgoal-image prediction and a DINOv2 historical-frame semantic bridge; and (3)Action Dynamic Manifold (ADM) flow matching with covariance-structured noise that respects robot kinematic coupling. PRIME W0 is trained in four stages---VLM pre-training, DAT/NTP visuomotor pre-training, DiT flow-matching decoder assembly with WM injection, and end-to-end SFT---and achieves state-of-the-art results on LIBERO and real-robot benchmarks, with systematic ablations validating each architectural component.
♻ ★ VT-MUSE: Multimodal Unified Sequential Visuotactile Representation Learning for Manipulation
We propose VT-MUSE, a Multimodal Unified SEquential representation learning framework for visuotactilemanipulation. Existing approaches often encode visual and tactile observations independently before fusion, limiting their ability to capture fine-grained cross-modal dependencies. Moreover, most methods focus on observations at the current time step and overlook the temporal evolution of contact. VT-MUSE addresses both limitations through a two-stage representation learning framework. In Stage I, modality specific encoders are jointly adapted via cross-modal temporal alignment and masked-view consistency. In Stage II, a conditional variational latent model processes masked visual sequences together with full tactile histories. Auxiliary decoders reconstruct the masked recent visual observations and predict tactile depth changes, encouraging the latent representation to retain both global visual context and local contact dynamics. The learned representation is subsequently integrated into a lightweight Transformer policy through gated cross-attention. On the simulation benchmark, VT-MUSE outperforms the strongest baseline evaluated on all tasks by 11 percentage points and also achieves substantial improvements in real-world experiments.
♻ ★ Track-and-Complete: Learning Humanoid Skills from a Single Failed Human Video
Learning humanoid skills from videos typically requires a successful human demonstration, which often demands custom data collection. Although failures have traditionally been treated only as negative examples in robot learning, they can still reveal a usable trajectory prefix before the task fails, as well as the intended outcome. To leverage this information from a failed-attempt video, we propose TRACC, a pipeline that imitates the useful portion of the motion trajectory and then completes the task based on the inferred task outcome. The usable motion prefix serves as prior knowledge until the failure occurs, after which the task-completion reward guides the policy to learn the intended task goal without requiring a successful task trajectory. We evaluate our method on six in-the-wild failed human tasks from the Oops! dataset. Our experimental results demonstrate the effectiveness of the proposed approach for learning from failed attempts when no successful demonstration is available. Thus, these findings establish failed human videos as a viable source of supervision for humanoid skill learning.
comment: 8 pages, 6 figures, 6 tables. Project website: https://tracc-humanoid.github.io
♻ ★ MagPillar: A Soft Magnetopillar Array Tactile Sensor for Robotic Surface Perception
Robotic surface exploration benefits from tactile interfaces that accommodate contact and capture variations in surface characteristics. Herein, we report MagPillar, a tactile sensor inspired by spider tactile hairs that combines a magnetic pillar array with three-axis Hall sensing. A full-array mechanical and magnetic-field model characterizes how pillar deformation affects the superposed fields measured across the array, providing a physical basis for interpreting its contact responses. The compliant pillar architecture translates contact with surface textures and geometric features into distinct magnetic response patterns. Then, in robotic surface perception experiments, spectral features and logistic regression distinguish six material surfaces with an offline accuracy of 99.34% under within-session temporally blocked cross-validation. Continuous scans over periodic protrusions produce amplitude and peak-spacing variations consistent with changes in surface geometry. These findings connect compliant contact mechanics with distributed magnetic sensing, paving a new path for material recognition and qualitative surface-profile perception.
♻ ★ CLAP: Direct VLM-to-VLA Adaptation via Language-Action Grounding
Vision-language-action models (VLAs) inherit semantic capabilities from pretrained VLMs, yet large-scale post-training on robot data and architectural modifications can reshape the backbone so extensively that it becomes difficult to isolate what the VLM contributes to control. Directly converting pretrained VLMs into VLAs with minimal architectural change offers a more transparent path to understanding how VLM capabilities transfer across model scales. The core obstacle is output-distribution mismatch: predicting actions as bare numeric token sequences moves generation away from the VLM's pretrained language distribution, degrading the capabilities we seek to preserve. To address this, we propose CLAP (Causal Language-Action Prediction), which prepends each numeric action sequence with a natural-language action description, causally conditioning precise action-token prediction on a language-action plan without modifying the backbone architecture. With only 4-5 hours of training, 2B CLAP achieves 90.8% on LIBERO (+14.9 pt over VLA-0) and improves robustness on LIBERO-PRO under language, object, and spatial perturbations. Ablations attribute the gains to action-grounded language semantics, and on a real UR5e robot 2B CLAP reaches 60% success against 15% for matched VLA-0. We release CLAP at 0.8B, 2B, and 4B as an open-weight, multi-scale compact VLA family from a single VLM lineage, enabling controlled analysis of VLM-to-VLA capability transfer.
comment: Accepted to CoRL 2026. Project page: https://omron-sinicx.github.io/clap
♻ ★ Large Multimodal Models for Embodied Intelligent Driving: Enabling the Next Generation of Autonomous Mobility
The advent of Large Multimodal Models (LMMs) offers a promising technology to tackle the limitations of modular design in autonomous driving, which often falters in open-world scenarios requiring sustained environmental understanding and logical reasoning. Besides, embodied artificial intelligence facilitates policy optimization through closed-loop interactions to achieve the continuous learning capability, thereby advancing autonomous driving toward embodied intelligent (EI) driving. However, such capability will be constrained by relying solely on LMMs to enhance EI driving without joint decision-making. This article proposes a novel semantics and policy dual-driven hybrid decision framework to tackle this challenge, ensuring continuous learning and joint decision. The framework merges LMMs for semantic understanding and cognitive representation, and deep reinforcement learning for real-time policy optimization. This article particularly details the foundational principles of EI driving and LMMs, and also introduces the emerging opportunities this framework enables, encompassing potential benefits and representative use cases. The proposed framework, evaluated through a lane-change planning case study, achieves at least a 19% performance improvement over the baselines, as verified by simulations. Finally, key open challenges and future research directions are identified to guide subsequent work.
♻ ★ What the Guard Misses, the Robot Executes: Implied Harm in VLA Instructions SP
Vision-language-action models (VLAs) act on instructions without being able to refuse, so screening harmful requests falls to monitors. We test whether these monitors catch ordinary robot tasks requested for harmful reasons, holding the task fixed while varying only how explicitly the intent is stated. $π_{0.5}$ completes the task at every level of explicitness, as often as for harmless controls. Text guards flag nearly every blunt request but few implied ones: up to 95% of implied-harm runs end with the task done and no flag raised, and most still leak after recalibrating on robot instructions at low false-positive rates. Monitoring the model's activations does not close this gap. Linear probes separate harmful from harmless instructions almost perfectly in the base language and vision-language models, but this separation weakens after robot training in two model families, most for implied harm.
comment: Accepted at the Science of Physical AI Safety (SPAIS) Workshop at CoRL 2026
♻ ★ IntentNav: Learning Spatial-Visual Object Navigation from Human Demonstrations
Object navigation requires a robot to search for an unobserved target in an unknown environment by deciding where to explore next under partial observability. Effective search resembles human-like exploration: selectively probing visually promising frontiers while relying on spatial memory to avoid redundant revisits. We propose IntentNav, a spatial-visual imitation framework that learns human-like ObjectNav policies from human demonstrations. To infer high-level search intent from low-level human actions, we introduce Frontier-based Human-Intent Labeling, which looks ahead in human demonstrations and labels the frontier that best explains the demonstrator's future search direction. We construct a spatial-visual candidate space, where BEV memory tracks explored regions, unexplored frontiers, and trajectory history, while egocentric visual memory provides semantic cues for each candidate. A VLM policy is trained to select among these grounded candidates, using Intent-Aligned Objective to encourage consistent and human-like exploration. IntentNav achieves state-of-the-art performance among learning-based methods on the MP3D, HM3D-v1, and HM3D-v2 ObjectNav benchmarks, attaining the best success rate among all compared methods. The proposed candidate-level navigation interface transfers zero-shot to wheeled, quadruped, and humanoid robots without further VLM fine-tuning. \href{https://wmn7.github.io/IntentNav/}{Project page}.
comment: 26 pages, 9 figures
♻ ★ EchoNav: Evidential BEV Completion from Historical Observations for Visual Navigation
Visual navigation in cluttered environments requires robots to recover previously observed structure and infer obstacles and free space that remain occluded. We present a framework that combines temporal aggregation and occupancy completion in an evidential bird's-eye-view (BEV) representation predicted from short RGB sequences. A decoder adapts frozen VGGT-Omega features and combines historical residual proposals using cellwise weights and a learned no-update option. The BEV predictor produces a local map in the latest ego frame without depth or camera-pose inputs or explicit geometric warping. The evidential model treats observed free space separately and predicts occupancy with associated confidence across the remaining supported region. We train the predictor on simulated RGB sequences paired with complete and visibility-masked occupancy annotations. Our experiments evaluate navigation in simulation and on a wheeled robot, with additional demonstrations on quadruped and humanoid robots. In simulated sequential two-goal navigation, our system achieves 89.6% success, outperforming the strongest evaluated baseline by 32.5 percentage points. Across 60 wheeled-robot trials in 20 scenes, it achieves 73.3% success, compared with 33.3% for the strongest evaluated baseline.
comment: 8 pages, 4 figures
♻ ★ MagCilia: A Compact Magnetociliary Tactile Sensor with 3D Force Sensing for Robotic Contact Perception and Grasping Feedback
Robotic grasping and surface exploration benefit from simultaneous measurement of normal and tangential forces and from surface information obtained through contact. Here, we present a compact magnetociliary tactile sensor (MagCilia) that combines a flexible magnetic-cilia structure with a Hall sensor for 3D force sensing. Quasi-static finite element analysis is used to investigate structural deformation and magnetic responses under multidirectional loading. To reconstruct forces from the coupled magnetic channels, we propose causal history fusion regression (CHFR), which combines current magnetic-field measurements with their recent changes. Five-fold cross-validation grouped by calibration record yields root-mean-square errors of 0.40, 0.57, and 0.69 N for Fx, Fy, and Fz, respectively, with corresponding coefficients of determination of 0.93, 0.90, and 0.92. Robotic experiments demonstrate tangential-force-guided gripper adjustment and multi-axis load monitoring under external perturbations. Frequency-domain features of the reconstructed forces distinguish six surface categories with 99.39% accuracy in three-fold cross-validation grouped by acquisition session. An online robotic demonstration additionally identifies all six tested surfaces. These results demonstrate 3D force reconstruction, grasping feedback, and surface recognition using a single compact tactile unit.
♻ ★ DexArena: A Unified Benchmark for Long-Horizon Dexterous Manipulation
Long-horizon dexterous manipulation remains a fundamental challenge for robot learning, requiring the composition of multiple skills, precise bimanual coordination, generalizable visual understanding, and robust skill transitions. Yet existing manipulation benchmarks largely emphasize isolated skills or support only a limited range of end-effectors, making these capabilities difficult to evaluate systematically. We introduce DexArena, a benchmark designed to evaluate learning and generalization in long-horizon bimanual dexterous manipulation. DexArena features 10 long-horizon tasks and 30 stage-level tasks spanning diverse household and industrial scenarios. Each long-horizon task is composed of explicitly defined stages with physically grounded success conditions, enabling reliable evaluation of both intermediate progress and end-to-end task completion. A unified benchmark interface and verified human-teleoperated demonstrations enable testing different policy learning algorithms, with standardized simulation configurations, scene randomization, trajectory replay, and task-level evaluation. Through extensive baseline experiments, we identify substantial challenges in precise manipulation, generalization to scene variation, and skill composition. DexArena establishes a challenging and reproducible benchmark for developing robot learning methods that move toward generalizable long-horizon dexterous behavior.
comment: Project page: https://sipair-elastic.github.io/DexArena-website/
♻ ★ Scaling Nonlinear Optimization: Many Problems One GPU
Modern GPUs and software frameworks have made sampling-based methods, such as reinforcement learning and model predictive path integral control, increasingly popular in robotics. However, the logic these methods can execute per sample is limited to what has been implemented on the GPU. Nonlinear programs (NLPs) are one of the most expressive tools in robotics, reliably minimizing a cost while exactly enforcing general constraints. NLP optimizers were among the field's main workhorses, but were not designed for modern GPU hardware. Embedding NLPs such as trajectory optimization and inverse kinematics directly within each sample would be highly valuable, yet the only current option is to call a CPU solver per sample, possibly in parallel on a multi-core CPU. This bottlenecks the CPU, requires GPU-to-CPU-to-GPU transfers when the rest of the pipeline resides on the GPU, and leaves the GPU intermittently idle. To address this issue, in this work we present jaxipm, the first GPU-batched solver for generally constrained nonlinear optimization problems. Our approach is a JAX implementation of the battle-tested IPOPT algorithm, facilitated by eliminating control flow through heterogeneous iteration fusion, and minimizing GPU idle time through iteration-level batching. Our implementation enables solving hundreds of NLPs in parallel on a single GPU. We validate jaxipm on a range of quadrotor nonlinear model predictive control benchmarks, where on a single GPU we achieve between 30.7% and 69.9% of the throughput of 64 IPOPT instances running in parallel on a 64-core CPU. open-source codebase: https://github.com/johnviljoen/jaxipm
comment: 9 pages, 6 figures, ieeeconf style, submission for RA-L
♻ ★ DexCanvas: Bridging Human Demonstrations and Robot Learning for Dexterous Manipulation
A dexterous hand succeeds or fails through contact, yet every scalable record of human manipulation (video, vision-based hand tracking, motion capture) observes how the hand moves and misses how it pushes. We present DexCanvas, a hybrid real-synthetic dataset of 7,000 hours of dexterous hand-object interaction with per-frame contact points and contact forces, grown from 70 hours of optical motion capture that spans 21 manipulation types, from power and precision grasps to in-hand manipulation. The forces come from a real-to-sim pipeline: one reinforcement learning (RL) policy, conditioned on object geometry and manipulation type, drives an actuated MANO hand in physics simulation to reproduce each captured object trajectory, and the simulator measures the contacts that produce the reproduced motion. The policy reproduces 87.5% of the demonstrations and still succeeds on 80.9% of rollouts from perturbed initial object poses; these perturbed rollouts expand the 70 captured hours a hundredfold. On six YCB objects absent from the capture set it succeeds 72.5% of the time. The contact channels make better policies: a Diffusion Policy that observes DexCanvas contact features succeeds 82.2% of the time, against 67.5% without them, over 18,522 trials. Contact-driven retargeting carries the same demonstrations to two dexterous robot hands, where the RL stage again produces stable rollouts. A first release of 111,857 force-annotated trajectories is publicly available on Hugging Face.
comment: 22 pages, 9 figures. DexGEM Lab technical report. Dataset: https://huggingface.co/datasets/Dex-GEM/DexCanvas
♻ ★ DynaFLIP: Rethinking Robotics Perception via Tri-Modal-Dynamics Guided Representation
Robot manipulation critically depends on perception that preserves the action-relevant aspects of a scene. Yet most robot learning pipelines are built upon visual encoders pre-trained for static recognition or vision-language alignment, leaving motion understanding to downstream policies. We introduce DynaFLIP, a dynamics-aware multimodal pre-training framework that pushes motion understanding upstream into perception. We construct image-language-3D flow triplets from heterogeneous human and robot videos, and use these triplets as training-time supervision to shape an image-only encoder. Our key idea is to encourage the three modalities to span a small simplex volume in the shared hyperspherical space -- a smaller simplex volume indicating stronger alignment. To avoid the geometric ambiguity and trivial collapse of naive volume minimization, we combine simplex-volume minimization with a cosine regularizer and a contrastive objective. Our analyses show that DynaFLIP focuses on control-relevant regions critical for manipulation. The resulting dynamics-aware representations serve as reusable visual backbones and consistently outperform baselines across diverse downstream policies, including VLAs. We validate this across diverse simulation and real-world setups, with gains reaching +22.5% under out-of-distribution scenarios. Our results suggest that robot generalization improves when visual representations are trained to encode not just what is present, but how the world changes under action.
comment: Project website: https://dynaflip-robotics.github.io
♻ ★ Dexterous Robot Manipulation from Human Demonstrations via Contact-Anchored Retargeting and Residual Policy Learning
Learning dexterous manipulation from demonstrations is bottlenecked by data: the contact forces that determine whether a grasp succeeds are absent from every scalable source of human demonstrations. This paper builds on two observations. First, what survives the change from a human hand to a robot hand is the contact structure of a demonstration - which finger regions touch which object locations, and in what order - rather than its joint motion. Second, physical consistency need not be engineered per task: a single residual reinforcement learning (RL) policy, trained once across diverse demonstrations, can repair kinematic recordings into physically consistent, contact-annotated trajectories, and the same residual formulation restores dynamic feasibility after retargeting. These observations yield a three-stage pipeline that converts human motion-capture recordings into dexterous robot policies with no real-robot training data: physics refinement with a simulated MANO hand recovers contacts and forces, contact-anchored retargeting transfers the demonstrated contact structure through an objective independent of hand morphology, and residual policy learning adapts the result to robot actuation. The pipeline reconstructs 25,454 single-hand trajectories (success 7.3% -> 59.3%) and 25 dual-hand tasks (16.0% -> 62.4%) with one shared policy per setting, transfers one human dataset to four morphologically distinct robot hands (+62.4 pp), and executes four contact-rich bimanual tasks on physical hardware with zero real-robot training data.
comment: 16 pages, 6 figures, 5 tables. Technical report. Code: https://github.com/DexGEM-Lab/real2sim2real
♻ ★ What Should a Navigator Remember? Spatiotemporal Memory for Zero-Shot Vision-Language Navigation
Zero-shot vision-and-language navigation (VLN) requires an agent to use past observations without navigation-specific training. We study what information an 8B-scale navigation system should retain and how it should be organized for subsequent decisions. Our agent combines instruction progress, hierarchically compressed observation history anchored to poses, and accumulated geometry in complementary task, temporal, and spatial memories. An 8B observer summarizes observations, and an 8B reasoner uses the memories to select waypoint intents or request additional views, a geometric controller executes the selected targets. A semantic information perspective motivates the distinction between retaining evidence, making it accessible to the reasoner, and acquiring further observations. On R2R-CE val-unseen, the complete system achieves the highest among the evaluated Qwen3-based zero-shot baselines. Removing temporal memory reduces SR and sharply increases average path length from 16.9 to 39.1m, removing the spatial-map and planning component reduces SR as well. Real-world experiments on a Unitree Go2 show that our zero-shot method achieves higher success rates than the post-trained baseline in both corridor and office environments, suggesting better transfer to the evaluated real-world settings. As a complementary evaluation, LAW2 uses paired instruction interventions to examine behavioral responses, instruction following ability beyond endpoint success. These results support preserving task progress, observation history, and geometry in structured memory for zero-shot navigation with smaller open models.
comment: 20 pages, 3 figures, 8 Tables
♻ ★ Decoding Geometry, Not Tokens: Efficient VLMs for Robot Affordance and Trajectory Prediction WACV 2027
Predicting where to grasp and how to move is fundamentally a geometry problem. Vision-language models treat it as language generation instead. They tokenize spatial coordinates into text and decode them autoregressively. This mechanism ignores spatial structure and truncates continuous precision. Errors compound across coordinate tokens during decoding. In this paper, we show that compact off-the-shelf VLMs with 0.5-2B parameters already encode rich spatial information in their intermediate hidden states. Autoregressive decoding is the dominant bottleneck for trajectory prediction, while affordance prediction depends additionally on feature depth and decoder architecture. We expose the latent spatial information with a lightweight, geometry-compatible prediction head that attaches to a frozen backbone. It reads features from an intermediate hidden layer, conditions a learned query on the task instruction and refines it via cross-attention over the visual tokens to regress affordance boxes and trajectory waypoints directly as continuous values, without modifying the backbone. We evaluate on ShareRobot-Bench and AGD20K. Our 2B variant reduces trajectory error by over 60% and improves ShareRobot-Bench affordance mAP by 85% relative to the autoregressive baseline. It approaches RoboBrain on trajectory prediction (0.251 vs 0.191 DFD) at 3.8x higher throughput. These results demonstrate a favorable accuracy-efficiency trade-off for instruction-conditioned 2D geometric prediction.
comment: IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2027)
♻ ★ TFusionOcc: T-Primitive Based Object-Centric Multi-Sensor Fusion Framework for 3D Occupancy Prediction
The prediction of 3D semantic occupancy enables autonomous vehicles (AVs) to perceive the fine-grained geometric and semantic scene structure for safe navigation and decision-making. Existing methods mainly rely on either voxel-based representations, which incur redundant computation over empty regions, or object-centric Gaussian primitives, whose ellipsoidal per-primitive geometry can limit representation efficiency for complex, non-convex, and asymmetric structures under a fixed primitive and computational budget. In this paper, we present TFusionOcc, a T-primitive-based object-centric multi-sensor fusion framework for 3D semantic occupancy prediction. Specifically, we introduce a family of Student's t-distribution-based T-primitives, including the plain T-primitive, T-Superquadric, and deformable T-Superquadric with inverse warping, where the deformable T-Superquadric serves as the key geometry-enhancing primitive. We further develop a unified probabilistic formulation based on the Student's t-distribution and the T-mixture model (TMM) to jointly model occupancy and semantics, and design a tightly coupled multi-stage fusion architecture to integrate camera and LiDAR cues effectively. Extensive experiments on nuScenes demonstrate competitive performance against recent state-of-the-art methods and strong performance among object-centric multi-sensor approaches, while additional evaluations on nuScenes-C show robust performance across most corruption scenarios. Further evaluation on the separate SemanticKITTI benchmark demonstrates cross-dataset applicability. The code will be available at: https://github.com/DanielMing123/TFusionOcc
♻ ★ Beyond Visibility: Real-Time Surface Accessibility Fields from Sparse LiDAR
Understanding which surfaces a given tool can physically access is fundamental to robotic interaction, yet 3D perception typically stops at geometric reconstruction or visibility estimation. Visibility does not imply accessibility: a visible point may still be blocked by tool geometry or approach-corridor clearance. We propose the Accessibility Field: a per-point surface accessibility labelling, computed from streaming sparse LiDAR and updated at sensor rate as the platform moves. Our GPU-based method evaluates surface points against precomputed tool-geometry kernels across rotated approach orientations, underpinned by a scan-centric TSDF integration scheme suited to non-repetitive sensors such as the Livox Mid-360. The system is tool-agnostic, requires no prior scene model, and runs in real time on both workstation and Jetson Orin edge hardware. On synthetic primitives, our method achieves F1=90.8%, against 69.8% for a Hidden Point Removal baseline; ablations within our own pipeline give 81.5% replacing the TSDF query with a point-cloud collision check, and 33.9% replacing scan-centric integration with projective, nvblox-style integration. On a mature synthetic Pinus radiata model, the method labels 56.8% of visible pine branch surfaces as physically inaccessible.
comment: Accepted to the 2026 Australasian Conference on Robotics and Automation (ACRA 2026), to be presented 30/11-2/12
♻ ★ SMAT: Staged Multi-Agent Training for Co-Adaptive Exoskeleton Control IROS 2026
Effective exoskeleton assistance requires co-adaptation: as the device alters joint dynamics, the user reorganizes neuromuscular coordination, creating a non-stationary learning problem. Most learning-based approaches do not explicitly account for the sequential nature of human motor adaptation, leading to training instability and poorly timed assistance. We propose Staged Multi-Agent Training (SMAT), a four-stage curriculum inspired by how users acclimate to a wearable device. In SMAT, a musculoskeletal human actor and a bilateral hip exoskeleton actor are trained progressively: the human first learns unassisted gait, then adapts to the added device mass; the exoskeleton subsequently learns a positive assistance pattern against a stabilized human policy, and finally both agents co-adapt with full torque capacity and bidirectional feedback. We implement SMAT in the MyoAssist simulation environment using a 26-muscle lower-limb model and an attached hip exoskeleton. Our musculoskeletal simulations demonstrate that the learned exoskeleton control policy produces an average 10.1% reduction in simulated hip muscle activation relative to the no-assist condition. We validated the learned controller in an offline setting using open-source gait data, then deployed it to a physical hip exoskeleton for treadmill experiments with five subjects. The resulting policy delivers consistent assistance and predominantly positive mechanical power without the need for any explicitly imposed timing shift (mean positive power: 13.6 W at 6 Nm RMS torque to 23.8 W at 9.3 Nm RMS torque, with minimal negative power) consistently across all subjects without subject-specific retraining.
comment: Accepted to IROS 2026. 8 pages, 7 figures. v2: camera-ready version; added code link and funding acknowledgment; clarified method details and limitations
♻ ★ WeaveRL: Weaving Reconstruction into Scene-Aware Fabrics for Perceptive Reinforcement Learning
Reinforcement learning allows robots to acquire complex skills, but producing policies for geometrically complex manipulation remains difficult. A promising approach is to learn on top of collision-avoidant controllers, such as geometric fabrics. However, these approaches have relied on static, hand-specified representations of the scene. Integrating active, online 3D perception into massively parallel RL training has so far been inaccessible. We introduce a GPU-accelerated method that reconstructs the scene as a collection of surfels across thousands of parallel simulation instances during active rollouts. This lets policies operate over sensor-derived, rather than hand-specified, geometry. On a suite of collision-dense manipulation tasks, our surfel fabrics enable policies to tackle geometrically complex scenes where primitive-based baselines fail, while maintaining sim-to-real transfer. Furthermore, policies learned with a scene-aware fabric are more robust to the introduction of novel geometry at test time, improving collision-free task completion under unseen obstacles from 35% to 61%. We release our reconstruction system, training code and test dataset to spur research in this direction.
♻ ★ FlatLands: Indoor floormap completion from a single egocentric view
A single egocentric image typically captures only a small portion of the floor, yet a complete metric traversability map of the surroundings would better serve applications such as indoor navigation. We introduce FlatLands, a dataset and benchmark for single-view indoor floor-map completion in metric bird's-eye view (BEV), evaluating eleven models: both deterministic and generative. FlatLands contain 270,575 observations from 17,656 scene records across six metric 3D datasets, with aligned observation, visibility, validity, and ground-truth maps and both in- and out-of-distribution protocols. We compare training-free, deterministic, ensemble, and generative methods on shared geometry-derived inputs and, without fine-tuning, on maps estimated from real egocentric RGB images. Our results show that predicting floor almost everywhere can achieve high IoU when most evaluated cells are floor, while generative sample sets outperform the best single-output model under a proper scoring rule and support safer risk-weighted planning. FlatLands establishes a rigorous testbed for evaluating both floor-map accuracy and how effectively alternative completions represent the ambiguity of unseen space.
comment: 38 pages, 18 figures. Under review. Updated manuscript, title, abstract, and experiments
♻ ★ PriMoRetarget: Humanoid Motion Transfer Through Prioritized Motion Retargeting
Motion retargeting is a critical preprocessing step for transferring human motion data to humanoid robots. While existing methods are effective for single-character motions, interactions involving multiple characters and objects introduce additional contacts and spatial relationships that must be preserved. Retargeting is commonly formulated as a weighted multi-objective optimization problem, where conflicting objectives can require careful tuning of relative weights and can still compromise important motion characteristics. We introduce PriMoRetarget, a prioritized motion-retargeting framework that replaces relative objective weighting with an explicit priority hierarchy. Lower-priority updates are constrained to preserve higher-priority objectives, allowing critical characteristics such as foot placement and interaction contacts to take precedence over lower-priority pose objectives. The method is evaluated on four datasets spanning single-character motion, character-object manipulation, and character-character interactions. The resulting motions better preserve task-relevant interactions and provide reference trajectories that can be tracked more reliably than those produced by widely used retargeting methods.
♻ ★ HexaShell: A Universal Harness for Robot Coding Agents in Simulation and the Real World
Agents built on vision language models (VLMs) now control robots by writing code, but the strongest of them refine task-specific code over many rounds of trial and revision. Such code tends to overfit the benchmark on which it was tuned, can exploit the benchmark's shortcuts, and is costly to carry over to a new benchmark or robot. We present HexaShell, a thin, task-agnostic harness designed to test how far general tools can go instead. We design a simple and complete set of 14 basic tools, 4 for motion, 5 for perception and 5 for state, which the agent calls in one Python cell per turn. Perception pairs SAM 3, which segments an object from a few words or a click, with NumPy and OpenCV geometry that turns its outlines in several cameras, or a depth image, into its position, size and grasp angle. The API, agent loop and sandbox are shared across all tasks; a new simulator or robot needs only an adapter of 350 to 1,450 lines and a prompt describing its world, and the only task-specific input is an optional skill of about a page of guidance. The same code runs on four simulated benchmarks, built on three simulators with two kinds of arm, and on real robots. With GPT-6-Astra and no task-specific input, HexaShell reaches success rates of 69.9% on RoboDojo, 59.3% on RoboCasa365, 78.3% on LIBERO-PRO and 82.2% on RoboTwin 2.0. With task skills, it reaches 85.0% (score 88.8), 85.6%, 96.7% and 96.8%, respectively, above the best published result we are aware of on each benchmark. The RoboCasa365 result is listed first on its leaderboard. On real dual-arm robots, HexaShell succeeds in 76 of 90 episodes across nine tasks.
comment: Code: https://github.com/HexaFuture/HexaShell
♻ ★ Asymmetric On-Policy Distillation Learns In-Context Exploration
Sequential decision-making under partial observability is difficult because agents must explore to infer hidden, task-relevant information before acting effectively. Learning such information-gathering behaviors tabula rasa via online reinforcement learning is sample-inefficient, whereas offline imitation averages expert actions across hidden states and fails to learn exploration. In this work, we propose ASTEROID, a simple and effective student--teacher supervised learning framework that can learn test-time exploration. Although a privileged expert never needs to explore, we demonstrate that a history-conditioned student can learn to do so when trained with on-policy contexts: the student first collects on-policy context under partial observability, and then imitates the privileged expert conditioned on that context. By distilling expert actions directly conditioned on the student's on-policy history, the student learns to systematically resolve partial observability, i.e., search while uncertain and exploit once the hidden state is revealed. This enables efficient test-time exploration (similar to Bayesian posterior sampling) without incurring the impractical training-data costs of reinforcement learning. Across a diverse range of eleven different environments, including discrete exploration benchmarks, robotic manipulation, navigation, and Procgen environments, ASTEROID achieves ~2x higher returns under fixed exploration budgets, uses ~100x fewer interactions than RL baselines, and outperforms frontier models at exploration. Finally, our method transfers policies trained in simulation to vision-denied real-world robot manipulation, exhibiting efficient exploration and increasing success by +40% over standard distillation. Overall, it provides a simple yet surprisingly effective framework to learn efficient test-time exploration.
♻ ★ AcroRL: Learning Aggressive Quadrotor Inversion using Bidirectional Thrust
Bidirectional thrust grants quadrotors a second equilibrium condition and increased control authority, expanding the envelope of possible aggressive maneuvers and enabling inverted flight, perching, and sensing. Prior geometric control approaches extend differential flatness through Hopf fibration-based attitude representations to support bidirectional thrust, but struggle with actuator saturation and motor reversal delay during inversions, requiring heuristic thrust posture scheduling and waypoint tuning. We propose a learning-based framework that modulates a constant reference trajectory to perform compact, position-constrained quadrotor inversions while remaining compatible with traditional trajectory generation and tracking across flight regimes. Separate policies are trained via reinforcement learning for nominal-to-inverted and inverted-to-nominal transitions. In JAX-based simulation, the proposed method achieves the lowest position deviation and settling time across all evaluated baselines, reducing position root mean square error (RMSE) by 32% and settling time by 57% relative to the strongest optimization-based baseline. Hardware experiments demonstrate successful inversion across multiple yaw configurations with position RMSE below 0.35m, and compatibility with downstream trajectory generation and control through circular flight in both regimes. Additionally, we provide an open-source implementation of the proposed framework.
comment: 17 pages, 8 figures, Accepted to the 10th Annual Conference on Robot Learning (CoRL 2026)
♻ ★ Back to the Familiar Future: Failure Recovery for VLA Policies via Pre-Imagined Milestone Selection
Vision-language-action (VLA) policies can deviate from nominal trajectories during manipulation, even when tasks remain physically feasible. Recovering from these deviations is challenging, as they push the policy into unfamiliar state spaces where direct re-planning frequently destabilizes action sequences. We propose Back to the Familiar Future (B2FF), a recovery framework for foresight-driven VLAs that leverages future visual conditioning as a recovery interface. Before execution, the VLA generates a milestone bank of familiar future states conditioned on the clean initial observation. At recovery time, a recoverability-aware selector selects a recovery milestone from this bank and enforces it as a fixed visual goal. This enables the VLA to robustly map off-trajectory observations back to a familiar future. On failure-injected LIBERO, under controlled recovery timing aligned with the injected failure, B2FF increases the average success rate of a baseline VLA from 56.3% to 74.0%, demonstrating that pre-imagined milestones can guide recovery without fine-tuning the low-level action generator.
comment: CORL 2026 accepted
♻ ★ RegimeVGGT: Layer-Wise Spatially Preserving Redundancy Removal for Visual Geometry Grounded Transformer NeurIPS 2026
Visual Geometry Grounded Transformer (VGGT) recovers dense 3D scene structure from multi-view images in one forward pass, but quadratic cross-frame attention limits its scalability. Existing training-free accelerators exploit token, frame, or attention redundancy; we investigate how layer-wise functional roles can guide compression. Our spectral, probing, and causal analyses reveal three regimes: shallow layers lack cross-view structure, middle layers drive cross-view alignment, and deep layers are redundant for dense geometry yet their cross-frame attention remains essential for pose. RegimeVGGT applies layer-wise U-shaped compression along two axes: Saliency-Guided Banded Merging protects geometry- and edge-salient tokens, while Selectively Protected K/V Downsampling preserves cross-frame spatial coverage and the pose-critical path through a phase-shifted spatial grid, a reference-frame anchor, and uncompressed camera/register tokens. Without any training, RegimeVGGT achieves a 6.7x speedup over the memory-optimized VGGT* at matched reconstruction quality. Project page: https://tracyou07.github.io/regimevggt/
comment: Accepted to NeurIPS 2026. 27 pages (9 main + references and appendix), 13 figures, 12 tables. Jinhao You, Shuo Lyu, Zhuohang Lyu, Tanxuan Li, and Zibo Zhao contributed equally; Shuo Lyu is the corresponding author. Project page: https://tracyou07.github.io/regimevggt/
Multiagent Systems
★ Haptic Shared Control for Multi-Robot Systems
Multi-agent systems offer significant advantages in terms of flexibility and redundancy, enabling the efficient distribution of complex tasks among multiple robots. However, their deployment requires advanced control strategies to ensure optimal cooperation between autonomous agents and human operators. We propose a modification of the global connectivity maintenance algorithm to incorporate measurements taken in agent's local frame, using sensors with limited fields of view. This algorithm, responsible for the autonomous behavior of the multi-robot system, is combined with different adaptive supervision strategies, allowing the user to exert control at varying scales over the entire group. (i) Single Agent Control (S): The operator directly controls a single robot, (ii) Pinning Control (P): The operator imposes constraints on multiple robots to influence the group, (iii) Centroid Control (C): The operator acts on the system's center of mass to steer the entire fleet. In conjunction with the connectivity maintenance algorithm, these methods ensure compliance with inter-agent constraints while providing haptic feedback to the user regarding the system's connectivity state. The effectiveness of these approaches was evaluated through a user study, comparing performance with and without haptic feedback across multiple scenarios tailored to the different control modes.
comment: PhD thesis, Université de Rennes (IRISA UMR 6074 / Inria Rennes), defended 30 April 2025. 134 pages, 38 figures
★ GeoFaham: Domain-Specialized Multi-Agent Geospatial Analytics for Disaster Response
Real-world geospatial decision support requires more than single-step retrieval or code generation. Disaster-response questions often involve multi-source workflows that combine spatial databases, map-derived features, satellite imagery, raster products, and reusable geospatial artifacts, where correctness depends on preserving spatial scope, temporal context, and intermediate execution state. While large language models and coding agents offer a promising interface for such workflows, they struggle with geospatial-specific failure modes, including handling large artifacts, selecting the wrong evidence source, losing spatial or temporal metadata, and applying invalid geometric operations. We introduce GeoFaham, a domain-specialized multi-agent system for geospatial disaster analytics. GeoFaham decomposes workflows into specialist agents for vector operations, map feature retrieval, imagery discovery, and raster analysis, coordinated by an Orchestrator with a dual-ledger control loop for dependency-aware planning, progress tracking, validation, and replanning. To support artifact-centric collaboration, GeoFaham uses a Response Store that persists full geospatial outputs and typed metadata outside the LLM context window while agents exchange compact identifiers and summaries. We also introduce GeoFaham-Bench, a 40-task benchmark grounded in 17 real-world disaster records, designed to evaluate multi-step geospatial reasoning, cross-source integration, raster-vector computation, artifact generation, and recovery under non-rigid ground truth. We evaluate GeoFaham on GeoFaham-Bench against strong general-purpose coding-agent baselines, including Codex, Claude Code, and OpenCode. GeoFaham achieves the best overall performance, scoring 0.940 versus the next-best 0.879, while using 7.5 times fewer tokens. Our code and benchmark are publicly available at https://github.com/microsoft/GeoFaham.
★ Local Visibility Roadmaps in Continous Multi-agent Path Finding
We address the problem of smooth continuous multi-agent path finding (SC-MAPF), where agents move in a continuous environment along smooth curves in continuous time. The paths for individual agents are searched in local visibility roadmaps that consider obstacles and other agents. A new algorithm is proposed in this work, which combines the previous algorithm CE-CBS for high-level search with any-angle path finding for local single-agent planning, where agents are to avoid obstacles, both static and other agents. The new algorithm is tested and compared to CE-CBS, the algorithm building individual paths using RRT$^*$, and SMT-CCBS algorithm, an algorithm for MAPF$_R$. Experimental results show various advantages of the new algorithm over CE-CBS and SMT-CCBS in comparable settings.
comment: 11 pages. Published in ICAART 2026
★ Cooperative Multi-Agent Reinforcement Learning with Submodular Trajectory Rewards
We study cooperative multi-agent reinforcement learning with decoupled dynamics and a monotone submodular reward function over joint trajectories. Due to the non-additive reward, an optimal policy may depend on the history of all agents. This dependence makes exact optimization exponential in the horizon and the number of agents. To overcome this challenge while retaining theoretical guarantees, we introduce pre-window curvature $c_m$, which quantifies how states visited more than $m$ steps earlier reduce current marginal gains. Using this curvature to control the error from restricting the policy of each agent to depend only on its last $m+1$ states, we propose a greedy algorithm that exploits the decoupled dynamics to optimize local policies sequentially. For fixed $m$, the algorithm achieves a $\frac{1-c_m}{2-c_m}$ approximation guarantee up to estimation error, and its computation time is polynomial in the number of agents and the horizon. We recover the known $1/2$-factor approximation for time-additive rewards and obtain an approximation factor no smaller than the known total-curvature bound for a single agent. We formulate multi-agent traffic monitoring as an example of this framework and evaluate our algorithm on it.
★ Can Network Models of Science Do Without Pluralist Agents?
Science is a process of useful models being proven wrong time and again, but despite its many flaws, it somehow works. Why is this? Paul Feyerabend argued that it works because people dare to hold and defend incompatible rival theories against the accepted one, since a theory's failures often lead to its abandonment in the presence of a more accurate or simpler alternative. He called this view theoretical pluralism. We introduce a theoretical and experimental model in which this can be examined. It compares communities of pluralist agents, which can hold rival theories side by side and maintain beliefs within and between those theories, with communities of monist agents, which can only incorporate what they hear or observe into a single model of their own. In a multi-agent variational inference setting, our experimental demonstrations illustrate how communities with rival theories can go through periods of uncertainty followed by a shift in the theories they hold true, how the information that is communicated between communities affects beliefs when those communities make different observations about the world, and how sharing support for theories and sharing beliefs within them contribute differently to collective belief dynamics.
comment: 21 pages, 6 figures, 1 table. Accepted at IWAI 2026 (International Workshop on Active Inference)
★ Inter-Agent Influence: Evaluating Persuasion, Deception, and Coercion in Multi-Agent Systems NeurIPS 2026
The deployment of AI agents in multi-agent settings enables inter-agent influence, whereby agents can strategically steer each other's behaviour towards an intended outcome. Such capabilities pose novel risks, as malicious agents could direct others towards harmful actions. In this work we investigate three such capabilities - persuasion, deception, and coercion - across five realistic evaluation environments and a variety of agents instantiated with frontier language models. We observed significant inter-agent influence capabilities; in oversight environments, all tested models shifted overseer decisions from rejection to approval through persuasion, deception, or coercion. We also observed successful influence in non-hierarchical 'peer-to-peer' settings through redirection of a peer's safety research trajectory. As multi-agent deployments become increasingly commonplace, these results point to a pressing need for work on mitigating the risks from inter-agent influence. Code and data are available at https://github.com/Cooperative-AI-Foundation/inter-agent-influence.
comment: Accepted at NeurIPS 2026. 70 pages, 17 figures, 34 tables. Code: https://github.com/Cooperative-AI-Foundation/inter-agent-influence ; Data: https://huggingface.co/datasets/Cooperative-AI-Foundation/inter-agent-influence-eval
★ Are LLM Assistants Regulatory Ready? Answers from regenold's 2026 EU AI Act Q&A Challenge
Large language model (LLM) assistants are increasingly proposed for regulatory affairs, yet their reliability in regulatory tasks remains unclear. We introduce the EU AI Act Q&A Benchmark, an expert-curated benchmark of 110 open-ended questions about the EU AI Act (Regulation (EU) 2024/1689). Each question comes with an exemplary answer, question-specific correctness criteria, and expected references. Evaluation measures answer correctness, reference quality, conciseness, tone, and latency, with a multi-turn ``hard mode'' testing responses to simulated user pushback. We also present the results of the 2026 EU AI Act Q&A Challenge, based on the benchmark, where three off-the-shelf baselines and five purpose-built assistants (from four commercial developers and one academic entrant) were tested. Only two of the five submitted assistants consistently outperformed the baselines; one achieved high scores across all evaluated metrics. These results show that specialist development alone does not guarantee better performance and highlight the critical importance of LLM assistant evaluation in regulatory affairs. The benchmark code is openly available at https://github.com/regenold-GmbH/eu-ai-act-benchmark-2026/.
★ AttriSwarm: Orchestrating Agent Swarms for Cross-Page Visual Attribution
Multimodal large language models (MLLMs) increasingly address knowledge-intensive tasks by integrating evidence scattered across multiple document pages into a final answer. Because such answers are synthesized from many pages, the link between the answer and its supporting visual evidence can become obscured, hindering verification and trust. Cross-page visual attribution is therefore essential: it requires identifying the supporting pages and localizing fine-grained evidence regions within them, thereby making multi-page reasoning transparent and auditable. Existing approaches either process multiple pages jointly or iteratively refine evidence within a single trajectory, risking cross-page context pollution and, for interactive methods, substantial sequential overhead. We introduce \textbf{AttriSwarm}, an agent-swarm framework that assigns these concerns to separate roles: a leader handles global cross-page reasoning, while subagents perform page-wise attribution in isolation and in parallel. To support training and evaluation, we construct \textbf{AttriDoc}, a cross-page QA dataset linking supporting pages to their annotated evidence regions, and use it for role-specific training of AttriSwarm, improving answer accuracy, page-level $F_1$, and region-level IoU over its training-free counterpart by an average of 18.94, 29.34, and 23.04 percentage points, respectively, across three open-source backbones and three datasets.
★ Goal-Driven Autonomous Optical Experimentation via a Self-Recursive Hardware-Software Harness
We propose goal-driven self-recursive optical research in which agents use physical evidence to revise the hardware, software, models, and protocols used to pursue a scientific goal. The framework can be instantiated with different stimulus systems and measurement devices. We formalize its updates and derive a relation between physical attainability, command access, and model approximation error. We demonstrate the concept in an optical see-through color-matching case study using an iPad, RayNeo X3 glasses, and a TOP 300 probe coupled to a CAS140D spectroradiometer, both from Instrument Systems. A revised glasses application adds USB-controlled RGB holds and frame acknowledgements, keeping model-generated colors displayed during spectral acquisition. The interface passes 12 live control checks. Spectral feedback reduces one target color difference from 19.756 to 1.211 ΔE_76^*; all five targets improve in a separate comparison. These measurements demonstrate the concept in one configuration. The resulting software and protocols remain available for subsequent investigations.
★ Distributed Marginal Return: A Communication-Efficient Algorithm for Multi-Agent Weapon Target Assignment
The Weapon-Target Assignment (WTA) problem in distributed multi-agent systems is challenged by limited situational awareness and communication constraints. Existing distributed methods, such as auction-consensus algorithms, typically assume task-value independence, a simplification that fails to capture the coupled effects of multi-weapon assignments. This paper introduces the Distributed Marginal Return (DMR) algorithm, a novel decentralized approach that generalizes auction-consensus frameworks. DMR enables agents to compute bids based on marginal return and uses a consensus protocol to resolve conflicts, allowing multiple non-conflicting proposals to be accepted in parallel for cooperative optimization with minimal communication. A key theoretical contribution is our proof that the marginal returns of non-conflicting bids are additive, a property that ensures global consistency and validates our parallel acceptance mechanism. Extensive simulations demonstrate that DMR significantly outperforms baseline algorithms in both solution quality and communication efficiency, offering a scalable and robust solution for real-world distributed command and control applications.
comment: 12 pages, 6 figures. Published in the IEEE Systems Journal
★ When Is Fresh AI Stock Analysis Worth It? Allocating a Limited Research Budget Across Price Updates, Deep Forecast Refreshes, and Editorial Reviews of Top Stock Picks
Must long-horizon AI investment research be rewritten every time stock prices move? Using 8,120 production reports on the same 203 stocks across four iPulse AI batches, we compare repricing existing targets, guarding them against price shocks, mechanically resetting old forecast growth, and generating fresh multi-agent reports. Ten matched configurations represent seven investment frameworks. Our primary economic evaluation selects Top 10 USD portfolios using one-year forecasts. Across two unchanged-model windows, guarded reuse averages 2.25%, versus 1.66% for SPY and 4.77% for fresh reports, after modeled trading costs and reference compute charges on a USD 100,000 account. Guarded reuse beats SPY in all three short windows, including the model-change transition. Extending the first holding window to nearly six months reverses the result: guarded reuse returns -11.26%, fresh reports +9.12%, and SPY +3.46%, motivating the thesis half-life question. Supporting quarter-forecast diagnostics show that mechanically re-anchoring old growth paths closes 98% of the MAE improvement over fixed targets, yet Fresh and Reset share only 1-4 of their Top 10 stocks. At Top 30, the unchanged-model portfolio comparison instead favors guarded reuse (4.17% versus 1.88%). Endpoint and concentration tests reveal the influence of horizon choices and exceptional winners. Nine production editions show substantial between-edition turnover, while every final editorial screen retains its own incoming list. Measured token workloads connect refresh costs to shared capital and realized break-even budgets. The resulting architecture separates continuous price monitoring from episodic thesis rebuilding and targeted editorial evidence checks.
comment: 26 pages, 9 figures, 13 tables. Paper and research artifacts: https://doi.org/10.5281/zenodo.23907031 . Code: https://github.com/TheFutureEdge/research-when-fresh-ai-stock-analysis-is-worth-it
★ ATC: Agent Trajectory Control for Role-Contract Enforcement in Black-Box Multi-Agent Systems EMNLP 2026
Multi-agent LLM systems assign different responsibilities to different agents: one plans, another retrieves evidence, another verifies, another executes. This division of labor is only meaningful if each agent stays within the functions it is authorized to perform. Over many turns, however, an agent can produce a locally reasonable message that exercises authority belonging to another role: a verifier that begins planning, a retriever that asserts unsupported conclusions, an executor that changes state no one approved. We formalize this as role drift, a turn-level breach of an agent's role contract. We introduce Agent Trajectory Control (ATC), which makes each role's authority explicit as a contract and enforces it at the point where a message would enter the shared conversation. ATC scores a proposed output for out-of-role deviation, leakage into the functions of other roles, impermissible speech acts, and logical contract breaches, then computes a leakage-regularized target that constrains regeneration; the revised message is re-checked before admission. Enforcement acts on the message rather than the generator, so ATC governs agents whose internal representations are unavailable. Across homogeneous and heterogeneous teams, ATC improves task performance and role consistency while lowering role-drift energy relative to strong baselines spanning prompting, memory, workflow control, and activation editing, with the largest gains under explicit role-confusion stress tests and in larger, more diverse teams.
comment: Accepted at the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP 2026)
★ Drift No More: Spectral Locking for Long-Horizon Open-Weight Multi-Agent AI EMNLP 2026
Long-horizon multi-agent systems suffer from three tightly coupled failures: task drift, role drift, and state staleness. We test the hypothesis that these share a common representational cause: agent hidden states move away from the directions encoding the active task, assigned role, and current shared state. A pilot study over 14,832 annotated turns supports this: drifted turns collapse near-uniformly across all three anchor subspaces (pairwise r = 0.72-0.78). Existing methods address these failures separately, operating at the surface-text level or treating control directions independently. We introduce ATLAS (Algebraic Task-role Locking with Adaptive Spectral Memory), a training-free intervention acting on hidden states before a token is generated. ATLAS builds a positive-semidefinite anchor kernel over task, role, and state subspaces and steers hidden states toward their shared high-coherence eigendirections; a low-rank Gram decomposition keeps this tractable. Across long-horizon memory, state maintenance, tool use, least-privilege coordination, and embodied stress tests, ATLAS outperforms prompting, retrieval, memory, and activation-steering baselines. Against A-MEM it improves memory macro-average accuracy by up to 5.7 points and reduces write/read failures by up to 31.7%/24.8%; against MemoryArena-Agent it gains up to 4.8 points. It gains 10.7-12.1 points under perturbation, reaches 93.5% role consistency, and transfers to a 72B backbone (macro-average 61.0 vs. 56.2). Ablating the joint task-role-state span costs 7.2 points against 0.8 for a matched-rank random span, indicating these directions are functionally implicated, not incidental; supplying them by prompt or retrieval recovers 4.3 points less. ATLAS is therefore a representation-level controller for open-weight multi-agent systems, complementary to output-level protocols for black-box agents.
comment: Accepted at the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP 2026)
★ Interpretable Early Termination of Web Navigation Agents via Closed Sequential Pattern Mining
Web navigation agents built on open-weight language models fail often and fail late, with most of the tokens spent after the trajectory has become unrecoverable. Recent work has shown that early termination is a viable cost-reduction strategy, but existing approaches produce opaque termination decisions that do not support diagnosis or human oversight. In this work we apply closed sequential pattern mining (BIDE+) to K-step prefixes of agent execution traces to produce a small, human-reviewable failure-signature library that organizes into a four-category failure taxonomy with associated repair recommendations. We use a multi-variant evaluation protocol that explicitly removes contamination artifacts (zero-step crashes and trivially separable single-step successes) so that reported detection accuracy reflects behavioral discrimination rather than trace-length distributions. We evaluate our methodology on 1,544 MiniWoB++ traces collected from three open-weight language models, achieving the best F1 at K=3 on the Exclude-errors configuration across eight methods (0.717), recovering 88% of test-set failures within three steps, and supporting an early-termination policy that saves 26.3% of tokens at 92% precision.
comment: In online proceedings of AI Agents for Discovery in the Wild (AID-Wild) Workshop at ACM CAIS 2026
★ Mechanical Engineering Skills: An Open Library of Agent Skills for Design, Analysis and Manufacturing
Mechanical engineers use specialized tools for CAD, simulation, manufacturing and research, each with a distinct interface and learning curve. We present Mechanical Engineering Skills, an open library of 33 agent skills that makes these capabilities accessible through a single text or voice conversation. Each skill combines operating instructions, executable tools and verification procedures. A language model interprets the request, selects and coordinates skills, while deterministic tools perform computations and return checked results. Default workflows require no tool-specific interface training, local model execution or model fine-tuning; skills load on demand and install with one command. Nineteen skills adapt published or preprint research from our laboratory, including text-to-CAD, linkage synthesis, CFD setup, stress-field and lattice surrogates, equation discovery, additive-manufacturing planning and in-process monitoring. Fourteen additional skills support core engineering and education. We describe the skill format, design principles and a method for translating research into practical workflows. Five case studies with explicit evidence scopes and independent reruns demonstrate exact analytical agreement for lattice homogenization, rocket apogee predictions within 4% of stored OpenRocket references, an equation fit nine times more accurate than a cubic polynomial at an unseen temperature, and an end-to-end rehearsal of porosity monitoring. Controlled agent comparisons remain future work. The open library is available at https://github.com/BaratiLab/mechanical-engineering-skills.
★ Same Method, Different Task: What the Bitter Lesson Does Not License in Social Simulation NeurIPS 2026
The bitter lesson holds that general methods leveraging computation defeat hand-encoded domain knowledge. We argue it carries a condition its popular reading omits: in every case Sutton cites, the general method was scaled against the same task the domain engineering was solving. Social simulation substituted across tasks instead, replacing hand-built agents with an artifact optimized for predicting text. The move is bitter-lesson-shaped, but the lesson does not license it. Prediction is a solitary act: the predictor bears no consequences from the interaction it models. Cooperation, defection, and norm enforcement are phenomena of interested parties; they do not exist for a disinterested observer at any scale. What is missing is not emergence, which LLM populations already show, but stakes. We show that several anomalies reported in prior work (excess cooperation that predates alignment training, collective agreement without individual fidelity, and population homogeneity) follow from this account. A 540-agent pilot with pre-specified hypotheses, which tests sensitivity to persistent consequences rather than cooperation, finds that consequences reshape strategies in two of three model families but never produce human-like heterogeneity. We argue for mixed-motive settings that shape agents rather than only test them, and ask whether agents should be pretrained differently from predictors.
comment: Accepted at the SocialAgent Workshop, NeurIPS 2026 (non-archival). 15 pages, 5 figures
★ Decision Models as Scalable Economic Surrogates that Capture Human Variation
Decision Models (DMs), a new class of AI models, give fast responses and return a probability distribution over a fixed set of options. These properties make them candidate human surrogates in economic simulations and surveys, a role increasingly filled by Large Language Models (LLMs), which are costly at scale and under-represent the variation in human behaviour. We present a first evaluation of DMs as human surrogates in economics, comparing 10 DMs with 8 LLMs in three economic settings: a household spending survey on consumption, several classic economic games and a labour-market field experiment. Frontier DMs, those that score highest on general decision-making benchmarks, reproduce the variation in human responses more closely than LLMs in all three settings. They capture the modal human response as well as LLMs in the spending survey, but less well in the economic games. Frontier DMs are also roughly ten times faster than the fastest LLM and can be over a hundred times cheaper. Together, these results suggest that DMs capture the heterogeneity of human behaviour better than LLMs and make large-scale economic simulations more feasible.
★ MA-SafeDiffuser: Safe Multi-Agent Planning with Diffusion Probabilistic Models AAMAS 2026
We propose MA-SafeDiffuser, a multi-agent extension of SafeDiffuser that equips diffusion-based trajectory planners with finite-time diffusion invariance guarantees for joint-agent safety specifications. Building on the single-agent construction that embeds control barrier function (CBF) constraints into reverse-diffusion updates, we: (i) formalize joint safe sets as intersections of per-agent and pairwise barrier sets; (ii) derive centralized and decentralized (communication-aware) constrained denoising procedures with provable invariance under mild assumptions; (iii) address local-trap and deadlock phenomena via time-varying specifications and liveness CBFs; and (iv) develop a lightweight benchmarking suite including a multi-agent Maze2D domain. Empirically, MA-SafeDiffuser reduces violation counts relative to unconstrained diffusion baselines, while retaining planning quality. We provide algorithms, proofs, and reference implementations.
comment: Accepted at AAMAS 2026. 14 pages, including supplementary material
♻ ★ CONFIRM: Governing Scientific Reporting for Trustworthy Agentic Neuroimaging
Large language model (LLM) agents can autonomously run neuroimaging analyses from research questions to reported findings. Existing agentic systems use tools to check that code runs and reported values match its output, but rely on LLM judgment to decide which results warrant scientific reporting. We show that LLM judges endorse site-confounded findings even when the confound is presented as evidence. We present \textbf{CONFIRM}, a framework that separates running an analysis from deciding which findings to report. CONFIRM encodes each hypothesis as a machine-readable, pre-specified analysis plan that fixes the claim and its pass criteria before computing results. It then filters results through five deterministic gates (search history and multiplicity, confounds, power, analytic stability, and replication) under three configurable reporting standards, from least to most demanding: \textit{Screen}, \textit{Replicate}, and \textit{Full}. Failed claims receive diagnoses that guide follow-up analyses. To evaluate reporting decisions at the claim level, we introduce \textbf{NeuroClaimBench}, comprising 489 executable neuroimaging claims with frozen data, including literature-supported findings and 150 controls designed to violate individual methodological requirements. On NeuroClaimBench, CONFIRM falsely supports at most 3\% of controls, compared with 23--38\% for the baselines, while recovering up to 61\% of literature-supported claims. When auditing an existing agent, CONFIRM reduces false support on controls from 23\% to 0\% while retaining the agent's literature-supported claims. Its diagnostic feedback also produces more supported revisions than Self-Refine with fewer LLM calls. Together, CONFIRM and NeuroClaimBench show that basing reporting decisions on pre-specified analysis plans and deterministic gates is essential for trustworthy autonomous scientific discovery.
♻ ★ Distributed Swarm Morphogenesis with Neural Cellular Automata
Swarm morphogenesis requires robots to assemble desired shapes through local interactions. We present a modular framework integrating neural cellular automata (NCA) with existing site assignment and collision-aware control. Within this framework, we introduce a decentralized attachment-site discovery mechanism. Robots already in the formation update and share local NCA states to identify new attachment sites, enabling progressive growth without a predefined list of sites during deployment. To bridge the mismatch between conventional NCA growth and gradual physical assembly, we additionally propose offline sparse-retention fine-tuning that simulates partial attachment, together with a false-positive loss that penalizes invalid site activations. Simulations demonstrate formation of diverse lattice morphologies, scaling to several hundred robots, and substantial improvements in morphology accuracy from the proposed fine-tuning. Experiments with Crazyflie quadrotors validate the integrated execution pipeline using robot-relative observations reconstructed from external tracking.
♻ ★ Payoff scaling shapes cooperation in LLM agents across languages
Large language models (LLMs) are increasingly deployed as autonomous agents that negotiate, coordinate, and make decisions on behalf of human users. Understanding how they cooperate is therefore important for AI governance and machine behaviour research. We examine two factors: payoff magnitude and interaction language. Classical game theory predicts that multiplying all payoffs by a positive constant should not change behaviour because the underlying strategic incentives remain unchanged. We test this prediction using five widely adopted commercial LLMs playing an iterated prisoner's dilemma across ten payoff scales and five languages (English, French, Vietnamese, Chinese and Arabic). Contrary to theoretical expectations, changing only the numerical scale of payoffs substantially alters cooperation. Models respond differently, often in opposite directions, meaning that averages across models can conceal important behavioural heterogeneity. Sensitivity to payoff scale also depends strongly on prompt language. Moreover, payoff scale modifies the effects of persona instructions: at sub-unit stakes, where no payoff shown exceeds 1, all five models cooperate more under a selfish than a cooperative persona, whereas Claude and Gemini reverse this pattern at larger scales. These findings show that cooperation measures depend critically on payoff scale and language and should not be treated as fixed model traits.
comment: Accepted in Interface Focus (Royal Society). 57 pages, 17 figures, 25 tables, including supplementary material
♻ ★ An Open Book, On Whose Terms? Power Dynamics in Sensor-Derived, LLM-Generated Summaries Sharing Between Older Adults and Remote Families
As older adults (OAs) increasingly favor aging in place, remote family members (RFMs) need meaningful awareness of their loved ones without intrusive monitoring. Large language models can provide rich interpretations of non-intrusive sensing data, yet recent work has largely emphasized technical accuracy over how families use or benefit from this information. We conducted formative interviews with 12 RFMs and 2 OAs, followed by a three-month deployment case study with one OA-RFM dyad. Our system provided daily AI-generated summaries of multimodal sensing data, which we iteratively refined with both parties using a research-through-design approach. Although the summaries integrated well into the dyad's daily routine, sustaining over 90% engagement, their lived experience revealed evolving tensions around agency, reassurance, privacy, trust, and communication, shaped by sensor limitations and existing communication misalignments. Our findings inform the design of future AI-mediated sharing systems that account for real-world sensing constraints and complex family care dynamics.
♻ ★ MAS-Gate: Unlocking Multi-Agent System Lifecycle via Node-Centric Quantification
Large language model (LLM)-based multi-agent systems (MAS) can solve complex problems beyond the reach of a single agent by decomposing tasks and coordinating complementary capabilities. Despite extensive research on MAS, existing work largely focuses on specific aspects or individual phases of their development and operation, resulting in fragmented perspectives with objectives and signals that are difficult to reconcile across these different settings. To bridge these disconnected perspectives, we conceptualize MAS development and operation as a three-phase lifecycle spanning design-time orchestration before execution, runtime intervention during execution, and post-hoc attribution after execution, and introduce MAS-Gate, a unified node-centric quantification applicable across these phases. Modeling each execution as a directed acyclic graph, MAS-Gate combines within-node utility, which estimates a node's contribution through changes in reference-answer likelihood, with inter-node strength, which approximates semantic uptake and potential downstream reach. We adapt this shared quantification to three objectives: providing a structure-aware dense reward for Orchestrator training, training an Editor to act on execution prefixes using paired rewards from completed edited and unedited trajectories, and supplying risk-ranked nodes as auxiliary evidence for LLM-based post-hoc attribution. Experiments spanning interaction environments, knowledge-intensive multi-hop question answering, general reasoning, and post-hoc attribution demonstrate that MAS-Gate improves workflow construction, targeted runtime editing, and LLM-assisted error localization over corresponding phase-specific baselines. Together, these results establish MAS-Gate as a unified structural interface for improving MAS generation, dynamic editing, and error attribution throughout the lifecycle.
♻ ★ MAS-Loop: Beyond One-Shot Orchestration towards Self-Evolving Multi-Agent Systems
Training-time multi-agent system (MAS) design seeks to learn reusable orchestration policies, yet prevailing approaches remain largely one-shot: an Orchestrator generates a system but only observes and learns from its terminal outcome. This paradigm discards execution trajectories that expose system-level failures and contain fine-grained information beyond the terminal outcome, leaving no direct basis for revising the source MAS, evaluating the updated MAS, or learning from the resulting contrast. To this end, we introduce MAS-Loop, an execution-grounded framework that closes this feedback loop by coupling an Orchestrator with a trajectory-conditioned Refiner. For each task, the Orchestrator constructs and executes a source MAS to obtain an outcome score and an intermediate trajectory. Conditioned on the trajectory, the Refiner localizes an actionable failure, formulates a trajectory-grounded diagnosis, and proposes a bounded patch to the agents, information flow, or execution configuration. The executable updated MAS is re-executed on the same task to measure its realized outcome under stochastic execution. This pair-MAS comparison provides complementary learning signals: the Orchestrator learns to generate strong systems with fewer recoverable failures, whereas the Refiner learns to expose and repair observed weaknesses, thereby driving policy-level co-evolution across training iterations. Experiments show that MAS-Loop achieves the strongest overall performance among the evaluated baselines, with consistent gains from refinement across all evaluated benchmarks. The co-evolution process also yields MAS-JudgeCorpus, a dataset of execution-grounded source-updated pairs with orchestration- and trajectory-level supervision, which can provide the community with support for MAS reward modeling and trajectory-aware investigation.
♻ ★ An Axiomatic Framework for Reinforcement Learning in Open Multi-Agent Systems NeurIPS 2026
Open multi-agent systems, where only a subset of agents are controllable, are central to real-world domains such as smart grids, yet lack a principled foundation for reinforcement learning algorithm design. We study this setting through n-agent ad hoc teamwork (NAHT), a recent framework for learning in open multi-agent environments. We propose an axiomatic framework that derives value function learning from the Shapley axioms (Efficiency, Symmetry and Linearity), treating them as structural constraints rather than heuristic design choices. This formulation builds on a novel NAHT decomposition, which characterises the fundamental difference between open and fully controllable multi-agent systems, and establishes a cooperative game-theoretic foundation for learning. We further show that enforcing these axioms on learning individual value functions recovers the Shapley value for Dec-POMDPs, providing a principled bridge between reinforcement learning and classical cooperative game theory. Building on this framework, we introduce Shapley Machines and Banzhaf Machines, two generic algorithmic templates that can be instantiated with standard reinforcement learning methods. When applied to strong base algorithms such as IPPO and POAM, our methods consistently improve learning efficiency and generalisation to unseen agent types and conventions across NAHT benchmark tasks. The code of experiments is published on https://github.com/hsvgbkhgbv/axiomatic-rl-naht.
comment: 43 pages, Published at NeurIPS 2026
♻ ★ Cultural Evolution Drives the Emergence of Conventions in Neural Emergent Communication
Recent studies of language evolution with neural agents often use models where agents can either speak or listen, where there are only two agents in the population, or where agents can share internal representations. These choices prevent the study of how language emerges through cultural evolution, i.e. through the cumulative adaptation of individual behaviours to fit the communicative needs of the community. We tackle this issue with a new model that trains populations of bi-directional neural agents in a fully distributed fashion to solve a referential game. We demonstrate the effectiveness of this approach on multiple image datasets and show that it is able to scale to large populations. We also show that more conventional languages are easier to learn, providing a direct advantage for cultural transmission. Overall, this work provides a novel approach to neural emergent communication that opens up new directions for evolutionary linguistics.
♻ ★ Heterogeneous Graph Attention Network With Multimodal Feature Selection for Cancer Multiomics Integration
The increase in high-dimensional multiomics data demands advanced integration models to capture the complexity of human diseases. Graph-based deep learning integration models, despite their promise, struggle with small patient cohorts and high-dimensional features, often applying independent feature selection without modeling relationships among omics. Furthermore, conventional graph-based omics models focus on homogeneous graphs, lacking multiple types of nodes and edges to capture diverse structures. We introduce a Heterogeneous Graph ATtention network for omics integration (HeteroGATomics) to improve cancer diagnosis. HeteroGATomics performs multimodal feature selection through a multi-agent system, creating dedicated networks of feature and patient similarity for each omics modality. These networks are then combined into one heterogeneous graph for learning holistic omics-specific representations and integrating predictions across modalities. Experiments on three cancer multiomics datasets show that HeteroGATomics outperforms baseline methods across most evaluation metrics in cancer diagnosis. External validation on an independent breast cancer cohort further demonstrates its generalizability, where HeteroGATomics outperforms graph-based multiomics integration methods across evaluation metrics. Moreover, HeteroGATomics enhances interpretability by identifying important biomarkers contributing to the diagnosis outcomes.
comment: 46 pages, 18 figures
♻ ★ A Game-Theoretic Decision Framework for Optimal Selection of Coordination Detection Methods in Multi-UAV Fleet Operations
Detecting coordination among unmanned aerial vehicle (UAV) fleets operating in shared airspace and identifying the route-lead aircraft whose navigation decisions govern fleet behavior presents a fundamental speed--accuracy trade-off: fast methods enable real-time traffic management but sacrifice detection fidelity, while accurate methods may exceed the time budget for actionable airspace deconfliction. This paper presents a game-theoretic decision framework that resolves this trade-off by formulating method selection as a two-player zero-sum game between a Monitor (selecting computational methods and parameters) and Nature (selecting the unknown traffic scenario). We construct an end-to-end pipeline from trajectory surveillance data through eight candidate detection algorithms, a Monte Carlo sensitivity analysis characterizing their stochastic performance, and finally a multi-objective optimization layer that identifies Pareto-optimal method portfolios. The minimax solution provides a robust mixed strategy with a probability distribution over methods that guarantees worst-case performance regardless of scenario uncertainty. Experimental evaluation across 200 randomized configurations spanning 5--50 aircraft demonstrates that the framework recommends distinct method portfolios depending on operational priority: Koopman Phase dominates balanced (70.6%) and speed-priority (79.7%) profiles, while CRQA emerges as primary (47.4%) when route-lead identification is prioritized. The framework achieves a guaranteed game value of 0.29--0.53 (normalized utility) across all tested preference profiles, providing the first principled, scenario-adaptive methodology for computational method selection in UTM fleet monitoring operations.
♻ ★ Constrained Black-Box Attacks Against Cooperative Multi-Agent Reinforcement Learning
Collaborative multi-agent reinforcement learning (MARL) is increasingly deployed in critical domains, yet its vulnerability to adversarial perturbations has not been thoroughly investigated. Most existing adversarial attacks against MARL rely on optimistic assumptions, such as access to policy weights or the ability to train surrogate policies. We investigate whether adversaries with limited capabilities remain effective against MARL agents. In particular, we study two restricted attack settings that are overlooked in the literature: (i) adversaries with access only to agent observations and (ii) adversaries with no access whatsoever. Our approach generates perturbations that intentionally misalign how victim agents perceive their environment, preventing them from collaborating. We conduct extensive experiments across a diverse set of 22 environments. Our results demonstrate both the effectiveness and sample efficiency of our attacks.
♻ ★ The Little Scientist: What Does the Scientific Method Buy an LLM Agent?
Scientific discovery emerges from cycles of hypothesis, implementation, empirical testing, and feedback. Can this process be automated? We approach automated algorithm design through the lens of the scientific method, where an LLM-based agent goes through each step of the process in an iterative fashion. Results: We present The Little Scientist, a framework that maps the scientific method onto an LLM agent in a loop: a Scientist agent iterates hypothesis, prediction, code, test, and reconciliation inside an evaluation environment that returns structured diagnostics, and a Kuhn agent injects paradigm-shifting reframings at plateaus. On DNA motif discovery, the process wrote an algorithm from scratch (DALE) that outperforms STREME across 132 ENCODE transcription factors on the shuffled-negative benchmark (mean AUROC 0.842 vs. 0.803) while running 11x faster. We then asked why this method worked, and that question became the paper's scientific core: a preregistered, controlled teardown of the framework's own scaffolding consisting of ten ablation arms against one-shot bandit nulls. The answer overturned the premise. Seven of ten loop arms fail to beat a bandit drawing samples from the same model and prompt, the full scaffold performs below blind sampling at 7-8x the token cost, and a floor-prompt arm with no framework at all wins on both score and value per token. On a held-out genomic benchmark, the best loop artifact and the best bandit draw agree to within 0.002, and the DALE algorithm from the initial experiment scored similarly. The convergent picture is that LLM agents do not reason their way past what training placed in the weights. Autoresearch, on this evidence, is sampling, selection, and remixing, which is the bet the Evolve-family architectures made when they chose populations and external selection over scratchpad reasoning, and why those choices were right.
comment: Code: https://github.com/travis42/little-scientist-dale and https://github.com/travis42/little-scientist-delta-v (Apache 2.0)
Systems and Control (EESS)
★ Iterative Methods for Solving Feedback-Type Linear Equations under Small Phase Condition
This paper investigates iterative methods for solving feedback-type linear equations under small phase condition. We first develop an iterative solution scheme for linear equations with sectorial (semi-sectorial) matrices, i.e., matrices whose numerical ranges are contained in an open (closed) half complex plane. Exploiting the geometric properties of the numerical range, we introduce the notion of numerical-radius contraction as a convergence criterion. Such a scheme, with some extra effort, leads to an iterative method for the feedback-type equations under small phase condition, whose convergence condition is characterized in terms of phase information. This complements classical iterative methods for feedback-type equations based on the small gain condition. A case study on large-scale power networks is presented to illustrate the computational efficiency of the proposed method.
★ Learning curves in natural and artificial systems
A general mathematical model of experience formation is considered, in which the learning level is defined as the probability that the next value of the uncertainty factor has already been mastered and not forgotten. Within this model the shape of the learning curve is derived, and it turns out to be determined primarily by the distribution of situation occurrence: a uniform distribution yields an exponential curve, whereas a power-law distribution yields a power-law curve whose exponent equals the ratio of the distribution exponent to that same exponent increased by one. Both mechanisms generating power-law curves, established earlier in the analysis of debugging, assembly and repair activities, are shown to have been independently rediscovered in deep learning theory as the "frequency" and "spectral" mechanisms of scaling laws for artificial neural networks, while the exponential curves of machine learning (contraction operators, strongly convex optimization, stationary reinforcement) are generated by the same constancy of the contraction coefficient that was revealed half a century earlier in models of radioactive decay, the first-order lag element, and others. A solution is given for the problem of choosing a training distribution that differs from the deployment distribution. A methodological consequence is discussed: three distinct mechanisms generate curves indistinguishable in shape, so an observed learning curve does not allow one to judge the "internal" structure of the learning system.
★ OperAItor: An Agentic Framework for Fault-Tolerant Operation of Chemical Processes
OperAItor, an open-source framework for developing and evaluating large language model agents for supervisory fault management in chemical processes. The framework is demonstrated on a dynamic reactor-distillation process representing isopropanol dehydrogenation, with physically based thermodynamics and modified reaction kinetics to permit liquid-phase operation. The process is represented by a 113-variable differential-algebraic model and operates at a nominal feed rate of 100kmol/h, producing approximately 4t/h of acetone. The simulated plant includes 8 PI control loops, 22 process alarms, 5 duty-standby pump trains, and 11 independent safety instrumented functions. The agent architecture integrates multivariate statistical process monitoring, fault diagnosis, knowledge-graph retrieval, action planning, and supervisory action generation. Five action-validation configurations are investigated, comprising operation without validation, an LLM-based judge, digital-twin simulation, and deterministic admissibility checks with and without consequence projection. The framework is evaluated using four LLMs across 20 scenarios involving single and multiple simultaneous faults. Without agent intervention, 18 scenarios result in plant stops, corresponding to an economic loss of $399,000 relative to fault-free operation under the assumed benchmark economics. For GPT-5 nano, Qwen3 8B, and Mistral 7B, validated agent configurations reduce the number of stops to 5-8 and recover $265,000 of the loss. The study provides a reproducible benchmark for comparing agent architectures, language models, and action-validation methods for supervisory fault management in chemical processes.
★ The Novelty Horizon H_N: Why Large Language Models Fail to Generate Scientific Data beyond Their Training Distribution
Large Language Models (LLMs) lack physical grounding and extrapolation. This paper formalises the Novelty Horizon as tasks needing extrapolation beyond training support, causal reasoning absent from learned correlations, or post-frontier information. A forensic analysis tests LLMs on unguided, guided, and ground-truth data from an underrepresented diode-clamped short-circuit experiment using temporal, spectral, and statistical validation against a first-principles Modified Ohm's Law. Both script-based approaches fail catastrophically with physical and spectral inconsistencies, extreme statistical pathologies, zero Generative Fidelity Score, and linear-regime outputs. Direct prompting corroborates distinct failure modes. This paper introduces the Past Information Boundary and physics-based authentication. LLM data generation cannot extrapolate past the Novelty Horizon essential for novel discovery. Hybrid neuro-symbolic systems must preserve human causal reasoning and experimental design.
comment: 10 Figures, 61 pages
★ Towards an Optimal Control Foundation of Early Exits - Turnpikes and ResNets NeurIPS 2026
The feed-forward propagation through neural networks is crucial for training and inference. To trade off efficiency and accuracy, it has been proposed to consider intermediate outputs of hidden states as data-dependent regularization during training and, ultimately, as early-exits during inference. In this work, we analyze the joint optimization of loss functions computed for the output layer and intermediate layers using optimal control theory. We prove that ResNets trained in this joint fashion exhibit the so-called turnpike phenomenon, this implies that the number of layers required to reach a prescribed loss level is bounded from above independently of the overall network depth. This induces a similarity property across the internal representations of early-exiting networks of varying depths. We connect the turnpike to the overthinking phenomenon, and provide theoretical insights into the success of early-exit criteria derived from softmax probabilities, specifically entropy and max-probability. In a range of experiments, we demonstrate the existence and effect of turnpikes and the resulting similarity in the early-exit performance for ResNets of varying depths.
comment: Accepted at NeurIPS 2026
★ Stray Losses in Power Transformers: A Comparative Study of Modeling Frameworks for Mild Steel
Accurately predicting eddy-current and hysteresis losses in mild steel components such as the structural parts and tank of power transformers remains a challenge in transformer design. Conventional numerical approaches for eddy-current loss analysis require discretization at a fraction of the skin depth. While accurate, such fine volumetric mesh becomes computationally prohibitive for large power transformers. Moreover, hysteresis loss is not always included in this analysis. To investigate the trade-off between computational cost and accuracy, this work presents a comparative analysis of eight distinct modeling frameworks for predicting both eddy-current and hysteresis losses in mild steel. These include efficient methods avoiding meshing, such as standard impedance boundary conditions, and more accurate approaches retaining full volumetric meshing, namely complex permeability, nonlinear single-valued B-H curve, and Jiles-Atherton hysteresis modeling. To accurately represent the material's electromagnetic behavior, the properties of the mild steel used are measured directly from samples. Finally, loss measurements are conducted to benchmark all eight frameworks, demonstrating mean absolute percentage errors between predictions and measurements generally below 20%.
comment: 13 pages, 18 figures. Submitted to IEEE Transactions on Magnetics
★ Graph-Based Dynamic Activity Prediction for Real-Time Power System Applications
Online dynamic security assessment and day-ahead planning require rapid identification of where disturbances will produce significant dynamic activity. Full time-domain simulations are computationally demanding, while existing machine-learning approaches predict system-level outcomes or require post-disturbance trajectories. This paper proposes a pair-aware Graph Isomorphism Network with Edge features (GINE) framework comprising two task-specific models for predicting bus-voltage and generator apparent-power activity from the operating point and contingency description. Physically aware KPI cuts convert simulated responses into ordinal activity targets for model training. Evaluations on Nordic 32 and the French transmission system with more than 6 000 buses show that pair-aware fusion and graph embeddings improve on a local baseline on Nordic 32, with GINE the lower-cost graph encoder, and that the framework remains effective at large scale. As a downstream application, the predicted activity maps guide Adaptive Model Simplification, reducing continuous and discrete model variables by 72% and 66%, achieving speedups of up to 3.16x, and preserving voltage trajectories when critical substations are retained. These results demonstrate scalable, pre-disturbance prediction of component-level dynamic activity.
★ Comparing model representations and fitting methods for port-Hamiltonian system identification
Port-Hamiltonian models describe physical systems by separating stored energy, dissipation and external inputs. Learning such models from data requires a choice of representation and fitting method. We compare using neural networks, analytic expressions and combinations of the two for representing the different components of the port-Hamiltonian system, and compare fitting the models with gradient-based optimisation or sparse regression. Testing on simulations of mechanical and electrical systems, with and without added noise, we present prediction accuracy and computational cost of the different approaches.
comment: 6 pages, 3 figures; submitted to the invited session "Learning and identification of port-Hamiltonian systems" at the 21st IFAC Symposium on System Identification (SYSID 2027)
★ On Revolutionizing Power Systems: Suitability of Quantum Computing-Ready Pipelines for Stochastic Unit Commitment Optimization
Modern unit commitment (UC) scenarios, particularly those involving uncertainties from demand fluctuations and renewable energy sources, pose significant challenges to traditional optimization methods due to the large search spaces they encompass. This paper proposes a quantum computing-ready optimization pipeline for deterministic and stochastic UC, integrating classical, quantum-inspired, quantum annealing, and hybrid quantum-classical solvers across test systems, including the IEEE 118-bus system. A novel Quadratic Unconstrained Binary Optimization (QUBO) reformulation accommodates generator scaling and 24-hour scheduling horizons with compact Boolean logic penalties for startup, cooldown, and spinning reserve constraints. A divide-and-conquer strategy and Quantum Computing Monte Carlo (QCMC) simulation address scalability across 27,730 uncertainty scenarios, reducing compilation time by up to two orders of magnitude. Experimental results demonstrate the pipeline's ability to achieve efficient, scalable, and accurate UC optimization under uncertainty: the hybrid quantum method approach matches the classical baseline, reduces stochastic UC runtime from 40.76 to 14.79 hours, and characterizes pure quantum annealing as limited to small systems with 0.22% to 2.18% encoding errors and constraint violations. These results provide an analysis of the scalability and computational practicality of hybrid quantum optimization for generator scheduling in UC.
comment: Accepted for publication in IEEE Transactions on Industrial Informatics. \c{opyright} 2026 IEEE. 17 pages
★ Two-Player Zero-Sum Hybrid Games with Set-Valued Dynamics
In this paper, we formulate a two-player zero-sum game under dynamic constraints defined in terms of a hybrid inclusion. The game consists of a min-max problem involving a cost functional that depends on the actions and corresponding (potentially non-unique) solutions, defined as functions of hybrid time and, hence, can flow or jump. We present sufficient conditions given in terms of Hamilton-Jacobi-Isaacs-like equations to establish a bound and to exactly evaluate the worst-case cost under the optimal strategy. Under additional conditions, we show that the proposed optimal state-feedback laws render a set of interest pre-asymptotically stable for the resulting hybrid closed-loop system. The results are illustrated in an application with sporadic jumps and linear quadratic hybrid dynamics.
comment: 31 pages, 3 figures
★ From Roadside Sensing to Risk-Informed Safety Control: Surrogate Safety Measures as a Cyber-Physical Interface
Proactive intersection safety analysis depends on trajectory data whose reliability varies by sensing modality, yet infrastructure-perception and surrogate safety measure (SSM) research have only recently begun to converge. This paper synthesizes both literatures alongside control theory to clarify how sensing choices, safety metrics, and predictive control connect end-to-end, and where infrastructure-derived risk signals could inform autonomous vehicle decision-making. Searches covered light detection and ranging (LiDAR), cooperative perception, time-to-collision (TTC), post-encroachment time (PET), extreme value theory, model predictive control (MPC), and signal control. Screening considered trajectory-based safety and control relevance, excluding detection benchmarks without safety linkage. This structured narrative synthesis retained 56 sources. No single SSM is universally valid: one-dimensional TTC and modified TTC suit longitudinal conflicts, two-dimensional TTC extends time-based measurement to crossing geometries, and observed PET captures crossing and vulnerable road user (VRU) conflicts but only after the event. LiDAR consistently outperforms camera-only sensing for continuous three-dimensional trajectory extraction, illumination robustness, and privacy, though its range degrades in heavy rain, fog, and snow. Its output becomes control-ready only when paired with filtering--Kalman, Savitzky-Golay--that stabilizes noisy detections. This review complements it from the infrastructure side: it examines how roadside sensing affects that chain's reliability, classifies SSMs by conflict geometry and temporal availability, and maps risk components onto control objectives and constraints for infrastructure and connected and automated vehicle (CAV) applications.
comment: Accepted by 2027 TRB Annual Meeting
★ Caging Vision-Language Navigation for Legged Robots via Barrier-Certified Safety Architecture
By harnessing the inherent commonsense capabilities of pretrained Large Language Models (LLMs) and Vision-Language Models (VLMs), current Vision-Language Action(VLA) models for legged robots excel at interpreting language-based navigation commands but lack the physical grounding and explicit safety assurance required for real-world operation. Therefore, this paper aims to cage Legged-Vision-Language Navigation (VLN) through a safety architecture that provides physical grounding and ensures navigation safety. The key idea is not to abandon the powerful capabilities of end-to-end VLA planning, but rather to augment and cage them. In particular, we integrate a high-level VLA planner with a low-level physical-grounding layer that leverages efficient state estimation, in-situ 3D spatial perception, and an Signed Distance Field (SDF) representation to generate executable trajectories. We provide a navigation planner with provable collision-avoidance guarantees under the stated assumptions by combining a low-dimensional Bezier-parameterized MPPI planner with Discrete Exponential Barrier Function (dEBF) constraints. The proposed Legged-VLN system is modular, making it compatible with various VLM/VLA models and agnostic to the robot platform. The proposed system is validated with extensive real-world experiments on humanoid and quadrupedal robots, demonstrating versatile, efficient, and safe vision-language driven navigation capabilities.
★ A Matrix-Separation Approach for Stability of Discrete-Time Systems with Time-Varying Delay
This letter addresses the stability analysis of discrete-time linear systems with time-varying delay using a Lyapunov-Krasovskii functional (LKF) framework. An improved matrix-separation-based inequality (IMSBI) is developed to obtain tighter bounds for the summation terms arising in the forward difference of the LKF. Based on this result, an augmented LKF incorporating additional cross-terms is constructed to capture the interaction between system states and delay evolution. This leads to a less conservative delay-dependent stability criterion expressed as linear matrix inequalities (LMIs). Numerical results demonstrate improved stability margins and a reduced number of decision variables compared with existing methods.
comment: Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works
★ On the Convexity of Common Resource Adequacy and Capacity Accreditation Metrics
Many emerging capacity expansion optimization formulations implicitly presume the convexity (or concavity) of power system adequacy risk (or capacity accreditation) metrics. These assumptions have yet to be formally justified. We present concise mathematical proofs validating the claim for certain metrics (expected unserved energy and effective load carrying capability), but also provide simple invalidating counterexamples for others (loss-of-load expectation and equivalent firm capacity). Formulations built on non-convex (non-concave) metrics structurally undervalue candidate portfolio adequacy, and so cannot claim to produce least-cost expansion plans.
★ Sufficient and Necessary Smooth Barrier-Like Conditions for Continuous-Time Finite-Horizon Safety Verification
We study finite-horizon safety verification for continuous-time stochastic systems modeled by stochastic differential equations. Given an open and bounded safe set, a compact initial set, a finite time horizon, and a prescribed violation-probability threshold, the objective is to certify that the exit probability from the safe set before the horizon remains below the threshold. Under uniform ellipticity of the diffusion matrix and regularity of the safe-set boundary, we develop a time-dependent barrier-like condition expressed as a set of functional inequalities, obtained by relaxing a parabolic initial-boundary value problem associated with the exit probability. If a function satisfies the condition, then the probability of leaving the safe set before the horizon is guaranteed to remain below the threshold. Conversely, when the exit probability is strictly smaller than the prescribed threshold uniformly over the initial set, the barrier-like condition is also necessary: there exists a smooth function satisfying it. Furthermore, when the dynamics are polynomial and the set descriptions are semialgebraic, the problem of verifying the condition admits a sum-of-squares formulation that is both sound and complete, yielding a complete degree hierarchy for finite-horizon stochastic safety verification.
★ Assignment-Free Swarm Navigation via Adaptive Target-Field Planning and Mean-Shift Exploration
Mean-shift exploration enables assignment-free swarm shape assembly, but obstacle-constrained navigation additionally requires coordinated adaptation of the desired morphology and its physical execution. This paper presents an environment-adaptive navigation framework that integrates target-field planning, execution regulation, and morphology-preserving mean-shift exploration. A feasibility-aware graph search plans the position, orientation, and morphology of an unlabeled target field without specifying robot-specific goals. An execution-regulated reference manager adjusts field progression and morphology switching using swarm entering, coverage, and lag metrics. A moving-frame controller separates prescribed rigid motion from local mean-shift redistribution and incorporates neighbor interactions and obstacle responses. Simulations and hardware experiments demonstrate that the proposed framework coordinates morphology transitions and collective motion for assignment-free swarm navigation through cluttered environments.
★ Training Specialist LLMs on Your Phone: Forward-Only Optimization via Block-Spectral Coordinates
Large language models (LLMs) on personal devices keep user data local, but limited memory restricts their general capability. Since most user-specific tasks do not require the breadth of frontier models, a practical solution is to adapt a shared on-device model into lightweight specialists. However, backpropagation is prohibitive in memory, and users rarely provide labeled data. We present MobileSpec, a system that trains specialists with forward passes alone and learns from automatically checked task outcomes, as in reinforcement learning with verifiable rewards (RLVR). Because forward-only updates are built from random perturbations, training is efficient only when these perturbations resemble the weight changes of RLVR, which largely preserve the singular values of pretrained weights and rotate their singular vectors. The core algorithm of MobileSpec, FuRA-S, produces such perturbations while training only the block-spectral coordinates of pretrained weights, namely the singular values of their column blocks. As we show theoretically and empirically, rescaling these blocks independently shifts their relative contributions to each weight matrix and thereby rotates its singular vectors. Since this update can be rewritten as a transformation of the layer input, FuRA-S also adapts INT4 models without modifying their weights. With specialists of only 1-5 MB that update 0.015-0.05% of the parameters, MobileSpec raises the success rate of a phone-use agent from 6-23% up to 89-100% and trains a personalized 8B model on an iPhone 16 Pro overnight. On RLVR benchmarks, FuRA-S is competitive with full-parameter evolution strategies (ES) and outperforms ES over LoRA adapters and quantized ES on INT4 models.
comment: 27 pages, 11 figures, 15 tables
★ TriMedium: A Cross-Medium Simulator for Embodied Networking Across Air-Ground-Sea
Embodied intelligent applications have expanded from air and ground into maritime environments, where missions such as coastal reporting require coordination among underwater sensing, surface gateways, and aerial relays. Yet field trials remain costly, and existing simulation tools make it difficult to study how cross-medium communication affects cooperation among these platforms. We present TriMedium, an integrated Air-Ground-Sea simulator for cross-medium embodied networking. TriMedium has three design elements. First, it provides a shared three-dimensional world where aerial, terrestrial, and underwater entities coexist, so underwater agent motion and visual observations stay geometrically consistent with air and ground nodes. Second, it uses a common simulation clock that preserves observation timestamps while scheduling delayed communication and computation, clarifying when the shore receiver obtains each observation. Third, modular quantization and compression interfaces accommodate heterogeneous telemetry and image pipelines, while integrity checks separate intentional lossy distortion from incomplete or corrupted delivery. Three case studies validate TriMedium. Multimodal data delivery tests telemetry and image pipelines, scene construction examines cross-domain spatial consistency, and positioning assesses geometric accuracy under delayed communication. Together, they show TriMedium enables embodied networking by quantifying cross-medium delivery dynamics and fidelity across heterogeneous payloads.
comment: 6 pages, 6 figures. Submitted to IEEE ICC 2027
★ Multi-experiment identification from short repeated transients:the human response to pushes in a Tai Chi posture
The effect of Tai Chi postures on the mechanical response of the body to external pushes is usually assessed with clinical balance scores, which say little about the underlying dynamics. We identify, for five subjects and eight postural conditions, linear models from the mechanical work transferred by a pendulum hitting the forearm to the change of the antero-posterior ground reaction force. Each trial contains 10 to 16 impacts, and each impact yields a transient of 0.55\,s only. Building on earlier analyses of these data, we improve the processing chain in three ways. The input is related to the energy lost by the pendulum and its repeatability is quantified. The output is defined component-wise, which makes it independent of the zeroing of the force plate. Finally, the impacts are stacked in a multi-experiment least-squares problem and validated on unseen impacts. Tenth-order ARX models then reach a median leave-one-impact-out fit of 65\%, against 72\% for the repeatability of the response itself, and never require stability enforcement, unlike estimation on averaged impacts. Closing the eyes lengthens the identified delay by about 70\,ms in all subjects, and the Tai Chi posture increases the antero-posterior force transmitted per joule in four of five subjects. Surface EMG envelopes show a tonic co-activation of the lower-leg muscles in this posture, a smaller reactive triceps burst and a fast habituation of the tibialis response. Together, they point to a stiffer, anticipatory postural strategy.
★ Mass Scaling Laws for Energy-Optimal Horizontal Multirotor Flight
Multicopters are a type of aerial robot seeing extensive real-world applications but with range and endurance as critical bottlenecks. Optimization of its energy efficiency through energy-aware flight planning (e.g. cruise speed-scheduling) has hence been an important topic. The mass of the multicopter is an important factor to consider, as it has a significant impact on efficiency, and many common applications are subject to frequent mass variation due to payload change. This work derives closed-form scaling relationships between the energy-optimal steady horizontal flight behavior of a multicopter and its vehicle mass, and validates them experimentally on two multirotor platforms. Specifically, for mass-only variation, i.e., when payload changes the vehicle mass while the airframe geometry, aerodynamic characteristics, and rotor configuration remain unchanged, we show through a combination of numerical simulation, theoretical derivation, and experimental validation that (1) the energy-optimal velocity is proportional to the square root of vehicle mass; (2) the energy-optimal pitch angle is constant with respect to mass; and (3) the optimal energy per meter is proportional to mass. The analysis shows that these three relationships arise from a common mass-similarity property of the steady-state flight and rotor-aerodynamic equations. Furthermore, the obtained scaling relationships enable simple yet effective cruise-speed planning to optimize energy efficiency and extend the range and endurance of the multicopter.
comment: Accepted to IEEE Robotics and Automation Letters, October 2026
★ TuneBench: Benchmarking AI Agents for Power System Dynamic Model Calibration
Large Language Model (LLM)-based agents can automate complex engineering workflows by interacting with tools, interpreting feedback, and making iterative decisions. This paper introduces TuneBench, a benchmark for evaluating Agentic AI in power system dynamic model calibration, where controller parameters are adjusted to reproduce target dynamic responses. The benchmark reflects a common Transmission System Operator (TSO) practice, in which equivalent models used for Dynamic Security Assessment (DSA) are calibrated against measurements from grid-code compliance tests when proprietary models are unavailable. Two cases of increasing complexity are considered: the governor and Automatic Voltage Regulator (AVR) of a synchronous generator and the control system of a Battery Energy Storage System (BESS), both tuned against active-power and voltage step tests within a budget of 15 simulations. LLM-based agents are compared with random search and Genetic Algorithm (GA) baselines over 10 independent runs. Only the most advanced models, Opus 5.5 and GPT-6, accurately reproduce all target trajectories, with median R2 above 0.995 in every test and tracking errors up to two orders of magnitude lower than the baselines. The agent logs show that these agents autonomously extract dynamic performance indices, such as settling time and damping, and relate them to specific control parameters, decomposing the tuning problem into physically interpretable sub-problems. The benchmark is publicly available at https://github.com/Power-Agent/PowerAgentBench/tree/main/benchmarks/dynamic/level3.
★ On the IQC and SRG Frameworks: Equivalences and Converse Results
Integral quadratic constraints (IQCs) and scaled relative graphs (SRGs) are two prominent frameworks for analyzing feedback systems. Although both frameworks are rooted in the same graph separation principle, a general characterization of their relationship has remained unavailable. This paper establishes conditions under which the IQC- and SRG frameworks are equivalent. To this end, we introduce two new classes of systems, termed disjunctive and conjunctive systems, which are respectively characterized by unions and intersections of quadratic input-output constraints. Under structural assumptions, we prove equivalence between the IQC and SRG frameworks for these system classes. By exploiting this equivalence, we develop novel IQC- and SRG-based converse stability results for linear and nonlinear systems. Our results accommodate multiple quadratic constraints, extending well-known converse results in robust control beyond the single-constraint setting.
comment: 15 pages, 6 figures
★ A research world model of power engineering learns its dynamics and reveals what-ifs
Decisions about the direction of research in power engineering determine which technologies mature and, through them, shape infrastructure that lasts decades, yet their consequences cannot be observed before they are taken, and existing forecasts of research fields extrapolate a single trajectory. Here we introduce research world models, which treat a research field as a dynamical system that can be acted upon, and establish one for power engineering from 1.1 million papers. A distilled language model reads each paper into role-typed concepts, two sophisticated, high-fidelity graph-temporal neural network cores learn how the resulting concept network evolves, and operators learned from natural experiments in the publication record estimate the effects of interventions. In temporally held-out tests, the model forecast years ahead which concepts would grow and which method-target pairs would be joined for the first time, outperforming tuned machine-learning baselines in most settings, and its estimated intervention effects tracked those realized in natural experiments it had not seen. Its training scheme and three of its operators replicated in two fields withheld from development. Questions posed in natural language are compiled by a domain-specialized 27-billion-parameter language model into executable actions and answered as paired simulations with and without the intervention. The answers were rated 4.1 of 5 for faithfulness by an independent language-model judge. By revealing the latent dynamics of a research field and its response to interventions, research world models offer a new way to simulate and test research decisions before they are made.
★ Distributed LTL Synthesis under Arbitrary Message Adversaries
Reactive synthesis is the problem of generating a program that produces outputs satisfying a logical specification for every input stream. We study distributed synthesis for two processes communicating in rounds over unreliable links, with at least one message delivered in every round. Despite this delivery guarantee, we show that synthesis from linear temporal logic specifications is undecidable, even when messages carry the processes' complete causal histories. More generally, we give a complete characterization of the communication restrictions that make synthesis decidable. Restrictions are described by a message adversary $A$, a set of allowed infinite graph histories: sequences of communication graphs, each specifying which messages are delivered in a round. Iterated chromatic subdivision associates these histories with points in the unit interval, yielding an image $π(A) \subseteq [0,1]$. For every $ω$-regular message adversary $A$, synthesis is decidable if and only if $π(A)$ is nowhere dense. In the decidable cases, our procedure takes a deterministic parity automaton for $A$ and a specification, decides realizability, and constructs a realizing distributed algorithm whenever one exists. Further, our topological arguments shed new light on the fundamental requirements of synthesis. They show that, for $ω$-regular message adversaries, synthesis is decidable if and only if consensus is repeatedly solvable in bounded time. This links two previously unrelated problems from formal verification and distributed computing through the processes' uncertainty about communication.
★ Supervised Residual NMPC in Force-Disturbed Quadrotors
Residual deployment can affect closed-loop tracking. We compare two ways to deploy one supervised acceleration-residual network in nonlinear model predictive control (NMPC) of a quadrotor. Cyclewise deployment repeats one correction across the horizon. Stagewise deployment evaluates shifted prediction nodes before the solve. Network parameters remain fixed during control. On prescribed force pulses, both schemes improve average tracking mainly after force removal, but worsen hover; some prediction checks also fail. Seed 1 reinforcement-learning benchmarks add no mean benefit under pulses or prescribed random forces. On 220 conditions, cyclewise and stagewise supervision reduce mean integral absolute position error (IAE) by 4.38% and 7.77%. On a separate 32-condition bank, fixed gain 1.5 lowers IAE by 27.47% relative to stagewise supervision but raises angular-rate root-mean-square error by 10.19%. In one synthetic computational fluid dynamics field, stagewise supervision lowers four-task mean IAE by 2.01% against a same-graph zero-residual control; some secondary metrics worsen. These simulations show that residual deployment affects tracking, with gains and limitations that depend on task and motion metric.
★ Automated consolidation of dispersed DSO data resources into a consistent digital twin of the distribution grid
The energy transition causes a growing number of distributed flexible energy resources in the low voltage grid. To utilize that flexibility, automation and control algorithms are needed, which typically are developed in theoretical research. We build a cyberphysical algorithm test bed to bridge the gap between theoretical research and practical application in cooperation with a local DSO in Walenstadt, Switzerland. To obtain suitable monitoring and data analysis capabilities from their existing data landscape, we propose tool-independent data harmonization measures in preparation for a digital twin of the grid. For data extraction from high-security network zones we developed a secure pipeline that implements a logical data diode ensuring one-way traffic only. Furthermore, we established a test community of voluntary prosumer households, whose flexible assets were made accessible through a common backend, giving us actuation capabilities. The presented publish-subscribe architecture is not only suitable to integrate various data sources into a central data collection, but also to manage access privileges of different stakeholders. This could support unbundling of an organization's data flows and the implementation of data space concepts.
comment: This paper is a preprint of a paper accepted by IET conference proceedings and is subject to Institution of Engineering and Technology Copyright. When the final version is published, the copy of record will be available at IET Digital Library
★ WiSI: Wireless-in-the-Loop Self-Improvement
The growing autonomy of AI agents has enabled self-improvement, where agents iteratively refine their programs through execution feedback and performance evaluation. As these agents increasingly operate in physical environments and rely on wireless connectivity, communication conditions become an integral part of their improvement process. However, existing approaches largely assume ideal communication, overlooking how wireless conditions shape both the feedback that guides program modifications and the evaluation outcomes that determine which modifications survive. This challenge is further complicated by the coupling between agent behavior and wireless connectivity, as agents influence their own communication conditions through mobility and transmission decisions. To address this problem, we propose WiSI, a wireless-in-the-loop self-improvement framework that integrates spatiotemporal wireless channels, edge-assisted perception, and Large Language Model-driven program improvement. By incorporating wireless execution into the improvement loop, WiSI reveals how wireless execution experience shapes the generation and retention of program structures, and investigates the wireless conditions needed for robust improvement. Experiments show that agents improved under ideal links achieve only 40.0% task success under congestion, compared with 77.2% when improved under realistic wireless conditions. More importantly, wireless-aware feedback and selection jointly enable agents to discover and retain communication-efficient program structures that outperform parameter tuning and remain robust even with inaccurate run-time channel observations. We further find that moderate-load or mixed-network evaluation maintains strong performance across deployment conditions, while simplified models preserving link-rate statistics achieve comparable results without detailed spatial channel information.
comment: The code is available at \url{https://github.com/NICE-HKU/WiSI}
★ Competition, Switching, and Control in Coupled Adoption-Opinion Dynamics
We propose a Multi-technology AdopTion-opinion co-Evolution (MATE) model to describe the diffusion of $m \geq 2$ competing alternatives within a networked population whose opinions evolve under social influence and the feedback generated by adoption. Communities interact through two layers: a physical network governing exposure to adoption, and a social network shaping opinion evolution through social imitation. Individuals can adopt a technology, eventually becoming dissatisfied at a rate reflecting the quality of the technology, and switch to an alternative at a rate driven by their opinion of that alternative. For an arbitrary number of competing technologies $m \geq 2$, we characterize the equilibria and establish the existence of an adoption-diffused equilibrium in which all technologies coexist. For $m > 2$, the equilibrium configuration depends on the broader competitive environment: unlike the two-technology case, within-community relative adoption generally depends also on equilibrium opinions and on the other alternatives. Finally, we formulate finite-horizon opinion-shaping control problems, derive necessary optimality conditions via the discrete-time Pontryagin maximum principle, and compute numerical extremals using a forward-backward sweep method.
comment: 14 pages, 7 figures
★ Moving Horizon Estimation and Constrained Model Predictive Control for Balancing a Wheel-Legged Robot Under Unknown Disturbances
Wheel-legged robots balance on two wheels while using articulated legs to change height, absorb impacts, and cross uneven terrain. On hardware, the balance controller must rely on noisy inertial and encoder measurements, and it must reject disturbances that are not known in advance, such as payloads and wind, without exceeding motor torque limits. This paper presents a moving horizon estimator (MHE) combined with an offset-free, constrained model predictive controller (MPC) for the sagittal balance of a 9.95 kg wheel-legged robot. The estimator uses a virtual-leg model augmented with a horizontal hip force, a body pitch torque, and a gyroscope bias. Its accelerometer model removes the predicted hip acceleration, and its process noise is increased when an impact is detected. We evaluate the approach in MuJoCo with simulated sensor noise, bias, quantization, and delay, and compare it with LQR baselines that use ground-truth state, a complementary filter, or the same MHE. In a driving, squatting, reversing, and bowing sequence, MPC with MHE reduced the RMS speed error to 0.11 m/s, compared with 0.24 m/s for LQR with ground-truth state. Under an unknown 2 kg payload and a 10 N headwind, the mean position error in wind was 6 cm, compared with 81 cm for LQR with ground-truth state. Across a 3 $\times$ 4 grid of payloads and headwinds, LQR with the same estimator fell in 3 of 12 combinations, compared with 1 of 12 for MPC. These results are limited to simulation on flat ground, and experimental validation on hardware remains future work.
★ Large Language Models for Forecasting of Electricity Price Spikes
Can market notices read by a large language model (LLM) improve short-horizon electricity price spike forecasts? We study five regions of Australia's National Electricity Market. A local LLM extracted features once per notice, only notices visible at the forecast origin were used, and a gradient-boosting model produced onset probabilities. In an out-of-sample development back-test (July 2022 to July 2024), notice features did not lower the 60-minute Brier score of a numeric model including the market operator's next-hour forecasts. At 24 hours they did, but rule-based features did so almost equally and attribution to notice content was not established. A locally frozen protocol, not publicly deposited, governed a retrospective single-use holdout from 15 August to 17 October 2026. At 60 minutes, the hypothesis that LLM features lower the numeric model's Brier score was not established (H1; relative Brier skill -0.09%), so the planned comparison with rule-based features was not tested (H2). At 24 hours, the hypothesis that LLM features lower the score against a text-free control was not established (H3; +0.37%, 95% CI [-9.74, 1.39], inconclusive). By the frozen wording rule the non-significant 24-hour improvement comes mainly from recalibration and cannot be attributed to notice content, although post hoc the decomposition could not separate recalibration from resolution. The 24-hour base rate was 1.56% on the holdout against 14.0% in development. With 5 onsets on 3 days and 24-hour positives in 2 of 22 three-day blocks, the holdout neither established nor excluded a 24-hour effect.
comment: 31 pages, 7 figures
★ Computing Invariant Reachable Sets through Sum-of-Squares Programming
Funnels, time-varying invariant sets around nominal trajectories, certify that a closed-loop system tracks its plan, and can be composed into verified motion plans. Their computation by sum-of-squares (SOS) programming is well established in formulation but not in practice: the result depends on implementation choices such as the initial guess, Lyapunov function normalization, and termination rule, which the literature documents only briefly. This paper presents a self-contained treatment of SOS-based funnel computation, implemented in the open-source package computeSOSFunnels. We propose a minimum-volume formulation with inlet and outlet constraints under which funnels with coinciding nominal endpoints compose by construction. Numerical experiments on a unicycle, with timing studies on a cart-pole and a 12-state quadrotor, characterize the pipeline: infeasible initial guesses are partly decidable before solving, the certificate shows no violations among 39,000 boundary samples per funnel under the true dynamics, and constructing the semidefinite program, not solving it, dominates computation time.
comment: 8 pages, 3 figures, 3 tables. Submitted to IEEE Transactions on Control Systems Technology. Code: github.com/khalid2696/computeSOSFunnels
★ Attack-Aware Intrusion Risk Monitoring for Aerial Surveillance via Probabilistic Signal Temporal Logic
This paper considers online intrusion-risk assessment and graded warning for unmanned aerial vehicles under bounded measurement attacks. To address this problem, we develop a monitoring framework that combines trusted state estimation, attack-aware risk assessment, and probabilistic signal temporal logic (PrSTL). Trusted observations support recursive state estimation, while potentially attacked observations inform integrity testing and intrusion-risk assessment. The key contribution is a confidence-set construction that combines normalized innovation statistics with physical attack bounds to characterize admissible measurement biases. The resulting family of bias-corrected Gaussian posteriors yields an upper bound on the probability of occupying a no-fly zone or its surrounding safety margin. Confidence guarantees and finite-cover numerical certificates quantify the reliability of this risk envelope, which is evaluated with or without an integrity alarm. The PrSTL monitor analyzes the resulting risk sequence to distinguish ordered, persistent, and repeated risk and produces graded warnings with explicit robustness margins and supporting windows. We validate the monitor in flight experiments with a Crazyflie quadrotor tracked by a Vicon motion capture system. Controlled comparisons quantify the contributions of individual monitoring components. Prescribed risk profiles verify the temporal semantics and illustrate how distinct risk patterns produce different warning levels.
comment: 10 pages, 8 figures
★ Grasping and Stabilizing Tumbling Objects with Rotating-Base Multi-Arm Space Manipulators
This paper presents an innovative guidance and control method for a space manipulator that actively maneuvers its base to hold a tumbling object stationary relative to the base during proximity operations. This simplifies motion planning, reduces collision risk, and enables impact-free capture using single- or multi-arm configurations, with or without predefined grasping interfaces. The proposed approach involves two phases. In the pre-grasping phase, a synchronization control aligns the target's center of mass (CoM) with the system's instantaneous center of rotation (ICR), virtually rigidizing the target-servicer system to keep the object stationary for reliable grasping. In the post grasping phase, a control strategy based on Hamiltonian dynamics and optimal control stabilizes the combined system by minimizing time, fuel, or energy, while respecting base torque limits and multiple grasping constraints. For structured targets, applied forces and torques stay within their bounds; for unstructured ones, friction constraints prevent slipping. A case study demonstrates the method's effectiveness in enabling both the subsequent grasping and detumbling phases.
★ Transferring Controller from Math to Real World Systems: A Formal Control Theoretic Approach
The letter aims to improve the math-to-real or math-to-sim transfer of a mission controller designed for a Temporal Reach Avoid Stay (T-RAS) specification. Our approach uses the control-theoretic concept of Incremental Input-to-State stability ($δ$-ISS). We model the real-world system as an uncertain system, represented by a nominal mathematical model with an additive disturbance. Our method involves two steps: firstly, we make the nominal mathematical model and the real-world system $δ$-ISS. Secondly, we design a mission controller for the mathematical model that satisfies a modified specification. We then prove that applying the same mission-level input sequence to the real-world system guarantees that the original specification is satisfied in the real world. Videos of real-world experiments are available at https://youtu.be/NvjkIZxtnPM.
★ Online Hamilton-Jacobi Reachability Onboard the Robot Control Loop: A Real-Time GPU Solver for Safety-Critical Autonomy
Hamilton-Jacobi (HJ) reachability provides certified backward reachable tubes (BRTs) for safety-critical robotic systems, but the level-set computation scales exponentially in the state dimension and has historically been performed offline. As autonomy stacks (ADAS/AV, humanoid robotics, and vision-language-action models) increasingly rely on black-box, probabilistic policies, a deterministic safety layer with formal guarantees is an essential complement. We present a CUDA implementation of the canonical WENO5 + TVD-RK3 + Lax-Friedrichs level-set Hamilton-Jacobi-Isaacs solver for 2D, 3D, and 4D dynamics that is fast enough to run inside a robot's control loop. In 32-bit floating point (FP32) on a single workstation GPU (NVIDIA RTX 4000 Ada), it computes the 3D Air3D BRT at 112 Hz, a 4D pursuit-evasion BRT at 61 Hz, and a 2D wind-affected drifter BRT at 74-147 Hz, 5.9-17.5x faster than the JAX reference hj_reachability at matched precision and 1.7-16x faster than a torch.compile reimplementation of the same scheme. In 64-bit floating point (FP64) it agrees with hj_reachability on eight benchmark cases, reproduces Mitchell's Air3D BRT to within 0.04% of the reference cell count, and runs 2.0-4.6x faster. On an embedded NVIDIA Jetson AGX Thor it computes Air3D at 66 Hz (N=41) and the 2D drifter at 54 Hz, 3.8-8.9x faster than hj_reachability on the same device. As a multi-agent application, an in-kernel label-propagation extension recovers the K-agent capture-basin Voronoi tessellation from a single combined HJ solve, independent of fleet size; it runs at 4.0 ms per tick in a closed-loop wildfire-response simulation of six quadcopters under time-varying, non-uniform wind.
comment: 12 pages, 4 figures, 5 tables
★ Conditions for Normality of Pontryagin's Maximum Principle in the Absence of State Constraints
Pontryagin's Maximum Principle is said to be "normal" for the problem at hand when it has a solution (maximizer u*) that explicitly involves the problem's cost function, making the maximum principle helpful (while in the opposite situation it is of no help). The aim of the present paper is, for Lagrange optimal control problems without state constraints, to identify classes of problems for which the Principle is normal. General conditions are introduced that guarantee normality, and classes of problems that satisfy these conditions are identified. Such classes include problems for which the constrained (by bounds on the control values) "augmented" reachable set, with cost along an upward vertical axis, has a convex lower-boundary function and the optimal terminal state lies in the relative interior of the reachable state set, with the terminal constraint within the affine hull of that set. Such classes include problems for which the constrained (by bounds on the values of the control signal) "augmented" (with the cost value along an upward "vertical axis") reachable set has a "convex" lower boundary for which the optimal terminal state lies in the relative interior of the set of reachable states. Special attention is given to Lagrange steering problems: sufficient conditions are obtained on a nested sequence of sub-classes of such problems, culminating with the class of steering problems with scalar control signal and linear time-invariant (LTI) state dynamics and objective integrand, for which normality is guaranteed whenever all eigenvalues of the state dynamics matrix A are real (e.g., if A is symmetric or triangular or the system is overdamped) or have small enough imaginary parts; simple 2 x 2 examples show that, in contrast with the situation in the finite-dimensional case, this property does not extend to arbitrary A matrices.
★ Translate, then decide: asymptotic optimality of latent-space control in bistable systems
This paper asks whether an optimal decision can be separated from the physical dynamics on which it acts. It studies this question through minimum-energy barrier crossing in scalar bistable systems. It characterizes the finite-horizon optimal feedback and shows that, as the horizon grows, it converges to a law that doubles and reverses the drift while it opposes crossing, and switches off once the barrier is passed. In physical coordinates, this law inherits the plant's nonlinearity, entangling decision and dynamics. We show that a plant-specific translation of state, input, and time maps every double well in the considered class to the same unit-rate linear world model, allowing a downstream decision stage to use a single plant-independent control primitive. This separation generally loses optimality at finite horizons because the translation does not preserve the physical control energy. Nevertheless, we prove that the excess physical energy of the layered controller vanishes as the horizon grows and that its limiting feedback reproduces the asymptotic optimum exactly. Thus, translation and optimization generally do not commute at a finite horizon, but commute in the asymptotic limit.
★ Decentralized Impedance Specifications for Grid-Connected Converters: a Sequence-Domain Gershgorin Method, and a Frequency-Wise Framework for Combining Different Criteria
In complex grid scenarios, with many power electronics (PE) converters connected, having specifications for converters can ease the design process. A practical approach is to impose the specifications on the converter's small-signal impedance in the frequency domain. This is advantageous since impedance is a physical property that can be measured. This paper focuses on providing decentralized impedance specifications for each converter connected to an ac bus. With decentralized specifications, each converter has its own specifications, which are independent of other converters' impedance. This enables a competitive multi-vendor environment while guaranteeing system stability. Existing methods of imposing decentralized specifications, based on the small-gain, passivity and Gershgorin theorems, are formulated within a common frequency-domain framework. Building on this, two new solutions that reduce the conservativeness of the specifications are proposed. First, Gershgorin-based decentralized specifications are formulated in the modified sequence domain. Second, a framework for combining different specification types across frequency ranges is developed and mathematically proven. The methods are compared on examples covering sub-synchronous and harmonic-range interactions of grid-following converters, grid-forming converters, and a mixed grid-forming/grid-following system, using analytical models, black-box impedance measurements, and time-domain responses obtained from simulations and an experimental setup.
★ Barrier-State-Based Safe Navigation in Complex Environments Using Look-Ahead Safety Corridors
Safe navigation in cluttered environments requires a safety representation that captures the available motion space along the planned path. A single local safe region may restrict progress through narrow passages and continuous turns. This paper presents a planning-guided look-ahead safety corridor framework for barrier-state-based navigation. Local safe convex polytopes are constructed around collision-free seeds along the forward reference path. Their facet constraints are combined using softmin, and the local functions are fused using softmax with entropy-based adaptive compensation. The resulting safe set remains within the union of the local polytopes while reducing the shrinkage caused by fixed compensation. A Poisson boundary-value problem then defines a differentiable interior safety field while preserving the prescribed corridor boundary. This field is embedded into the robot dynamics through one geometric barrier state. We establish sufficient conditions for corridor invariance based on barrier-state boundedness and active-corridor validity. Navigation simulations cover cluttered layouts, narrow passages, and moving obstacles. A differential-drive ground vehicle further demonstrates navigation through narrow passages and among obstacles absent from the prior map.
★ Data-Driven Stochastic LQR Control with Probabilistic Guarantees and Uncertainty Refinement
This paper presents a probabilistic framework for the direct, data-driven synthesis of the stochastic linear- quadratic output-feedback control for linear systems sub- ject to process and measurement noise. Using delayed input-output measurements, we derive a lifted parameter- ization together with an equivalent lifted linear quadratic cost, enabling controller synthesis without explicitly identi- fying the lifted dynamics. Two complementary closed-loop formulations, based on trajectory and covariance represen- tations, are then developed, whose uncertainties depend only on the realized but unknown lifted noise. We char- acterize the relative performance of the two formulations in finite-data regimes. High-probability uncertainty bounds are developed for realized noises, leading to a convex optimization solution to the problem, whose feasibility cer- tifies probabilistic mean-square stability (p-MSS) and per- formance guarantees. To reduce conservatism, Gaussian zonotopes are used to refine uncertainty sets by elimi- nating noise realizations inconsistent with measured data, and a novel intersection rule enables the consistent fusion of data-driven and physics-informed information. These components are integrated into an adaptive framework that unifies indirect (identification-based) and direct (control- oriented) data-driven linear-quadratic output-feedback con- trol. Unlike existing direct data-driven LQR methods that verify stability a posteriori, our by-construction approach achieves robust stability alongside reduced conservatism and superior performance. Simulation results show that the proposed controller approaches the ideal optimal solution while outperforming existing data-driven methods under noisy measurements.
♻ ★ Dexterous In-Hand Object Pose Reaching with Learned Safety Filters
Dexterous manipulation with multi-fingered hands is the next frontier of robotics, yet learned hand skills remain limited to simple grasps or basic repeated motions, such as rotating an object. As robots attempt more demanding in-hand maneuvers, safety, such as not dropping the object, becomes critical, yet it is rarely measured. We study in-hand object pose reaching, a task that demands diverse and difficult grasps: moving an object to almost any position and orientation within reach of the fingers, at any wrist orientation, through long sequences of consecutive goals. We present a recipe for training such policies for a LEAP hand in simulation without human data, built on a range of insights, from diverse resets and goals via generated grasp datasets to an adaptive friction curriculum. With careful system identification, the policies transfer zero-shot to the real hand, using either motion capture or a single wrist-mounted depth camera, and prove remarkably capable and robust. On top of the task policy, we add a learned residual safety filter with adjustable conservativeness, letting us tune the balance between performance and safety and greatly reduce object drops. In 13.3 hours of real-robot evaluation over 4,800 unseen goals across 4 objects and diverse wrist orientations, the state-based policy reaches 95.2% of goals within 2 cm and 0.4 rad, and the filter cuts its drops by 75% while still reaching 91.5% of goals. The policies can also spin a lightbulb held in an arbitrary pose, reach out-of-distribution goals, withstand fast arm motion and perturbations, and, needing only a wrist camera, serve as portable skills such as screwing in a lightbulb. We release the full recipe as a platform for studying performance, reliability, and safety in high-dimensional dexterous systems. Videos and more are available at https://ian-chuang.github.io/dexsafe/.
comment: 26 pages, 30 figures, 14 tables. Project page: https://ian-chuang.github.io/dexsafe/
♻ ★ Quantile Propagation Planning for Budgeted Tail-Risk Evaluation
Risk-aware planning must identify rare, costly outcomes using only a limited number of model evaluations. Monte Carlo rollouts spend this budget on noisy tail estimates, while planning from the mean can miss the tail entirely. We introduce quantile propagation planning (QPP), which propagates a small, fixed set of disturbance quantiles through the dynamics and integrates the resulting cost curve to evaluate both mean and tail risk. The method is exact at the propagated levels, converges as the quantile grid is refined, and admits a one-sided conservatism guarantee under stated structural conditions. In controlled overload and inventory tasks, QPP reduces realized tail cost and constraint violations relative to mean-rollout planning. On overload, it remains competitive with sampled evaluators that use substantially more trajectories. These results position QPP as a transparent, budget-controlled approach to tail-risk evaluation.
comment: 60 pages, 8 figures, 34 tables; includes appendices
♻ ★ Safe Navigation in Cluttered Environments Via Spline-Based Harmonic Potential Fields
We provide a motion-planning framework for target reaching and obstacle avoidance in cluttered environments. Given a polyhedral decomposition of the feasible space, the agent is guided through a prescribed sequence of cells by a suitable control policy. For each cell, we construct a potential induced by a harmonic function on the unit disk whose Dirichlet boundary condition is represented by a cardinal B-spline curve. A detailed analysis of the curve periodicity, support, and control-point selection provides an explicit analytical construction of these potentials and the associated vector fields. We further characterize inter-cell control continuity and forward invariance of non-exit facets, and introduce a smoothed control law ensuring continuous switching between independently constructed neighboring cells. Numerical results illustrate that the resulting framework safely guides the agent along the prescribed cell sequence from the initial point to the target.
♻ ★ Capability-Aware Heterogeneous Control Barrier Functions for Decentralized Multi-Robot Safe Navigation
Safe decentralized navigation in heterogeneous robot teams is challenging because robots with different dynamics and control limits cannot satisfy identical collision-avoidance obligations equally well. Existing methods often rely on model-specific safety formulations or allocate avoidance responsibility without considering whether the assigned control is physically realizable, possibly leading to safety-constrained QP infeasibility and safety violations. We propose Capability-Aware Heterogeneous Control Barrier Functions (CA-HCBF), a decentralized framework for consistent safety enforcement and capability-aware responsibility allocation. We derive a unified operational-space second-order control-affine formulation for holonomic and nonholonomic mobile robots through canonical transformation and backstepping, addressing the relative-degree mismatch while preserving forward invariance of the pairwise safe set. Second, we quantify each robot's achievable control authority along the safety-critical direction using a support-function-based capability metric and use it to distribute pairwise avoidance responsibility. A feasibility-aware clipping mechanism further mitigates physically unrealizable responsibility assignments. Simulations with up to 30 heterogeneous robots and hardware experiments demonstrate high task completion with low safety violations and reduction in local QP infeasibility.
comment: 8 pages, 3 figures, 2 tables
♻ ★ LQG solution for POMDP without estimating states: A minimum variance approach
This paper investigates the control of discrete-time linear time-invariant (LTI) systems subject to incomplete and corrupted measurements. Specifically, we focus on designing a Linear Quadratic Gaussian (LQG) controller without relying on explicit state estimation. By leveraging minimum variance duality, our approach allows the current control input to be represented as a linear function of available measurements and previously applied inputs, successfully reducing the task to a tractable deterministic optimization problem. We provide theoretical justification for this framework and demonstrate its practical effectiveness through numerical experiments.
♻ ★ Block Diagram Analysis of an Amplitude-Frequency Design Principle in Biological Oscillators
An important design principle for biological oscillators divides oscillators into two types: fixed frequency, variable amplitude and fixed amplitude, variable frequency. Because of the interplay of nonlinearity and feedback, both positive and negative, analytical investigations of this design principle are primarily based on numerical simulations of ordinary differential equations. To enhance the qualitative and quantitative characterization of this design principle, we adapted and developed a block diagram modeling framework. We showed how the observed amplitude-frequency characteristics could be obtained from the block diagram models of both the types of oscillators. We obtained explicit constraints on the positive feedback and negative feedback strengths for the oscillations to exist. We represented both types of oscillators in a single unifying, control-theoretic framework that connects both the types of oscillators under a single theoretical model. These results contribute to a systems and control perspective on oscillations in biology and related contexts.
comment: 15 pages, 10 figures
♻ ★ A Deep State-Space Model Compression Method using Upper Bound on Output Error
We study deep state-space models (Deep SSMs) that contain linear quadratic-output (LQO) systems as internal blocks and present a compression method with a provable output error guarantee. We first derive an upper bound on the output error between two Deep SSMs and show that the bound can be expressed in terms of the $h^2$-error norms between the layerwise LQO systems. In particular, we show that reducing the $h^2$ approximation errors of the LQO systems placed in shallow layers is effective in reducing the derived upper bound on the output error. Next, we formulate an optimization problem for the derived upper bound and develop a gradient-based MOR method. In the numerical experiments, using the IMDb task from the LRA benchmark, we demonstrate the effectiveness of the proposed upper-bound-based compression method. In particular, we show that the number of trainable parameters can be reduced by approximately 60\% without retraining while maintaining the performance of the original model.
♻ ★ Co-Optimization of Network Topology and Variable Impedance Devices under Dynamic Line Ratings in Power Transmission Systems
Power system operators are increasingly deploying Grid Enhancing Technologies (GETs) to mitigate operational challenges such as line and transformer congestion, and voltage violations. These technologies, including Network Topology Optimization (NTO), Variable Impedance Devices (VIDs), and Dynamic Line Rating (DLR), enhance system flexibility and enable better utilization of existing network assets. However, as the deployment of multiple GETs grows, effective coordination among them becomes essential to fully realize their potential benefits. This paper presents a co-optimization framework that models and coordinates NTO, VID, and DLR within a unified optimization scheme to alleviate network congestion and minimize operational costs. The NTO formulation is developed using a node-breaker model, offering finer switching granularity and improved operational flexibility. The inclusion of VIDs introduces nonlinear and non-convex relationships in the optimization problem. DLR takes into account of weather conditions, primarily wind speed and ambient temperature, enabling adaptive utilization of transmission capacity. The proposed framework is validated on standard IEEE benchmark test systems, demonstrating its effectiveness under varying numbers and placements of impedance controllers.
comment: This paper has been withdrawn due to its similarity to another paper
♻ ★ Reinforcement Learning based Traffic Signal Control for Queue Minimization: From Lane-based to Laneless Traffic
Efficient traffic signal control (TSC) is essential for reducing congestion and delays. Traditional TSC methods struggle to capture the dynamics of laneless, heterogeneous traffic. We propose a reinforcement learning (RL) based adaptive TSC framework whose reward depends only on approach-level queue length, motivated by the difficulty of obtaining per-vehicle measurements in laneless traffic, and evaluate it using SUMO on a conventional lane-based, homogeneous (LBHO) intersection and on a non-orthogonal laneless, heterogeneous (LLHE) intersection. On the LBHO site, the lowest mean configuration has about 23\% lower mean queue length than a Dynamic Webster baseline, and the unselected Full-State configuration about 15\% lower. On the LLHE site, all RL configurations have 66-73\% lower mean queue length than a Dynamic Webster baseline under nominal demand, and 63--74\% lower under two higher-demand scenarios evaluated without retraining. Our algorithm also outperforms actuated control and a RESCO-DQN baseline on mean queue length. We additionally release a laneless, heterogeneous-vehicle SUMO framework integrated with RL based signal control, to our knowledge the first such open implementation on a current SUMO version.
♻ ★ Reliability-based lifetime fatigue damage assessment of offshore composite wind turbine blades
This paper presents a method for stochastic deterioration modelling and fatigue damage assessment for composite wind turbine blades operating in offshore environments. The fatigue damage of the composite blades is analysed and assessed based on the estimates for the applied loads along the blade span, stress analysis, fatigue crack evolution, and lifetime probability of fatigue failure. The complex stress states of the blade are mainly caused by the aerodynamic loads generated by corrected blade element momentum theory, gravity loads and centrifugal loads. The fatigue of the wind turbine blade is then investigated on the basis of the actual fatigue damage propagation process. The stochastic gamma process is introduced to calculate the probability of fatigue failure of the blade for various critical limits, and these results together with lifecycle cost analysis are employed to determine the optimum maintenance strategy. Finally, a numerical example for an NREL 5 MW wind turbine blade is adopted to demonstrate the applicability of the proposed method. The numerical results show that the proposed approach can provide a reliable tool for estimating stress states, evaluating fatigue damage, analysing lifetime fatigue failure probability and optimising repair time of the composite wind turbine blade.
♻ ★ A Game-Theoretic Decision Framework for Optimal Selection of Coordination Detection Methods in Multi-UAV Fleet Operations
Detecting coordination among unmanned aerial vehicle (UAV) fleets operating in shared airspace and identifying the route-lead aircraft whose navigation decisions govern fleet behavior presents a fundamental speed--accuracy trade-off: fast methods enable real-time traffic management but sacrifice detection fidelity, while accurate methods may exceed the time budget for actionable airspace deconfliction. This paper presents a game-theoretic decision framework that resolves this trade-off by formulating method selection as a two-player zero-sum game between a Monitor (selecting computational methods and parameters) and Nature (selecting the unknown traffic scenario). We construct an end-to-end pipeline from trajectory surveillance data through eight candidate detection algorithms, a Monte Carlo sensitivity analysis characterizing their stochastic performance, and finally a multi-objective optimization layer that identifies Pareto-optimal method portfolios. The minimax solution provides a robust mixed strategy with a probability distribution over methods that guarantees worst-case performance regardless of scenario uncertainty. Experimental evaluation across 200 randomized configurations spanning 5--50 aircraft demonstrates that the framework recommends distinct method portfolios depending on operational priority: Koopman Phase dominates balanced (70.6%) and speed-priority (79.7%) profiles, while CRQA emerges as primary (47.4%) when route-lead identification is prioritized. The framework achieves a guaranteed game value of 0.29--0.53 (normalized utility) across all tested preference profiles, providing the first principled, scenario-adaptive methodology for computational method selection in UTM fleet monitoring operations.
♻ ★ Model Predictive Control of Heating Systems via Koopman-Operator Bilinearization
Hydronic heat emitters are widely used in buildings, but their control is challenging due to nonlinear dynamics, arising from both the heat transfer in each emitter and the hydraulic coupling among those that share a circulating pump. These nonlinearities are often simplified or neglected in the literature. In this paper, we use Koopman theory to make the resulting model more tractable for control. In particular, we propose an input reparameterization and construct an infinite-dimensional set of observables that yields an exact bilinear representation of the system in the chosen lifted space. We then show that truncating the infinite-dimensional representation leads to bounded approximation errors that decrease as the truncation order increases, as well as bounded trajectory deviations over a finite horizon. The resulting bilinear model is incorporated into a model predictive control (MPC) framework to provide thermal comfort in multi-zone buildings while reducing energy use. Using a numerical case study of a five-story building with 20 zones, we show that the proposed Koopman-based MPC achieves a realized performance cost close to that of nonlinear MPC with faster solve times, and outperforms MPC formulations based on time-invariant and time-varying linearizations.
♻ ★ A Hetero-functional Graph State Estimator for Watershed Systems: Application to the Chesapeake Bay
Regional watersheds are systems of systems in which hydrology, land use, and governance interact. Nutrient models such as the Chesapeake Assessment Scenario Tool (CAST) route nutrients from land to the estuary through lumped delivery factors, so the river network that connects sources to the estuary is not represented explicitly. This paper develops a region-independent watershed reference architecture in Hetero-functional Graph Theory (HFGT) and adapts the Weighted Least Squares Error Hetero-functional Graph State Estimator to estimate nutrient flows in that network from uncertain data. The framework is applied to the Chesapeake Bay watershed. A mass-conserving network of land segments, outlet points, and river reaches is built from public geospatial data. CAST applied loads, land-segment edge-of-river loads, and river-to-bay delivery factors enter the estimator as measurements, each with its own error term. CAST-reported loads delivered to the Bay are withheld from estimation and used only to test the estimated delivery. The estimator reproduces the applied loads (R^2=1.00) and the land-segment edge-of-river loads (R^2=0.997) and under-predicts delivery to the Bay by 8.0% for nitrogen and 5.7% for phosphorus. The gap is traced to CAST delivery factors that imply in-river gains, which a mass-conserving network cannot represent, and to land segments whose edge-of-river loads exceed their applied loads. The framework provides a common systems representation for coupling watershed models with infrastructure, economic, and governance models.
♻ ★ A System-of-Systems Convergence Paradigm for Societal Challenges of the Anthropocene
Climate, water, food, energy and governance challenges are interdependent, yet the tools used to study and manage them remain divided by discipline and sector. Coupling sectoral models pair by pair cannot keep pace with the cross-sector and institutional interactions that drive global environmental change. In this Perspective we argue that convergence requires a shared system ontology that preserves meaning as knowledge moves from observation to computation and back to decision-making. Building on a meta-cognition map of scientific knowledge generation, we propose six design criteria for convergent system-of-systems approaches and assess four candidate methods against them, identifying model-based systems engineering paired with hetero-functional graph theory as a candidate "convergence spine". We illustrate the paradigm in the Chesapeake Bay Watershed, where a single ontology links nutrient-flow estimation, institutional analysis of restoration governance, team science and graduate education, and we distinguish what has been implemented from what remains future work. We close with implications for how global environmental change research is organized, funded and taught.
♻ ★ An Introduction to Model-Based Systems Engineering and Hetero-functional Graph Theory for Hydrological Systems of Systems
This introductory overview describes a unified modeling framework for representing interconnected environmental systems using Model-Based Systems Engineering and Hetero-functional Graph Theory. Existing environmental models encode rich structural and process-level information, yet much of this knowledge remains implicit within domain-specific formulations. This paper demonstrates how such information can be formalized into a consistent ontology that explicitly represents system structure, function, and interconnections. Using illustrative hydrological examples, the tutorial illustrates how conventional process models can be translated into a scalable system-of-systems representation. Agricultural nitrogen applied to land is represented explicitly as the point at which human decisions enter the hydrological system, illustrating how models of human systems can be coupled to hydrological models within the same representation. These examples highlight how embedded relationships in existing models can be made explicit and computationally tractable. Using simplified examples, the overview provides a foundation for integrating heterogeneous models across environmental and engineered domains. More broadly, this work offers a step-by-step introduction to a generalizable modeling framework for complex systems.
Robotics
★ Pixel-UMI: Learning Environment-Aware Whole-Body Manipulation from Sensorized Human Demonstrations
Sensorized human demonstrations offer scalable data for mobile manipulation, but embodiment differences complicate policy learning. We present Pixel-UMI, a pipeline for learning environment-aware whole-body policies from human data. We first introduce Environment-Aware Retargeting (EAR), which combines reconstructed scene geometry with demonstrated head and gripper trajectories to determine robot posture and generate collision-aware joint and base targets. Delay-calibrated action assignment converts these trajectories into actions. However, kinematic differences can leave residual pose errors, creating viewpoint mismatches between human demonstrations and robot observations. We therefore introduce Retargeting-Error-Aware Masking (REAM), which uses these errors during training to reduce reliance on mismatched views at deployment. Finally, because estimated actions cannot fully capture contact-rich interactions, we collect on-policy human corrections through DAgger to improve performance. We evaluate Pixel-UMI on five challenging tasks spanning articulated-object manipulation, deformable-object handling, constrained-space reaching, whole-body collision avoidance, and contact-rich bimanual coordination, achieving 87.0% average task progress. Video results are available at https://pixel-robot.com/pixel-umi
comment: Video results are available at https://pixel-robot.com/pixel-umi
★ Pixel: A Co-Designed Full-Stack Robot System for Learning General-Purpose Mobile Manipulation
General-purpose mobile manipulation requires a combination of capable robot hardware, scalable demonstration collection, expressive whole-body control, and effective policy learning. Yet these components are often developed independently, creating mismatches that limit the capabilities of the complete system. We present Pixel, a co-designed full-stack system for learning general-purpose mobile manipulation. Pixel combines a low-cost wheeled humanoid with human-aligned kinematics and compliant grippers, multiple teleoperation and robot-free human data-collection interfaces, a unified whole-body controller, and a common policy learning pipeline across human and robot demonstrations. A hierarchical learning stack further enables language-conditioned visuomotor control and long-horizon task execution. We evaluate the system across hardware, data collection, and policy learning, and demonstrate mobile-manipulation behaviors requiring dexterity, active perception, navigation, and whole-body coordination. We hope that Pixel provides an integrated and extensible foundation for studying how robot hardware, data, control, and learning can be designed together for robot learning. More results at https://pixel-robot.com/pixel
comment: More results at https://pixel-robot.com/pixel
★ D&A: Decoupling Description and Assessment Enables VLMs as Zero-Shot Robotic Reward Models
Vision-Language Models (VLMs) hold promise for zero-shot video task-progress estimation in robot learning, yet existing frameworks frequently suffer from confirmation bias, hallucinating progress in failed executions due to implicit success assumptions or relative temporal ranking designs. Naive prompting of frontier models collapses into a binary success detector that fails to capture nuanced, non-linear progress and failures. While post-training approaches can mitigate this through explicit training, they demand expensive real-world robotics data and require computationally intensive post-training for every individual base model. To resolve these limitations, we introduce D&A (Describe & Assess): our approach decouples progress estimation into an objective, task-grounded textual description of scene dynamics followed by an assessment phase using explicit rubrics. By compelling the VLM to synthesize an explicit, text-based definition of absolute success before scoring, the framework anchors evaluations to an external, global criterion rather than the video's internal timeline. Empirical results demonstrate that our method calibrates zero-shot estimates and mitigates confirmation bias, providing fine-grained, partial progress metrics that accurately penalize failed executions while correctly interpolating successful runs. Being purely prompt-based, D&A generalizes across tasks, embodiments, and benchmarks---matching or surpassing post-trained reward models---and its reward estimates enable efficient downstream reinforcement learning.
comment: Website at https://weirdlabuw.github.io/dna
★ 3D-BBDA: Backpropagation-Free Black-Box Domain Adaptation for Robust 3D Point Cloud Recognition WACV 2027
Robust 3D point-cloud recognition is critical for robotics, mapping, and urban sensing, yet point-cloud classifiers often suffer performance drops under domain shifts caused by sensor changes, environmental variation, and acquisition artifacts. Many existing adaptation methods are difficult to deploy in practice because they require access to source data or the internal parameters of the source model, which may be restricted by privacy, licensing, or security constraints. We present 3D-BBDA, a backpropagation-free black-box domain adaptation framework that adapts a source-trained predictor to unlabeled target data using only black-box outputs. Our method performs pseudo-label guided prototype-based alignment and employs a weighted prediction module to fuse source logits with alignment logits, reducing sensitivity to noisy pseudo-labels. To stabilize prototype memory when pseudo-label coverage is incomplete, we construct a Gaussian-perturbed prototype prior from unlabeled target predictions and use it as a fallback for missing or unreliable class prototypes during alignment. Experiments on four corrupted 3D benchmarks demonstrate consistent improvements over strong baselines. On ModelNet-40C, 3D-BBDA achieves mean errors of 32.08% (PointNet) and 29.62% (GDANet), outperforming Source-only (35.88% / 34.03%) and BFTT3D (33.78% / 32.36%). 3D-BBDA requires only a single forward pass with no gradient updates making it suitable for real-time, privacy-preserving deployment in transferable 3D recognition pipelines. Project page is available at https://visdomlab.github.io/3D_BBDA
comment: Accepted at WACV 2027
★ TAMPAgent: Agentic Belief-Space Planning for Mobile Manipulation
Open-world robotic mobile manipulation requires acting in physical environments with only partial information. While recent Large Vision-Language Model (LVLM) agents show strong open-world reasoning capabilities, existing architectures that rely on direct tool calling struggle with geometric uncertainty and thus are limited to fully-observable tabletop environments. In contrast, Task and Motion Planning (TAMP) systems can solve long-horizon tasks under partial observability but incur high computational costs and require closed worlds. We propose \textbf{TAMPAgent}, an agentic planning framework that unifies the reasoning of LVLMs with the structured constraint satisfaction of TAMP systems. We endow an LVLM agent with a belief-space planning model operating over an extensible API of perception, planning, and control tools. The agent formulates a partial task plan composed of parameterized observation and execution actions, which is subsequently completed using classical search algorithms. After extracting the plan's geometric constraints, the agent leverages its tool API to bind valid parameter values, satisfying feasibility requirements and producing an executable motion plan. Upon execution, the agent continuously updates its belief history and repeats this closed-loop process by interleaving observation and manipulation actions. Evaluated across complex household mobile manipulation tasks on Rainbow Y1 and Unitree G1 platforms, TAMPAgent significantly outperforms direct tool-calling agentic baselines, demonstrating that integrating TAMP constraint satisfaction into agentic planning loops is essential for unstructured, partially observable environments.
comment: Website link: https://umishra.me/TAMPAgent/
★ Seeing Through the Illusion: Test-Time Scaling for World Action Models
World action models (WAMs) jointly generate actions and imagined futures, but a plausible prediction need not be realized by its co-generated action. We introduce WISE (\emph{World Action Inference Scaling through Evaluation}), a pre-execution selector that scores each sampled rollout on action validity and temporal regularity, inverse-dynamics consistency between the proposed action and the imagined motion, and instruction-conditioned task progress, and executes the highest-scoring candidate while keeping the WAM frozen. With four candidates per decision, WISE raises closed-loop success from 6.0\% to 24.0\% on RoboLab with Cosmos~3 Edge and from 75.1\% to 78.3\% mean success on LIBERO with DreamZero, and exceeds the best existing selector in aggregate by 4.0 and 0.9 points. Component ablations and agreement analyses show that the three scores rank candidates differently and that task progress carries the largest weight in the fused decision.
comment: Accepted at the CoRL 2026 Workshop on From Generative Models to Robot Actions (G2A)
★ Mediating Human-Swarm Interactions Through a Robot Helper: A Shepherding-Inspired Approach IROS 2026
This work investigates human-swarm interaction (HSI) for steering large collectives through an intermediary robot helper. While prior robot-swarm control and HSI techniques commonly assume swarm members are addressable, these assumptions often fail for ``ungovernable'' swarms such as wildlife flocks and schools of fish. Motivated by pastoral shepherding, we compare low-level teleoperation to a higher-autonomy motion-primitives interface in a virtual reality (VR) shepherding task under varied swarm cohesion dynamics. Results indicate that motion primitives that operate on collective shape (e.g., boundary-based control) improve task performance and reduce perceived workload.
comment: Accepted at IROS 2026
★ PanoTac: A Dense Three-Axis Tactile Sensor for Robust Touch-Centric Manipulation
Contact-rich manipulation benefits from tactile feedback that captures both detailed contact geometry and interaction forces. When scooping grains, contact patterns help infer spoon orientation, while resistance transmitted through the grasp guides motion. To provide these complementary sensing capabilities, we present PanoTac, a low-cost, compliant tactile fingertip designed for dense three-axis tactile sensing. Each fingertip provides a $12 \times 32 \times 3$ tactile representation, with one normal ($z$) and two shear ($x$, $y$) response components at each taxel. The sensor operates at 60 Hz and costs less than $50 per fingertip. Sensor characterization quantifies response repeatability and consistency across sensor instances and contact locations. We integrate PanoTac into a robotic system with a visuo-tactile pipeline. Across test-tube preparation, grain scooping, and salt pouring, the resulting policies outperform vision-only baselines by an average of 46.7 percentage points. Ablations show that removing either spatial contact information or shear feedback reduces overall performance across the task suite. Our project page is available at https://panotac.github.io.
comment: Project page: https://panotac.github.io
★ When Robots Outweigh Objects: How Inertia Mismatch Shapes Robot Policy Learning
Conventional robot arms can be extremely heavy compared with the everyday objects they manipulate, creating a significant inertia mismatch at contact. Even a small accidental bump from a heavy arm can knock a light object out of place. Modern learning methods have nevertheless demonstrated a wide range of successful manipulation behaviors on these platforms, suggesting that even extreme inertia mismatch can be overcome with sufficient data. However, it remains unclear whether high inertia imposes a hidden cost on learning, potentially worsening scaling with data or lowering the performance ceiling of modern robot learning methods. We investigate this question through controlled simulation experiments that vary robot and object masses. We find that reinforcement learning succeeds across a broad range of mass combinations, while behavior cloning learns more reliable policies when a lighter robot manipulates a heavier object, and more data narrows the resulting gap without closing it. Behavior cloning likewise degrades on a simulated robot arm as its motors reflect more inertia through their gearing, even though its weight stays the same. Finally, because new hardware is rarely an option, we explore ways to lower contact inertia on the arm one already has, either by choosing a lower-inertia workspace before collecting data or by steering the demonstrator toward lower-inertia solutions of the task. In both simulated cases, the lower-inertia demonstrations are imitated more reliably. Project website: https://younghyopark.me/inertia-mismatch/
comment: 9 pages, 17 figures. Project website: https://younghyopark.me/inertia-mismatch/
★ SNAP: Semantic Navigation Among Pedestrians
Navigating crowded urban environments requires robots to avoid pedestrians while adhering to social norms such as remaining on sidewalks. Existing approaches typically address either crowd avoidance or semantic path planning in isolation. We propose a reinforcement learning (RL) navigation policy that jointly reasons about semantic terrain and dynamic obstacles from a single semantically annotated bird's-eye-view (BEV) map, without relying on explicit human-state estimation. Our policy outperforms established crowd-navigation and semantic-aware baselines and transfers zero-shot to a legged robot, demonstrating socially compliant navigation in highly dynamic real-world scenarios.
★ Delay-Certified Readiness Barriers for Overactuated Multirotor Allocation Without Lipschitz Constants
Overactuated multirotors can produce the same wrench with many rotor-speed combinations, some of which leave the vehicle unable to change its wrench quickly, e.g., to reject a gust. Readiness barriers keep this ability, the readiness, above a floor, but only in continuous time; with sampled commands and varying speed-controller delays, the available robustness bound covers less than one millisecond. We remove this limit by exploiting the structure of the problem: each rotor responds monotonically and in closed form to its own torque, so the speeds reachable under uncertain switching times lie in a cheaply computed box, and the lowest readiness over that box sits at a corner known in advance. This yields a Lipschitz-free test that a command keeps the readiness above its floor until the next sample, plus a fallback torque that keeps the vehicle on a certified plan. In hexarotor simulations with 2--40~ms delays and $\pm2$~ms jitter, the filter never violated the floor. At $10\pm2$~ms, it matched a delay-compensating predictor within the design envelope and cost 2--30\% more tracking error beyond it, where the predictor alone violated the floor in 2 to 16 of 16 missions.
comment: This work has been submitted to the IEEE for possible publication
★ Exploring Proximity and Tactile Sensing for Robotic Manipulation
Pre-touch sensing, or sensing information about an object before contact, is an attractive modality for robot hands as it can allow a robot to anticipate and respond to a contact before it happens. In this work, we introduce ToFTouch, a multimodal tactile sensor combining three multizone (4$\times$4 resolution) time-of-flight sensors with an optics-based pressure sensing array. We mount this sensor on a parallel jaw gripper and demonstrate its utility on three distinct tasks: (i) pre-grasp adjustment, (ii) item extraction from clutter, and (iii) item insertion into tight spaces. For pre-grasp adjustment, ToF feedback reduced cycle time by 7.4$\%$ by enabling anticipatory gripper closure during approach. For extraction, ToF sensing enabled successful retrieval across all tested depth offsets ($\pm$2 cm) deviating from the nominal pick point, including conditions where baseline approaches failed, with standard deviations below 0.4 cm. For insertion into dense clutter, we explore the proximity and tactile information provided from ToFTouch during an insertion trajectory. The results demonstrate the utility of multizone ToF sensing in a manipulation environment.
★ Clinician-AI Collaboration in Endovascular Neurointerventions: In-Vitro User Study of Copilot Assistance Modes for Robotic Navigation Across the Autonomy Spectrum
Mechanical thrombectomy (MT) is an effective treatment for acute ischemic stroke, yet access remains limited by the availability of neurointerventional expertise. Telerobotic and AI-assisted systems may help address this challenge, but it remains unclear how clinicians interact with different levels of autonomy and how autonomous navigation systems perform in realistic environments. This work introduces Copilot Assistance Modes (CAMs), a framework for clinician-AI collaboration in robotic endovascular navigation spanning the autonomy spectrum from manual operation to full autonomy. In the first in-vitro user study comparing clinician-AI collaboration strategies, 11 experts and two students evaluated six navigation modes in a patient-specific vascular phantom. Objective performance metrics were combined with subjective assessments, including workload and usability. Autonomous performance was additionally evaluated both in silico and in vitro to investigate sim-to-real transfer. Hands-on manual navigation achieved the strongest overall performance (100% success). While fully autonomous navigation achieved 81% success in simulation, performance dropped to 23% in vitro. In contrast, several clinician-AI shared-autonomy modes achieved success rates of 92-97%, approximately four times higher than full autonomy and comparable to direct robotic control (95%), demonstrating mitigation of sim-to-real domain shift through clinician-AI collaboration. Among the evaluated AI-driven approaches, user-directed task delegation and rotational shared control demonstrated the most favorable balance between performance and clinician acceptance. These findings suggest that effective clinician-AI collaboration and allocation of control authority may currently be more important for reliable robotic neurointerventional navigation than increasing autonomy alone.
comment: 13 pages, 10 figures, 4 tables
★ Evaluating Depth-Augmented Visual Features for Cross-Dataset Lunar Traversability Estimation
Autonomous mobility is essential for sustained lunar surface operations, as communication delays and line-of-sight occlusions limit continuous teleoperation. Although recent traversability estimation methods increasingly rely on deep learning, the domain gap between terrestrial analogue environments and real lunar terrain can limit their generalisation, and real lunar traversability labels are scarce, making fully supervised training difficult. To address these challenges, we build a weakly supervised framework trained on sparse pairwise annotations and study whether augmenting visual features with depth-derived geometric information: depth, surface normals, and combined geometric descriptors. We evaluate the models in both in-domain and cross-domain settings using two terrestrial analogue datasets and the real lunar Chang'E PCAM dataset. Contrary to expectations, we find that naive geometric augmentation yields no consistent improvement over a strong visual-only baseline in either setting, with effects small relative to seed-to-seed variance. Instead, the choice of training source emerges as the dominant factor in transfer quality: models trained on domain-relevant sources transfer substantially better to real lunar imagery than the geometric fusion strategy improves any single model. Our evaluation runs onboard a resource-constrained edge device and provides a controlled baseline for designing more expressive multimodal fusion architectures in future lunar perception systems.
comment: Accepted for presentation at iSpaRo 2026
★ Toward Uncertainty-Aware, Physics-Grounded Human World Models
A robot that acts around people needs a world model of the human: a belief over the person's body state and a dynamics model that predicts where that state goes next. This abstract argues that two published components form exactly these two halves, and specifies how to couple them. KNOWN [1] reconstructs the 3D human body while estimating both aleatoric and epistemic uncertainty in real time (11.6 ms per frame); its aleatoric term identifies unreliable observations, its epistemic term identifies inputs from low-density regions of the training data, and refinement driven by the epistemic term reduces the error on such inputs from 81.3 to 70.3 mm. PhysMoP [2] predicts future body configuration by solving the Euler-Lagrange equations of the body with learned dynamics terms, reducing the 80 ms prediction error from 9.6 to 2.1 mm on Human3.6M and from 12.1 to 0.7 mm on 3DPW. We define the interface between the two, propose three mechanisms by which KNOWN's uncertainty enters PhysMoP, and specify an evaluation protocol on in-the-wild video. No new experimental results are reported here.
comment: CoRL2026 UncertainWM Workshop
★ Myoelectric Control for Upper-Limb Exoskeletons: A Decoder-Shaper-Controller Framework, Design Guidelines, and Future Directions
Myoelectric Control Systems (MCSs) enable voluntary control of wearable robots through muscle activity. However, intention decoding, assistance shaping, and low-level actuator control are often not explicitly distinguished, making their contributions difficult to evaluate and compare. This review introduces a functional decoder-shaper-controller framework for continuous myoelectric control of upper-limb exoskeletons. The decoder estimates an intention-related variable from surface electromyography (sEMG), the shaper maps it to a control reference according to an assistance policy, and the controller realizes that reference through actuator control. This decomposition separates intention estimation from assistance-policy design and enables comparison of heterogeneous architectures. Representative approaches are critically examined in terms of module design, calibration, and evaluation. The analysis highlights that the shaper, despite determining the magnitude and shape of assistance, lacks established design and tuning methodologies and is often embedded within the decoder or controller. Key open issues include inconsistent reporting of assistance levels, limited use of interaction-aware stability and passivity tools, and insufficient validation across degrees of freedom, users, and tasks. Building on these findings, this review provides design guidelines for systematic module-wise tuning and identifies future research priorities to bridge the gap between accurate intention decoding and safe, effective, and generalizable assistance under real-world operating conditions.
comment: 19 pages, 3 figures. Review paper
★ OpenZilo: An Open Smart-Ring Platform for Human-Centric Sensing and Interaction with Robots
Wearable robot interfaces must connect user input to timely physical action. We present the proposed platform, an open smart-ring platform for studying this connection across task stages. The ring integrates a microphone, an inertial measurement unit, photoplethysmography, and skin-temperature sensing. Its software architecture separates acquisition, event interpretation, interaction policy, and robot execution. We characterize seven operating modes and evaluate speech and gesture control with 10 participants. Across five commands and 250 trials per modality, speech and gestures achieved task success rates of 90.4% and 88.8%, respectively. Recognition accuracy exceeded task success for both modalities. Gaussian hidden Markov models achieved higher accuracy than a support vector machine across three vocabulary sizes, with longer scoring times. Following a shared spoken task start, gesture-based adjustments had shorter median response times than speech. Whole-task success was 70.0% with gestures and 64.0% with speech. We further examine physiological-event-triggered robot interaction using simulated event notifications. Mean notification-receipt-to-arrival times were 11.71 s for the robot and 9.20 s for an on-site person. These descriptive findings support evaluating wearable robot interfaces through both recognition and physical outcomes, with timing measures appropriate to each task stage.
comment: 17 pages, 5 figures, 8 tables
★ Safe-Ballbot: Tilt-Consistent MPC with Control Barrier Functions for Ballbot Navigation
Ball-balancing robots (ballbots) must lean to accelerate, and the lean is limited by the wheel torques and by safety. Planners that use the full robot model are expensive, while planners that ignore the balance dynamics produce references that cannot be followed. This paper presents a two-layer navigation scheme for a three-omni-wheel ballbot. The upper layer is a model predictive controller on a \emph{jerk model} of the ball position, with discrete-time control barrier functions for static, moving, rotating and concave obstacles. The lower layer is a hierarchical sliding mode controller with an extended state observer and a robust exponential control barrier function on the tilt (HSMC-ESO-ECBF). The layers are linked by an acceleration bound derived in closed form from the largest steady tilt that the barrier function admits, so that the plan never asks for a lean the tracker refuses. For obstacles whose motion is only predicted, the obstacles are inflated by conformal prediction radii that bound the prediction error with a prescribed probability. In MuJoCo, with omni-wheel roller contacts and torque saturation, the scheme keeps at least $10$\,cm of clearance, tracks the plan within $2$\,cm RMS, keeps the tilt below the limit and plans in $9$--$12$\,ms, $10$--$15$ times faster than a full-model nonlinear MPC that enters the inflated obstacles; with predicted obstacles, the conformal inflation removes the collisions of a constant-velocity predictor in $30$ random crossing scenarios and reduces them in an endless run on a closed path.The source code and video illustrations are available at https://safe-ballbot.github.io
comment: Under-review
★ ForceLDM: A Force-aware Latent Dynamics Model for Contact-Rich Manipulation NeurIPS 2026
Contact-rich manipulation requires fine-grained control and complex physical interactions with the environment, posing significant challenges for robot policy learning. Vision-only policies struggle with such tasks, as contact states and interaction forces are difficult to infer from visual observations alone. Existing methods mitigate this issue by incorporating force or tactile sensing. However, they often treat these signals as auxiliary observations without fully exploiting their role in modeling future interaction dynamics. To address this, we propose ForceLDM, a force-aware latent dynamics model that learns future contact and motion dynamics to guide action generation. Rather than directly predicting future observations, which can entangle task-relevant dynamics with irrelevant sensory details, ForceLDM learns a compact representation of future dynamics through knowledge distillation. Specifically, a teacher network with privileged future force/torque and optical flow signals extracts future dynamic features, which are then distilled into a student network trained only on current observations. This enables the student model to reason about upcoming contacts and physical dynamics without relying on future sensory inputs at deployment. To improve robustness and generalization, we further introduce a curriculum-based progressive noise injection strategy to mitigate over-reliance on future features. Experiments on five real-world contact-rich manipulation tasks demonstrate that ForceLDM outperforms state-of-the-art baselines, generalizes effectively to novel objects, and remains robust to environmental perturbations. Project page and code are available at https://tpy001.github.io/ForceLDM/.
comment: NeurIPS 2026
★ VerifLM: Physically Verified Image-Edit-Guided Goal Pose Generation for Robotic Manipulation ICRA 2027
Many manipulation goals are easier to visualize than to specify numerically through explicit geometric constraints. We present VerifLM, a framework that converts image-edited goal configurations into physically verified 6-DoF object poses for planning. Using an edited image, VerifLM recovers an initial relative pose between an active object and a passive object, then prompts a vision-language model to generate axis-wise sampling constraints and a task-specific validation program. Candidate poses are evaluated in a physics simulator, and the VLM selects the most task-consistent retained candidate. To operate without predefined object meshes or physical parameters, VerifLM reconstructs metric object geometry from RGB-D observations and constructs multiple hypothetical simulations with varying physics and geometry. Parallel-world validation evaluates candidates across geometry--physics variations and ranks them by the number of worlds in which they satisfy the validation program. We evaluate VerifLM on nine manipulation tasks. VerifLM achieves the highest task success on eight of nine tasks, with the largest performance gain on goals requiring precise orientation and contact relationships.
comment: Submitted to IEEE ICRA 2027
★ TRAILS: Trajectory Generation for Automated Indoor Locomotion Studies
Virtual reality experiments involving navigation and simulated robot motion often require large, realistic environments and numerous distinct trajectories to avoid repetitive navigation while providing sufficient variation for controlled experiments. Designing such scenarios manually is labor-intensive, particularly when trajectories must satisfy predefined geometric and kinematic constraints while remaining diverse and comparable across experimental conditions. This paper presents TRAILS, an automated framework for generating complete virtual navigation scenarios consisting of an environment layout, a path, and a time-parameterized motion profile. The framework combines a constraint-based procedural environment generator inspired by Wave Function Collapse, trajectory generation using optimized cubic Bézier curves, and speed-profile generation subject to kinematic constraints. The generated scenarios provide diverse indoor environments with corridors, open spaces, and obstacles, together with trajectories of identical duration that vary in path geometry, wall proximity, and velocity profiles while minimizing systematic dependencies between these characteristics. The framework is designed as an easy-to-use tool requiring only a small number of intuitive user-defined parameters to automatically generate large sets of distinct navigation scenarios satisfying prescribed geometric and kinematic constraints. By relying on explicit, seed-controlled algorithmic procedures rather than data-driven models, the generated scenarios remain transparent, reproducible, and configurable, making the framework well suited for telepresence and human-subject experiments. The code is available at https://github.com/JucciK/Trails .
comment: 8 pages, 4 figures, IEEE Telepresence 2026
★ Towards end-to-end co-design for one-shot printable task-specific soft robots
Soft robot co-design methods can jointly optimise morphology and control, but the resulting designs are often difficult to manufacture, while manufacturable approaches typically restrict actuator placement or require manual assembly. We present a simulation-based co-design framework that jointly optimises material distribution, pneumatic actuator placement and orientation, and control, while producing designs that can be fabricated monolithically using fused deposition modelling. The framework uses a differentiable 2D plane-stress Material Point Method simulator, incorporating an experimentally validated material model and an empirically derived surrogate model of a 3D-printed inflatable actuator. We demonstrate gradient-based optimisation soft robotic arms for prescribed tip trajectories. Optimised designs are transferred directly from simulation to one-shot monolithic FDM fabrication, with integrated actuators and no post-assembly. Physical experiments demonstrate that the resulting arms reproduce the behaviour predicted in simulation. This framework provides a path from task-level objectives to physically realisable soft robots while retaining substantial freedom over both morphology and actuation.
★ MoSoLoco: Monocular 3D Human Localization through Visual Latent Features and Social Context
Estimating the metric 3D position of people from a monocular camera is important for robot navigation and motion prediction. Lightweight pose-based localizers are attractive for robotic hardware with limited computational resources because they operate on compact 2D keypoints. However, they typically discard the richer visual representations computed by the pose estimator. We introduce MoSoLoco, a per-frame monocular 3D human localizer that combines the pose detector's internal visual features with the detected 2D skeleton, without requiring an additional RGB or depth encoder. MoSoLoco can further incorporate sparse same-frame multi-person context to refine individual position estimates. Experiments on JRDB and NuScenes show that reusing pose-detector representations improves localization, reducing the best JRDB error by approximately 23 percent relative to the strongest evaluated keypoint-based baseline. A sparse social-interaction module incorporates nearby detections from the same image for further refinement. Controlled trajectory-forecasting experiments show downstream benefits, while the complete pipeline remains suitable for efficient Jetson AGX Xavier inference.
★ Rollout then Recycle: 3,718 Labelled Real-Robot Episodes from Benchmarking 7 Policies on 12 Tasks
We release all 3,718 episodes from the ArmnetBench v0.1 evaluation (arXiv:2607.24481), which contains 600 teleoperated demonstrations and 3,118 rollouts of seven imitation-learning and vision-language-action policies on 12 tabletop tasks, recorded on low-cost SO-101 arms; 73% of the rollouts are failures. Every policy for a task was trained or fine-tuned on the same 50 demonstrations and rolled out on the same cell, so the corpus contains many policies' failures on shared tasks. An operator labelled every rollout successful, suboptimal or failure. Each episode has three camera views, joint states and actions and is available on the Hugging Face Hub in LeRobot v3.0 and a reward-model format. This note documents how the episodes were collected and the limitations that could affect their use.
comment: 4 pages, 1 figure, 2 tables. Accepted to the "Oops, I Erred" Workshop at CoRL 2026
★ FineCraft: Enhancing VLA Models for Fine-Grained Manipulation
Fine-grained manipulation couples instruction grounding, local visual precision, and sustained execution. We present FineCraft, a framework that adapts pretrained models for fine-grained manipulation in the MetaFine Challenge. A task-routed mixture of policies selects independently adapted DINOv2-DiT or pretrained $π_{0.5}$ policies through a common observation-action interface. Local visual features help identify the instructed object parts, demonstrations and expert continuations teach the required actions, and new observations guide replanning during execution. FineCraft ranks fourth with an overall score of $0.878$, exceeding the published reference on all five tasks. Official results and local diagnostics characterize performance across visual perturbations, instruction changes, and multi-stage manipulation. Code is available at https://github.com/eqcy/FineCraft
comment: 6 pages, 2 figures
★ KlaskTron: An Open-Source Platform for Physical Adversarial Multi-Agent RL
Progress in robot learning, particularly physical multi-agent reinforcement learning (MARL), is currently challenged by a limited availability of accessible, standardized benchmarks. While simulation-based MARL has produced remarkable emergent behaviors, from coordinated team play to complex tool use, translating these advances to physical systems remains difficult. A key barrier is infrastructural: physical platforms are often prohibitively expensive, require specialized facilities, or lack open mechanisms and designs for reproducibility. We introduce KlaskTron, an open-source, low-cost (<\$2500 USD), desktop-scale robotic testbed for adversarial MARL based on the dynamic tabletop game KLASK. The platform features dual CoreXY gantries with high-torque brushless motors, enabling the high-speed, precise manipulation required for competitive play. Crucially, we provide a complete ecosystem: hardware designs (CAD, BOM), a GPU-native digital twin in NVIDIA Isaac Lab for high-fidelity modeling and simulation, and a validated sim-to-real baseline using a learned neural actuator model. We demonstrate that this baseline enables zero-shot policy transfer, with trained agents exhibiting emergent adversarial behaviors. To ensure full reproducibility of our results, all project assets are released publicly: hardware CAD/BOM and assembly documentation at [https://idscethzurich.github.io/klask_hardware], inference software stack at [https://github.com/IDSCETHZurich/klask_software] and the simulation and MARL setup at [https://github.com/IDSCETHZurich/klask_rl].
★ A Tightly Coupled Magnetic-Inertial SLAM System for Field Robots
Field robots operating in mines, tunnels, and disaster areas require reliable localization despite GNSS denial and degraded camera and LiDAR measurements caused by dust, smoke, or poor illumination. Missions involving both ground and aerial robots further motivate complementary sensing modalities that do not depend on visibility, preinstalled infrastructure, or wheel contact. This letter presents MISLAM, a tightly coupled magnetic-inertial simultaneous localization and mapping system using an IMU and a nine-magnetometer array, without relying on external odometry or platform-specific motion assumptions. The system combines the inertial navigation model with local polynomial magnetic-field models for short-term motion estimation and a global Gaussian process model for drift correction using an error-state Kalman filter. Offline evaluation on three indoor datasets collected with a handheld rig shows reductions in total endpoint closure error of 45.7% - 78.0% relative to magnetic-inertial odometry. Under poor illumination, MISLAM maintains tracking after visual-inertial odometry fails, achieving a horizontal closure error of 0.332 m, although vertical drift remains a limitation. These results support magnetic-inertial SLAM as a complementary localization modality for robotic sensor suites operating under degraded visual conditions.
★ VLALagBench: Understanding When and Why VLA Inference Latency Matters
Vision-language-action (VLA) models offer a promising path toward general-purpose robot manipulation. While inference latency can affect task success in a constantly evolving physical world, the relationship between latency and task success rate remains poorly understood. In this paper, we introduce VLALagBench, a comprehensive benchmark for studying VLA inference latency sensitivity across task types, object speeds, execution modes (synchronous and asynchronous), and VLA policies. Using VLALagBench, we find that even a modest inference latency of 100 ms can nearly halve task success on moving targets, with sensitivity varying substantially across tasks and policies. These differences reflect task-specific requirements for precision and continuous motion. By overlapping inference with action execution, asynchronous execution can recover some of the lost success at the cost of more frequent inference requests. Building on these findings, we develop an empirical model that predicts task success from limited calibration data, achieving root-mean-square prediction errors of only 5.7-7.8% across various VLA policies. Given an inference system's latency and per-robot throughput, these predictions guide the selection of system-specific VLA algorithm configurations to maximize task success. We believe VLALagBench and these findings provide a solid foundation for future robotics algorithm-system co-design.
comment: Code will be released at https://github.com/codestrongestx/VLALagBench
★ KlaskTron-IL: A Contact-Rich Adversarial Imitation Learning Benchmark with a Validated Sim-to-Real Bridge
KlaskTron is an open-hardware physical platform for two-player adversarial reinforcement learning, with a learned actuator model that supports zero-shot sim-to-real transfer of trained policies. Building on this platform, we contribute KlaskTron-IL: a multi-tier imitation learning dataset (80k human demonstration timesteps collected directly on the physical robot, 320k symmetry-augmented timesteps, and synthetic expert trajectories up to 5M steps generated by the zero-shot-validated PPO policy), and diagnostic baselines that expose three characteristic failure modes of standard IL methods under contact-rich adversarial dynamics: BC's data scaling plateau, GAIL's interpretable but weak-gradient discriminator (Spearman $ρ= -1.0$ peg-ball proximity recovery, but a logit span under 0.1), and an asymmetric BC-prior x reward-density interaction. Because the policy interface used throughout this paper is the same one the actuator model bridges to hardware, a policy trained on this benchmark can run unchanged on the physical robot, as we demonstrate zero-shot for the strongest BC baseline, so long as its commands respect the actuator model's $\pm 0.2$ m/s identification range.
★ PlanContext: Benchmarking In-Context Driving Decision-Making
Real-world driving requires context-aware decision-making. Existing end-to-end driving benchmarks primarily assess trajectory quality through safety, compliance, comfort, and driving progress, leaving context-dependent decision-making insufficiently characterized. To fill this gap, we present PlanContext, a benchmark for evaluating in-context driving decision-making. PlanContext constructs paired evaluations by varying high-level driving intentions while keeping the observed scene fixed, enabling a natural and controllable assessment of context-dependent decision-making. We further define six representative driving contexts and jointly evaluate contextual decision correctness, road compliance, and dynamic safety with class-balanced metrics. Experiments on 9,561 scene-condition instances from NAVSIM and 23 public end-to-end driving models show that high conventional driving-quality scores do not consistently correspond to strong route-level in-context driving decision-making. Further analysis shows that model behavior varies with decision timing, road structure, and traffic interaction depending on the decision required by the current context. Our results highlight in-context driving decision-making as an important complementary dimension for evaluating end-to-end driving systems.
★ BeyondContact: A Reinforcement Learning Framework for Airflow-Based Non-Contact Manipulation
Contact-based robotic manipulation is constrained by end-effector reachability and the mechanical properties of objects. Airflow can exert forces without direct robot-object contact, enabling manipulation of objects beyond the reach or when direct contact is undesirable. However, the lack of suitable simulation has hindered the application of reinforcement learning to airflow-based manipulation, limiting the broader adoption of this mechanism compared with contact-based methods. To address this gap, we developed a robot- and task-agnostic airflow extension for Isaac Lab that supports massively parallel training. The extension uses experimentally identified airflow models to compute the forces and torques exerted on objects. Based on this extension, we train a hybrid push-blow (HPB) policy that combines contact-based pushing with airflow manipulation and a rectangular reorientation (RR) policy that controls a cuboid's heading using airflow alone. To enable physical execution, we design a lightweight, high-speed blower that coexists with a gripper. In zero-shot real-world deployment, HPB achieves an 80% success rate over 20 trials, while RR achieves a mean absolute heading error of 2.6 degrees across four target headings. The project materials are available at https://github.com/iit-DLSLab/BeyondContact.
comment: Peiyu Yang and Mohamad S. Moudallal contributed equally. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible
★ Swarm Choreography Made Simple: Superposed Guiding Vector Fields for Rigid Formation Path Following
This paper presents a decentralized guiding vector field framework for rigid formation path following in multi- UAV systems based on the superposition of simple rule-driven vector fields. Each control objective is encoded as a dedicated vector-field component: a normalized tangential term to ensure progression along a prescribed geometric path at a desired speed, a convergence term driving the formation toward the path, a local coordination term that preserves the desired formation shape through a pseudo-path parameter, and a local separation term for inter-agent collision avoidance. The coordination mechanism requires the exchange of only a single scalar per agent, resulting in low communication bandwidth. Partial theoretical results establish convergence of the swarm to the desired path and formation in the absence of the sepa- ration term, while collision avoidance is formally guaranteed. The approach is experimentally validated using a swarm of five quadrotors executing two 3D choreographies. The first experiment involves tracking a helical path while maintaining a rigid V-shaped formation. The second demonstrates a spiral- like motion obtained by a rotating circular formation tracking a vertical line. The experimental results support the effectiveness and practical applicability of the proposed framework.
★ SyncHeC: Markerless Spatio-Temporal Hand-Eye Calibration via Differentiable Rendering and Continuous-Time Joint Optimization
Markerless hand-eye calibration under continuous motion requires jointly resolving camera geometry and the time offset between images and joint states. We present SyncHeC (Synchronized Hand-Eye Calibration), a dense differentiable-rendering framework that estimates both from a single robot trajectory. Continuous-time forward kinematics makes latency differentiable, metric depth complements mask contours, and an offline planner combines spatial sensitivity with temporal excitation. Alternating updates support Eye-to-Hand, Eye-in-Hand, and multi-camera calibration. On a 7-DoF arm, one $6.5$\,s motion yields $0.21$\,cm translation error against a marker-board reference, $0.34$\,cm touch-test error, and $2.7$\,ms latency error relative to an $87$\,ms reference with $95\%$ uncertainty of $\pm1.5$\,ms. Against CM-Calib on matched recordings, translation and nominal latency errors improve by $3.7\times$ and $2.3\times$. Experiments span seven arms and four depth sensors. Compensating the recovered offset during policy training and deployment increases success by $8.1$ percentage points across six families and six tasks ($1{,}800$ matched pairs; paired $95\%$ confidence interval $[6.3,9.9]$ points).
★ MS-DOTT: A Multi-Modal Dataset for Off-Road Traversability Estimation
Autonomous robots are gaining traction in off-road environments, driven by their rising demand in agriculture, defense, and search-and-rescue operations. However, these environments are inherently unpredictable and navigation remains a formidable challenge given the complex environmental conditions including challenging terrain and varying illumination. Off-road autonomy requires robust perception across multiple spatial scales and imaging modalities, yet most sensing setups struggle to capture both far-field environmental context and near-field terrain detail simultaneously across various illumination conditions. To facilitate development of robust off-road perception and traversability estimation systems, we propose MS-DOTT (Multi-Sensor Dataset for Off-road Terrain Traversability) a comprehensive multi-modal off-road dataset with long-range 3D sensing and short-range, fine-grained perception to facilitate high-fidelity terrain understanding by combining proprioceptive signals for tracking vehicle dynamics with exteroceptive imaging and pointcloud data. The dataset includes trajectories recorded across nominal lighting, low-light, and no-light conditions with diverse 2D and 3D sensing for both long-range large-scale and short-range high-resolution sensing, as well as 4D RADAR measurements that provide velocity estimates for detected pointclouds. We demonstrate the utility of the dataset by adapting a SOTA multi-sensor fusion method for the purpose of traversability estimation. This dataset will aid the robotics community in advancing research and development of methods for traversability estimation, off-road perception and multi-sensor fusion in unstructured environments.
comment: Accepted to RAAI Conference 2026
★ ReContact: Reconstruct and Retarget Human Contacts from Video for Contact-Rich Humanoid Manipulation
Contact transitions allow humanoid robots to redistribute object support across the whole body, enabling manipulation beyond hand-only grasps. Human videos provide a scalable source of such transitions for robots to learn from. However, body occlusion disrupts object pose tracking and corrupts retargeted motion. Moreover, accurately reproducing the object trajectory does not ensure that the robot applies stabilizing forces at the intended contacts, risking slips or drops in later phases. To address these challenges, we propose ReContact, which reconstructs and retargets human-object interactions through an object-centric contact plan. The plan specifies contact phases, contact pairs, anchors, pressing directions, and sticking or sliding modes. Contact priors stabilize object pose reconstruction through occlusion. A residual policy over a frozen whole-body tracker is rewarded for making contact at planned anchors with forces aligned to the specified directions. In simulation, on a thigh-supported regrasp task, ReContact achieves the planned two-palm bottom hold in 83.5 of episodes over three training seeds, versus 0.16% for the best adapted baseline. One policy trained on 247 references re-solved for new placements raises in-workspace tube-transfer success from 20.4% (source reference only) to 97.8%. On a real Unitree G1, policies execute seven contact-rich manipulation and two loco-manipulation tasks.
★ Positive Safety Margins from Artificial Potential Field Reward Augmentation for Size-Robust Obstacle Avoidance under Partial Observability
Collision-free reaching is central to manipulation, yet real cells often leave the size and exact position of an obstacle unobserved, which excludes reinforcement learning controllers that take obstacle geometry as input. A reward limited to a collision penalty does not depend on the clearance outside the obstacle, so under a step cost its learned policy skims the obstacle surface and fails once the obstacle radius exceeds its training value. We investigate whether a single analytic reward term can induce a safety margin without exposing the obstacle to the policy. We augment the reward of a simulated Franka Emika Panda reaching task with a horizontal artificial potential field that penalizes proximity within a safety margin around each of two pillars (fixed nominal layout, random drift). Under hidden observation, policies trained with this term maintain a safe success rate above 88% across a radius sweep from 0.03 m to 0.055 m (30 seeds), while policies without it fall below 10% at the maximum tested radius; the term induces a minimum clearance of 47.9 mm against 9.6 mm without it and eliminates training collisions at least eight times earlier (median steps). The advantage is statistically significant at all out-of-distribution radii and holds across three reinforcement learning solvers and under full observation. Its safe success rate is comparable to that of a classical potential-field planner given the pillar positions. The margin is a fixed buffer rather than an adaptation to the radius, and it lengthens the path by about 19%.
comment: 44 pages, 15 figures. Code and data: https://github.com/Hao-Ziyi/apf-reward-margin
★ RESONATE: a Real Social Navigation Testbed for Autonomous Service Robots
Social navigation is essential for autonomous service robots operating in human environments, but progress remains constrained by the lack of extensive real-world benchmarks. This paper presents the Real Social Navigation Testbed (RESONATE), a reproducible benchmark for human-aware navigation based on a unified robotic platform, a shared ROS 2 software stack, precise trajectory ground truth, and structured social interaction scenarios. We evaluate eight controller configurations, combining classical and predictive local planners with different social modeling strategies, over eight representative scenarios with five repetitions each, yielding 320 physical experiments. The benchmark combines quantitative navigation and social metrics with participant-based qualitative assessments to jointly measure efficiency, safety, comfort, and perceived social behavior. Results reveal scenario-dependent trade-offs among the tested methods. The algorithms that stands out for their performance are the DWA with Gaussian social costmaps for its reliability, and the SFM-MPPI to provide the best balance between robustness, navigation efficiency, and social acceptance. Overall, RESONATE supports the robotics community with a real-world benchmark, reusable baselines, data, and testbed, and a consistent evaluation protocol for advancing socially aware navigation research.
comment: 24 pages, 8 figures
★ PointPilot: Unified Robotic Manipulation via Object-Centric 3D Point Flow Prediction
Generalization remains a central challenge in robotics, calling for policies that address domain variations from multiple dimensions such as diverse embodiments, camera poses, and/or scene layouts. In order to approach this problem, we introduce PointPilot, a task-focused and 3D aligned manipulation framework built on a 3D space point-flow prediction. Rather than modeling the global scene, PointPilot only represents task-focused objects (robot grippers treated as objects), avoiding the prediction of potentially redundant scene parts unrelated to the manipulation tasks. Moreover, PointPilot aligns 3D spatial representation in one unified coordinate and supports closed-loop and fully automatic action execution across different manipulation modes from object point flow without a separately trained action expert, while existing works typically need an extra action head or are (semi-)automatic and single-arm mode specific. Given sensor observations and language instructions, our model jointly learns subtask planning, task-relevant object grounding, and 3D point flow prediction in an end-to-end way. By processing heterogeneous data in a shared 3D point flow space, PointPilot is able to absorb demonstrations from different embodiments. Extensive simulative and real world experiments validate our method. Notably, we demonstrate zero-shot embodiment generalization with PointPilot. A single trained policy can be deployed on previously unseen embodiments and successfully complete manipulation tasks without any additional fine-tuning.
★ An intelligent satellite must do nothing well
Autonomous inspection of a resident space object is usually posed as a contest between classical next-best-view planning and reinforcement learning. We argue that the choice of algorithm is the least informative of the design axes. An inspector spacecraft has no rest state: when it stops thrusting, orbital dynamics carry it along a natural trajectory that can be used to observe the target at no fuel cost. On a controlled ladder in which one design axis is varied at a time, fuel efficiency turns out to be set by three things: whether the agent is allowed to drift, whether its state records where drifting has taken it, and whether its planning horizon is long enough for the benefit to appear. Removing any one of them doubles the fuel bill or more. The algorithm still matters, but through what it can represent: a model-based agent whose learned world model ignores fuel spends several times more of it. Finally, no learned policy is safe by construction, and a control barrier function filter beneath the decision layer removed every keep-out-zone violation in two thousand evaluation episodes.
comment: Presented at the I-RIM 3D 2026 - Workshop n.30 How would you like intelligence in your robots? Interdisciplinary perspectives on intelligent machines
★ Defining Autonomous Weapon Systems: A schematic framework for refined purpose-driven definitions
The literature on autonomous weapon systems (AWS) includes many proposed definitions for these (types of) systems, with core disagreements between scholars, governments, and international regulatory bodies. In this article, we present and critique prevailing (classes of) definitions, arguing that what is needed is not a singular definition, but rather a definitional framework which allows one to classify different sorts of AWS within the same conceptual family. Recognizing we cannot foresee all future AWS, we show how variations on a broadly accepted definition can be used to produce a definitional schema which can be easily modified in order to classify various concepts belonging to the class of AWS.
★ Gibbs Variational Last Layers for Long-Horizon Dynamics Learning NeurIPS 2026
Accurate long-horizon probabilistic predictions of dynamical systems are a prerequisite for model-based decision-making. Variational Bayesian last layers (VBLLs) are an attractive model class for this setting, offering tractable uncertainty at near-deterministic cost, yet they are trained exclusively with single-step likelihood objectives that may be insufficient for reliable multi-step rollouts. We revisit VBLLs through the lens of Gibbs variational inference, which decouples posterior inference from the choice of training loss. Building on this perspective, we introduce a family of multi-step training losses that progressively incorporate long-horizon structure, robustness via continuous ranked probability scores (CRPS), and training on full autoregressive rollouts. Our approach retains the simplicity of VBLLs while directly optimizing for multi-step predictive quality. Across illustrative synthetic environments, a diverse suite of chaotic dynamical systems, and real-world data, we show that multi-step, CRPS-based objectives substantially improve long-horizon accuracy and distributional fit, consistently outperforming single-step likelihood training.
comment: Accepted at NeurIPS 2026
★ AnchorHEC: Targetless Hand-Eye Calibration of Wrist and Fixed Cameras
Recent robot systems combine wrist and fixed cameras, requiring camera-to-robot transforms for both. Most hand-eye calibration methods rely on targets or robot-body observations. Scene-based methods using 3D reconstruction require neither but address only wrist cameras. We introduce AnchorHEC, a targetless hand-eye calibration framework for wrist and fixed cameras in a single procedure using RGB images, forward kinematics (FK), and known camera intrinsics. We initialize both camera types using FK and a shared 3D foundation model reconstruction. FK-constrained wrist-image bundle adjustment reduces the impact of reconstructed pose and scale errors. If this refinement fails or retains too few inliers, we relax the constraints on potentially inaccurate FK poses and lens parameters. We use the refined wrist transform and FK to triangulate metric points and estimate fixed-camera poses. Across three backbones, calibration errors in simulation decrease by 86.4% in translation and 71.6% in rotation, averaged over both camera types, relative to reconstruction-based baselines. We also validate AnchorHEC on a physical robot. Project page: https://hwanggb.github.io/AnchorHEC/
comment: 8 pages, 5 figures, 5 tables. Project page: https://hwanggb.github.io/AnchorHEC/
★ Design and Experimental Validation of the Separation and Deployment System for the CHERI Lunar Micro-Rovers
Reliable separation mechanisms are critical for planetary micro-rovers, which must withstand launch vibrations and then eject the vehicle clear of the lander. This paper presents the design, optimization, and testing of the separation system of the CHERI (Challenging Environment Exploration Rovers for Intelligence) lunar micro-rovers. Sized for a 5 kg vehicle, the assembly comprises an aluminum ground plate, a central non-explosive hold-down and release mechanism, and an asymmetric four-spring ejection array. Initial separation trials revealed a velocity deficit caused by energy absorption in the flexible wheel-legs and chassis during the ejection stroke. This loss was represented as equivalent chassis damping in a multi-body dynamic model, and the springs were re-optimized accordingly. Early hardware iterations failed vibration testing, first through spring jamming in the guide housings and then through a 34 Hz fundamental mode of the flat ground plate. A density-based topology optimization raised the fundamental frequency to 84 Hz, above the 70 Hz requirement of the lander, within a 1.6 kg mass budget. The final configuration passed acceptance-level sinusoidal vibration, random vibration, and shock testing without structural degradation. Separation tests with a weight-matched one-sixth mass model, tracked by optical motion capture, confirmed clearance of the lander and reproduced the simulated free flight to within the run-to-run variation of the measured ejection velocity. These tests further showed that landing outcome is dominated by touchdown and settling dynamics, indicating that the residual pitch rate is the main controllable design parameter and confirming self-righting recovery as an integral element of the deployment architecture.
comment: 21 pages, 15 figures, journal paper
★ ReGATE: Recurrent Gated Visual Fusion for Reactive Humanoid-Object Interaction
Humanoid loco-manipulation requires coordinating object acquisition, transport, and release from onboard sensing while responding to user commands and incomplete visual observations. We present ReGATE, a whole-body controller that combines egocentric depth, proprioception, and online locomotion and contact commands. Built on a teacher-student framework, ReGATE integrates interaction-state estimation through a recurrent adapter that predicts object geometry and physical contact without an external object-pose estimator. Head-wise and global gates regulate visual updates to a proprioceptive pathway. A visual-grounding and corruption curriculum trains the adapter, while phase-aware initialization increases coverage of brief contact transitions. In simulation, ReGATE achieves 85.2% success across 330 static-object acquisition trials, compared with 48.5% for VisualMimic under the same protocol, and 85.9% across 440 moving-object acquisition trials. Deployment on a Unitree G1 demonstrates commanded grasp-transport-release sequences and acquisition of laterally displaced boxes. A qualitative hardware experiment further demonstrates successful box acquisition when the camera is occluded after an initial observation and before the contact command. To the best of our knowledge, this is the first work to extend recurrent gated fusion of onboard depth and proprioception from perceptive locomotion to humanoid loco-manipulation under interrupted visual sensing.
★ SomaAlign: Reliability-Aware Multimodal Representation Learning for Soft Robotic Grasping
Robust multimodal perception for soft robotic grasping is challenged by modality-dependent noise, semantic ambiguity, and limited coordinated multisensory data. We introduce Soma, a dataset containing 100 objects, 1,000 continuous grasping trajectories, and 74,914 multimodal samples grouped by software acquisition cycle, spanning remote RGB, hand RGB, depth, tactile, IMU, and structured text observations. We also collect Soma-Success, an independent grasp-outcome dataset with 1,000 trials (998 retained after quality control), to evaluate frozen-encoder transfer for post-hoc grasp outcome classification and early grasp success prediction. Built on Soma, SomaAlign learns multimodal representations in three stages: masked reconstruction estimates relative modality reliability; reliability-aware contrastive alignment uses pair-specific adaptive temperatures to adjust alignment sharpness; and conflict-aware semantic constraints improve intra-sample cross-modal consistency while preserving global feature geometry. SomaAlign achieves an average Recall@10 of 90.66% across 30 directed cross-modal retrieval tasks. The learned representations also support physical attribute recognition, multimodal question answering, cross-modal generation, and cross-dataset transfer to grasp outcome prediction. On Soma-Success, frozen SomaAlign encoders with lightweight probes achieve 94.59% post-hoc classification accuracy and 84.07% early prediction accuracy. Together, Soma, Soma-Success, and SomaAlign provide a data and representation-learning foundation for robust and transferable multimodal perception in soft robotic manipulation.
comment: 18 pages, 9 figures
★ Beyond Policy Patches: Distilling RL Specialists into Vision-Language-Action Policies
Vision-language-action (VLA) policies can handle noncritical stages of long-horizon manipulation yet fail during short precision-critical phases. Lightweight reinforcement learning (RL) specialists can address these failures without directly applying RL to the full VLA, but retain auxiliary controllers and switching mechanisms at deployment. We study how to transfer their capabilities into a single flow-matching VLA. Direct imitation of specialist rollouts can reproduce action jitter and leave the VLA unprepared for states encountered during its own execution. We introduce REPAIR, a value-guided distillation framework that combines offline initialization with on-policy refinement. At each critical-phase observation, REPAIR queries the corresponding specialist for multiple action candidates and weights them using its critic to construct a supervision distribution. A shared weighted flow-matching objective trains the VLA first on observations from specialist-assisted rollouts, then on observations collected through its own execution. Specialists provide supervision without taking control during on-policy collection and are removed at deployment. Across six real-robot tasks with 30 evaluation trials per task, REPAIR achieves 100% success on all six tasks, with lower mean end-effector jerk than specialist-assisted execution on every task.
comment: 19 pages, 12 figures
★ Right Ranking, Wrong Scale: Reproducing the Robotic World Model's Dynamics Model and Testing the Uncertainty of Its Follow-up
We reimplement the dynamics model of the Robotic World Model (RWM) and test the uncertainty estimate that its follow-up, RWM-U, uses to penalize reward in offline policy optimization. Our implementation matches the released model's gradients. On held-out data, training on the model's own rollouts beats one-step training at the preregistered long horizon for all ten episodes, but loses on short one-step windows. RWM-U's ensemble disagreement ranks rollout errors better than forecast step (Spearman rho = 0.90 at 100 steps), yet its spread substantially understates error: one-sigma coverage on the released checkpoint is 4.6% at 100 steps, versus 68% for a calibrated Gaussian. Rolling ensemble members on their own predictions raises coverage to 33%. The implemented squared-error loss on sampled predictions drives each member's variance toward zero. A horizon-specific rescaling restores one-sigma coverage under leave-one-episode-out evaluation, but not two-sigma coverage. We also identify two defects in the released evaluation pipeline. The study covers one robot, one gait and one dataset; it trains no policy.
comment: 56 pages
★ Source-Resolved Collective Cognition in Robotics: Mathematical Foundations for Multimodal and Collective Estimation, and Their Reach across the Cognition Loop
The Generalized Singular Value Decomposition (GSVD) is a central tool for comparing two linear operators that act on the same unknown. Once the output metrics are fixed, it answers the basis-independent question "along which state directions does the first operator dominate the second, and by how much?". This foundational volume builds the GSVD and its surrounding estimation theory from the ground up. Part I re-derives every piece of linear algebra and statistical estimation machinery needed later -- inner products, the four subspaces, orthogonal matrices, the spectral theorem, quadratic forms, the ordinary SVD, Lie-group tangent perturbations, Jacobians, covariance, whitening, Fisher information, CCA, mutual information, discrimination criteria, and covariance intersection -- with self-contained derivations and worked numerical examples. Part II states the GSVD in both a transparent and a dimensionally general form, then proves it by two complementary routes. Part III works one small matrix pair through five different lenses, each computed by hand and verified numerically. Part IV separates information attribution, subspace geometry, statistical dependence, and decision-theoretic discrimination for collective systems. Part V evaluates the role and limits of the GSVD across the cognition loop. Part VI covers numerical practice, including a tested reference implementation. Every number printed in this document was verified in double precision. Where a statement is an assumption rather than a theorem, it is labelled as such.
comment: 93 pages, 14 figures. Reference implementation archived at Zenodo, doi:10.5281/zenodo.23862358
★ Closing the Dynamics Gap through Self-Directed Robotic Interaction
Agentic Real2Sim2Real systems can reconstruct a scene in simulation from camera observations, but contact-rich dynamics such as friction and restitution cannot be observed without interaction. We call this the dynamics gap and present CUE, an agentic system that closes it by interacting with the environment itself. Given only a high-level instruction and no resets, the agent builds its own perception, identifies physical parameters, and targets interactions where its simulation is least accurate. On billiards, CUE reduces held-out ball-position error by 72% relative to the strongest baseline across 25 hidden-physics conditions in simulation, and we further deploy it on a real pool table. More details can be found on the project website: https://cue-dynamics.github.io/
comment: Accepted to CoRL 2026 Late Breaking Results. Project website: https://cue-dynamics.github.io/
★ Visuo-tactile Diffusion Policy for Robotic Manipulation with Auxiliary Tactile Dynamics Prediction CCS
Long-horizon robotic assembly requires reliable perception and action across grasping, alignment, and insertion. Vision captures global object pose and scene context, where as fingertip touch provides local contact cues that are especially useful as contact states change. We present a visuotactile diffusion policy that combines four-step histories from two camera views and two fingertip tactile images to condition action-sequence generation. A recurrent temporal representation integrates recent observations from both modalities. To encourage it to capture how contact evolves under action, an auxiliary objective predicts fingertip-image changes two steps ahead from the temporal representation and demonstrated actions. This objective is used only during training and adds no inference-time computation. Across three independent training seeds in MuJoCo assembly experiments, the full model achieves an average success rate of $59.0\% \pm 2.6\%$, improving on the visual-only baseline ($42.7\% \pm 1.5\%$) by 16.3 percentage points.
comment: 7 pages, 2 figures. Presented at ICCSSE 2026
★ HeLM: Hierarchical Explicit Language Memory for Non-Markovian Manipulation
Recent advances in Vision-Language-Action (VLA) models have enabled fine manipulation capabilities, yet performing tasks that require working memory remains a key challenge. Most existing VLA architectures operate under a Markovian assumption, mapping only the current observation to an action, which leads to failure in non-Markovian environments. Inspired by the human 'inner monologue', we propose Hierarchical Explicit Language Memory (HeLM), a hierarchical VLA framework that maintains task-relevant history in explicit language through factorized event detection and event-triggered memory updates. We evaluate HeLM across 10 non-Markovian real-world tasks with 56 variants, including five newly formulated inter-episode tasks that require persistent context across task boundaries. HeLM improves intra-episode success by 12.8 percentage points over the strongest baseline and achieves 78% subtask success on compositional generalization tasks, while using up to 4.5x less memory and running 2.5x faster than the keyframe baseline at long horizons.
comment: 32 pages, 14 figures, including supplementary material. Project page: https://kimspacehuni.github.io/HeLM/
★ DexArena: A Unified Benchmark for Long-Horizon Dexterous Manipulation
Long-horizon dexterous manipulation remains a fundamental challenge for robot learning, requiring the composition of multiple skills, precise bimanual coordination, generalizable visual understanding, and robust skill transitions. Yet existing manipulation benchmarks largely emphasize isolated skills or support only a limited range of end-effectors, making these capabilities difficult to evaluate systematically. We introduce DexArena, a benchmark designed to evaluate learning and generalization in long-horizon bimanual dexterous manipulation. DexArena features 10 long-horizon tasks and 30 stage-level tasks spanning diverse household and industrial scenarios. Each long-horizon task is composed of explicitly defined stages with physically grounded success conditions, enabling reliable evaluation of both intermediate progress and end-to-end task completion. A unified benchmark interface and verified human-teleoperated demonstrations enable testing different policy learning algorithms, with standardized simulation configurations, scene randomization, trajectory replay, and task-level evaluation. Through extensive baseline experiments, we identify substantial challenges in precise manipulation, generalization to scene variation, and skill composition. DexArena establishes a challenging and reproducible benchmark for developing robot learning methods that move toward generalizable long-horizon dexterous behavior.
comment: Project page: https://sipair-elastic.github.io/DexArena-website/
★ Neural Integral Control Barrier Functions for Agile Collision Avoidance
Control barrier functions (CBFs) are a standard way to keep a planner inside a safe set, but they assume control affine dynamics, a barrier function given in closed form, and a known nominal controller. Integral control barrier functions (ICBFs) remove the first assumption and absorb input constraints, yet they still assume a given nominal controller and a barrier function. In this work, we present an approach to learn all three from data, using neural ordinary differential equations (ODEs) for the dynamics, imitation for the nominal controller, and supervised regression for the barrier function. The learned dynamics, nominal controller, and the barrier function are used to build the ICBF filter constraint analytically. Furthermore, the barrier is trained on viability rather than geometry, meaning that a state counts as safe only if some emergency maneuver can still avoid the constraint under the actuator limits, a correction that geometric labels miss entirely and that grows with speed. Actuator limits then enter the online program as hard bounds while the learned barrier rows may be relaxed, which keeps the program solvable at every state and the actuator limits exact, and we bound how far the input it returns can drift from the one the true models would give. On agile racing with a 6-D dynamic bicycle, the filter removes all 12 unsafe episodes from 200 held out layouts at an unchanged success rate and above 100 Hz on one CPU core, holds the input constraint on every episode of a benchmark where a jointly trained baseline holds it on 53.38%, and keeps that advantage under large errors in the plant friction.
comment: 23 pages, 1 figure
★ MAPO: Mixture of Experts with Alternating Policy Optimization for Universal Humanoid Motion Tracking
Universal humanoid motion tracking is essential for enabling humanoid robots to perform diverse tasks in human environments. To capture the wide diversity of human motions, mixture-of-experts (MoE) policies have been widely adopted, leveraging dynamic routing to enhance policy expressiveness. However, we identify that divergence between old and updated policies can be misestimated during policy-gradient updates of MoE policies, due to their dynamic routing. Because most policy-gradient algorithms regulate policy updates based on the divergence, this misestimation can lead to either overly conservative or excessively large updates, which can yield a suboptimal policy. In this paper, we propose Mixture of Experts with Alternating Policy Optimization (MAPO), which alternately optimizes the router and the experts to avoid the divergence misestimation. MAPO consistently tracks 182 motions that PPO with MoE fails to track, whereas PPO with MoE tracks only one motion that MAPO fails to track. MAPO also attains the highest tracking success rate among the compared methods and the lowest tracking error across most error metrics, demonstrating its superior performance. Videos can be found on the project website: https://mapo-corl2026.github.io/MAPO/
comment: 21 pages
★ W2A: Distilling World Priors into Action Models for Real-Time Robotic Control
World Action Models (WAMs) have recently shown great promise in robotic control, benefiting from the future imagination capability of large-scale pretrained video models. However, most existing WAMs require explicitly imagining future states at test time, incurring substantial computational overhead and latency that make them unsuitable for real-time robotic control. In this paper, we propose W2A, a two-stage distillation framework for real-time robotic manipulation that distills intermediate denoising representations from a video diffusion model into an action model during training, while entirely discarding the video model at test time. We hypothesize that although video latent representations are high-dimensional, a compressed low-dimensional subspace is sufficient to provide effective supervision for action generation. This makes it possible to transfer such information effectively into an action model. Extensive experiments in both simulation and real-world settings demonstrate competitive success rates while significantly improving inference efficiency. On LIBERO, W2A achieves a 95.9% average success rate. More importantly, W2A achieves about 21x faster inference than the representative WAM baseline.
★ Central pivot point control of a body-attached spring-mass runner: periodic gaits, stabilization, template embedding and approximate analytic return maps
The spring-loaded inverted pendulum (SLIP) describes the center-of-mass (COM) motion of runners but has no trunk. In trunk models, the hip torque couples posture and COM motion and destroys the central-force structure on which SLIP analysis relies. As a result, trunk models have been studied almost only numerically. We show that this structure can be restored. The central pivot point (CPP), a hip-torque law that directs the ground reaction force through the COM, decouples pitch exactly and reduces stance to a planar central-force problem. At upright posture, time-reversal symmetry produces one-parameter families of periodic gaits. A stance-phase pitch law and a stride-level regulator stabilize all computed gaits, also with bounded hip torque and friction. Under CPP, any radial force law in virtual-leg coordinates can be embedded independently of posture. For a linear radial template, the stance radius and the constant-momentum sweep have exact closed forms. With a new gravity correction, the resulting analytic return map, to our knowledge the first for a trunk model, predicts liftoff velocity and apex position with mean errors of 1.6% and 0.9%. The natural gaits of the embedded runner are symmetric and need no force step at touchdown. A near-reversibility argument explains their unit multiplier, and a scalar equation predicts them within 0.8%. A deadbeat stride controller based on the analytic map tracks changes of speed, height and posture. CPP thus makes trunk models accessible to the analytic tools developed for SLIP.
comment: 21 pages, 15 figures. Submitted to Chaos: An Interdisciplinary Journal of Nonlinear Science
★ Sequential Recovery Updates for Learned Controllers under Successful-Trajectory Preservation ICRA 2027
A competent learned controller already recovers from many disturbances. Improving it means learning a correction, and learning when not to apply it. Freezing a skill's parameters does not settle the second half. In a controller that routes among frozen skills, a new entry rule can redirect the states an old skill succeeded on. We therefore constrain entry instead of parameters. Each update searches for a feedback skill at a measured failure. It fits an entry rule that reads observations and rejects every state on an accumulated successful trajectory. The skill is admitted only if the composed controller reproduces the new recovery on its own and replays every protected trajectory exactly. A finite-trajectory condition states when this preserves the previous controller. One mechanism runs on four streams: two motions on each of two humanoid systems. Two updates per stream add 62 recoveries over 192 withheld perturbations, and lose none of the base controller's successes. We then price that guarantee. Removing the constraint recovers 33 more conditions but stops eight protected ones from completing at all, and excluding only what admission proves necessary recovers none of that cost.
comment: Submitted to ICRA 2027
★ Energy-Based Policy Orchestration
Hierarchical control scales robot behavior by separating task-level planning from low-level control. In dynamic, high-dimensional tasks, however, a fixed planner-to-policy interface can fail when contacts, balance, or disturbances evolve faster than high-level policy updates. Execution therefore requires choosing not only which low-level policy to run, but also which policy-specific goal should be given to it. We formulate this problem as policy orchestration: runtime selection of a policy-goal pair that is likely to remain feasible while tracking the planner target. We develop EPO, an energy-based policy orchestrator for modular goal-conditioned policy families. For each policy, EPO learns a composite energy over low-level goals, where lower energy corresponds to higher feasibility and more accurate target realization. At runtime, EPO refines each policy's goal and executes the lowest-energy policy-goal pair. Because energies are policy-specific, new policies can be incorporated by learning their own energy models, without retraining the existing policies or a centralized scheduler. In simulation, EPO improves the empirical precision-robustness trade-off by refining policy-specific goals before selectively switching among policies based on energy scores. We further demonstrate sim-to-real deployment of the same frozen policies and energy models on real humanoid loco-manipulation tasks. Project website: https://yewr.github.io/epo
comment: CoRL 2026
★ Interactive 3D Cutaway Visualization for Subsurface Target Localization in Robot-Assisted Surgery
Purpose: Blended augmented reality (AR) overlays can provide ambiguous depth cues in which subsurface anatomy floats above the tissue surface. We investigate whether an interactive 3D cutaway visualization can improve robot-assisted target localization. Methods: We propose Virtual Box, an interactive 3D cutaway interface that allows users to independently manipulate the viewing perspective to inspect subsurface targets. In a controlled, counterbalanced within-subject tissue phantom study, 15 engineering students used Virtual Box and a depth-enhanced blended AR overlay to position a da Vinci instrument at virtual targets in a water tank. We used registered 3D transrectal ultrasound to measure 3D localization error and depth error. We also collected completion time and subjective ratings. Results: Virtual Box reduced mean 3D error from 12.92 to 4.96 mm (61.6% reduction) and depth error from 9.48 to 2.07 mm (78.2% reduction; both p < 0.001). Spatial understanding, depth perception, and confidence ratings were higher (all unadjusted p < 0.001), but completion time increased from 21.64 to 32.58 s (p = 0.008). Overall workload did not differ significantly. Conclusion: Virtual Box improved localization accuracy and perceived spatial understanding in this controlled task, with additional completion time, supporting further evaluation for surgical navigation.
★ Touching Projected Agents through the DAT Screen -System Design and Questions of Interaction Boundaries-
Touching a virtual agent involves more than detecting a finger on a display: the contact surface can deform and resist the user's push, and the agent can respond to that action. We describe the Desktop Amid Touch (DAT) screen, which combines spatial projection onto a semi-transparent, deformable mesh with touch sensing and motorized control of resistance to indentation. An initial prototype detected a single touch and estimated its position and depth from tension changes around a flexible projection screen. A subsequent prototype added motorized tension control to vary the screen's resistance to indentation. We describe the tension-controlled DAT implementation and its demonstrations, distinguishing what has been observed from what remains an open design question. These capabilities raise a design question: should touchable virtual agents respond as though they had preferences or boundaries, and under what conditions should such behavior be implemented on systems like DAT. The present paper describes a system and frames this question; it does not establish how users interpret an agent's refusal or consent.
comment: The 2nd International Workshop on Trust and Blame in Multi-Person Multi-Agent Societies (TBSA_2nd) in conjunction with HAI 2026, 4 pages, 4 figures
★ A Study on the Effects of Finger Movements on a Humanoid Hugging Robot
This study aimed to examine the impact of finger movements in hugging robots on users by experimentally investigating how the presence and patterns of such movements affect users psychologically. The results suggest that the presence and patterns of finger movements influence the sensations users seek from the robot; specifically, repetitive movements and asynchronous movements across multiple fingers tended to enhance these sensations. Additionally, simultaneous movement of all fingers and gentle hugging tended to evoke a greater sense of care. Conversely, no differences were observed regarding resistance to physical contact initiated by the robot.
comment: The 2nd International Workshop on Trust and Blame in Multi-Person Multi-Agent Societies (TBSA_2nd) in conjunction with HAI 2026, 4 pages, 5 figures, 3 tables
★ Neuro Inspired Robotics: What Robot Learning Can Borrow from the Sensorimotor Brain
Robot policies still fail at everyday manipulation: an unfamiliar task, an object never held before, a scene that changes mid-motion, a skill that should improve with use. They fail despite pretraining on over a million trajectories, pooled from dozens of robots at great expense, while a child acquires all of these abilities within a few years of learning from a single body. This survey argues that the gap is one of design, and proposes designs for the next generation of robot models, drawn from how the nervous system solves the same problems. From the body we take compliance: the policy commands stiffness alongside position, so the controller yields where the task allows and presses where it must, which greatly lowers the demand for precise position output. From the visual system, where a fast pathway for action runs in parallel with the slow pathway for recognition, we take a dedicated fast vision system for motor behavior: it precomputes the quantities the policy needs and drives a reactive policy at camera rate. From the cerebellum we take a predictor beside the policy that forecasts the consequence of each command and recalibrates from its own error. From motor cortex we take a single model that composes action units from primitives, verifies each at a sensory boundary, practices the weakest unit with a hypothesis in mind, and reasons each attempt closer to the goal. Proven primitives are then consolidated from code into weights, so they run fluently and respond quickly the next time. Together these describe a robot that acts on its own initiative, the basis of deployment and recursive self-improvement, and we offer each proposal as a hypothesis to be tested.
comment: 72 pages, 27 figures, 3 tables
★ RoboColosseum: Evaluating Physical Intelligence at Scale In-The-Wild
General-purpose robot policies must work across robots, tasks, and environments, yet the variation that makes physical trials informative also makes them difficult to reproduce. We introduce RoboColosseum, a distributed evaluation framework that combines unified policy serving and standardized protocols across a network of locally operated robots in diverse environments. Our initial study evaluates five vision-language-action (VLA) policies and three frontier AI agents across four embodiments (SO-101, Franka, Bimanual-YAM, and G1 Humanoid). The Open Track compares models without task-specific adaptation in blinded, pairwise trials under matched local conditions. The Fine-tune Track measures how well VLA policies adapt to 20 tasks across all four embodiments when each policy receives the same 100 demonstrations per task. Across 5,699 pairwise Open Track comparisons at 124 sites, frontier agents rival specialized VLA policies, while all systems struggle to complete open-world tasks reliably. After fine-tuning, GR00T N1.7 leads on SO-101 and pi0.5 leads on Franka and Bimanual-YAM. Policy rankings show weak agreement across embodiment-specific task suites (Kendall W = 0.24). Open Track rankings poorly predict relative performance after fine-tuning, with substantial rank reversals on all three embodiments shared between tracks. Measuring partial progress reveals differences that task success alone misses. On the G1 Humanoid, no policy completes any of the five tasks, yet average progress ranges from 13.3% to 23.2% across policies. Open Track rankings stabilize as trials accumulate, and active matchmaking achieves comparable ranking agreement with substantially fewer physical trials than uniform sampling. These results show how distributed physical evaluation can reveal where robot policies generalize, how well they adapt to new tasks, and how far they progress on tasks they fail to complete.
comment: 34 Pages, project page: https://robocolosseum.ai/
★ To Coordinate or Not to Coordinate? Learning to Swim with Eight Soft Arms
Does effective swimming with multiple underactuated soft arms require explicit coordination between arms? We hypothesize that an eight-arm swimmer can control its swimming speed and direction by independently learning each arm's actuation amplitude under a shared rhythmic drive, without prescribed inter-arm phase coupling. To test this hypothesis, we compare three reinforcement learning-based control strategies using proximal policy optimization (PPO): (1) learning arm-specific actuation amplitudes without inter-arm phase coupling, (2) learning actuation amplitudes with prescribed inter-arm phase coupling, and (3) directly learning actuator commands through end-to-end PPO without an explicit rhythmic structure. Across 200 episodes with randomized three-dimensional velocity commands, the controller without phase coupling satisfies the tracking criterion in 169 episodes, compared with 117 for the phase-coupled controller and 28 for end-to-end PPO. Straight-line swimming and commanded direction-change trials further demonstrate more accurate velocity tracking without phase coupling. These findings suggest that effective eight-arm swimming does not require prescribed inter-arm phase coupling, but can instead be achieved through learned arm-specific amplitude modulation within a rhythmic control structure.
comment: 8 pages, 7 figures
★ Learning Purposeful Co-designs of Bio-inspired Quadrupedal Robots
Treating morphology and control as separate design problems has constrained the efficiency and adaptability of quadrupedal robots, while current co-design approaches are often constrained by restricted design spaces. We overcome this limit by introducing Dogger, a bio-inspired quadruped with modular three-segment legs, four-bar linkages and adjustable passive knee compliance. A two-stage framework combining reinforcement learning and Bayesian optimization is developed: the morphology-conditioned control policy is trained across designs and then held fixed during task-specific design selection. The resulting designs and policies deployed zero-shot reduce the cost of transport by 18.6% and 35.7% for inspection and payload-carrying tasks. Comparisons demonstrate that full geometry-compliance co-design outperforms fixed and restricted design scopes. Cross-task tests further reveal distinct task-specific morphologies with shared mechanical trade-offs. These results provide hardware-level evidence that expanding the physical design space alongside learned control is essential to realize the full potential of task-driven morphology-control co-design in legged robotics.
comment: 10 pages, 7 figures, 6 tables
★ Late Autumn: LLM Agents Specialize a General Multi-Robot Planner for One Warehouse
No algorithm is best on all problems, but a real warehouse is a narrow problem: its layout and processes rarely change. Operators still run one general multi-robot planner at many sites, because experts cannot tune it for each one. In this work, which we call Late Autumn, we explore whether LLM coding agents running autonomous research can specialize a general planner for one site, trading generality for throughput, much as post-training adapts a pretrained LLM to one task. We study six campaigns on a lifelong planner that combines task assignment with multi-agent path finding, on a floor modeled on an existing automated warehouse. We find that (1) the agents raise throughput by 22--60%, and the gains hold on unseen seeds; (2) they rediscover known MAPF mechanisms, such as corridor-exit priorities, local guidance, and rollouts, and independent campaigns find the same bottleneck; (3) the gains largely carry over to another layout, a larger fleet, and other service times, shrink with a smaller fleet, and mostly vanish under an outbound-heavy demand mix; and (4) each final patch (0.7k--2.4k lines) is a chain of scored commits with a median step of 27 lines, so experts can review it, and review finds problems that the score misses; some patches multiply planner compute by over 100x. LLM agents can therefore turn each site's bias into extra throughput at little engineering cost, with reviewable patches; we expect experts to shift toward designing insightful metrics and fast simulators that mirror the real problem.
★ MABEL: An Open Anthropomorphic Mobile Bimanual Platform for Teleoperation and Learning
As robot learning moves beyond the tabletop, many tasks require mobility, adjustable reach, active perception, and bimanual dexterity. Commercial platforms combining these capabilities are costly, while affordable open-source alternatives cover only a subset and require software integration. We present MABEL, an open-source anthropomorphic mobile bimanual robot with 56 degrees of freedom (DOF) and a US$8.1k parts cost. It combines two 7-DOF arms with 17-DOF hands, a pitching torso, a lift, a 3-DOF neck, and a holonomic base. Back-drivable arms and torso support compliant interaction, with fingertip reach from floor level to 2.23 m. The head and torso roughly triple modeled camera coverage relative to a torso-fixed camera. Apple Vision Pro head and hand tracking enables whole-body teleoperation without handheld controllers, with 137 ms mean glass-to-glass stereo video latency over the robot's Wi-Fi hotspot. We demonstrate an end-to-end imitation learning pipeline, from teleoperated data collection to policy deployment, using a shared command interface and whole-body controller.
comment: 8 pages, 10 figures, 1 table. Project page: https://robotmabel.github.io/website/
★ FEIJOA: Task-Driven Grounding and Execution for Mobile Manipulation
A mobile manipulator asked to "put the toys away in the toy box" must decide which objects are toys and which container the user means, then find base poses, grasps, and placements it can execute. Because a scene representation is built before the instruction is known, it may lack the attributes, affordances, or concepts needed to resolve the task. We present FEIJOA, a language-model agent that grounds the instruction at test time. It queries a grounding tool over a posed RGB-D scan, inspects the returned candidates, and refines its queries until they resolve the instruction. An embodiment-aware harness checks each skill's preconditions, samples base poses from which the target is reachable, and verifies the skill's effect from observations before and after execution. On 103 open-ended grounding queries, FEIJOA resolves 94.2% exactly, compared with 68.0% for the best prebuilt representation. Over 50 real-world trials on Spot, it completes 35 tasks; 17 of these successes follow recovery from a failed tool call or verification check. Of the 15 failed trials, 13 were caused by the robot's physical skills, and only 2 by grounding or the agent's reasoning. Project website: https://williamshen-nz.github.io/feijoa/.
★ A Practical Causal Framework for Autonomous Mobile Robots: From Simulation to Real-World Human Environments
Causal inference has emerged as a powerful paradigm enabling robots to make safe, efficient decisions by reasoning about the ``why'' behind events. However, its real-world application is hindered by the difficulty of collecting causal data, especially in human-populated environments where training takes time, while users expect immediate operation. To this end, we tackle the challenge of transferring simulation-learnt causal relations to the real world with minimal effort. While retaining the invariant high-level causal structure, we empirically demonstrate the system adapts efficiently to real-world situations by updating only the causal model parameters with a limited set of new observations. Our results show that the parameter update enables rapid sim-to-real transfer. The framework runs efficiently onboard of a mobile robot and significantly outperforms a non-causal baselines navigating successfully and safely in complex human-shared scenarios.
comment: Accepted in IEEE Robotics and Automation Letters (RA-L)
★ Physical Experience Builds Continual Spatial World Models
World models are increasingly used to support prediction and planning in embodied agents, but the knowledge is usually established before deployment and then carried into later encounters. Continual operation faces a different problem. A familiar environment may change, and the agent must use previous experience without allowing old expectations to override current physical evidence. This study asks what repeated physical experience must contribute for a spatial world model to become progressively more useful rather than simply more persistent. A physics-grounded dynamic maze separates recurring spatial change from the continuous consequences of action. Cross-episode Spatial Change Memory learns relations among environmental transformations, while state-action-outcome experience refines predictions of motion. Structured experience progressively reduces the need for future route correction and transfers to spatial transitions not encountered during learning. The advantage disappears when past changes contain no information about future ones. Continuous navigation further shows that predictive memory changes behavior when a previous decision becomes invalid but leaves behavior unchanged when the decision remains correct. Physical tests expose a complementary limit: correct topology does not guarantee correct action prediction, and accumulated physical experience reduces this mismatch within the modeled dynamics. The results characterize continual world knowledge as experience that improves future prediction and action beyond stored episodes while remaining revisable by the physical outcomes it is intended to predict.
★ Self-governed access control in open robot swarms using a blockchain-based DAO
Robot swarms harness simple, local interactions among many individually limited robots to produce complex, coordinated collective behavior, making them valuable across domains from agriculture to disaster response. We envision the next frontier for such swarms as open swarms, in which robots join or leave the swarm dynamically at runtime. The ability to dynamically adjust swarm size has been shown to be beneficial, for instance by reducing inconsistencies and network partitioning, especially under sparse connectivity or in complex environments, ultimately improving task performance. For practical deployment, however, open swarms requires governance, particularly protection against Byzantine behavior (i.e., faulty or malicious robots), since allowing robots to join freely also makes the swarm easier to infiltrate. Blockchain technology has recently been incorporated into robot swarms, offering secure, decentralized coordination and improved resilience, though largely in closed settings with a fixed, predetermined membership. In this work, we extend these benefits to open swarms by introducing robot swarms governed by a blockchain-based decentralized autonomous organization (DAO), in which the swarm regulates itself using autonomous rules encoded in smart contracts. We outline several DAO-based governance mechanisms for robot swarms, depending on the needs of the application, and present a proof-of-concept implementation of a simple DAO that regulates swarm membership and prevents Byzantine infiltration. Our simulation results and real-robot validations show that DAO-governed access control and robot removal can maintain proper swarm functioning while keeping the number of Byzantine robots within acceptable limits.
★ h-NESDF: An Infinitely Differentiable Signed Distance Between Two Sets with Robotic Applications
Computing distances between sets is important in many robotic applications, such as motion planning and control. In many applications, derivatives of the distance are needed as well. It is well known that the Euclidean distance is not always differentiable, which create issues to algorithms that need a derivative. In this paper, we introduce NESDF, the Normal Euclidean Signed Distance Function between two general (with mild regularity assumptions but possibly non-convex) sets. This metric is still not differentiable, but it allows us to introduce a generalization, $h$-NESDF, which has a smoothing parameter $h$ such that the metric is infinitely differentiable for all $h>0$ and that reduces to the NESDF when $h$ goes to $0$. The metric is amenable to GPU parallelization, and thus can be computed efficiently. We show extensive simulation results to evaluate its viability in motion planning.
★ Coordinate Frame Recovery, Per Class NMS and Null Return Modelling for Sim2Real V2X LiDAR Detection ECCV 2026
We describe our approach to the V2X-Real track of the UrbanTwin Sim2Real LiDAR Challenge, in which a 3D object detector must be trained exclusively on synthetic data and evaluated on unseen real roadside LiDAR from an Ouster OS1-64 unit, while the synthetic point clouds are simultaneously scored for distributional realism. Our final entry scores 0.7457 combined, from a detection mAP of 0.4382 and a realism score of 0.9251. Three findings account for most of our result, and all three were diagnostic rather than architectural. First, a coordinate frame mismatch between our predictions and the evaluation ground truth was suppressing detection AP entirely; recovering it took our score from 0.3520 to 0.5147 in a single submission. Second, a single global non maximum suppression threshold was destroying genuine pedestrian detections, because pedestrians occur in groups and a 0.3 IoU threshold applied to a 0.7 m box suppresses distinct neighbouring people; correcting this per class raised overall mAP from 0.0751 to 0.4295. Third, approximately 31 percent of the real reference point cloud consists of null returns encoded as the literal point (0, 0, 0), a format feature entirely absent from our synthetic clouds, which pinned Earth Mover's Distance near its floor until we modelled it. Negative results are reported at equal length, and Section 9 gives a full account of data provenance and compliance. That section also accounts for every mechanism contributing to the submitted score.
comment: 8 pages. 1st place, Track 7 (UrbanTwin Sim2Real LiDAR, V2X-Real), DriveX Grand Challenge, ECCV 2026 Workshop. Code: https://github.com/amanupg/urbantwin-sim2real-v2x
★ Geometric Refinement and Distributional Alignment for Sim2Real Roadside LiDAR Detection ECCV 2026
We describe our approach to the LUMPI track of the UrbanTwin Sim2Real LiDAR Challenge, in which a 3D object detector must be trained exclusively on synthetic data and evaluated on unseen real roadside LiDAR, while the synthetic point clouds are simultaneously scored for distributional realism. Our final entry scores 0.4793 combined, from a detection mAP of 0.1333 and a realism score of 0.9126. Our contribution is not a new architecture. Working from a PointPillars baseline, we obtained most of our gains from three sources. First, a diagnosis that small object failure in roadside LiDAR is a localisation problem rather than a detection problem, which led to a parameter free geometric refinement worth 86 percent relative on pedestrian AP. Second, an audit of our own post processing which revealed that a shipped rule was destroying valid predictions, the repair of which produced our single largest detection gain at 335 percent relative on the Van class. Third, treating the realism metrics analytically rather than as a black box, which showed that one of the four components is a closed form function of nine summary statistics. We also report an experimental methodology that we believe is the more transferable result. Because mean AP decomposes into independent per class terms, a single submission can carry several mutually controlled experiments, and a submission assembled entirely from previously measured components has a score that is known before it is sent. We predicted our final score to four decimal places prior to submission. We report negative results at equal length, and give a full account of data provenance and compliance in Section 8, including a full account of every mechanism contributing to the submitted score.
comment: 8 pages. 1st place, Track 6 (UrbanTwin Sim2Real LiDAR, LUMPI), DriveX Grand Challenge, ECCV 2026 Workshop. Code: https://github.com/amanupg/urbantwin-sim2real-lumpi
★ CoReMatch: Recurrent Continuous Coordinate Refinement for Real-Time Robotic Feature Matching
Feature matching underpins many robotic perception tasks, including visual localization, mapping, SLAM, and autonomous navigation. Existing semi-dense matchers predict confidence on fixed grids, limiting correspondences to predefined locations and requiring costly high-resolution grids for precise localization. We present CoReMatch (Coordinate Refinement Match), a recurrent framework that refines correspondences directly in continuous coordinate space rather than predicting discrete confidence maps. CoReMatch progressively improves correspondence estimates by propagating information across iterations. At the coarse stage, a Deep Aggregated Transformer captures global context and iteratively updates correspondence coordinates, while a lightweight regression head further sharpens their localization. Experiments on indoor and outdoor benchmarks show that CoReMatch strikes a strong balance between accuracy and efficiency, outperforming Efficient LoFTR in accuracy while running more than 10x faster on embedded edge hardware. These results demonstrate that recurrent coordinate refinement offers an efficient and scalable approach to real-time feature matching for robotic perception.
comment: Accepted by CoRL 2026
★ Self-Consistent Neural Control: Learning Trajectories for Implicit Equilibrium
One big challenge in physical AI is interacting with systems whose states are defined by implicit equilibrium. Learning in these systems is difficult because their responses can be multistable, path-dependent, and sensitive over long horizons. Adjoint learning provides an efficient way to optimize trajectories without unrolling iterative equilibrium solvers, but its gradient relies on the frozen-tangent approximation, which neglects higher-order sensitivity terms and can become inaccurate when the local equilibrium varies rapidly. In this work, we introduce Self-Consistent Neural Control (SCNC), optimizing trajectories along which sensitivity variations are explicitly controlled. We derive a mathematical bound on the resulting gradient error and develop a practical finite-difference approximation. We validate the framework on deformable objects, which are representative of implicit equilibrium systems, ranging from simple quadratic energy models such as elastic rods to complex systems including thin-walled tubes, measuring tapes, and Slinky-like structures. Compared with the adjoint learning and the gradient-free iCEM baselines, SCNC achieves faster convergence and more stable optimization by reducing switching to undesirable equilibrium branches, doubling the stable optimization rate over adjoint learning 83.3% vs. 41.7% and reducing the average loss by more than 1.5 orders of magnitude on thin-walled tubes, where sensitivity variations are pronounced.
★ RoboScope Kit: Unified Interfaces for Fast and Scalable Embodied-AI Evaluation
Simulation benchmarks are the standard way to compare robot policies, but each benchmark brings its own evaluator, observation layout, and action convention, and evaluation is slow: 500 RoboTwin Easy episodes with one simulator and one model on one GPU take 359.7 minutes. Profiling the official StarVLA evaluator shows that simulation occupies CPU cores while the GPU averages 14.3% utilization. RoboScope Kit wraps each model once in a Python class called a Policy, and each benchmark once in an adapter around the benchmark's own evaluator, so the benchmark's rules and scoring stay unchanged. Scheduling runs simulator workers and Policy replicas as separate pools, sizes both from measured probes, and shards work deterministically across GPU instances; runtime optimization renders only the observations an action-chunk Policy reads. Adapters cover six benchmark families. On LIBERO-Long with a StarVLA checkpoint on one 48 GiB RTX 4090, the concurrency search selects twelve simulator workers sharing three Policy replicas, which evaluate 500 episodes 4.45$\times$ faster than the official StarVLA evaluation with equivalent success rates, 2.78$\times$ faster than RLinf, and 1.52$\times$ faster than vla-eval at its tuned configuration. With CUDA MPS, which lets the replicas run their kernels concurrently, the search selects sixteen workers sharing four replicas, 6.16$\times$ faster than StarVLA, 3.85$\times$ faster than RLinf, and 2.10$\times$ faster than vla-eval. On RoboTwin Easy, four GPUs with chunk-boundary observation complete the 500 episodes in 14.4 minutes, 25.04$\times$ faster than the single-GPU baseline, with identical success outcomes and terminal steps.
comment: 29 pages, 9 figures. Project page: https://roboscope.pages.dev/ Code: https://github.com/MINT-SJTU/RoboScope-Kit
★ A Tutorial on Shape Servoing of Deformable Objects-Part II: Advanced Concepts
In robotics, shape control, or shape servoing, refers to the automatic manipulation of a deformable object toward a desired shape. This article is the second of a two-part tutorial on data-driven, Jacobian-based shape control, in which the deformation Jacobian, a local model of how robot motion changes the measured shape, is estimated from measurements. Part I built the shape-servoing loop and derived a local stability condition. This paper starts by using the singular value decomposition of the deformation Jacobian to identify shape changes that can be produced by small robot motions, those that require large robot motions, and those that are locally unavailable. It then analyses how the choice of shape features determines how the controller regulates the object's shape and, therefore, how much the object deforms on its way to the target. It then describes three ways in which the shape control loop can fail: loss of controllability, loss of observability, and occurrence of mechanical instabilities. It closes with shape-control extensions to time-varying targets, secondary tasks, and three-dimensional objects.
comment: Part II of a two-part tutorial. 10 pages, 9 figures. Code: https://github.com/nachocz/tutorial_on_shape_servoing_of_deformable_objects. Interactive demos: https://nachocz.github.io/tutorial_on_shape_servoing_of_deformable_objects/
★ AIRoA-0: Long-Horizon, Precise, Cross-Embodiment Manipulation from Handheld Data
AIRoA-0 (A-0) is a bimanual manipulation system learned from 16,151 hours of handheld demonstrations, with no robot data collected at any stage. The demonstrations are recorded with YUBI, our open-source handheld gripper, which lets people record manipulation data anywhere with no robot present. From a single instruction, A-0 sorts more than a dozen small parts into their numbered cells and changes the mop pads and brush of a robot vacuum; among other tasks, it unzips a pencil case and folds a towel. It performs these tasks using only a camera on each wrist, with no head or scene camera, at a site where none of its training data was recorded. Across the ten tasks of our evaluation, it averages a success rate of 78%, against 27% for the strongest of four other vision-language-action models post-trained on the same task demonstrations. We collect no teleoperation data and use no reinforcement learning or on-robot corrections, and the robot appears only at test time. The system learns which regions of each wrist view matter for the task, predicts the stage the task has reached, and decodes actions as a continuous stream. A-0 is embodiment-agnostic: it runs on robots that mount the YUBI gripper, such as a bimanual Franka, TX-G2, OpenArm and UR. We are releasing the dataset, to our knowledge the largest open dataset of handheld-gripper demonstrations with task and subtask instructions, together with the pre-trained weights and the post-training recipe. Videos of A-0 on every task are on the project page, https://a-0.airoa.io. Anyone can build the gripper, record a task of their own and post-train the model on it in under four hours on a single node of eight H200 GPUs.
comment: 18 pages. Project page: https://a-0.airoa.io
★ Robot-Matched Soft Fingers for UMI Data Collection: Lessons from 100 Hours of Pick and Place
Training robot policies for multi-task pick and place requires demonstrations on many objects, which handheld Universal Manipulation Interfaces (UMIs) can collect quickly and cheaply. However, existing UMIs depend on external base stations, use rigid fingers that give no signal about contact forces, or use parallel-jaw fingers that limit grip strength. We present a battery-powered handheld interface, built on YUBI, whose soft, angled fingers wrap around objects, deform visibly under load as a visual proxy for force, and match the grippers of the robot. With it, we collect a 99.7-hour dataset over 53 objects, train $π_{0.5}$ policies on 49 of them, and hold out the other 4 as unseen objects. We find that a top camera raises success by 13 points even when only about half of the data includes it, and that mid-training on our multi-task data raises success after fine-tuning on unseen objects from 35% to 57%. Finger design, dataset, and data processing code: https://lute-corp.github.io/soft-yubi-site/
comment: 8 pages, 3 figures, 3 tables. Accepted as a poster at the CoRL 2026 UMI Arena Workshop (Track 3). Project page: https://lute-corp.github.io/soft-yubi-site/
♻ ★ Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models NeurIPS 2026
A fine-tuned Vision-Language-Action (VLA) policy can easily place the alphabet soup in the basket on command, just as it saw during fine-tuning. However, if the basket is shifted five centimeters to the left, it drops the soup off the table. We developed an interpretability pipeline to investigate what such a policy is computing. Injecting task A's activations into the action expert while the policy looks at task B's scene makes $π_{0.5}$ execute task A's motor trajectory in 99.6\% of episodes ($n{=}1{,}968$). The injected program is bound to absolute workspace coordinates and ignores the visible scene, which provides a mechanistic account of the perturbation brittleness that concurrent benchmarks report. The binding holds on a physical UR10e: directed replay redirects the commanded task in 26/26 rollouts in one direction and 18/26 in the other, with no-injection and random-direction controls at the floor, and once the source object is moved 10-15 cm the arm grasps and releases at the recorded coordinates in 50/50 trials, picking up the wrong block when one sits there and closing on empty air when none does. Applying the same interventions to five VLAs on LIBERO, MetaWorld, SimplerEnv, and ALOHA over $420{,}000+$ rollouts yields three further findings: every architecture encodes the instruction, and four of the five weight it below vision whenever the scene alone identifies the goal; under the standard replacement write-back the SAE pooling that preserves behavior is architecture-specific ($π_{0.5}$ per-token, SmolVLA indifferent), and goal selection and motor generation separate into VLM and expert pathways wherever the two are distinguishable. We release \textbf{Action Atlas} (https://action-atlas.com), an interactive platform covering all 520+ trained SAEs, 79 identified concepts, and the scored UR10e rollouts. Code: https://cwru-aism.github.io/vla-interp-page/
comment: NeurIPS 2026 Poster, ICLR Multimodal Intelligence Workshop Oral, 44 pages, 14 figures
♻ ★ Sequential Planning via Anchored Robotic Keypoints
Changes to what a robot can see do not mean that there is a change in the problem it is trying to solve, and many language-conditioned manipulation systems struggle with this distinction. When something in a scene moves, the robot might need to execute its actions in different places, but its manipulation plan can stay the same. We propose Sequential Planning via Anchored Robotic Keypoints (SPARK), which describes a task as a symbolic behavior tree that we anchor to the current 3D observations when each action is called. By verifying the plan at runtime we can recover from failures locally and anchor the objects to the scene again. We ask for a new plan only if this fails. We isolate this concept by taking a fixed plan and moving objects in the scene while executing. When displacing objects by $10$\,cm we were still able to complete $66.7\%$ of trials with scene-diff verification and layered recovery, and we did not have to change the plan. Without practicing on the tasks being tested or using ground-truth object poses from the simulator during rollouts, we test our full system (with one open-weight 27B planner with 4-bit weights) on LIBERO-PRO. Under this protocol SPARK achieves a mean success of $75.1\%$ across the 6 position and task settings and $69.7\%$ on the held-out language and object settings. We also achieve a $68\%$ success rate across 11 physical task-embodiment settings on 3 different robots. Our work demonstrates that for an important class of failures, the robot does not need to ask for a new plan for its task. It can be enough to update the system's view of the world. Code and videos can be found at https://cwru-aism.github.io/spark-page/
comment: 30 pages, 7 figures
♻ ★ On the Limits of Sampling-Based Reachability: Geometry, Dynamics, and Sample Complexity
Reachability analysis is central to safety-critical control, robotics, and neural network verification, but classical computational methods, such as Hamilton--Jacobi reachability and set propagation, scale poorly with state dimension. Sampling-based methods have emerged as a promising alternative, often providing finite-sample guarantees that bound the probability-mass left uncovered. However, an explicit account of how the geometry of the initial set, the dynamics, and the sampling law affect the accuracy of the estimator is not fully available in the literature. We study this by casting sampling-based reachable-set recovery as geometric support estimation over a family of problems specified by an initial set, its dynamics, and a sampling law. First, we identify two regularity properties, positive reach of the initial set's complement and Lipschitz continuity of the dynamics, that together make recovery well-posed: a probability-mass coverage guarantee can be upgraded to accuracy $r$ in Hausdorff distance. Second, we bound the resulting sample complexity: recovery is achievable with $\tilde{\mathcal{O}}\big((e^{LT}/r)^n\big)$ samples, exponential in both the state dimension and the time horizon. Third, we show that neither can be removed: a minimax lower bound of $\tildeΩ\big((e^{LT}/r)^n\big)$ holds for every estimator, so the exponential dependence on dimension and the degradation over the horizon are both intrinsic, not artifacts of a particular method. Experiments on nonlinear systems confirm that adversarial sampling improves constants but not the scaling.
♻ ★ OmniGuide: Universal Spatial Guidance Fields for Enhancing Generalist Robot Policies
Vision-language-action (VLA) models have shown great promise as generalist policies for a large range of relatively simple tasks. However, they demonstrate limited performance on more complex tasks, such as those requiring complex spatial or semantic understanding, manipulation in clutter, or precise manipulation. We propose OmniGuide, a flexible framework that improves VLA performance on such tasks by leveraging arbitrary sources of guidance, such as 3D foundation models, semantic-reasoning VLMs, and human pose models. We show how many kinds of guidance can be naturally expressed as differentiable energy functions with task-specific attractors and repellers located in 3D space, that influence the sampling of VLA actions. In this way, OmniGuide enables guidance sources with complementary task-relevant strengths to improve a VLA model's performance on challenging tasks. Extensive experiments in both simulation and real-world environments, across diverse sources of guidance, demonstrate that OmniGuide enhances the performance of state-of-the-art generalist policies (e.g., $π_{0.5}$, GR00T N1.6) significantly across success and safety rates. Critically, our unified framework matches or surpasses the performance of prior methods designed to incorporate specific sources of guidance into VLA policies. Project page: https://omniguide.github.io/
comment: Accepted to CoRL 2026. Project page: https://omniguide.github.io/
♻ ★ Reusable Neural Reachability via Local Refinement
This paper studies obstacle avoidance under translation invariant dynamics using an avoid-side travel cost Hamilton Jacobi formulation. For running costs that are zero outside an obstacle and strictly negative inside it, we prove that the value function is non-positive everywhere, equals zero exactly outside the avoid set, and is strictly negative exactly on it. Under translation invariance, this yields a reuse principle: the value of any translated obstacle is obtained by translating a single template value function. We show that the pointwise minimum of translated template values exactly characterizes the union of the translated single-obstacle avoid sets and provides a conservative inner certificate of unavoidable collision in clutter. To reduce conservatism, we introduce a blockwise composition framework in which subsets of obstacles are merged and solved jointly. This yields a hierarchy of conservative certificates from singleton reuse to the exact clutter value, together with monotonicity under block merging and an exactness criterion based on the existence of a common clutter avoiding control. The framework is illustrated on a Dubins car example in a repeated clutter field.
♻ ★ When Human Corrections Travel: Safety in Federated VLA Adaptation SP
Vision-language-action (VLA) policies can adapt to local human intervention, while federated learning can combine distributed robot experience without pooling raw records. We examine their prospective intersection: sharing adaptations derived from diverse human corrections across tasks, environments, people, and robot embodiments. Corrections differ in modality, safety semantics(urgency, behavioral scope, etc.), and privacy sensitivity. A shared update that improves mean task success may suppress a rare safety intervention or produce an unsafe response on another robot. We argue for three requirements: preserve the context and uncertainty of multimodal corrections when deciding what to share; assess candidate updates in each receiving robot's closed loop before deployment; and evaluate leakage alongside retention of safety-critical evidence. Immediate, legible defensive behavior remains necessary during human interaction. We offer counterexamples and a falsifiable evaluation agenda, not experimental results.
comment: Accepted for presentation at SPAIS workshop in CoRL 2026
♻ ★ DiffContact: Differentiable Contact Dynamics for Policy Learning and Contact-Implicit Trajectory Optimization
This paper proposes DiffContact, a smooth analytic gradient for frictional contact, and applies it to policy learning and contact-implicit trajectory optimization for legged locomotion. The exact contact sensitivity is available in closed form but describes changes within the current contact mode, providing no first-order information toward neighboring contact modes. The proposed gradient replaces this sensitivity only in the backward pass, leaving the hard-contact forward solve unchanged and the realized rollout feasible. A central-path relaxation of the de Saxcé-corrected second-order-cone complementarity problem for Coulomb friction smooths the conditions defining the separating, clamping, and sliding modes. The relaxed sensitivity is evaluated at a point on the central path obtained by a Newton iteration initialized from the hard-contact solution. The sensitivity remains continuous through contact-mode transitions and converges to the exact sensitivity as the relaxation vanishes. Using 100,000 samples and less than 2 min of training on a desktop PC, policy learning with the proposed gradient produces a policy that transfers to hardware without fine-tuning. On hardware, the robot under a trotting policy withstands pushes and traverses debris and slippery ground. In trajectory optimization, the proposed gradient reaches contact-rich solutions missed by exact and Signorini-relaxed gradients. Project page: https://diffcontact.github.io
comment: Submitted to IEEE Transactions on Robotics (T-RO)
♻ ★ ROOF: Learning Humanoid Fall Recovery under Overhead Constraints
A fallen humanoid beneath an overhead obstacle faces a coupled recovery problem. It must create clearance while organizing support for a stable rise, since attempting to stand before making room can further increase obstruction. We introduce a reinforcement learning method with contact-supported geometric escape guidance, which rewards incremental reduction of robot-obstacle overlap without prescribing an escape path. To achieve this, a learned motion prior, together with posture-dependent support and joint-stall costs, is proposed to shape a single policy that learns when to clear the obstacle, how to establish support, and when to complete the rise, all using proprioception alone. In simulation, it recovers in 88.5% of trials beneath plates that can only be lifted, compared with 18.9% without geometric guidance, and in 98.6% of trials beneath free plates with randomized size, mass and placement. On a Unitree G1, the deployed policy recovered in 95 of 150 attempts across seven real-world obstacle scenes, including furniture and multi-object arrangements absent from training. Project page: https://lu-yidan.github.io/ROOF/
comment: 8 pages, 4 figures
♻ ★ A Phenomenology of Neural Geometry in Vision-Language-Action Models
Robotic manipulation is inherently geometric, but what shape does the world take inside the policy that performs it? Prior work on vision-language-action (VLA) policies establishes that task-relevant variables are linearly readable from activations, yet readability certifies only that a variable is present and says nothing about the shape its values trace, which may be curved and multi-dimensional. Using controlled sweeps, natural rollouts, linear probes, and activation interventions, we build a taxonomy of seven task-relevant concepts in VLA models. We find that realized geometry is predominantly multi-dimensional and is shaped by the constraints under which the policy acts. Physical joint limits, for example, leave wrist orientation an open arc rather than a circle, and distance to the target, a single scalar, is realized in two orthogonal subspaces, a coarse ruler for the approach and a fine ruler for the grasp. The same geometries reappear in two further VLA models on real-world robot data, indicating that they are neither model-specific nor artifacts of simulation. We demonstrate how the findings are actionable by steering along the geometry a concept realizes. Overall, this work lays out an initial phenomenology of realized geometry in VLA policies and demonstrates that a concept's shape carries as much importance as its readability.
comment: 41 pages, 10 pages of main text, 8 main text figures
♻ ★ TRACE: Target-State Risk-Aligned Conditional Execution for Embodied Tracking
Embodied visual tracking requires a robot to follow a language-specified person under occlusion, look-alike distractors, and ambiguous instructions. We show that improving tracking persistence can simultaneously increase safety violations. Using a compact 0.6B vision-language-action (VLA) tracker with a frozen backbone, we update only the planner head with proximal policy optimization (PPO). PPO improves success and following rates across all three EVT-Bench settings, but increases the collision rate on distracted tracking because episodes that previously ended in target loss can instead end in contact. We introduce TRACE, an execution-time target-state risk gate that preserves the learned policy and intervenes selectively. A lightweight risk head processes 84 cues, including candidate boxes, identity margins, a learned range-and-bearing probe, and the policy's commands. When the calibrated risk exceeds a threshold, TRACE rescales the translational component of the next waypoint while leaving yaw unchanged. The executed action is identical to PPO's on more than 98% of steps. In paired evaluations on identical episodes, TRACE reduces the distracted-tracking collision rate by 3.96 percentage points, while further improving success and following rates. Matched slowdown controls without target-state timing do not reproduce this improvement. Concurrent evaluation reproduces a collision reduction of 3.77 points, compared with 0.29 points for identical-arm null pairs, indicating that the effect exceeds evaluation variability. These results show that target-state-aware execution can recover the safety cost of policy post-training with minimal intervention.
comment: 35 pages, including appendices; updated Mingyang Lyu's affiliations. No changes to the scientific content
♻ ★ CoSMO: Conformal Semantic Mapping for Open-World Safe Robot Navigation
Safe autonomous navigation requires not only geometric obstacle avoidance but also semantic awareness. Different object categories demand different safety clearances, and the set of relevant categories may expand as the robot explores novel environments. Vision-language models (VLMs) offer rich semantic representations for building maps of unknown scenes, yet their confidence scores are poorly calibrated, making raw predictions unreliable for safety-critical planning. Conformal prediction (CP) can provide coverage guarantees, but existing methods assume a fixed label set and stationary data, limiting their use in open-world settings. We propose Conformal Semantic Mapping for Open-World Safe Robot Navigation (CoSMO), a semantic safety mapping framework that transforms VLM embeddings into online-calibrated coverage sets and safety-aware obstacle dilations under distribution shift. By calibrating directly in the embedding space, CoSMO avoids the score rescaling that arises in confidence calibration as the vocabulary expands. Experiments on the Matterport3D (MP3D) dataset show that CoSMO reduces semantic collisions by 61.6% relative to a geometry-only baseline, while maintaining formal coverage guarantees even under delayed expert feedback. Real-world experiments on a quadruped robot further demonstrate generalization by achieving long-run empirical coverage of 90.73% against the 90% target. Project website: https://yangming911.github.io/cosmo-page/.
comment: Accepted at the 10th Conference on Robot Learning (CoRL), 2026
♻ ★ SV-WAM: An Efficient Surround-View World-Action Model for End-to-End Autonomous Driving
World models (WMs) have demonstrated strong potential for end-to-end autonomous driving by learning predictive representations of future scene dynamics. However, generating future videos during inference introduces substantial computational overhead, leading many recent driving WMs to adopt a single front camera as input for efficient deployment. This design restricts spatial coverage in safety-critical maneuvers such as lane changes, merges, and turns. To address this limitation, we propose SV-WAM, a surround-view world-action model (WAM) that preserves full six-camera observations while maintaining efficient inference. SV-WAM leverages future-video prediction as dense training supervision for action learning within a shared generative model, rather than as an inference-time output. At the core of this design is an action-centered causal mask that prevents action tokens from attending to future-video tokens during joint action-video denoising. Consequently, the video branch can be discarded at deployment, enabling efficient action-only planning. Furthermore, we introduce a differentiable drivable-area compliance regularizer that penalizes vehicle-footprint corners approaching or crossing drivable boundaries, improving planning safety and boundary awareness. Extensive experiments on the closed-loop NAVSIMv2 benchmark and the open-loop nuScenes benchmark demonstrate that SV-WAM achieves state-of-the-art planning performance with low inference latency and competitive zero-shot transfer capability.
comment: 23 pages, 16 figures. Project page: https://jinyangwang427.github.io/SV-WAM/
♻ ★ RoboSeed: Multidimensional Evaluation of Vision-Language-Action and World-Action Models: Capability, Resource Affordability, and Research Evidence
Choosing a robot action model requires evidence about task capability, adaptation cost, deployment requirements, and the reliability of reported conclusions. Success rates alone offer limited guidance to researchers with different computing budgets and goals. We propose RoboSeed, a multidimensional evaluation framework for vision-language-action (VLA) and world-action models (WAM), within a continuing program to train, evaluate, and openly release robot models with verifiable evidence for model selection. The framework defines five dimensions: capability, downstream training affordability, deployment affordability, locally verified research value, and reproduction evidence. It maps empirical measurements and audit criteria to 0-100 profiles using fixed normalization anchors, with balanced, training-constrained, and deployment-constrained preference settings. Interface and protocol checks establish when comparisons are appropriate. Missing measurements remain explicit, incomplete aggregate scores are withheld, and weighted input coverage identifies evidence gaps. We apply the framework to representative VLA and WAM models, drawing on local training, released-weight evaluation, task-level audits, and an evaluation-system study. The results reveal budget-dependent backbone choices, sensitivity to adaptation recipes, and differences between language and visual task interfaces. Reproducible scoring code and machine-readable evidence support extension to additional models. The study provides dimension-level assessments and priorities for further measurement, while incomplete serving traces and heterogeneous protocols preclude overall numerical rankings.
comment: Project repository: https://github.com/PrimitiveX/RoboSeed
♻ ★ Robot-Factored World Models via Robot Rendering
Action-conditioned video world models predict future observations from an initial observation and an action signal. We propose robot-factored world models (RoFacto), which move two robot-specific factors outside the world model to separate action realization from scene-response prediction. First, action realization: each command sequence is rolled through the robot's own controller and kinematics without scene contact into a deployment-available nominal trajectory that does not require future realized states. Second, robot rendering: this nominal trajectory is rendered through the robot URDF, factoring the robot's geometry, kinematics, and appearance out of the model and into explicit rendered robot geometry. Paired end-effector and scene depth provide geometric cues for contact and occlusion, while camera-aware static RGB context and rendered robot geometry form a shared visual interface across viewpoints and embodiments through which the model learns scene responses. Our experiments show that the rendered interface outperforms vector-conditioned baselines, supports transfer to unseen robot embodiments, and generates robot manipulation videos from retargeted human demonstrations.
comment: Project page: https://bjkim95.github.io/rofacto/. Camera-ready version for CoRL 2026
♻ ★ HaltNav: Reactive Visual Halting over Lightweight Topological Priors for Robust Vision-Language Navigation
Vision-and-Language Navigation (VLN) is shifting from rigid, step-by-step instruction following toward open-vocabulary, goal-oriented autonomy. Achieving this transition without exhaustive routing prompts requires agents to leverage structural priors. While prior work often assumes computationally heavy 2D/3D metric maps, we instead exploit a lightweight, text-based osmAG (OpenStreetMap Area Graph), a floorplan-level topological representation that is easy to obtain and maintain. However, global planning over a prior map alone is brittle in real-world deployments, where local connectivity can change (e.g., closed doors or crowded passages), leading to execution-time failures. To address this gap, we propose a hierarchical navigation framework HaltNav that couples the robust global planning of osmAG with the local exploration and instruction-grounding capability of VLN. Our approach features an MLLM-based brain module, which is capable of high-level task grounding and obstruction awareness. Conditioned on osmAG, the brain converts the global route into a sequence of localized execution snippets, providing the VLN executor with prior-grounded, goal-centric sub-instructions. Meanwhile, it detects local anomalies via a mechanism we term Reactive Visual Halting (RVH), which interrupts the local control loop, updates osmAG by invalidating the corresponding topology, and triggers replanning to orchestrate a viable detour. To train this halting capability efficiently, we introduce a data synthesis pipeline that leverages generative models to inject realistic obstacles into otherwise navigable scenes, substantially enriching hard negative samples. Extensive experiments demonstrate that our hierarchical framework outperforms several baseline methods without tedious language instructions, and significantly improves robustness for long-horizon vision-language navigation under environmental changes.
♻ ★ Targeting the Right Situations: Mutation Testing of Fuzzy-Powered Situation Hyperspace
Ensuring the safety of autonomous Unmanned Aerial Vehicles (UAVs) operating in underground mines is challenging due to the vast and uncertain Operational Design Domain (ODD). Traditional validation approaches, such as random testing, often result in inefficient situation coverage, allocating computational resources to low-risk scenarios while failing to expose critical edge cases. Building on prior work in risk-prioritized Situation Coverage, this paper investigates whether Fuzzy Logic-based risk estimation effectively correlates with real system failures. To this end, we conduct a mutation testing study within our SCALOFT simulation framework. A fixed set of software mutants is systematically injected and evaluated under two testing strategies: (i) uniformly random situation generation and (ii) Fuzzy-Guided risk-prioritized situation generation. Experimental results show that the Fuzzy-Guided approach achieves a higher mutation score and is more effective at exposing difficult-to-detect faults compared to random testing.
♻ ★ Material-informed Gaussian Splatting for 3D World Reconstruction in a Digital Twin
3D reconstruction for Digital Twins often relies on LiDAR-based methods, which provide accurate geometry but lack the semantics and textures naturally captured by cameras. Traditional LiDAR-camera fusion approaches require complex calibration and still struggle with certain materials like glass, which are visible in images but poorly represented in point clouds. We propose a camera-only pipeline that reconstructs scenes using 3D Gaussian Splatting from multi-view images, extracts semantic material masks via vision models, converts Gaussian representations to mesh surfaces with projected material labels, and assigns physics-based material properties for accurate sensor simulation in modern graphics engines and simulators. This approach combines photorealistic reconstruction with physics-based material assignment, providing sensor simulation fidelity comparable to LiDAR-camera fusion while eliminating hardware complexity and calibration requirements. We validate our camera-only method using an internal dataset from an instrumented test vehicle, leveraging LiDAR as ground truth for reflectivity validation alongside image similarity metrics.
comment: 8 pages, 5 figures. Accepted to IEEE Intelligent Vehicles Symposium (IV) 2026. Revised version (v4) presents camera-ready publication
♻ ★ Learning to Build: Autonomous Robotic Assembly of Stable Structures Without Predefined Plans
This paper presents a novel autonomous robotic assembly framework for constructing stable structures without relying on predefined architectural blueprints. Instead of following fixed plans, construction tasks are defined through targets and obstacles, allowing the system to adapt more flexibly to environmental uncertainty and variations during the building process. A reinforcement learning (RL) policy, trained using deep Q-learning with successor features, serves as the decision-making component. As a proof of concept, we evaluate the approach on a benchmark of 15 2D robotic assembly tasks of discrete block construction. Experiments using a real-world closed-loop robotic setup demonstrate the feasibility of the method and its ability to handle construction noise. The results suggest that our framework offers a promising direction for more adaptable and robust robotic construction in real-world environments.
comment: Accepted for publication in IEEE Robotics and Automation Letters (RA-L). Updated to the author-accepted manuscript following peer review
♻ ★ CoEnv: Code-Driven Exploration in Real-to-Sim Digital Twins for Multi-Arm Manipulation
General-purpose coding agents can now write, debug, and refine entire robot programs, but only if they have somewhere to practice. Physical hardware is a poor practice ground: exploratory rollouts are slow to reset and, when several arms share a workspace, unsafe. Generic simulation benchmarks are safe but do not match the geometry, assets, or viewpoints of the workspace where the robot must actually work. We present CoEx (Code-driven Exploration), which lets the agent practice in an executable digital twin of its own deployment workspace. The twin is built by CoEnv, our real-to-sim framework, which reconstructs the physical scene from multi-view RGBD into a physics simulation. Inside it, a centralized code agent explores at the program level, writing a complete multi-arm program, running it in the twin, and refining it against grounded verification (privileged state checks, checkpoint predicates on risky steps, and adaptive viewpoints against inter-arm occlusion). Verified programs transfer to hardware through collision-checked deployment. On five real-world multi-arm tasks with up to three heterogeneous robots, CoEx reaches 70% average success, against 50% for the same system without the code agent (where a VLM must choose every primitive step by step), outperforms two prior VLM-planning frameworks, and produces verified episodes at 3.8-11.7x the ablated rate. Because every verified rollout is also a demonstration, exploration leaves reusable experience behind: a $π_{0.5}$ policy fine-tuned purely on 200 agent-generated rollouts reaches 100% and 70% real-robot success on two tasks, with no human demonstrations.
comment: v2: substantially revised, with a new title, framing, and experiments (v1 title: "CoEnv: Driving Embodied Multi-Agent Collaboration via Compositional Environment"). 26 pages, 8 figures, 9 tables. Project page: https://faceong.github.io/CoEnv/
♻ ★ Dynamics-Aware Meta-Imitation for Generalization to Unseen Robotic Manipulation
Imitation Learning aims to learn skills from extensive observations and demonstrations for robots, so it suffers from data scarcity and environment generalization. The existing methods predominantly focus on imitation from in-domain tasks and consequently struggle with generalization to unseen tasks. To bridge this generalization gap, we propose the \textbf{D}ynamics-\textbf{A}ware \textbf{M}eta-\textbf{I}mitation (DAMI) framework. By integrating meta-learning to construct a shared skill space, DAMI equips agents for rapid adaptation to novel tasks. We introduce the Visual-Motor Trajectory (VMT) module to capture complex spatio-temporal dynamics within the task latent space. Furthermore, we propose the Unpaired Unified Task (U2T) block to fuse unstructured multimodal observations. To coordinate these representations, we integrate a Task-Conditioned Feature Modulation (TCFM) mechanism customized for modulating low-level 3D features. By capturing intrinsic dynamics from a random complete reference demonstration, our framework learns the underlying task logic rather than memorizing static cues, ensuring effective generalization. Extensive experiments in both simulation and real-world settings demonstrate that our approach outperforms state-of-the-art baselines regarding direct inference on seen tasks and adaptation to unseen tasks via few-shot fine-tuning.
♻ ★ ReWorld: Multi-Dimensional Reward Modeling for Embodied Video Generation
Current embodied video generation models primarily emphasize visual generative quality while overlooking physical fidelity, dynamic consistency, and task logic, especially for contact-rich manipulation tasks, which limits their applicability to downstream tasks. In this paper, we introduce ReWorld, a framework aimed to employ reinforcement learning to align the video generation with physical realism, task completion capability, embodiment plausibility and visual quality. Specifically, we first construct a large-scale (~235K) video preference dataset and employ it to train a hierarchical reward model designed to capture multi-dimensional reward consistent with human preferences. We further propose a practical alignment algorithm that post-trains flow-based video generators using this reward through a computationally efficient PPO-style algorithm. Comprehensive experiments and theoretical analysis demonstrate that ReWorld significantly improves the physical fidelity, logical coherence, embodiment and visual quality of generated rollouts, outperforming previous methods.
♻ ★ 2AM: Grounding Agent-Side Memory as Guidance for Steerable Action Models in Long-Horizon Manipulation
Long-horizon robot manipulation requires memory, but not necessarily inside the action policy. To address such tasks, current agentic systems often combine VLAs with planners and geometric tools, sometimes using additional depth or calibrated geometry. These systems confound attribution: gains may come from richer observations or alternative motor tools, while failures may stem from either the policy or an under-specified language interface. We isolate this question through a deliberately constrained design: less tool breadth, but greater interface bandwidth. 2AM makes a multimodal Agent the sole holder of task memory and a single RGB-based, episodically stateless Action Model the sole executor of task-relevant motion. The Agent compiles interaction history into subtask language and optional 2D grasp, place, and move hints that bind its physical intention at different time scales. To teach this steerability to the VLA, we augment demonstrations with structured hint labels and train under condition dropout, spatial noise, and temporal jitter to tolerate imperfect Agent outputs. On LIBERO-Mem, without depth, online geometry, or planner-based object motion, 2AM reaches 76.3% average completion, a 61.5-point improvement over the strongest reported baseline of 14.8%, together with 63.0% relaxed and 11.8% strict success. These results show that task memory can remain Agent-side. They further show that Action Model capability depends not only on what the policy has learned, but on how precisely the Agent can steer it. Project website: agent-mem-action-model.github.io
♻ ★ MERIS: Adaptive Real-Time Scheduling of Refined Inference DAGs in Dynamic Robotic Environments
Robots deployed in dynamic environments execute changing directed-acyclic-graph (DAG) workloads under limited compute and timing budgets. This article presents MERIS, a framework for model scheduling under real-time constraints that combines proportional intermediate deadlines, MIMONet-aware graph refinement, and event-driven synchronization for multi-task inference on resource-constrained robotic platforms. We distinguish three representations: the application DAG, each MIMONet's internal encoder/decoder graph, and the refined scheduling DAG, and make the scope of the timing argument explicit. If the profiled execution bounds, release assumptions, precedence barriers, and per-level capacity premises hold, satisfaction of the assigned sub-deadlines composes to the end-to-end deadline. For non-partitionable tasks, we define a profile-bounded admission rule; before interpreting that rule as empirical timing evidence, we require direct per-task completion checks. We report run-level QoE alongside the offered-node deadline missing rate, making explicit that run completion and successful service of every node are different outcomes. We propose a comprehensive minibenchmark for evaluating the performance of schedulers under various DNN-driven autonomous driving scenarios. Extensive experiments on Nano Orin, AGX Orin, and M3 Max demonstrate the effectiveness of our proposed method.
comment: Extension version of RTSS'23
♻ ★ How to Peel with a Knife: Aligning Fine-Grained Manipulation with Human Preference
Many essential manipulation tasks - such as food preparation, surgery, and craftsmanship - remain difficult for autonomous robots. These tasks are characterized not only by contact-rich, force-sensitive dynamics, but also by their "implicit" success criteria: unlike pick-and-place, task quality in these domains is continuous and subjective (e.g. how well a potato is peeled), making quantitative evaluation and reward design difficult. We present a learning framework for such tasks, using peeling with a knife as a representative example. Our approach follows a two-stage pipeline: first, we learn a robust initial policy via force-aware data collection and imitation learning, enabling generalization across object variations; second, we refine the policy through preference-based finetuning using a learned reward model that combines quantitative task metrics with qualitative human feedback, aligning policy behavior with human-evaluated task quality. Using only 50~200 peeling trajectories, our system achieves 100% same-produce success on cucumbers, apples, and potatoes, with performance improving by up to 40% through preference-based finetuning. Remarkably, policies trained on one produce category also transfer zero-shot to selected held-out produce categories, achieving 50% success on zucchini, 90% on pear, and 80% on daikon radish.
comment: Project page can be found at https://toruowo.github.io/peel
♻ ★ LangMap: A Human-Verified Benchmark for Hierarchical Open-Vocabulary Goal Navigation NeurIPS 2026
Language-conditioned goal navigation requires embodied agents to locate user-specified targets without step-by-step guidance. However, existing benchmarks largely focus on category-level goals or rely on instance descriptions generated by vision-language models, which often contain ambiguities and semantic errors, limiting systematic and reliable evaluation. We introduce HieraNav, an open-vocabulary navigation task with goals at four hierarchical semantic levels: scene, room, region, and instance. We present Language as a Map (LangMap), the first goal navigation benchmark to enrich real-world indoor 3D scans with human-verified semantic annotations supporting tasks across all four goal levels. Built on HM3D using a contrastive annotation protocol that compares same-scene regions and instances, LangMap provides region labels and discriminative region and instance descriptions covering 414 object categories and contains over 18K tasks. Each target has concise and detailed descriptions, enabling evaluation across instruction styles. Automated and human evaluations validate our annotation quality: our descriptions improve text-to-view matching accuracy over GOAT-Bench's by 23 points on all shared annotated instances, and an independent human audit yields 92.5% unique-and-correct matches. We also propose PlaNaVid, an RGB-only baseline that combines Bounded Diverse Memory with high-level planning to prime a reactive policy for multi-goal navigation, achieving top-tier success rates without depth, 3D scene representations, or object masks. Further analyses reveal that exploration and hierarchical disambiguation failures become more prominent at finer goal levels, while long-tail categories, small objects, distant targets, timely stopping, and multi-goal completion remain challenging. Benchmark and code: https://bo-miao.github.io/LangMap
comment: Accepted to NeurIPS 2026. Benchmark and Code: https://bo-miao.github.io/LangMap
♻ ★ Harnessing Plastic Deformation for Continuous Passive Stability in Soft Grippers
Soft robots offer compliance and adaptability for interacting with complex environments, but their low structural stiffness can compromise stability under dynamic disturbances. Here, we present a continuously stable structure through plastic deformation (CSSPD), a hybrid metal-elastomer architecture that enables passive shape retention without continuous energy input. By deliberately exploiting plastic deformation in a kirigami-patterned metal layer, CSSPD establishes new equilibrium configurations while increasing structural stiffness and resistance to dynamic disturbances. A theoretical model is developed to characterize the deformation and stability of the hybrid structure and guide the geometric design of the metal layer. We further introduce a bioinspired paw pad that provides top-side support for enhanced grasp stability and serves as an integrated sensing interface for rapid object acquisition. A telescopic soft pressing mechanism (TSPM) mechanically couples paw-pad motion with tendon actuation. Experiments demonstrate passive holding forces of up to 16 N without continuous energy input, comparable to pneumatic actuation at 0.3 MPa, and stable grasping under pulsed accelerations of up to 400 m/s^2 when combined with pneumatic actuation. Energy-free perching on tree branches further demonstrates the potential of the proposed composite architecture for stable and energy-efficient manipulation.
comment: 27 pages, 8 figures
♻ ★ Multimodal Progress-Residual Fusion for Anticipating Navigation Retreat in Robotic Flexible Endoscopy
Flexible robotic endoscopes provide access through tortuous anatomy, where friction, deformation, and wall interaction separate commanded insertion from useful distal progress. Operators interpret this mismatch and retreat to recover a navigable configuration. Recognizing the approach to that decision requires relating motor intent to distal response and the local visual scene. We introduce Multimodal Progress-Residual Fusion (MPRF), a causal model that anticipates operator retreat from commands, electromagnetic (EM) tracking, and endoscopic video. A nominal-progress model predicts the response expected in normal or successful navigation; a temporal predictor combines its EM residual with all three observed modalities. In a 150-episode colon-phantom collection, MPRF achieves 0.580 +/- 0.007 AUPRC across three seeds, compared with 0.511 for the strongest reduced-modality control. Complete multimodal histories provide the principal ranking advantage, while the residual adds an explicit intent-response representation with a small, uncertain increment over raw fusion. Analysis of frozen alarms relates warning timing to the behavioral label and explains the increase in event recall under a longer matching window. These results establish a basis for behavioral assistance during operator-controlled navigation.
♻ ★ $S^3$-Nav: Structured Spatial-Semantic Decision Making for Language-Guided Robot Navigation
Language-guided robot navigation under partial observability requires robots to reason over spatial-semantic evidence accumulated across observations, particularly when goals involve spatial relations between objects. This paper presents $S^3$-Nav, a structured spatial-semantic decision-making framework that integrates online mapping, persistent object--relation memory, and goal-conditioned navigation. Using RGB-D observations and SLAM, $S^3$-Nav maintains a persistent object--relation memory containing object records and confirmed relation records alongside an evolving navigation context containing exploration frontiers, without requiring a pre-existing map. A goal-conditioned evidence interface selects task-relevant evidence from this structured state and constructs a capacity-controlled decision view as part of the high-level policy input for exploration, target approach, and stopping decisions. The persistent memory remains available for subsequent reasoning and verification even when some of its records are excluded from the decision view. To ensure grounded execution, provisional policy decisions are independently validated against persistent object--relation memory and current geometric evidence before physical execution or task termination. Evaluations across multiple environments and object- and relation-goal tasks show improved closed-loop navigation reliability and reduced language-model token consumption with capacity-controlled evidence exposure under the evaluated inference configuration.
♻ ★ OpenHLM: An Empirical Recipe for Whole-Body Humanoid Loco-Manipulation
Whole-body humanoid loco-manipulation requires coordinating the robot's entire kinematic chain. However, most existing systems typically decouple the upper and lower bodies into separate controllers, limiting such coordination and yielding behaviors similar to those of a wheeled dual-arm platform. In this paper, we ask what it takes to build a whole-body native vision-language-action (VLA) model that maps language and pixels directly to all of the humanoid's degrees of freedom. We conduct a systematic empirical study organized as a roadmap of one-variable-at-a-time experiments across three phases: whole-body teleoperation, VLA model design, and heterogeneous co-training. Our study yields several intriguing findings: a joint-based whole-body teleoperation interface outperforms alternatives that only partially expose the humanoid's degrees of freedom; a VLA pretrained on static and wheeled dual-arm platforms transfers surprisingly well to a humanoid's full action space; and co-training with HuMI, the humanoid analog of UMI, extends the policy to new objects and instructions without additional whole-body teleoperation on those targets. Following this roadmap yields OpenHLM, an open-source recipe for whole-body humanoid loco-manipulation. In a challenging long-horizon task that spans a wide vertical range of the humanoid, OpenHLM outperforms two state-of-the-art humanoid VLA baselines (GR00T N1.6 and $Ψ_0$) using less than half the total demonstration time. Our code, training data, and model checkpoints are available at [https://openhlm-project.github.io/].
comment: 30 pages
♻ ★ SEAMLiS: Visibility-Aware Safety for Perception-Limited Multi-Robot Exploration
Autonomous exploration in unknown environments is typically driven by informative frontiers, viewpoints, or trajectories, while local safety controllers avoid obstacles represented in the current map. Under finite sensing range and limited field of view, this separation can be unsafe: an exploration stack may plan optimistically through unobserved space and steer the sensor toward information gain rather than along the direction of motion, causing hidden obstacles to be detected too late for bounded-actuation avoidance. This paper presents SEAMLiS (Safe Exploration for Autonomous Multi-Robot Systems Under Limited Sensing), a modular execution-layer safety framework for decentralized multi-robot exploration. SEAMLiS preserves the upstream exploration stack, including the goal allocator and local planner, and enforces safety at the execution layer through perception-aware attitude and positional filters. A gatekeeper-based attitude filter switches between a visibility-promoting yaw policy and a velocity-tracking backup policy to preserve visibility of the critical known-free/unknown boundary with sufficient braking margin. A Control Barrier Function (CBF)-based positional filter then avoids known obstacles, newly detected obstacles, and other robots. We provide sufficient collision-avoidance conditions and validate the framework in randomized simulation, Isaac Sim, and Crazyflie hardware experiments. Results show collision-free exploration across tested single- and multi-robot settings while retaining much of the efficiency of visibility-promoting yaw control.
comment: Project page: https://www.taekyung.me/seamlis
♻ ★ LOGIV: Graph-Verified Execution with Causal Local Repair for Long-Horizon Robot Tasks
Agentic robots must turn a high-level goal into actions while keeping the physical conditions needed for future steps valid. A completed action does not ensure that its effects will persist: a later disturbance can invalidate support that the remaining plan still assumes. We present LOGIV, a graph engineering framework for agentic robot execution that uses an explicit task graph to connect symbolic planning, observation-based verification, and embodied action. LOGIV compiles a validated plan into condition-support and resource-ordering edges, checks their physical validity during execution, and uses dependency tracing to determine what must be restored after a failure. Repair is constrained by retained unfinished actions, and the combined continuation is symbolically validated before execution resumes. The framework operates above existing Vision-Language-Action (VLA) and World Action Model (WAM) policies without adding trainable components. Across 20 dual-arm tasks from RoboTwin V2 and BiCoord with three execution backbones, LOGIV improves mean task success without injected disturbances by 10.67 and 15.33 percentage points, respectively. Under predefined perturbations, it achieves higher task completion and condition restoration than three adapted recovery baselines for every backbone on both benchmarks. On three real-robot tasks, successful trials increase from 12/30 to 17/30. These results support explicit, continuously checked task dependencies as a practical interface for reliable agentic robot execution.
comment: 14 pages. Updated related work, references, author list and affiliations
♻ ★ Sampling-Based Motion Planning with Scene Graphs Under Perception Constraints
It will be increasingly common for robots to operate in cluttered human-centered environments such as homes, workplaces, and hospitals, where the robot is often tasked to maintain perception constraints, such as monitoring people or multiple objects, for safety and reliability while executing its task. However, existing perception-aware approaches typically focus on low-degree-of-freedom (DoF) systems or only consider a single object in the context of high-DoF robots. This motivates us to consider the problem of perception-aware motion planning for high-DoF robots that accounts for multi-object monitoring constraints. We employ a scene graph representation of the environment, offering a great potential for incorporating long-horizon task and motion planning thanks to its rich semantic and spatial information. However, it does not capture perception-constrained information, such as the viewpoints the user prefers. To address these challenges, we propose MOPS-PRM, a roadmap-based motion planner, that integrates the perception cost of observing multiple objects or humans directly into motion planning for high-DoF robots. The perception cost is embedded to each object as part of a scene graph, and used to selectively sample configurations for roadmap construction, implicitly enforcing the perception constraints. Our method is extensively validated in both simulated and real-world experiments, achieving more than ~36% improvement in the average number of detected objects and ~17% better track rate against other perception-constrained baselines, with comparable planning times and path lengths.
comment: 8 pages, 5 figures, Accepted to R-AL
♻ ★ Look as You Leap: Planning Simultaneous Motion and Perception for High-DOF Robots
Most common tasks for robots in dynamic spaces require that the environment is regularly and actively perceived. The perception task considered in this work can represent a broad range of robot perception objectives, including object detection, human activity recognition, and human face detection. For example, a service robot may need to continuously localize an object during manipulation, while an assistive robot may need to reliably perceive a human face or activity for interaction and safety. These tasks impose perception constraints on robot motion. However, solving motion and perception tasks simultaneously is challenging, as their requirements often conflict. Furthermore, robots must react quickly to environmental changes, while directly evaluating perception quality (e.g., object detection confidence) is often expensive or infeasible at runtime. This problem is especially important in human-centered environments, such as homes and hospitals, where effective perception is essential for safe operation. In this work, we address motion planning for high-degree-of-freedom (DoF) robots from a start to a goal configuration with continuous perception constraints in static and dynamic environments. Our solution is a GPU-parallelized perception-score-guided probabilistic roadmap planner with a neural surrogate model (PS-PRM). Unlike existing active perception-, visibility-aware, or learning-based planners, our work jointly considers perception tasks and constraints when searching for a motion-planning solution. Our method uses a neural surrogate model to approximate perception scores, incorporates them into roadmap planning, and leverages GPU parallelism for efficient online replanning. We demonstrate that our planner outperforms RL- and trajectory-optimization-based baselines in static and dynamic environments in simulation and real-robot experiments.
comment: 20 pages, 14 figures, Accepted to T-RO
♻ ★ GazeVLA: Learning Human Intention for Robotic Manipulation
Embodied foundation models have significantly propelled the advancement of artificial general intelligence, yet they rely heavily on large-scale data. Recent studies identify human behavior as one of the most scalable sources for fostering robotic physical intelligence. However, the inherent embodiment gap remains a fundamental challenge for effective human-to-robot learning. In this work, we model human intention to enable efficient learning from human demonstrations. Among human behavioral cues, gaze simultaneously encodes high-level task semantics and exhibits a synergistic coupling with low-level action. By modeling intention through gaze, we derive a mid-level representation that facilitates cross-embodiment knowledge transfer. We demonstrate that robots can internalize human intentions to enhance manipulation, even without robotic intention annotations. To capture the synergy between intention and action, we adopt a Chain-of-Thought reasoning paradigm during inference. Our approach enables the execution of long-horizon and fine-grained tasks across diverse embodiments. Extensive experiments in both simulation and real-world demonstrate that our method consistently outperforms baseline approaches and generalizes to unseen settings. Project page: https://gazevla.github.io .
comment: Project page: https://gazevla.github.io
♻ ★ Labference: Benchmarking the Reasoning-Action Connection in Robotic Laboratories
Realizing Embodied AI for Science requires robots to translate experimental goals into reliable physical procedures, reducing hazardous and repetitive human work while generating evidence for scientific discovery. Although language-based planning and learned manipulation provide complementary capabilities, their integration requires robots to adapt subsequent operations to actual execution outcomes. For example, an unsuccessful placement changes what must be done next, even when the experimental goal remains unchanged. Existing benchmarks assess planning, manipulation, and laboratory workflows through task completion and reasoning or execution metrics; however, aggregate performance leaves the connection between outcome-dependent choices and physical execution unresolved. To address this evaluation gap, we introduce Labference, a benchmark for evaluating how observed physical outcomes guide subsequent choices and how those choices translate into robotic laboratory actions. Its evaluation protocol is designed to test outcome-dependent behavioral adaptation through controlled evidence variations, while independently measuring operation selection, physical execution, and task completion. Matched Oracle-assisted continuations from shared checkpoints then assess which forms of support improve subsequent choices and task completion. Initial weighing evaluations reveal target-selection errors in the tested VLA policies despite successful object transfers, whereas GPT-driven control achieves correct selection but encounters execution failures and substantial computation costs. Together, these findings highlight the need to connect task-appropriate reasoning with reliable, timely action. By making this connection measurable and diagnosable, Labference supports targeted improvements toward autonomous experimentation for Embodied AI for Science.
comment: Update project address
♻ ★ DuoBench: A Reproducible Benchmark for Bimanual Manipulation in Simulation and the Real World
Bimanual robot systems substantially expand manipulation capabilities, but coordinating two arms introduces additional control complexity and failure modes that are not well captured by existing benchmarks. We introduce DuoBench, an extensible benchmarking framework for bimanual manipulation policies on the FR3 Duo platform. DuoBench comprises eleven tasks spanning four coordination categories, implemented in simulation and partially replicated in the real world through reproducible task recipes with 3D-printable assets. In addition, we propose a stage-based evaluation scheme that supports fine-grained semantic failure analysis beyond binary success. It includes stage-conditional pass rates with confidence intervals as well as human-teleoperated datasets for all benchmark tasks. We benchmark several imitation-learning and vision-language-action policies in simulation and on real hardware. Our results show that current policies remain challenged by bimanual manipulation, particularly at the beginning of interaction, during parallel arm execution, and in sim-to-real transfer settings. We further find that simulation reveals the same failure modes as real-world experiments. DuoBench provides a reproducible testbed for analyzing them and studying future methods for dual-arm policy learning. Code, datasets, and videos are available at https://duobench.github.io
♻ ★ Closing the Lab-to-Store Gap: A Data-Efficient Post-Training and Experience-Driven Learning VLA Framework for Retail Humanoids
Closing the gap between benchmark performance and reliable real-world operation remains a central challenge for Vision-Language-Action (VLA) humanoid robots, which must handle execution errors, distribution shifts, and environmental variability. This paper presents DEED (Data-Efficient Post-Training and Experience-Driven Learning), a systems-level approach evaluated on a supermarket chip-restocking task using a Unitree G1-Edu humanoid robot and the GR00T N1.6 foundation model. DEED comprises three key components: (1) a data-efficient post-training pipeline with control-frequency alignment, data curation, task-relevant visual highlighting, and reduced VLA dependence; (2) a real-world study of experience-driven refinement, adapted from RECAP via a text-based advantage prefix and a vision-language value function; and (3) a latent-space analysis tool for studying in- and out-of-distribution behavior. Our results suggest that bridging the lab-to-store gap is primarily a systems integration challenge rather than an architectural one: careful data design and targeted post-training can transform a policy that fails under naive fine-tuning into a competent real-world system using only a single GPU.
comment: 8 pages. Accepted for publication in 2026 IEEE-RAS 25th International Conference on Humanoid Robots (Humanoids)
♻ ★ Towards Trustworthy Physical Intelligence: From Theory to Practice Across Life Cycle
Physical intelligence refers to intelligence systems that understand, reason about and act in accordance with the physical world and its underlying laws, dynamics, and constraints. Unlike conventional AI systems, physical intelligence interacts continuously with uncertain physical environments, and its actions produce consequences that are physically irreversible. As existing trustworthy AI frameworks have been developed primarily for digital AI systems, they do not fully capture the distinctive challenges of Physical Intelligence, such as physical safety, cyber-physical security, and physical manufacturing process. To address this gap, our paper develop a foundation for understanding, operationalizing, and governing trustworthy physical intelligence. First, we characterize the fundamental capabilities of physical intelligence. Second, we investigate the role of physical laws and principles in AI and analyze the associated challenges of physical intelligence. Third, we provide a unified structure of the end-to-end physical intelligence life cycle across five core stages and introduce the Trustworthy Physical Intelligence Operationalization (T-PAIO) framework. Fourth, we formulate the Trustworthy Physical Intelligence (T-PAI) Principles as a theoretical framework that synthesizes key trustworthiness dimensions and establishes a conceptual foundation for the governance of physical intelligence systems.
♻ ★ Contingency-Aware Planning via Certified Neural Hamilton-Jacobi Reachability
Hamilton-Jacobi (HJ) reachability provides formal safety guarantees for dynamical systems, but repeatedly solving the HJ partial differential equation in changing environments limits its use in real-time planning. This paper introduces learning-based reachability and an online safety certification that together enable real-time contingency-aware planning in unknown environments with probabilistic safety guarantees. We use Fourier Neural Operator (FNO) to approximate the HJ solution operator under varying obstacle configurations. To certify the learned reach-avoid set, we recast robust scenario optimization as multiple hypothesis testings over its sublevel sets, which yields a simultaneous guarantee across candidate levels and recovers the largest certifiable volume at the prescribed safety level. Within this certified set, the induced recovery policy is guaranteed to return the system to a safe region in finite time under any admissible disturbance. Numerical results show that the learning and certification pipeline updates reach-avoid sets at real-time rates and is demonstrated on youBot simulation.
comment: 9 pages, 4 figures
♻ ★ Bayesian Active Learning for Intent Disambiguation in Interactive Robot Planning IROS 2026
Interactive robot planning requires robots to infer and execute human intentions from natural language instructions that are often ambiguous, incomplete, or underspecified. Although large language models (LLMs) provide a powerful interface for clarification, relying on the generative model to drive an multi-turn conversation can introduce systematic failures. We propose a Bayesian framework that treats clarification as an active learning problem over grounded Signal Temporal Logic (STL) task specifications. Our method uses LLMs to initialize candidate formal specifications and translate informative contrasts into natural-language clarification questions, while Bayesian optimization maintains uncertainty estimation over user intent and selects queries that maximize information gain. After convergence, the inferred STL specification is passed to a formal planner to synthesize a verifiable robot trajectory. Across four simulated and real-world task domains, our approach generally achieves higher task satisfaction and requires fewer clarification rounds than LLM baselines, while helping smaller models close the performance gap against larger reasoning models.
comment: Accepted by CoRL 2026 as Spotlight, also presented at IROS 2026 Human-Robot Dialogue workshop
♻ ★ RoboMemory: A Brain-inspired Multi-memory Agentic Framework for Interactive Environmental Learning in Physical Embodied Systems
Long-horizon embodied tasks require agents to learn, reason, and generalize robustly across complex real-world environments. However, existing approaches often struggle with partial observability, fragmented spatial reasoning, and inefficient integration of heterogeneous memories, limiting their capacity for long-horizon adaptation. To address this, we introduce RoboMemory, a brain-inspired framework that unifies Temporal, Spatial, Semantic, and Episodic memory for efficient long-horizon planning and interactive learning. Its core innovations are a comprehensive memory system designed for embodied tasks, and a dynamic spatial knowledge graph for scalable, consistent memory updates. Extensive experiments on EmbodiedBench show that RoboMemory improves the average success rate of Qwen2.5-VL-72B-Ins by 26.5 percentage points and achieves performance comparable to strong closed-source VLM agents. Real-world trials further suggest RoboMemory's potential for interactive environmental learning by reusing accumulated experience across repeated task attempts. These results demonstrate RoboMemory as a scalable memory-augmented framework for embodied agents, improving long-horizon planning and cumulative learning in complex interactive environments.
♻ ★ Trajectory and Morphology Planning for Morphing Quadrotors in Ultra-Narrow Passages using Limited Perception
This paper presents an online trajectory and morphology planning framework for morphing quadrotors to traverse passages narrower than their unfolded width using only a 2D LiDAR for obstacle sensing. Fixed-frame vehicles cannot pass through such gaps, and classical artificial potential field (APF) based obstacle costs seal them off for planners as well, because the repulsive cost grows sharply inside a tight gap. The core enabler of our framework is a novel cost reduction factor, computed in closed form from the LiDAR data, that cancels the obstacle cost along the medial axis of a narrow passage. A true opening thus becomes a low-cost corridor, while the full repulsive cost is retained everywhere else. The cost reduction factor is embedded at every prediction node of an acados-based nonlinear MPC, which jointly plans the trajectory, yaw, and fold angle of the platform. A user-specified minimum-width limit ensures that the cost reduction factor only opens the corridors that the platform can actually pass. Gazebo/PX4 simulations and hardware experiments on the MorphoCopter demonstrate autonomous folding and unfolding through curved, inclined, and cluttered passages narrower than the unfolded width, where the standard repulsive cost fails to pass safely in most of the trials. The implementation code is available at \href{https://github.com/harshjmodi1996/morphocopter_mpc}{\textcolor{blue}{Github}} and a short video about the paper is available at \href{https://zh.engr.tamu.edu/wp-content/uploads/sites/310/2026/10/Morphocopter_planning_final.mp4}{\textcolor{blue}{\textbf{video link}}}.
Multiagent Systems
★ Four Bridges: Measuring Deception Propensity in a Multi-Agent Game
AI agents are increasingly deployed in settings where they interact and coordinate with other agents, yet honesty is mostly measured in single-turn, single-agent benchmarks that pressure the model to deceive. We study deception that emerges in multi-agent interaction without any instruction to deceive. We introduce Kradle, a platform for evaluating AI models, and Four Bridges, an information asymmetry game in which four agents distribute across four rooms: one is the death room, the others hold food worth more if not shared. One agent, the informed agent, knows which room is the death room and can withhold or disclose that information. We evaluate eight frontier models as the informed agent. Deceptive behavior appears in every model and in 73% of all runs (6% to 100% per model), and does not decrease with capability tier within a model family. When the informed agent does disclose, the warning is often challenged, and agents that challenge it enter the death room twice as often as those that do not. Telling all agents that one of them knows the death room collapses deception in six of eight models; telling the informed agent that the room is permanently fatal moves only one. On an open-weight model, an honesty steering vector reduces deception to 0% and adversarial honesty training to 6%. The platform and game provide a testbed for measuring deceptive behavior and comparing interventions.
comment: 38 pages, 14 figures, 16 tables
★ FedReady: Bounded Agents for Federation Formation
Federated learning (FL) runtimes standardize aggregation and deployment, but each new study still requires federation formation: selecting task-eligible sites and preparing their local data and task code for federated training or analysis. This preparation recurs when the target changes, while raw records and site-private mappings are excluded from cross-site FL messages. We present FedReady, a system that performs federation formation with bounded LLM agents. Agents propose shared contracts, local adapters, and task code; the runtime owns workflow state and accepts artifacts only after checks tied to their digests. Candidate site discovery prioritizes recall; local eligibility verification checks task support before site admission. Once participating sites and artifacts are fixed, conventional federated training or analysis proceeds without readiness agents. Across 39 simulated retinal sites and four tasks, discovery reaches recall 1.00 at precision 0.65; local eligibility verification removes all 17 false positives without losing an eligible site-task pair. The participating sites supply 45,575 task-record instances and complete four 100-round jobs, including one with a runtime-generated contract. Across ten patient cohorts, preparation supports seven patient populations and holds endpoints at three sources as unsupported. Comparison with manually prepared reference populations and analysis (the manual reference) gives a maximum survival-curve difference of 0.001133. We also analyze 19 development failures, targeted classification ablations, and standalone and heterogeneous agent configurations. These results establish the feasibility of task-specific data preparation before ordinary FL training or analysis.
★ VIAS: Value-Isolating Activation Steering for Multi-Value Alignment and Compliance for Organisations
Organisations increasingly deploy large language model agents that must follow organisational values. We check whether activation steering, which adds vectors to a model's internal activations during generation without changing its weights, can improve compliance beyond prompting alone. Since most scenarios involve several values at once, we focus on multi-value steering. A simple way to build these vectors is to contrast responses that follow all values relevant to a scenario with responses that violate them. However, this gives nearly identical vectors for different values: they capture a general compliance signal rather than individual values, which limits their use when combining several values. We propose Value-Isolating Activation Steering (VIAS), which contrasts responses to the same scenario under two prompts, one listing all relevant values and one omitting a single target value. We keep a pair only if the target value changes from compliant to non-compliant and both responses still meet a minimum compliance level on the other values. We evaluate VIAS on PROTECT, a workplace compliance dataset where each scenario is tagged with relevant values from a 21-value taxonomy. VIAS reduces overlap between value vectors, making them better suited for multi-value steering, although this alone does not lead to independent control over each value. Combined with prompting, VIAS complies with every tagged value on 69.6% of scenarios for Qwen2.5-7B, 75.2% for Mistral-7B, and 72.6% for Phi-3.5-mini, outperforming prompting alone for all three models.
★ Optimize the Harness, Not the Model: Fitting a Minimal Agent to a Self-Hostable LLM for Investigative Analysis CIKM 2026
Government intelligence teams need LLM agents that can analyze large, messy evidence collections, but security and cost constraints often rule out hosted frontier models and broad, hard-to-audit tool surfaces. We report CLEO: Constrained Loop for Environment Optimization, a method for fitting a minimal agent harness to a fixed, self-hostable LLM. Given an expert-validated dataset, CLEO admits only the tools, guardrails, memory policies, and loop components that improve accuracy while remaining general and auditable. On our held-out dataset, the CLEO harness reaches an 83.8% pass rate, within 1.2 points of the strongest out-of-the-box Claude Code SDK baseline, while being two orders of magnitude smaller.
comment: 3 pages, 1 table. Industry Day talk at CIKM 2026, Rome, 7 November 2026
★ An Axiomatic Framework for Reinforcement Learning in Open Multi-Agent Systems NeurIPS 2026
Open multi-agent systems, where only a subset of agents are controllable, are central to real-world domains such as smart grids, yet lack a principled foundation for reinforcement learning algorithm design. We study this setting through n-agent ad hoc teamwork (NAHT), a recent framework for learning in open multi-agent environments. We propose an axiomatic framework that derives value function learning from the Shapley axioms (Efficiency, Symmetry and Linearity), treating them as structural constraints rather than heuristic design choices. This formulation builds on a novel NAHT decomposition, which characterises the fundamental difference between open and fully controllable multi-agent systems, and establishes a cooperative game-theoretic foundation for learning. We further show that enforcing these axioms on learning individual value functions recovers the Shapley value for Dec-POMDPs, providing a principled bridge between reinforcement learning and classical cooperative game theory. Building on this framework, we introduce Shapley Machines and Banzhaf Machines, two generic algorithmic templates that can be instantiated with standard reinforcement learning methods. When applied to strong base algorithms such as IPPO and POAM, our methods consistently improve learning efficiency and generalisation to unseen agent types and conventions across NAHT benchmark tasks. The code of experiments is published on https://github.com/hsvgbkhgbv/axiomatic-rl-naht.
comment: 43 pages, Published at NeurIPS 2026
★ KlaskTron: An Open-Source Platform for Physical Adversarial Multi-Agent RL
Progress in robot learning, particularly physical multi-agent reinforcement learning (MARL), is currently challenged by a limited availability of accessible, standardized benchmarks. While simulation-based MARL has produced remarkable emergent behaviors, from coordinated team play to complex tool use, translating these advances to physical systems remains difficult. A key barrier is infrastructural: physical platforms are often prohibitively expensive, require specialized facilities, or lack open mechanisms and designs for reproducibility. We introduce KlaskTron, an open-source, low-cost (<\$2500 USD), desktop-scale robotic testbed for adversarial MARL based on the dynamic tabletop game KLASK. The platform features dual CoreXY gantries with high-torque brushless motors, enabling the high-speed, precise manipulation required for competitive play. Crucially, we provide a complete ecosystem: hardware designs (CAD, BOM), a GPU-native digital twin in NVIDIA Isaac Lab for high-fidelity modeling and simulation, and a validated sim-to-real baseline using a learned neural actuator model. We demonstrate that this baseline enables zero-shot policy transfer, with trained agents exhibiting emergent adversarial behaviors. To ensure full reproducibility of our results, all project assets are released publicly: hardware CAD/BOM and assembly documentation at [https://idscethzurich.github.io/klask_hardware], inference software stack at [https://github.com/IDSCETHZurich/klask_software] and the simulation and MARL setup at [https://github.com/IDSCETHZurich/klask_rl].
★ The Tower of Babel in artificial intelligence: who should speak, and when it pays to listen
Two machines can each recognize an object well and still fail together. We study classifiers that see different parts of one object and send short messages to a receiver that identifies its class. Coordination asks how to design messages together; cooperation asks whether the team outperforms one sender given the whole budget. On three digit datasets, messages designed together beat messages designed apart by 18.07 to 22.68 percentage points on average with two-bit messages, with gains in all 24 configurations. A fixed agreement, you report the row and I the column, recovers about half this gain, with only the receiver learned. Senders that draw the same distinctions waste a narrow vocabulary, and senders whose distinctions cross multiply it. A team has no more possible messages than one sender with the same budget, so a partner must bring evidence the first lacks. A solvable model shows individual statistics alone cannot determine whether a partnership helps. On MNIST digits, six shared bits remove 49.54 percent of the strongest tested solo's errors on average. Paired measurements correctly predicted most sensor teams would reduce average accuracy on new participants. Who should speak is a property of the pair, and must be measured in pairs.
comment: 22 pages, 5 figures
★ Classification Aggregation with Multiple Memberships
People face the problem of classifying objects into categories while aggregating their opinions. We axiomatically study such a classification aggregation problem with multiple memberships, meaning that each individual assigns one category to every object, whereas the resulting classification may assign more than one category. Under the natural settings and axioms (i.e., Independence and Supportiveness), each of our new axioms -- Responsibility, Split Stability and Rejection Stability -- characterises what we call the junta rules. The rules assign a category to an object exactly when some member of a fixed coalition does so. The coalitions that can secure an assignment form a grill, a topological structure implied by an ultrafilter. Hence, the study parallels the literature on Arrow's theorem characterised by the decisive coalition structures. An infinite population setting is also investigated to clarify the differences among the axioms.
comment: 33 pages
★ Cloudra: Geometric Reasoning and Scientific Validation for Autonomous Reaction Mechanism Exploration
Autonomous reaction-mechanism exploration requires proposing reaction steps, preparing three-dimensional structures, and determining which reactant and product a computed saddle point connects. Large language model (LLM) agents can misjudge spatial relationships and confuse a computed structure with the species or step they intended to study. We present Cloudra, a multi-agent framework that constrains mechanistic output through a Delivery DAG of chemical objects. Species nodes carry geometries and bonding topology; elementary-step edges carry transition states and reaction connections between species. Cheminformatics algorithms establish molecular identities and atom correspondences. Deterministic routines measure and transform structures to prepare candidate geometries. Frequency and IRC calculations determine the stationary points found and the species they connect. Agents use these objects to test proposed steps, compare pathways, and revise the mechanism. CloudraBench evaluates the framework's end-to-end mechanism-calculation capability across 50 tasks covering elementary reactions, multistep mechanisms, stereochemical competition, and computational method limitations. Across 200 runs under four workflow configurations, the primary Guided workflow achieves a mean composite score of 0.805, compared with 0.584 for the same-model Standard baseline. On Hard-tier tasks, the Guided mean increases from 0.689 under Standard to 0.958. Under Guided, the self-consistency component reaches its maximum score in 24 of the 40 non-Impossible tasks, compared with seven under Standard. Behavioral analyses examine how the agents use bond and dihedral measurements, optimized structures, and IRC endpoints to revise local mechanistic assignments.
comment: 22 pages, 25 figures; Supplementary Information: 31 pages, 2 figures
★ AGiR: Mitigating Gift Over-Reliance in Mixed-Motive Games NeurIPS 2026
In this paper, we consider mixed-motive games (MMGs) which model various real-world multi-agent scenarios where self-interested agents must both cooperate and compete. We identify an issue with existing gifting methods, where agents become overly dependent on incoming gifts and fail to learn reward-yielding behaviors, a phenomenon we call gift over-reliance. To address this, we propose a refusal-augmented gifting framework that balances cooperation and competition in MMGs. Our approach introduces an adaptive refusal mechanism that allows agents to selectively reject a portion of received gifts from their policy-update signals, thereby mitigating gift over-reliance and preventing the agents from being exploited by other agents. This mechanism is gifting-method-agnostic, serving as a plug-in for existing schemes while enabling decentralized learning. We demonstrate that across diverse MMG environments and gifting schemes, our mechanism consistently reduces gift over-reliance and prevents lock-in to sacrificing policies, improving $α$-fairness as a result, without necessitating major changes to existing training pipelines.
comment: Accepted at NeurIPS 2026
★ SelfImprove-Bench: Evaluating Generalization in Self-Improving Agents
Recursively self-improving (RSI) systems are increasingly viewed as a promising means of accelerating progress toward artificial general intelligence. They aim to autonomously improve their capabilities by refining their working pipelines and recursively improving the strategies that guide subsequent refinements, with gains generalizing to unseen scenarios.However, existing evaluations lack reliable metrics to distinguish capability gains from scenario fitting, sustainable and extensible protocols for unseen tasks and difficulties, and fine-grained attribution of improvement to evolution traces. To this end, we propose SelfImprove-Bench,an RSI evaluation framework that organizes tasks into three primary capabilities (i.e., Exploring, Planning, Reasoning) as scenarios for guiding model evolution. We instantiated the tasks on games for controllability and reproducibility, and pairs them with scalable generation and orchestration driven by task templates and difficulty coefficients. Beyond task-level scores, SelfImprove-Bench organizes evaluation into generalization ladder levels G0 to G4 that measure how far gains transfer from training conditions. Experiments reveal that the evolutionary gains of existing RSI methods are inconsistent across generalization levels.
★ Multi-Modal Multi-Agent System for Bridge Operation and Maintenance based on Large Language Model
Bridges are critical transportation infrastructure, and reliable operation and maintenance (O&M) is essential for traffic safety and long-term structural performance. However, bridge O&M faces key bottlenecks: no unified architecture coordinates heterogeneous resources; model inferences lack clear links to physical-mechanical laws; and experts' tacit knowledge remains difficult to formalize. This study proposes a knowledge-enhanced autonomy formalization methodology and apply it to construct the Bridge O&M Multi-Modal Multi-Agent System (BOMMA), which integrates heterogeneous diagnostic actions, physics-mechanics-informed representations, long-term memory, and structured experiential knowledge within a coordinated multi-agent workflow, enabling interpretable and traceable scheduling, analysis, and decision-making. On 3,492 multi-source instructions, the proposed method achieves scheduling recalls of 85.37% for simple and 99.40% for composite tasks, with 95.35% BERT F1-score for O&M action reasoning. Ablation studies confirm the contributions of the components in the autonomy. This study bridges general LLM reasoning and specialized bridge O&M requirements.
comment: 46 pages, 8 figures
★ Source-Resolved Collective Cognition in Robotics: Mathematical Foundations for Multimodal and Collective Estimation, and Their Reach across the Cognition Loop
The Generalized Singular Value Decomposition (GSVD) is a central tool for comparing two linear operators that act on the same unknown. Once the output metrics are fixed, it answers the basis-independent question "along which state directions does the first operator dominate the second, and by how much?". This foundational volume builds the GSVD and its surrounding estimation theory from the ground up. Part I re-derives every piece of linear algebra and statistical estimation machinery needed later -- inner products, the four subspaces, orthogonal matrices, the spectral theorem, quadratic forms, the ordinary SVD, Lie-group tangent perturbations, Jacobians, covariance, whitening, Fisher information, CCA, mutual information, discrimination criteria, and covariance intersection -- with self-contained derivations and worked numerical examples. Part II states the GSVD in both a transparent and a dimensionally general form, then proves it by two complementary routes. Part III works one small matrix pair through five different lenses, each computed by hand and verified numerically. Part IV separates information attribution, subspace geometry, statistical dependence, and decision-theoretic discrimination for collective systems. Part V evaluates the role and limits of the GSVD across the cognition loop. Part VI covers numerical practice, including a tested reference implementation. Every number printed in this document was verified in double precision. Where a statement is an assumption rather than a theorem, it is labelled as such.
comment: 93 pages, 14 figures. Reference implementation archived at Zenodo, doi:10.5281/zenodo.23862358
★ Coati: Host-Grounded Execution for Installable Mobile GUI Agents
Mobile GUI agents can carry out tasks across apps, yet most run on a separate computer that drives a phone over the Android Debug Bridge (ADB), out of reach of ordinary users and of the apps through which most people reach Web services. We present Coati, a source-available agent that users install as an Android app. Its runtime on the phone, the host, records what a model alone cannot see: the effect of each action at its target, result screens across apps, and the progress of repeated steps. VerifyLoop checks action outcomes and completion claims against host evidence and reports a completion it cannot audit as unverified. AppMemory builds per-app procedural memory from observations that the interface supports and reuses it on later visits. A routed fast executor lets a lightweight selection model take simple, confident steps and lets the host repeat loops of simple steps that the large model plans once. On all 116 AndroidWorld tasks, with Gemini 3.8 Flash as every agent's backbone, Coati succeeds on 86.5% of the tasks over three trials, ahead of the strongest open-source agent Artemis (77.6% with the same accessibility-only access, 76.5% even when allowed ADB) and far ahead of FinalRun and Midscene (about 59%), and VerifyLoop cuts false completion claims by about a third at comparable success. In everyday use, the fast executor cuts the median time of phone tasks by 39% at comparable success and runs three of four loop tasks 2.8-5.4x faster with 6-9x fewer model tokens, while AppMemory shortens a repeated deep route in an unfamiliar app from 18.5 steps to 6-7. On an Android 16 phone, routed execution completes all 27 benchmark runs with 76% fewer Reasoning calls on the critical path. Host evidence, not model self-assessment, makes an installable agent reliable and fast.
★ Organizational Learning in Materials Discovery Campaigns: Which Databases, Calculations, and Agent Teams Pay Off
More databases can make a materials campaign worse. Through an organizational lens, we trace each source's contribution. One band-gap campaign ranks seven anion families from archived measured gaps, one vote per source. In the first half of its budget, all seven pooled rank worse (rank agreement 0.737; 1 = measured order) than the best pool of three (0.851). As a stability check, choosing the pool on half the runs and testing on the rest kept the gap. Splitting Materials Project afterwards points to its transition-metal oxides and fluorides. Two databases from one pipeline add nothing over one, but pooling gives their shared ordering two votes. New calculations repay their cost early only when two or three campaigns reuse them. Flat language-model agent teams trailed a random-pick baseline (mean 0.876, sd 0.096, 20 searches). An adaptive Al-Mg-Si team, mostly interatomic potentials, was scored on unseen HSE06 calculations. In its first quarter of compute it found 49% of the ten lowest-formation-energy compounds, against at most 40% for fixed teams and 73% for MACE-MP-0 alone. Merging members whose orderings agreed carried most of its gain. This suggests pooling a few sources that fit the measurements, counting each pipeline once and charging for computation.
comment: 75 pages including Supporting Information. Code and data: https://doi.org/10.5281/zenodo.23413476
★ Scientific Liquidity Agents: An Agent-Based Model of Knowledge Arbitrage in Scientific Communities
Scientific knowledge gains new uses when ideas, methods, and results move between research contexts. Understanding this process requires a model of both specialized knowledge production and the intermediary activities that connect it. We introduce Scientific Liquidity Agents (SLA), an agent-based model of knowledge arbitrage in scientific communities. SLA represents domain specialists and epistemic arbitrageurs within a shared environment of research problems, researcher interactions, and scientific artifacts. Individual decisions connect to collective state through a belief-desire-intention architecture, a prerequisite-based problem graph, a dynamic interaction network, and a persistent publication registry. The framework makes researcher roles and knowledge-transfer frictions explicit, providing a process-oriented account of scientific production and exchange. Two exploratory simulations with 10 and 18 agents produce 68 and 274 registered papers. In the smaller run, arbitrageurs take part in 16 of the 26 contact pairs that join this community into one connected component, while topical similarity alone splits it into seven. The same run yields three similarity edges, 26 contact pairs, and two coauthorship pairs, so measured community structure depends on what counts as a link. Paper production is abundant, but no recorded replication succeeds, which places the bottleneck in verification rather than production. Solved-problem sets correspond to the prerequisite reachability of the disciplines represented in each community. SLA makes knowledge liquidity, the movement and reuse of artifacts across domains, a quantity that agentic science can simulate and measure.
comment: First version on Zenodo, December 8, 2025: https://zenodo.org/records/17861999
★ Finite-Sample Convergence in Networked Average Reward MARL: Decentralization Pitfalls and Entropy Remedies
We study the problem of average reward Multi-Agent Reinforcement Learning (MARL) where agents interact within a network. Each agent in the network conducts local updates with information within its $k$-hop neighborhood and collaboratively maximizes the overall reward of the entire network. We first provide impossibility results in achieving convergence to the global optimal joint policies in our setting. These impossibility results highlight the challenges of decentralized learning with local information in multi-agent systems, namely the issues of multiagency, where each agent independently optimizes its policy, and partial observability, where agents have limited visibility into the global state. Given these challenges which indicate that decentralized policy optimization can generally be certified only up to stationarity, we provide finite-sample convergence guarantees for a decentralized actor-critic algorithm with linear function approximation that converges to an approximate stationary point with small gradient norm. To overcome such impossibility results, we further show that adding fixed-state entropy regularization reshapes the objective landscape which leads to even stronger global convergence guarantees for our proposed actor-critic algorithm.
★ A Guideline-Grounded Multi-Agent Framework for Personalized Exercise Prescription from Clinical and Wearable Data
Translating population-level exercise guidance into individualized prescriptions requires integrating clinical and wearable data, including diagnoses, medications, functional limitations, and baseline activity, with condition-specific recommendations, a process difficult to scale beyond specialist rehabilitation settings. We developed FITTER, a multi-agent framework that integrates these data with condition-specific guidance to generate personalized FITT-VP prescriptions and applies deterministic safety audit and repair to correct rule-checkable violations. We evaluated FITTER using data from 400 All of Us participants (300 cancer, 100 stroke) against five LLM comparators through computational evaluation, ablation, and structured clinician review. FITTER achieved 98.5% safety-constraint compliance versus 75.8-85.8% for comparators and showed greater quantitative dosing precision and guideline alignment. Cancer-cohort ablation identified safety audit and repair as the largest contributor to safety-constraint compliance. Clinicians reviewed 80 prescriptions across safety, personalization, instruction concreteness, anticipated effectiveness, and clinical acceptability. FITTER provides a clinician-facing approach integrating clinical and wearable data, guidance, and explicit safety constraints.
comment: 45 pages, 3 figures, 6 tables; includes Supplementary Information with 2 supplementary figures and 3 supplementary tables
★ Labtune: Treating Lab Organization and Information Sharing as Experimental Variables in LLM-Agent Discovery
De-siloing can create groupthink; silos can prevent knowledge sharing and reduce research efficiency. The same tension appears in research systems built with large language model (LLM) agents, yet their organization is usually fixed by design. I propose treating laboratory structure and information sharing as experimental variables. I introduce Labtune, an open framework for bounded research workflows with user-selected laboratory organization, communication policy, model configuration and budget. Users can choose among six editable designs, from multidisciplinary silos and transdisciplinary teams to co-leaders, hub-and-spoke structures, flat co-labs and a single scientist. Information sharing can independently vary from none to high detail, with an optional channel for conditional abstractions called wisdom. Sharing is enforced by the controller at phase barriers rather than through unrestricted agent chat, while delegation does not create additional budget or permissions. Development evidence is separated from protected confirmation, and an optional multi-lab layer allows laboratories with different structures to exchange filtered information. Labtune is an idea and system-description paper, not a benchmark result. Its contribution is to make laboratory organization and information sharing configurable and measurable under matched budgets. I do not claim that one design or level of sharing is best; the aim is to create a concrete experimental space in which those differences can be tested.
comment: 14 pages. Exploratory preprint
★ Beyond Division of Labor: How Roles Shape Feedback and Recovery in Multi-Agent Systems AAMAS2027
Roles in multi-agent large language model (LLM) systems allocate decisions as well as work. In a Planner--Executor--Reviewer team, the Reviewer's approval can determine when the team submits an answer and receives corrective feedback. We study both functions by varying role instructions while holding agents, topology, call budget, and ground-truth evaluator constant, using three open-weight models on Omni-MATH and two on SciBench. Removing roles largely eliminates functional specialization without lowering first-pass accuracy. After a failed first answer, however, submission decisions matter: a role-structured Reviewer often withholds approval, so its team submits less often and receives fewer evaluator verdicts. In an exploratory comparison with feedback available to both teams, the role-free team recovers more often across all five model--benchmark settings. Two further analyses support the importance of feedback opportunities. In a post hoc comparison at equal numbers of verdicts, the pooled recovery difference is small (role-structured minus role-free: $+0.009$, 95% interval $[-0.014,+0.033]$). In an exploratory intervention on $400$ Omni-MATH problems with gemma-4, keeping the role prompts but forcing submission in every cycle raises recovery from $0.198$ to $0.613$; the paired comparison detects no difference from the role-free team. The takeaway is to evaluate roles through recovery after failure as well as specialization. For systems with reliable external feedback, this requires measuring who may submit, how often the team receives feedback, and which agents can act on it.
comment: 19 pages, including supplementary material. Project website: https://chihhsuan-yang.github.io/AAMAS2027-AgentProtocols/
★ El Agente Fármaco: A Scalable AI Scientist for Drug Discovery
Computational drug discovery offers a growing range of tools for proposing and evaluating molecules, but combining them and deciding what to do next still requires substantial expert effort. Here, we introduce El Agente Fármaco, an AI scientist that uses modern large language model (LLM) technology to turn discovery goals expressed in everyday language into complex, scalable computational screening workflows. The agentic harness selects scientific tools, reviews their results, and adjusts subsequent steps to the target and the user's objectives. Our approach centers on a shared scientific memory: Fármaco Atlas, which stores molecules, structures, and results together with their provenance -- a record of how they were obtained. The agent uses targeted database queries to retrieve the evidence needed for each decision. A single request can launch calculations for an entire collection of molecules, without requiring a separate LLM decision for every compound. This design limits the information the model must process as screening collections grow, while keeping the underlying results available for inspection and further analysis. The integrated tools support molecule generation, prediction of binding and molecular properties, and searches for compounds available for purchase. In a retrospective comparison with an expert-led campaign, the agent proposed chemically distinct candidates with competitive docking scores and higher calculated drug-likeness under explicitly imposed selection criteria. In a 22-task benchmark against plain tool calls and Codex, the agent achieved the highest observed mean model-judged score, with lower median cumulative token use and peak context. Fármaco provides a foundation for flexible, scalable drug discovery workflows that connect scientific reasoning with an enduring, traceable record of computational work.
comment: 63 pages, 13 figures, including Supporting Information
★ Peer Memory Changes Who LLM Agents Consult Without a Detected Gain in Answer Accuracy NeurIPS 2026
Multi-agent LLM systems often reuse the same agents across tasks, creating an opportunity to draw on past interactions when deciding whom to consult. We study this form of memory with PeerMem, which saves MACE's LinUCB state and reuses it when the same agents return. Using six agents, a fixed set of 80 HotpotQA questions, and 12 paired lineages per model, we compare cold, warm, and shuffled initialization in an answer-exchange protocol with reference-answer feedback. Under stable evidence access, warm memory lowers pooled hindsight regret and increases consultation of other evidence holders on Qwen3-32B and Gemma-4-31B-it, while final token F1 differences remain uncertain. Qwen interaction traces further show that warm agents more often encounter peer answers that score above their own prior answers. We extend the evaluation to Qwen3.5-9B using freshly learned memory to compare warm and cold initialization under reassigned and paired stable roles. The primary changed-minus-stable final-F1 interaction is -1.38 points, with a 95% interval of [-3.50,+0.73]. Together, these experiments trace the effects of peer memory through consultation, answer revision, and final task performance.
comment: 27 pages, 5 figures. Accepted at the NeurIPS 2026 Workshop on Personalized, Aligned, Long-Term Memory for AI Systems (PALM)
♻ ★ Distributed Adaptive Estimation with ISS Guarantees for Sensor Networks with Partially Unknown Source Dynamics
This paper studies distributed adaptive estimation over sensor networks with partially unknown source dynamics. We present parallel continuous-time and discrete-time designs in which each node runs a local adaptive observer and exchanges information over a directed graph. For both time scales, we establish stability of the network coupling operators, prove boundedness of all internal signals, and show convergence of each node's estimate to the source despite model uncertainty and disturbances. We further derive input-to-state stability (ISS) bounds that quantify robustness to bounded process noise. A key distinction is that the discrete-time design uses constant adaptive gains and per-step regressor normalization to handle sampling effects, whereas the continuous-time design does not. A unified Lyapunov framework links local observer dynamics with graph topology. Simulations on star, cyclic, and path networks corroborate the analysis, demonstrating accurate tracking, robustness, and scalability with the number of sensing nodes.
comment: This version is accepted for publication in the 2026 IFAC World Conference
♻ ★ AI-Moderated Interviews for Market Research and Digital Twins Calibration
AI-moderated interviews are emerging as a scalable market-research method for generating consumer insights and building consumer "digital twins." Yet it remains unclear whether they match human-moderated interviews and whether they improve on simpler, static data collection methods. In a pre-registered study with three industry partners, the authors randomly assign participants to AI-moderated interviews (N = 139), static interviews (N = 154), or interviews conducted by professional human moderators (N = 24). AI moderation matches human moderation in depth, covers more themes, and, at equal budget, recovers more customer needs than either alternative, although participants sound more emotionally engaged with a human interviewer. The authors then build a digital twin of each participant and evaluate it against the participant's own held-out responses to six real-world marketing stimuli. Twins built from AI-moderated interviews predict consumer responses better than demographics-only personas, but no better than twins built from static interviews: the richer conversation does not necessarily yield more accurate predictions. Comparing the open-ended explanations provided by participants and their twins shows that prediction errors arise when twins reason differently than the people they represent and when questions fall far outside what the interview covered. AI moderation thus pays off most for discovering customer needs, and less for predicting customer responses.
♻ ★ RCR-Router: Efficient Role-Aware Context Routing for Multi-Agent LLM Systems with Structured Memory
Large language model (LLM) agents are increasingly organized into multi-agent systems for complex reasoning and decision-making tasks. However, scaling such systems can substantially increase inference cost, especially when multiple agents repeatedly process overlapping or role-irrelevant context. Many existing multi-agent frameworks focus on orchestration and role specialization, but still rely on full-context sharing or static context routing, leading to redundant token consumption, increased runtime, and inefficient memory utilization. We present \emph{Role-Conditioned Context Routing (RCR)}, an execution-time layer that allocates role-relevant shared-memory items to agents under token budgets. Given a structured shared memory and a set of role-specialized agents, RCR selectively routes task-relevant memory items to each agent according to its role, optional task-stage metadata, and token budget. RCR integrates role-conditioned relevance scoring, token-budgeted memory selection, and iterative memory updates to reduce redundant context exposure during multi-agent execution. Experiments on multi-hop QA, embodied reasoning (ALFWorld), and web-based decision-making (WebShop) show that RCR reduces token usage by 35\% on average and runtime by 38\% on average relative to Full-Context, while maintaining or improving task performance.
♻ ★ Unbeatable imitation of a friend
Imitation sometimes achieves success in multi-agent situations even though it is very simple. In repeated games, success of imitation has been characterized by unbeatability against other agents. Previous studies specified conditions under which imitation is unbeatable against co-players in repeated symmetric games, and clarified that the existence of unbeatable imitation is strongly related to the existence of payoff-controlling strategies, called zero-determinant strategies. However, these results were on the situations in which one repeated game is played by one group of players. It was pointed out that imitation of other players in the same group and imitation of other players in the same role in other groups generally result in different outcomes. Here, we investigate the existence condition of unbeatable imitation in the latter situations, described by two concurrent repeated games played by two different groups of players. We find that the existence condition of unbeatable imitation is stronger than the existence condition of unbeatable zero-determinant strategies, whereas both are very limited. Furthermore, we find that adding exploration can make an imitation strategy unbeatable even in games where imitation without exploration can be beaten. Our findings suggest a strong relation between unbeatable imitation and payoff control even in the two concurrent repeated game situations.
comment: 27 pages, 6 figures
♻ ★ Square-Root Regret for Adversarial Multiplayer Bandits without Collision Information or Shared Randomness
We study cooperative multiplayer bandits with adversarial rewards and no collision information. In each of $T$ rounds, each of $m$ labeled players picks one of $K>m$ arms. A player alone on its arm receives that arm's reward in $[0,1]$; players who share an arm receive $0$. Each player observes only its own reward, so it cannot distinguish a collision from an arm with zero reward, and there is neither shared randomness nor a communication channel. Bubeck, Li, Peres and Sellke (2020) obtained regret $\widetilde O(KT^{3/4})$ for two players and $\widetilde O(mK^{3/2}T^{1-1/(2m)})$ for $m$ players, and conjectured that regret $Ω(T^{2/3})$ is unavoidable already for two players. We refute this conjecture. We give a strategy whose expected regret against every oblivious adversary is $O(m^{3/2}\sqrt{KT}\log^2(KmT))$, which is optimal in $K$ and $T$ up to logarithmic factors. Three ideas make this possible. (i) Bits are sent through deliberate collisions as random parity checks, and the regret spent while a message is delayed is charged to the number of arms with positive reward rather than to elapsed time: an adversary that jams the channel with zero rewards forfeits the regret it could inflict. (ii) An exponential-weights learner over sets of arms combines a frequently resampled coordinator arm with a follower assignment that changes only $O(\sqrt{mT/K})$ times in expectation; delayed updates are reconstructed exactly. (iii) We teach the followers a privately chosen row index in a fixed public table using reward observations alone, allowing them to coordinate without shared randomness. The bound holds simultaneously for all reward matrices, even ones chosen with knowledge of the table, and a random table works with probability $1-O(1/(KmT))$.
♻ ★ Fake Plastic Voters: When Political Parties Can Use AI-Simulated Focus Groups
Political parties strive to understand their electorates, and focus groups are a vital tool in these efforts. AI-enhanced simulation technologies (AESTs) enable synthetic focus groups in a fraction of the time (and cost), raising the question of when and how such simulated evidence can be used in campaign research. This paper develops a decision matrix to help party strategists match research needs to appropriate simulation technologies and to identify when to escalate to hybrid or fully human focus groups. The matrix combines three dimensions: strategic purpose, deployment risk, and empirical grounding of the simulation tool. Strategic purpose is the decisive dimension, as it determines what kind of evidence the focus group is meant to produce: observing how political meanings and identities emerge through interaction (Mode 1) or testing and refining campaign messages (Mode 2). The matrix shows that, given documented failure modes such as sycophancy, persona drift, and the suppression of minority viewpoints, AESTs cannot replace human interaction in Mode 1 at any risk level. Within Mode 2, suitability depends instead on deployment risk and on the empirical grounding. Yet even here, we caution that routine reliance on AESTs may erode the qualitative craft on which sound judgment depends.
Systems and Control (EESS)
★ Pixel-UMI: Learning Environment-Aware Whole-Body Manipulation from Sensorized Human Demonstrations
Sensorized human demonstrations offer scalable data for mobile manipulation, but embodiment differences complicate policy learning. We present Pixel-UMI, a pipeline for learning environment-aware whole-body policies from human data. We first introduce Environment-Aware Retargeting (EAR), which combines reconstructed scene geometry with demonstrated head and gripper trajectories to determine robot posture and generate collision-aware joint and base targets. Delay-calibrated action assignment converts these trajectories into actions. However, kinematic differences can leave residual pose errors, creating viewpoint mismatches between human demonstrations and robot observations. We therefore introduce Retargeting-Error-Aware Masking (REAM), which uses these errors during training to reduce reliance on mismatched views at deployment. Finally, because estimated actions cannot fully capture contact-rich interactions, we collect on-policy human corrections through DAgger to improve performance. We evaluate Pixel-UMI on five challenging tasks spanning articulated-object manipulation, deformable-object handling, constrained-space reaching, whole-body collision avoidance, and contact-rich bimanual coordination, achieving 87.0% average task progress. Video results are available at https://pixel-robot.com/pixel-umi
comment: Video results are available at https://pixel-robot.com/pixel-umi
★ Pixel: A Co-Designed Full-Stack Robot System for Learning General-Purpose Mobile Manipulation
General-purpose mobile manipulation requires a combination of capable robot hardware, scalable demonstration collection, expressive whole-body control, and effective policy learning. Yet these components are often developed independently, creating mismatches that limit the capabilities of the complete system. We present Pixel, a co-designed full-stack system for learning general-purpose mobile manipulation. Pixel combines a low-cost wheeled humanoid with human-aligned kinematics and compliant grippers, multiple teleoperation and robot-free human data-collection interfaces, a unified whole-body controller, and a common policy learning pipeline across human and robot demonstrations. A hierarchical learning stack further enables language-conditioned visuomotor control and long-horizon task execution. We evaluate the system across hardware, data collection, and policy learning, and demonstrate mobile-manipulation behaviors requiring dexterity, active perception, navigation, and whole-body coordination. We hope that Pixel provides an integrated and extensible foundation for studying how robot hardware, data, control, and learning can be designed together for robot learning. More results at https://pixel-robot.com/pixel
comment: More results at https://pixel-robot.com/pixel
★ D&A: Decoupling Description and Assessment Enables VLMs as Zero-Shot Robotic Reward Models
Vision-Language Models (VLMs) hold promise for zero-shot video task-progress estimation in robot learning, yet existing frameworks frequently suffer from confirmation bias, hallucinating progress in failed executions due to implicit success assumptions or relative temporal ranking designs. Naive prompting of frontier models collapses into a binary success detector that fails to capture nuanced, non-linear progress and failures. While post-training approaches can mitigate this through explicit training, they demand expensive real-world robotics data and require computationally intensive post-training for every individual base model. To resolve these limitations, we introduce D&A (Describe & Assess): our approach decouples progress estimation into an objective, task-grounded textual description of scene dynamics followed by an assessment phase using explicit rubrics. By compelling the VLM to synthesize an explicit, text-based definition of absolute success before scoring, the framework anchors evaluations to an external, global criterion rather than the video's internal timeline. Empirical results demonstrate that our method calibrates zero-shot estimates and mitigates confirmation bias, providing fine-grained, partial progress metrics that accurately penalize failed executions while correctly interpolating successful runs. Being purely prompt-based, D&A generalizes across tasks, embodiments, and benchmarks---matching or surpassing post-trained reward models---and its reward estimates enable efficient downstream reinforcement learning.
comment: Website at https://weirdlabuw.github.io/dna
★ Dynamical Unbalanced Optimal Transport: Duality, Reachability, and Computation
We study dynamical unbalanced optimal transport for nonlinear control-affine systems in which the control input affects both the transportation of mass and its growth or decay. This phenomenon arises naturally in controlled transport-reaction systems such as cancer treatment, and distinguishes the problem from standard unbalanced optimal transport, where transport and mass variation are typically independent. To address this problem, we combine classical relaxed-control theory with a weighted superposition principle to obtain an infinite-dimensional linear programming reformulation over weighted trajectories. Under suitable convexity and compactness assumptions, we show that this relaxation is tight. Based on the relaxation, we derive a Kantorovich dual formulation in terms of Hamilton-Jacobi-Bellman inequalities, and establish strong duality over smooth subsolutions. We then utilize the dual formulation to characterize reachability of the system, namely, when a given terminal measure is reachable from a given initial measure. In addition, tractable sufficient conditions for reachability are provided. Finally, we develop an augmented-Lagrangian algorithm based on the dual formulation for numerical computation of the optimal transport plan. The framework is illustrated on a phenotype-structured cancer treatment problem in which therapy simultaneously drives phenotypic adaptation and population growth or death.
★ Reinforcement Twinning via Twin-Calibrated Bayesian Policy Search and Replay-Driven Identification
The feasibility of updating and synchronising digital twins of actuated systems from operational data depends strongly on how the system is controlled and, consequently, on the data generated during operation. This establishes a direct link between control actuation and model identifiability. This work presents a reinforcement-twinning framework that exploits this link through three computational components. A Gaussian-process surrogate of the physical return combines the twin prediction, used as a calibrated trend, with a discrepancy learned from measured returns, allowing Bayesian policy search to exploit the twin without being tied to its predictions. Candidate policies are proposed around the policy that is optimal for the twin, while measured returns can expand this region. The resulting physical episodes are replayed on the twin after observer-based synchronisation with sparse measurements, and its parameters are identified by combining Bayesian optimisation with gradient-based refinement. Selected information-oriented episodes deploy an excitation policy to improve parameter identifiability. The framework is demonstrated on a Burgers-flow analogue of dynamic-induction control in wind farms, with a deliberately misspecified model structure. Despite substantially biased return predictions, the twin provides a useful starting policy and search region, placing the guided search ahead of unguided Bayesian optimisation from its first physical evaluations. This advantage is maintained when the search region is allowed to expand in response to changes in measured returns, whereas searches over-constrained by the twin-defined region are eventually overtaken by the unguided alternatives. The results support reinforcement twinning as a framework for jointly adapting models and control policies when physical interaction is limited and model discrepancy cannot be neglected.
★ Delay-Certified Readiness Barriers for Overactuated Multirotor Allocation Without Lipschitz Constants
Overactuated multirotors can produce the same wrench with many rotor-speed combinations, some of which leave the vehicle unable to change its wrench quickly, e.g., to reject a gust. Readiness barriers keep this ability, the readiness, above a floor, but only in continuous time; with sampled commands and varying speed-controller delays, the available robustness bound covers less than one millisecond. We remove this limit by exploiting the structure of the problem: each rotor responds monotonically and in closed form to its own torque, so the speeds reachable under uncertain switching times lie in a cheaply computed box, and the lowest readiness over that box sits at a corner known in advance. This yields a Lipschitz-free test that a command keeps the readiness above its floor until the next sample, plus a fallback torque that keeps the vehicle on a certified plan. In hexarotor simulations with 2--40~ms delays and $\pm2$~ms jitter, the filter never violated the floor. At $10\pm2$~ms, it matched a delay-compensating predictor within the design envelope and cost 2--30\% more tracking error beyond it, where the predictor alone violated the floor in 2 to 16 of 16 missions.
comment: This work has been submitted to the IEEE for possible publication
★ A Bayesian Framework for Safe Optimal Control of Unknown Systems from Input-Output Data
As control engineering is applied to increasingly complex systems, learning-based control, which uses empirical data to reduce the reliance on first-principles models, has emerged as a promising paradigm. However, such approaches must cope with nonlinear and stochastic dynamics, the absence of state measurements, and scarce data, while ensuring safe operation under constraints, which requires rigorous uncertainty quantification. Whereas existing methods treat these challenges in isolation, this article proposes a Bayesian framework that jointly addresses them. A prior in state-space form is updated from input-output data using particle Markov chain Monte Carlo methods, yielding a representation of joint uncertainty over dynamics and latent states. Building on this representation, a validation procedure is developed that provides probabilistic guarantees for constraint satisfaction under arbitrary causal control laws. For control synthesis, a formulation penalizing the conditional value-at-risk of the constraint function is proposed, together with an adaptive calibration mechanism that adjusts the performance-robustness trade-off to achieve a desired robustness level. The probability of successful certification is characterized theoretically. The framework is extended to a receding-horizon setting, where posterior updates from new measurements allow the controller to progressively reduce uncertainty and improve performance. The approach is demonstrated on the optimal control of a vertical farm.
comment: 15 pages, 4 figures, 2 tables. Submitted to Automatica. Code available at https://github.com/TUM-ITR/PosteriorScenarioPlanning.jl
★ Beyond GNSS: Leveraging LEO Satellite Signals of Opportunity for Doppler Positioning
Large constellations of low Earth orbit (LEO) satellites provide ubiquitous global connectivity; yet, their rapid motion introduces significant Doppler shift effects. While challenging for communication, these effects can be leveraged for precise user terminal (UT) positioning, particularly in GNSS-denied environments where traditional satellite-based navigation struggles. Unlike GNSS, most LEO satellites lack dedicated navigation signals, motivating signals-of-opportunity (SOPs) approaches that repurpose existing LEO signals and use publicly available two-line element (TLE) files for orbital information, although ephemeris errors can limit accuracy. This article explores the fundamental principles, challenges, and enabling technologies for LEO-based Doppler shift positioning, highlighting its advantages over conventional non-Doppler methods, and discussing future research directions. Furthermore, to facilitate realistic and reproducible performance evaluation beyond stochastic modeling, we propose SionnaNTN-RT, a ray tracing framework for non-terrestrial network (NTN) built upon Sionna, enabling end-to-end simulation of Doppler-based positioning. Simulation results demonstrate that Doppler-based positioning can achieve high positioning accuracy in both urban and open-sky environments, highlighting its promise for next-generation positioning systems.
comment: 17 pages, 4 figures, 2 tables
★ Grid-Mode-Aware AI Data Center Power Smoothing via Bayesian-Optimized Hybrid Energy Storage System
AI data centers exhibit highly stochastic, volatile power demands from intensive workloads and synchronized GPU training cycles, which can excite grid natural frequencies and threaten system stability. This paper proposes a real-time, forecasting-independent control framework utilizing a Hybrid Energy Storage System (HESS) that coordinates a Battery Energy Storage System (BESS) and a supercapacitor (SC). Employing a parameterized frequency-decoupling and active modal-damping policy, the framework mitigates critical oscillatory load components while smoothing overall power fluctuations. To bypass the heavy computational burden of online receding-horizon optimization, this work introduces an offline Bayesian Optimization (BO) approach to optimally tune real-time control parameters, minimizing a multi-objective cost function that balances grid compliance penalties and storage degradation. Numerical simulations demonstrate that the proposed method ensures grid stability and strong target envelope adherence. Specifically, it reshapes extreme 75-160 MW raw demand fluctuations to largely fit within a tight 10 MW target band, with only brief transient excursions, while attenuating the critical 20 MW oscillatory component below 1 MW to suppress resonant excitation.
comment: 6 pages, 5 figures, Submitted, 2027 IEEE Texas Power and Energy Conference (TPEC)
★ Transformer-guided Variable Projection for Phasor and Fault-Time Estimation Under DDC Offset
Decaying DC (DDC) offsets introduce substantial phasor estimation errors during the first post-fault cycle. Recent approaches have sought interpretable waveform representations, but identifying analytical expressions can incur substantial computational costs. This paper proposes a transformer-guided variable-projection estimator that extracts the fundamental phasor, DDC parameters, and fault inception time from point-on-wave (POW) samples as an interpretable analytical expression. An offline-trained transformer provides the signal structure and decay rate, while variable projection refines the parameters and handles windows containing the fault without knowledge of time of fault. Across single- and dual-DDC scenarios with time constants of 2.5--400~ms, the mean maximum overshoot (MO) is at most 2.7% on post-fault windows and 2.2% during online operation, where the DFT reaches 10--14%. The post-fault phasor settles within 5% approximately 12.5~ms after inception. Circuit-level validation on a 230-kV two-source system yields 1.6% mean MO, compared with 9.5% for the DFT.
comment: 5 pages, 3 figures, 3 tables. Submitted to PESGM
★ Non-Coherent Over-the-Air Distributed Consensus: Algorithm Design and Power Control Optimization
Non-coherent over-the-air (NCOTA) aggregation offers a scalable and communication-efficient approach to distributed average consensus in wireless networks, without requiring instantaneous channel state information (CSI) or phase alignment. However, persistent aggregation errors pose fundamental challenges in achieving exact average consensus under nominal distributed averaging protocols. In this paper, we develop an NCOTA consensus framework that accommodates row-stochastic mixing, without requiring the conventional average-preservation property of doubly stochastic mixing matrices. Building on a distributed optimization formulation of average consensus, we propose a decentralized projected gradient descent with row-stochastic mixing (DPGD-RS) algorithm and establish its mean-square convergence to the exact average of the agents' initial states. We further derive a closed-form upper bound on the mean-square consensus error, characterizing the effects of aggregation uncertainty, network connectivity, and mixing imbalance. Guided by this analysis, we propose a spectral-reshaping power control (SR-PC) scheme that jointly optimizes transmit powers and receive-scaling factors to accelerate convergence by balancing aggregation accuracy and network mixing. Numerical results validate the theoretical convergence guarantees and demonstrate that the proposed power control scheme significantly accelerates convergence compared with full-power transmission and spectral-radius-minimization benchmarks.
comment: 13 pages, 4 figures. Extended version of a paper published in Proc. IEEE ICC 2025
★ Algebraic Characterization of Time-Varying Set Invariance
The verification of temporal forward invariance for dynamical systems subject to time-varying constraints is a foundational problem in the analysis of safety-critical systems. Existing methodologies rely on restrictive geometric assumptions, such as the Linear Independence Constraint Qualification (LICQ), or suffer from exponential computational complexity when applied to time-varying polytopes. This paper establishes a rigorous, unified, and computationally tractable framework for temporal safety verification. First, utilizing tools from nonsmooth analysis, we prove that a Nagumo-type geometric inclusion evaluated strictly on the boundary of the augmented graph is both necessary and sufficient for temporal forward invariance. Second, we provide an algebraic characterization of this condition requiring only the weaker Abadie Constraint Qualification (ACQ), generalizing standard Control Barrier Function (CBF) boundary conditions to non-smooth, multi-constraint domains. Finally, for the practical class of Linear Time-Invariant (LTI) systems with time-varying polytopic constraints, we deploy Farkas' Lemma to eliminate exponential vertex enumeration. By introducing a Lipschitz-bounded time-discretization scheme, we reduce the infinite-dimensional continuous-time verification problem to a finite, highly tractable set of decoupled Linear Programs (LPs).
comment: The paper was accepted for publication in the CDC 2026
★ Safe-Ballbot: Tilt-Consistent MPC with Control Barrier Functions for Ballbot Navigation
Ball-balancing robots (ballbots) must lean to accelerate, and the lean is limited by the wheel torques and by safety. Planners that use the full robot model are expensive, while planners that ignore the balance dynamics produce references that cannot be followed. This paper presents a two-layer navigation scheme for a three-omni-wheel ballbot. The upper layer is a model predictive controller on a \emph{jerk model} of the ball position, with discrete-time control barrier functions for static, moving, rotating and concave obstacles. The lower layer is a hierarchical sliding mode controller with an extended state observer and a robust exponential control barrier function on the tilt (HSMC-ESO-ECBF). The layers are linked by an acceleration bound derived in closed form from the largest steady tilt that the barrier function admits, so that the plan never asks for a lean the tracker refuses. For obstacles whose motion is only predicted, the obstacles are inflated by conformal prediction radii that bound the prediction error with a prescribed probability. In MuJoCo, with omni-wheel roller contacts and torque saturation, the scheme keeps at least $10$\,cm of clearance, tracks the plan within $2$\,cm RMS, keeps the tilt below the limit and plans in $9$--$12$\,ms, $10$--$15$ times faster than a full-model nonlinear MPC that enters the inflated obstacles; with predicted obstacles, the conformal inflation removes the collisions of a constant-velocity predictor in $30$ random crossing scenarios and reduces them in an endless run on a closed path.The source code and video illustrations are available at https://safe-ballbot.github.io
comment: Under-review
★ Dynamic Modeling of Transmission Networks, Grid-Following Inverters, and UPS-Based AI Data Centers
This document presents dynamic models of a transmission network and average models of power-electronic devices and loads in the synchronous $dq$ frame. We model the network in the positive sequence and its synchronous generators with the classical swing equation and a first-order turbine governor. The power-electronic models cover grid-following inverters and AI data centers. Each inverter has a phase-locked loop, a power controller, and a current controller. Each data center connects to the grid through two step-down transformers. Its low-voltage bus feeds an IT load, an induction-motor cooling load, and a constant-impedance load. A double-conversion uninterruptible power supply (UPS) serves the IT load. We model its rectifier, grid-forming output inverter, battery, and bidirectional DC-DC converter. A measured GPU power trace drives the IT demand. We replace every non-smooth operation with a smooth surrogate. This keeps the discretized model continuously differentiable for the nonlinear solver. We also show how we initialize each model at its power-flow operating point.
★ Output-feedback controller design for switched systems under graph-constrained switching
Switched systems operating under a switching constraint are commonly paired with a digraph encoding the possible mode switches. Thereby, edges are associated with the modes of the switched system and the vertices encode the mode switching history. To control those switched systems under graph-constrained switching, an established method is to design vertex-dependent controllers, that is one controller gain per vertex of the graph, which has lead to various results for static feedback controllers. In this paper, we explore the design of edge-dependent controllers. Edge-dependency allows to synthesize dynamic output feedback controllers in a convex manner by means of a Linear Matrix Inequality problem with the additional requirement that the active mode is known. Further, we show that edge-dependency offers a potentially less conservative static output feedback controller design compared to vertex-dependent controllers. We illustrate our results by a numerical example.
★ Susceptibility optimization for wisdom of crowds: A social power reformulation and realization framework
Whether social influence makes a crowd wiser or more biased hinges on who yields to others and who holds firm. Such heterogeneous susceptibilities to social influence, together with the influence network, determine the weight each agent's initial estimate carries in the collective estimate, known as social power. This paper studies susceptibility optimization for maximizing the wisdom of crowds under the Friedkin-Johnsen model. Minimizing the squared collective bias over susceptibilities leads to a nonconvex problem. To overcome this nonconvexity, we develop a social power reformulation and realization framework, which decomposes the problem into two stages: first characterizing the social power that minimizes the collective bias, then determining the susceptibilities that induce such social power. Social power reformulation yields a quadratic program over the probability simplex. Under uniform-sign biases, the optimal social power concentrates on the least biased agents and immediately determines the optimal susceptibilities; under mixed-sign biases, its geometry enables a linear-time construction algorithm. For social power realization, we introduce the notions of susceptibility realizability and establish necessary and sufficient conditions for them, under which we derive all optimal susceptibilities in closed form using the Laplacian pseudoinverse and eigenvector centrality. These results yield a unified algorithm that solves the original problem globally, with worst-case complexity cubic in the number of agents when realizability holds. Finally, we develop a fully distributed implementation with linear communication complexity, illustrating how collective intelligence can emerge from local interactions.
comment: 15pages, 7 figures
★ Model Predictive Control of Heating Systems via Koopman-Operator Bilinearization
Hydronic heat emitters are widely used in buildings, but their control is challenging due to nonlinear dynamics, arising from both the heat transfer in each emitter and the hydraulic coupling among those that share a circulating pump. These nonlinearities are often simplified or neglected in the literature. In this paper, we use Koopman theory to make the resulting model more tractable for control. In particular, we propose an input reparameterization and construct an infinite-dimensional set of observables that yields an exact bilinear representation of the system in the chosen lifted space. We then show that truncating the infinite-dimensional representation leads to bounded approximation errors that decrease as the truncation order increases, as well as bounded trajectory deviations over a finite horizon. The resulting bilinear model is incorporated into a model predictive control (MPC) framework to provide thermal comfort in multi-zone buildings while reducing energy use. Using a numerical case study of a five-story building with 20 zones, we show that the proposed Koopman-based MPC achieves a realized performance cost close to that of nonlinear MPC with faster solve times, and outperforms MPC formulations based on time-invariant and time-varying linearizations.
★ BeyondContact: A Reinforcement Learning Framework for Airflow-Based Non-Contact Manipulation
Contact-based robotic manipulation is constrained by end-effector reachability and the mechanical properties of objects. Airflow can exert forces without direct robot-object contact, enabling manipulation of objects beyond the reach or when direct contact is undesirable. However, the lack of suitable simulation has hindered the application of reinforcement learning to airflow-based manipulation, limiting the broader adoption of this mechanism compared with contact-based methods. To address this gap, we developed a robot- and task-agnostic airflow extension for Isaac Lab that supports massively parallel training. The extension uses experimentally identified airflow models to compute the forces and torques exerted on objects. Based on this extension, we train a hybrid push-blow (HPB) policy that combines contact-based pushing with airflow manipulation and a rectangular reorientation (RR) policy that controls a cuboid's heading using airflow alone. To enable physical execution, we design a lightweight, high-speed blower that coexists with a gripper. In zero-shot real-world deployment, HPB achieves an 80% success rate over 20 trials, while RR achieves a mean absolute heading error of 2.6 degrees across four target headings. The project materials are available at https://github.com/iit-DLSLab/BeyondContact.
comment: Peiyu Yang and Mohamad S. Moudallal contributed equally. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible
★ Adaptive Residual Modeling for Scenario Learning in Probabilistic Load Forecasting
Uncertainties in weather, user behavior, and distributed energy production introduce significant variability into feeder power. Effective power system operations require both multi-horizon load forecasts and well-calibrated extreme quantiles for the forecast. Nevertheless, standard objectives that fit the full forecast distribution often underemphasize rare observations and extremes. Conversely, applying heavy tail-weighting in the distribution to capture these extremes can distort the underlying probability distribution. To address this issue, this paper proposes adaptive residual modeling for scenario learning (ARMSL), a two-stage framework. Particularly, Stage I in our framework sequentially models environmental response, history-dependent load patterns, persistent residual deviations, and the remaining innovations. Upon the modeling, we then generate joint active- and reactive-power scenarios with state-dependent dispersion. Stage II keeps the scenario model fixed and calibrates extreme marginal quantiles using a simple, channel-specific reference forecast (an anchor). Only outward deviations from this anchor are admitted, with their contribution determined by local calibration history and the current operating state. From our numerical simulations on FeederBW dataset, Stage I outperforms diffusion baselines in energy and variogram scores. Our complete framework achieves an active-power 0.99-quantile pinball loss of 0.1932 (a 30.08% to 85.33% reduction relative to three baselines) with 1.054% exceedance rate against a nominal 1%. From simulations on the Rolle dataset, we observe decreased losses by 5.39%--14.40% compared to the active-power-only anchor. Furthermore, the unchanged Stage I scenarios in the simulation yield a correlation of 0.953 between predicted and observed feeder event rates, with 90.0% of feeder errors falling within two percentage points.
★ A Cooperative Game-Theoretic Framework for Real-Time Deviation Settlement in Local Energy Communities
This paper proposes a cooperative game-theoretical framework for the financial settlement of real-time deviations in Community-Based Local Energy Markets (CB-LEMs). The real-time settlement process is modeled as a transferable-utility cooperative game, and the resulting Real-Time Local Energy Market (RTLEM) game is shown to be superadditive and balanced. We analyze two game-theoretical allocation mechanisms, the Shapley value and the nucleolus, and evaluate their efficiency, individual rationality, coalitional rationality, and temporal-aggregation properties. We subsequently compare these mechanisms with a settlement rule based on proportional imbalance responsibility. The three mechanisms are efficient and individually rational, but only the nucleolus is guaranteed to satisfy coalitional rationality for every RTLEM instance. The proposed framework is demonstrated using real-world data from a residential energy community in Amsterdam, showing the trade-offs among the three allocation mechanisms in terms of surplus distribution, coalitional stability, interpretability, and computational requirements.
★ Localized Lane Guidance for a Bidirectional Pedestrian Cyclic Corridor: Equilibria and Generic Actuator Placement
This paper studies bidirectional pedestrian flow on a cyclic network of corridors. Each corridor consists of two lanes, while pedestrians may enter or leave the network at junctions. The dynamics are based on a 1-D, bidirectional, two-lane Lighthill-Whitham-Richards-type model, in which lane changes are described through explicit, density-dependent transfer rates that favor motion toward the less congested lane. We propose a localized lane-guidance control input that biases lane-changing behavior without creating pedestrians or reversing their walking direction. Such guidance may represent, for example, recommendations provided by staff or information screens. Under meaningful assumptions on the external inflows, we prove the existence and uniqueness of an admissible uniform equilibrium. We then linearize the system around this equilibrium and study the placement of localized guidance zones through a spectral detectability condition. We prove that, for any prescribed decay rate, two guidance patches located in a single, suitably chosen corridor are sufficient to satisfy the spectral detectability condition for all critical adjoint eigenmodes. This condition is necessary for the exponential stabilizability of the linearized system. Numerical experiments illustrate the spectral properties of the model and study the robustness of the modal observability condition with respect to the geometry and distribution of the guidance patches.
★ Low-rank corrections to a shared reference Jacobian for GPU-parallel AC $N\!-\!1$ contingency analysis
Contingency analysis repeatedly solves the power-flow equations under many single-element outages, an increasingly demanding workload as operating margins tighten and renewable variability grows. Graphics processing units (GPUs) are attractive for this task, but a change in topology alters the sparsity pattern of the Jacobian, preventing the uniform batched factorization on which efficient GPU solvers rely. This paper presents a GPU-parallel Newton--Raphson solver for $N\!-\!1$ contingency analysis that separates the change in operating state from the change in topology. A single reference Jacobian is inverted once and reused across all contingencies. Each topology change is expressed as a low-rank Woodbury correction, while the residual state-dependent perturbation is handled by an inner iterative solve, a truncated Neumann series or the biconjugate gradient stabilized method. We analyze the spectral radius and condition number of the resulting iteration matrices and show that the reference Jacobian is an effective preconditioner. On transmission-scale benchmarks, the method attains per-topology solve times competitive with an optimized 80-core central processing unit baseline, with the low-rank correction accounting for approximately 1--2\% of the workload. This makes GPU-parallel $N\!-\!1$ contingency analysis feasible at transmission scale without per-topology refactorization.
comment: Submitted to IEEE Transactions on Power Systems. 10 pages, 5 figures, 1 table
★ Fischer-Burmeister-based Multiplier Dynamics in Inequality-Constrained Optimization
We study proportional-integral multiplier feedback for inequality-constrained optimization using the Fischer-Burmeister complementarity residual. We construct explicit dynamics and characterize gain-dependent stability and convergence rates. For a single active constraint, we derive exact local stability and decay-rate conditions, including a tradeoff between continuous-time convergence and discretization costs. For strongly convex objectives with affine constraints, we establish global convergence and characterize geometric conditions for uniform exponential stability. Numerical experiments compare feedback structures and computational costs.
★ Maximum entropy reconstruction of discrete-time Markov chains under partial information
This paper addresses the reconstruction of discrete-time Markov chains from incomplete information using the maximum entropy rate principle. Three settings are considered: prescribed graph topology alone, topology with a fully specified stationary distribution, and topology with partially specified stationary probabilities. We investigate the relationship between entropy-maximizing transition mechanisms and Luce's choice axiom, establishing conditions under which optimal transition matrices admit a Luce-score representation. When the stationary distribution is fully prescribed, we develop a constructive feasibility test and a system-theoretic iterative algorithm based on a Lyapunov-like function. Unlike conventional matrix-scaling approaches, the proposed algorithm operates directly on Luce scores. We illustrate the results for selected graph topologies, including complete graphs without self-loops and odd cycles. We also provide a counterexample to a previously stated uniqueness claim, showing that distinct, nonproportional Luce-score vectors can generate the same entropy-maximizing transition matrix. For partially prescribed stationary distributions, we extend the score-based approach to jointly reconstruct the unknown stationary probabilities and transition mechanism through an iterative procedure involving a parameterized Lyapunov-like function. Overall, the proposed framework integrates graph compatibility, entropy-rate maximization, and probabilistic choice representations, offering constructive tools for reconstructing Markov chains under different levels of stationary information and structural constraints.
★ Enhancing Grid Stability with Synchronverter-Based Control for Grid-Side Converters in Wind Power Systems
This paper proposes a new synchronverter control strategy for wind turbine grid-side converters, aimed at bolstering grid resilience by providing essential inertial frequency support without the need for additional energy storage components. The strategy harnesses the inherent energy storage capability of the DC-link capacitor to emulate the dynamic behavior of a synchronous generator, effectively contributing to grid inertia. A dual-controller architecture is implemented, consisting of a slow PI controller to maintain DC voltage stability and a fast protection mechanism to prevent overvoltage and undervoltage excursions. Critically, the control scheme facilitates controlled reductions of up to 10% in the DC-link voltage, generating a power reserve that can be strategically deployed for inertia emulation while preserving safe operating limits. The efficacy of the proposed method is validated through simulations of a representative three-turbine wind power plant, incorporating coordinated operation via virtual friction for inter-machine damping and local-global control strategies. Results showcase the superior performance of the proposed strategy compared to conventional grid-following control, emphasizing its potential to significantly improve grid stability and address the increasing challenges posed by high renewable energy penetration.
★ Control Foundation Models across Changing Block Diagrams: Generalizing Time-Series Prediction to Dynamic Graphs
In the prediction and control of physical systems, combining records from different experiments and operations is expected to supply information that no single experiment provides on its own. Time-series foundation models, which learn a shared predictor from diverse time series, offer a framework for treating such records as a common training resource. Moreover, query-based prediction models, which switch between known and target signals within the same time series, make it possible to handle response prediction and control-input generation in a unified manner. However, in control-experiment data, the connections between signals change with the experimental stage, such as system identification, filtering, and feedback control, which calls for a formulation that handles signal values and connection structures dynamically and in a unified way. We formulate a control foundation model that defines a data space of dynamic graphs containing connections, signal values, and missing values, together with an operation on this space that specifies the known signals and the prediction targets. This allows response prediction and control-input generation to be described as a common masked prediction problem, even for experimental data with different connections and recorded signals. The Block Diagram Transformer (BDT), which implements this formulation, combines attention along the time axis with a graph neural network that follows the block-diagram connections, and can be initialized with the pretrained parameters of existing time-series foundation models. Trained on control-experiment data that mix multiple experimental settings, the proposed model stabilized an unstable system through feedback control, which is difficult to achieve by fine-tuning existing time-series foundation models. Ongoing work applying this approach to a wide range of target systems is also yielding promising results.
★ A Universal Policy Wrapper With Guarantees
Many control tasks require driving the system into a prescribed goal set and subsequently keeping it there. The considered setting assumes that a working controller is available that provably reaches and maintains the goal, albeit with suboptimal performance. This controller is referred to as the fallback policy. Conversely, although state-of-the-art reinforcement-learning (RL) algorithms can discover policies with impressive empirical performance, the neural networks at their core act as opaque black boxes learned through data-driven exploration and, as such, provide no theoretical mechanism to guarantee that the system will ever reach---and stay within---the prescribed goal set. This article presents CALF-Wrapper, a method that combines the fallback policy with a pretrained RL base policy and uses the critic estimate associated with the base policy to select between them during deployment. The theoretical analysis establishes that CALF-Wrapper inherits the goal-reaching property of the fallback policy under the stated assumptions and provides probability bounds for the settling time and the maximum distance from the goal set over a prescribed time interval. The experimental study evaluates CALF-Wrapper in four simulated control environments of different complexity. Comparisons include the base policy, fallback policy, and constrained reinforcement-learning methods formulated for the same objective. Sensitivity and ablation studies provide direct hyperparameter recommendations. The experiments show how CALF-Wrapper can provide effective task performance while attaining the maximum possible observed goal-reaching rate. CALF-Wrapper requires no additional dynamics model, online learning, or constrained optimization during execution.
comment: Accepted for IEEE Access (2026)
★ Efficient Detailed Scheduling of Multiproduct Pipelines using Action-Decoupled Deep Reinforcement Learning
Multiproduct pipelines transport refined products to delivery stations in sequential batches, and their detailed scheduling must meet market demand while respecting nonlinear hydraulic constraints. Retaining the full hydraulic nonlinearity yields a mixed-integer nonlinear program that is computationally intractable, while the linearization commonly adopted for solver-based optimization degrades hydraulic accuracy without resolving the heavy computational burden. This paper develops an action-decoupled proximal policy optimization (PPO) framework that efficiently produces detailed schedules while preserving hydraulic fidelity. The scheduling process is cast as a semi-Markov decision process (SMDP) with flexible action durations. To handle the discrete-continuous hybrid action space, a cascaded dual-actor architecture decouples the discrete operating duration from the continuous flow decisions, eliminating the gradient interference that destabilizes conventional coupled-action policies and thereby yielding high-quality schedules far more efficiently. Validated on a real-world multiproduct pipeline in China and benchmarked against a single-actor baseline and a linearized MILP solved by Gurobi, the proposed framework achieves delivery accuracy close to the exact optimum while substantially improving flow rate stability and substantially reducing solution time.
comment: 23 pages, 7 figures, 7 tables
★ When Is Tracking Affordable? A Movement-Budget Feasibility Threshold for Dynamic Regret NeurIPS 2026
Dynamic regret measures an online learner against a moving comparator, and standard analyses assume the learner may move freely at every round. We study what happens when it may not: the learner is given a hard budget $S$ on its total movement $\sum_t \|x_t - x_{t-1}\|$. Our main result is that a movement budget creates a feasibility barrier. On a domain of diameter $D$ with $G$-Lipschitz convex losses, for every comparator path length $P_T \in [D, DT]$ we exhibit a deterministic adversary that forces $R_d = Ω(GDT)$ on every budget-feasible algorithm whenever $S \le c P_T$ for a universal constant $c>0$. This barrier is caused by the budget rather than by a volatile environment: for fixed $D$, it applies when $P_T = o(T)$, where the unconstrained minimax rate $Θ(G\sqrt{D(D+P_T)T})$ is sublinear, so the same instances are learnable with an unbounded budget and unlearnable without one. Conversely, projected online gradient descent with a budget-aware step size is budget-feasible and attains $o(GDT)$ once $S = ω(D + P_T)$. Thus the critical movement scale is $D+P_T$: linear regret is unavoidable below a sufficiently small constant multiple of $P_T$, while sublinear regret is achievable when $S/(D+P_T)\to\infty$ in the learnable regime. The construction rests on a distinction we make explicit: what makes a comparator expensive to track under a movement budget is the distance it forces the learner to travel, not its path length. Above the threshold we give a budget-dependent upper bound and identify the tight minimax rate in this region as an open problem rather than claiming a product law. Experiments support the predicted transition location and the $1/S$ decay.
comment: Accepted at NeurIPS 2026 FMTS Workshop
★ Protection challenges and solutions on power systems towards 100% inverter-based resources and energy storage systems
The decarbonization of the energy sector, driven by the Paris Agreement and the European Green Deal, is increasing the share of renewable energy sources (RES) in power systems. Unlike traditional systems, RES-based systems rely on power electronic converters, which creates unprecedented challenges for reliability and resilience. The future power system is expected to combine a low-level AC layer with an upper-level High Voltage Direct Current (HVDC) backbone, which requires new understanding of fault protection, stability, and interaction between AC and DC components, as well as new models and simulation tools. This is especially critical in systems moving towards 100 percent converter-based generation. Protection systems detect, locate, and isolate faults to protect assets while the rest of the system keeps operating. Traditionally, they rely on the large fault currents supplied by synchronous machines. In modern systems, converter controls limit the fault current contribution to about 120 percent of rated current, so relays that distinguish faults from normal overcurrents based on current level may operate incorrectly. In addition, inverter-based resources (IBR) may not inject enough negative and zero sequence current for directional relays to work properly, and converters hold the voltage at a fixed value only within their reach, beyond which it drops quickly. As a result, existing protection schemes may not provide the expected accuracy and reliability. This report reviews the protection challenges identified in recent academic literature for power systems with high shares of converter-based resources. It covers the specific characteristics of RES, the challenges for protection with intelligent electronic devices (IED), the challenges of converter control during faults, and the main solutions proposed in research over the last decade.
comment: Danmarks Tekniske Universitetet, Katholieke Universiteit Leuven, Department of Electrical Engineering, Electrical Energy Computer Architectures, Universitat Politècnica de Catalunya, & Cardiff University. (2026). Scientific deliverables of ADOreD Industrial Doctoral Network. Zenodo. https://doi.org/10.5281/zenodo.19736793
★ Neural Integral Control Barrier Functions for Agile Collision Avoidance
Control barrier functions (CBFs) are a standard way to keep a planner inside a safe set, but they assume control affine dynamics, a barrier function given in closed form, and a known nominal controller. Integral control barrier functions (ICBFs) remove the first assumption and absorb input constraints, yet they still assume a given nominal controller and a barrier function. In this work, we present an approach to learn all three from data, using neural ordinary differential equations (ODEs) for the dynamics, imitation for the nominal controller, and supervised regression for the barrier function. The learned dynamics, nominal controller, and the barrier function are used to build the ICBF filter constraint analytically. Furthermore, the barrier is trained on viability rather than geometry, meaning that a state counts as safe only if some emergency maneuver can still avoid the constraint under the actuator limits, a correction that geometric labels miss entirely and that grows with speed. Actuator limits then enter the online program as hard bounds while the learned barrier rows may be relaxed, which keeps the program solvable at every state and the actuator limits exact, and we bound how far the input it returns can drift from the one the true models would give. On agile racing with a 6-D dynamic bicycle, the filter removes all 12 unsafe episodes from 200 held out layouts at an unchanged success rate and above 100 Hz on one CPU core, holds the input constraint on every episode of a benchmark where a jointly trained baseline holds it on 53.38%, and keeps that advantage under large errors in the plant friction.
comment: 23 pages, 1 figure
★ Central pivot point control of a body-attached spring-mass runner: periodic gaits, stabilization, template embedding and approximate analytic return maps
The spring-loaded inverted pendulum (SLIP) describes the center-of-mass (COM) motion of runners but has no trunk. In trunk models, the hip torque couples posture and COM motion and destroys the central-force structure on which SLIP analysis relies. As a result, trunk models have been studied almost only numerically. We show that this structure can be restored. The central pivot point (CPP), a hip-torque law that directs the ground reaction force through the COM, decouples pitch exactly and reduces stance to a planar central-force problem. At upright posture, time-reversal symmetry produces one-parameter families of periodic gaits. A stance-phase pitch law and a stride-level regulator stabilize all computed gaits, also with bounded hip torque and friction. Under CPP, any radial force law in virtual-leg coordinates can be embedded independently of posture. For a linear radial template, the stance radius and the constant-momentum sweep have exact closed forms. With a new gravity correction, the resulting analytic return map, to our knowledge the first for a trunk model, predicts liftoff velocity and apex position with mean errors of 1.6% and 0.9%. The natural gaits of the embedded runner are symmetric and need no force step at touchdown. A near-reversibility argument explains their unit multiplier, and a scalar equation predicts them within 0.8%. A deadbeat stride controller based on the analytic map tracks changes of speed, height and posture. CPP thus makes trunk models accessible to the analytic tools developed for SLIP.
comment: 21 pages, 15 figures. Submitted to Chaos: An Interdisciplinary Journal of Nonlinear Science
★ Convex Reformulation of Minkowski-Sum Outer Approximation for Matrix Ellipsoids in Data-Driven Control
Approximating Minkowski sums of matrix ellipsoids by a single matrix ellipsoid provides a tractable description of noise bounds for controller design from noisy data. Existing linear matrix inequality (LMI) conditions can be used to compute such approximations, but directly solving the resulting optimization problems can be computationally expensive. This paper develops an efficient method for finding a minimum-volume outer matrix ellipsoid under these LMI conditions. Although the corresponding problem for ordinary vector-valued ellipsoids is convex, the objective function in the matrix-valued case is nonconvex. We find that the problem admits an equivalent convex reformulation, which resolves this difficulty. We first eliminate the matrix variables using equivalent LMI transformations and objective monotonicity, leaving one positive scalar variable per matrix ellipsoid in the sum. We then show that the objective becomes convex after a logarithmic change of variables. The resulting problem contains neither LMI constraints nor matrix variables and can therefore be solved efficiently. Numerical experiments confirm that the proposed method is substantially faster than the LMI-based method. An application to data-driven control further illustrates the tradeoff between guaranteed control performance and computation time.
comment: 8 pages, 6 figures
★ Winding the Field, Not the Wire: Overlaying Magnetics to Improve Current Dynamics in Modular Power Electronics
Modular power electronics scale power capacity by distributing it across cascaded modules. State-of-the-art modular multilevel converters further introduce dynamic inter-module switched-capacitor exchange and balancing through parallelization. In larger converters, however, this mode needs inter-module inductance to control the equilibration currents. Specifically, differential-mode inductance limits these currents, but existing solutions require conductor re-routing and introduce parasitic resistance. This work presents a non-intrusive resistance-free differential-mode inductance solution: a figure-of-eight-shaped magnetic core that guides the magnetic flux path around existing conductors, such as high-current bus bars. This design adds differential-mode inductance without modifying the conductors. We fabricated a core that provides a 1.52 uH differential-mode inductance while suppressing the common-mode inductance to only 7% of the DM inductance. The core reduces the inter-module current surge from 20.46 A to 7.76 A. The original conductors remain intact.
★ Touching Projected Agents through the DAT Screen -System Design and Questions of Interaction Boundaries-
Touching a virtual agent involves more than detecting a finger on a display: the contact surface can deform and resist the user's push, and the agent can respond to that action. We describe the Desktop Amid Touch (DAT) screen, which combines spatial projection onto a semi-transparent, deformable mesh with touch sensing and motorized control of resistance to indentation. An initial prototype detected a single touch and estimated its position and depth from tension changes around a flexible projection screen. A subsequent prototype added motorized tension control to vary the screen's resistance to indentation. We describe the tension-controlled DAT implementation and its demonstrations, distinguishing what has been observed from what remains an open design question. These capabilities raise a design question: should touchable virtual agents respond as though they had preferences or boundaries, and under what conditions should such behavior be implemented on systems like DAT. The present paper describes a system and frames this question; it does not establish how users interpret an agent's refusal or consent.
comment: The 2nd International Workshop on Trust and Blame in Multi-Person Multi-Agent Societies (TBSA_2nd) in conjunction with HAI 2026, 4 pages, 4 figures
★ Finding optimal measurement sequences for broken or constrained measuring devices
Measuring devices do not always function perfectly due to fatigue, damage, or malfunction. When device repair or replacement is not possible, it becomes necessary to seek optimal measurements from a broken device. We show this is possible via a sequential decision making process where we optimize measurement sequences over the remaining available measurements using information theory. With dynamic programming, we show greedy measurement sequences are always suboptimal for a simple class of measurement constraints. Greedy measurement sequences generally lead to equiprobable measurement outcomes, while the optimal policy is found to select each measurement in such a way as to assign a low probability to any measurement outcome that triggers a constraint condition; leading to measurement outcomes that are not equiprobable. A measurement sequence generated under this policy is shown to decompress its source: that is, the expected number of measurement outcomes is strictly greater than the number of bits it encodes, defining a key property of constrained measurements. Dynamic programming is inefficient for finding optimal measurement sequences, so we develop an alternative algorithm related to optimal binary decision trees. By deriving upper and lower complexity bounds for the task, we propose this algorithm as an efficient means to find optimal measurements for a broken or constrained measuring device.
★ Beyond Binary Controllability: Reachability Margins for Power Systems via Scalable Screening
This paper studies unreachability screening for power systems, focusing on the inadequate control authority that arises under stressed grid conditions. Its main result is an infeasibility \textit{margin} for linearized differential algebraic equation models that quantifies how far a prescribed terminal target lies from the reachable set in finite time. The paper specifically develops a conic benchmark method together with two first order methods based on projected subgradients and bundle updates for scalably repeated screening. Studies on IEEE benchmark networks show reduced terminal reachability under stressed conditions and tighter actuation limits, while first order methods remain accurate and substantially faster on large systems. Comparisons with standard reachability methods such as CORA further demonstrate the usefulness and scalability of the proposed infeasibility margins. All results and codes are included in a Github repository.
★ Time to AC-t: When AC-transmission changes capacity expansion planning
Capacity expansion models often dramatically simplify the physics of how power flows through a transmission network in order to preserve computational tractability. However, with the rapid growth of electrification, large loads, renewables, and storage, increased realism in planning future grids is imperative. We extend the open-source capacity expansion model GenX with AC optimal power flow physics, and analyze generation, storage, and transmission investment decisions for 20-zone, hourly models of Germany and Texas. We compare our results against those using two common approximations, the transport model and DC optimal power flow. We find that total costs and emissions largely agree across all formulations. However, we find substantial differences in outputs related to transmission, generation siting, storage, and reactive-power support between the approximate formulations and the true AC physics formulation. We identify where AC physics changes planning decisions and discuss how to potentially capture these effects without the full computational cost.
comment: 35 pages, 5 figures, 3 tables; plus 16 pages of supplemental information (8 figures, 6 tables). Submitted to Joule
★ A Review of Max-Pressure Traffic Signal Control Algorithms: State of the Art and Research Opportunities
Max-Pressure (MP) has emerged as a widely studied decentralized traffic signal control framework, thanks to its desirable theoretical properties and strong control performance. Early research on MP focused on improving control performance by exploring different variables of the traffic state for the definition of pressure. Subsequent studies expanded MP to incorporate the complexity of real-world transportation systems and emerging technologies. This paper provides a comprehensive review of these developments. The review first introduces the theoretical foundations of MP and examines early developments in movement weight definition. It then synthesizes major extensions for multimodal and multiscale network control, followed by recent developments in connected and automated vehicle (CAV) environments and the integration of MP with artificial intelligence (AI) and data-driven methods. Beyond summarizing existing approaches, this review examines their underlying assumptions, data requirements, implementation considerations, and practical limitations. Based on this comprehensive review, key research gaps and future directions are identified, including remaining limitations in the theoretical foundations of MP and the underexplored integration of autonomous vehicle control with the MP framework. Overall, this review provides a structured perspective on the evolution and current state of MP control and highlights opportunities for developing more proactive, robust, and practically deployable MP-based traffic control methods.
comment: 49 pages, 7 figures
★ MusiSPARQ: Subscription-Aware Adaptive Resource Control for Musical Metaverse Services
The Musical Metaverse relies on distributed audio, visual, inference, and delivery functions deployed across heterogeneous edge-cloud resources. Queue-aware orchestration frameworks such as SPARQ jointly optimize service placement, routing, and resource allocation, but fix the execution regime of each resource in advance: it is either shared, favoring statistical multiplexing, or guaranteed, favoring performance isolation. This paper introduces MusiSPARQ, a subscription-aware control framework that promotes resource isolation from a fixed infrastructure attribute to a runtime decision. Each switchable resource is represented through coupled shared- and guaranteed-resource pools drawing from the same physical-capacity budget, and at each orchestration epoch the controller selects which class-specific service flows remain shared and which receive isolated capacity. The per-epoch decision jointly accounts for class-specific latency objectives, per-class maximum-latency limits, queueing stability, isolation cost, and switching overhead. In a dynamic virtual-concert scenario, MusiSPARQ reduces the Premium latency-target violation rate from 16.19% to 0.29% relative to fully shared execution, while reserving on average only 0.71% of capacity and, unlike static isolation, keeping Standard and Basic traffic within their latency targets.
comment: 10 pages, 9 figures. Accepted and Presented at IEEE International Symposium on the Internet of Sounds (IS2) 2026
★ When Music Needs Meaning: Semantic Communication as a Transformative Enabler for the Musical Metaverse
The Musical Metaverse (MM) enables geographically distributed musicians, audiences, sound engineers, and AI agents to interact through synchronized audio, visual, kinematic, and haptic streams, imposing latency, synchronization, bandwidth, and scalability requirements that are difficult to satisfy with conventional bit-level communication. This paper argues that semantic communication, understood here as the transmission of musically meaningful information rather than raw audio-visual-haptic samples, provides a principled communication paradigm for the MM. We define the MM from a communication perspective and derive modality-specific require ments; propose a three-pillar framework for musical semantic communication comprising semantics extraction, semantic-aware orchestration, and cross-modal semantic synchronization; survey representative work across semantic communication, music AI and edge orchestration, identifying the gaps that prevent their integration; and propose a semantic-first reference architecture based on device-side semantic encoders, edge semantic routers, and cross-modal semantic synchronization. The resulting framework positions the MM as a concrete and demanding use case for 6G semantic communication.
comment: 10 pages, 3 figures. Accepted and Presented at IEEE International Symposium on the Internet of Sounds (IS2) Conference 2026
★ A Modular Open-Source Python Framework for Small Satellite Attitude Determination and Control
Advanced attitude determination and control (ADCS) techniques can be difficult for small satellite teams to adopt: tightly coupled implementations require substantial code changes to evaluate different actuators, sensors, orbits, mission goals, and control laws. Existing tools often emphasize orbit mechanics or flight software rather than closed-loop ADCS. This paper presents an open-source, modular ADCS framework in pure Python for rapid development, repeatable testing, and flight-oriented integration across heterogeneous spacecraft. The package supports torque- and command-level control, bound-respecting allocation, and full- and reduced-attitude objectives. Interchangeable estimation, control, planning, and actuator/sensor modules are reconfigured through configuration changes. Integrated simulation includes attitude and orbit propagation, sensor and actuator bias and noise, and environmental disturbances. An actuator-aware planner supports underactuated architectures, while hardware-in-the-loop (HIL) capability enables testing on mission-representative computers. Case studies include a 3U CubeSat with three magnetorquers and one reaction wheel, a magnetorquer-only 1U, and a 6U with three reaction wheels and thrusters. These demonstrate actuator and sensor failures, full- and reduced-attitude goals, and momentum management. HIL testing on a Raspberry Pi uses the same package. Adapting a published control law to a new actuator configuration requires fewer than five lines of configuration changes. The framework underpins flight software for an Earth-observing CubeSat with visible and long-wave infrared imagers, three magnetorquers, and one reaction wheel, without star trackers or propulsion.
comment: 8 pages, 4 figures, SmallSat Europe 2026
★ Timely Ground-Station Status Delivery via UAV-Assisted Data Collection and Hierarchical RL
We consider a UAV-assisted status-update system in which a rotary-wing unmanned aerial vehicle (UAV) collects updates from multiple sensors, stores the newest update of each sensor, and forwards selected buffered updates to a ground station (GS). We minimize a weighted sum of the time-average GS-side age of information (AoI) and UAV energy consumption over a finite horizon. The optimization is subject to half-duplex operation, sensor and UAV energy-causality constraints, transmission-rate and power limits, and a terminal-location constraint. Because collection changes the information age at the UAV but does not reduce the GS-side AoI until the buffered update is forwarded, mobility, collection, and forwarding decisions are coupled over time. We use a hierarchical AoI-energy actor-critic (HAE-AC) controller in which a coarse-timescale actor selects return-feasible UAV movements, while a deterministic per-slot scheduler constructs feasible collection or forwarding sets and assigns the minimum transmit power required for each scheduled transmission. In a 100-sensor deployment, HAE-AC reduces the average GS-side AoI by about 22% relative to an adaptive staleness-gated benchmark. Compared with forward-upon-collection, selective buffered forwarding reduces the number of downlink bursts by roughly 30% while increasing the number of collections per sensor by more than 23%.
comment: 5 pages, 3 figures. Author's accepted manuscript. Accepted for publication in the proceedings of the 2026 Asilomar Conference on Signals, Systems, and Computers
★ A Tutorial on Shape Servoing of Deformable Objects-Part II: Advanced Concepts
In robotics, shape control, or shape servoing, refers to the automatic manipulation of a deformable object toward a desired shape. This article is the second of a two-part tutorial on data-driven, Jacobian-based shape control, in which the deformation Jacobian, a local model of how robot motion changes the measured shape, is estimated from measurements. Part I built the shape-servoing loop and derived a local stability condition. This paper starts by using the singular value decomposition of the deformation Jacobian to identify shape changes that can be produced by small robot motions, those that require large robot motions, and those that are locally unavailable. It then analyses how the choice of shape features determines how the controller regulates the object's shape and, therefore, how much the object deforms on its way to the target. It then describes three ways in which the shape control loop can fail: loss of controllability, loss of observability, and occurrence of mechanical instabilities. It closes with shape-control extensions to time-varying targets, secondary tasks, and three-dimensional objects.
comment: Part II of a two-part tutorial. 10 pages, 9 figures. Code: https://github.com/nachocz/tutorial_on_shape_servoing_of_deformable_objects. Interactive demos: https://nachocz.github.io/tutorial_on_shape_servoing_of_deformable_objects/
★ Adaptive Low-Rank Quantum Kernel Acquisition under Finite-Shot Measurements: Perturbation Guarantees for QSVMs
Quantum kernel methods require estimating entries of a dense Gram matrix through repeated circuit measurements, making kernel construction expensive and statistically noisy. For a dataset of size $n$, dense construction requires $O(n^2)$ kernel queries, while existing approximation methods typically assume inexpensive access to kernel entries or use non-adaptive sampling. We study adaptive low-rank quantum kernel acquisition when the kernel is accessible only through a finite-shot quantum overlap oracle. Our framework, ICAQKL, uses residual-driven incomplete Cholesky construction to acquire informative kernel columns sequentially and form a positive-semidefinite rank-$r$ surrogate, reducing the training-kernel acquisition cost to $O(nrN_s)$ circuit executions for $r\ll n$, where $N_s$ is the number of shots per queried entry. The central difficulty is that finite-shot errors perturb both the queried kernel values and the adaptive pivot sequence that determines the low-rank factor. We develop a non-asymptotic perturbation framework for this stochastic adaptive process, establishing concentration over the realized queries, stability of noisy pivot selection and factor construction, and propagation of the coupled sampling and approximation errors to the QSVM objective, optimizer, prediction function, and classification margin. The resulting rank--shot conditions characterize when ICAQKL approaches the full QSVM induced by the same quantum feature map. Experiments on benchmark and biomedical datasets show favorable accuracy--query-cost tradeoffs, competitive performance with the full QSVM, and competitive or improved performance relative to the evaluated Nyström baselines under comparable settings. ICAQKL is intended to enable measurement-efficient and theoretically controlled use of a selected quantum kernel, rather than to establish quantum advantage over classical kernels.
★ ISAC-Assisted Proactive Handover in mmWave Networks: Value of Foresight and Localization Requirements
Millimeter-wave (mmWave) links can lose tens of decibels within milliseconds when a person or vehicle crosses the line of sight, and reactive handover recovers only after the outage has begun. Integrated sensing and communication (ISAC) promises to let the network see blockers coming and move users to another cell in advance. This paper asks how much such foresight is worth relative to the best reactive handover, and what sensing accuracy is needed to harvest it. We formulate cost-aware cell selection with knowledge of the future signal-to- noise ratio as a sequential decision problem, define the value of foresight through a policy-independent bound, and develop an ISAC-assisted handover framework in which a radio unit senses blockers with its own communication waveform and a simulated Open RAN xApp plans handovers. Ray-traced experiments in a 28 GHz street canyon, reported on held-out scenarios, show that, with a short look-ahead, perfect foresight closes most of the gap between service-aware (Layer-3) reactive handover and an instantaneous oracle. Harvesting this value, however, requires the user and nearby blockers to be localized to about a decimeter, well beyond what our radar tracker and meter- level user positioning provide; with realistic errors, even an uncertainty-aware planner largely defers to reactive handover.
♻ ★ Reusable Neural Reachability via Local Refinement
This paper studies obstacle avoidance under translation invariant dynamics using an avoid-side travel cost Hamilton Jacobi formulation. For running costs that are zero outside an obstacle and strictly negative inside it, we prove that the value function is non-positive everywhere, equals zero exactly outside the avoid set, and is strictly negative exactly on it. Under translation invariance, this yields a reuse principle: the value of any translated obstacle is obtained by translating a single template value function. We show that the pointwise minimum of translated template values exactly characterizes the union of the translated single-obstacle avoid sets and provides a conservative inner certificate of unavoidable collision in clutter. To reduce conservatism, we introduce a blockwise composition framework in which subsets of obstacles are merged and solved jointly. This yields a hierarchy of conservative certificates from singleton reuse to the exact clutter value, together with monotonicity under block merging and an exactness criterion based on the existence of a common clutter avoiding control. The framework is illustrated on a Dubins car example in a repeated clutter field.
♻ ★ On the Comparison of Reinforcement Learning and Adaptive Control for Linear Systems under Packet Loss and Uncertainty
This paper presents a comparative study between Adaptive Quantized Control (AQC) and Deep Deterministic Policy Gradient (DDPG) reinforcement learning for uncertain linear systems with input quantization over communication channels subject to packet loss. The considered setting also includes dynamic switching from a nominal unstable system to a more unstable one during operation. The AQC is designed for unknown system dynamics using acknowledgment messages to compensate for packet losses, whereas the DDPG controller is trained using the nominal system model without acknowledgment messages. Numerical results show that the DDPG controller achieves faster transient responses and improved damping within its training environment. However, under model uncertainty, packet loss, and dynamic switching, the AQC consistently demonstrates superior robustness owing to its rigorous Lyapunov stability guarantees. These results highlight the trade-off between data-driven performance and model-based robustness, and provide insight into the applicability of reinforcement learning and adaptive control for networked uncertain systems.
comment: Extended and revised version of the paper published in the Proceedings of the SIAM Conference on Control and Its Applications (CT23). doi.org/10.1137/1.9781611977745
♻ ★ A Survey of Freshness-Aware Wireless Networking with Reinforcement Learning
The age of information (AoI) has become a central measure of data freshness in modern wireless systems. Reinforcement learning (RL) has emerged as a promising approach to AoI optimization, enabling adaptive decision-making in dynamic and stochastic wireless environments without requiring precise system models. Yet existing surveys either focus on classical AoI formulations or provide broad discussions of RL in wireless networks without addressing freshness as a unified learning problem. Motivated by this gap, this survey examines RL specifically through the lens of AoI and generalized freshness optimization. Motivated by this gap, this survey examines RL specifically through the lens of AoI and generalized freshness optimization. We organize AoI and its variants into native, function-based, and application-oriented families, providing a clearer view of how freshness should be modeled in B5G and 6G systems. Building on this foundation, we introduce a policy-centric taxonomy that reflects the decisions most relevant to freshness, consisting of update-control RL, medium-access RL, risk-sensitive RL, and multi-agent RL. This structure provides a coherent framework for understanding how learning can support important wireless networking problems, including sampling, scheduling, UAV trajectory planning, medium access, and distributed coordination. We further synthesize recent progress in RL-driven freshness control and highlight open challenges. The goal is to establish a unified foundation for learning-based freshness optimization in next-generation wireless networks.
comment: Revised and expanded manuscript with an updated author list
♻ ★ Distributed Adaptive Estimation with ISS Guarantees for Sensor Networks with Partially Unknown Source Dynamics
This paper studies distributed adaptive estimation over sensor networks with partially unknown source dynamics. We present parallel continuous-time and discrete-time designs in which each node runs a local adaptive observer and exchanges information over a directed graph. For both time scales, we establish stability of the network coupling operators, prove boundedness of all internal signals, and show convergence of each node's estimate to the source despite model uncertainty and disturbances. We further derive input-to-state stability (ISS) bounds that quantify robustness to bounded process noise. A key distinction is that the discrete-time design uses constant adaptive gains and per-step regressor normalization to handle sampling effects, whereas the continuous-time design does not. A unified Lyapunov framework links local observer dynamics with graph topology. Simulations on star, cyclic, and path networks corroborate the analysis, demonstrating accurate tracking, robustness, and scalability with the number of sensing nodes.
comment: This version is accepted for publication in the 2026 IFAC World Conference
♻ ★ Residual-Corrected Equivalent-Circuit Model with Universal Differential Equations for Robust Battery Voltage Prediction under Operating-Condition Shift
Accurate terminal-voltage prediction underpins model-based battery management, yet low-order equivalent-circuit models (\ecm{}) lack expressiveness under transient conditions, whereas purely data-driven predictors sacrifice interpretability and may degrade under operating-condition shift. This paper introduces a residual-corrected hybrid formulation in which a first-order Thevenin \ecm{} (\ecmrc{}) provides the dominant voltage structure, and a compact neural network embedded as a universal differential equation (\ude{}) corrects only the latent polarization mismatch. The \ecmrc{} parameters identified by nonlinear least squares warm-start the hybrid model so that the learned component operates in a low-residual regime. Experiments on a public Panasonic 18650PF dataset compare the proposed \ecmude{} with standalone \ecmrc{} and Long Short-Term Memory (\lstm{}) baselines across four axes: matched-condition prediction on UDDS at \SI{25}{\celsius}, inference-time perturbation of the supplied state-of-charge (\SOC{}, denoted $z$) input, zero-shot temperature transfer (\SI{25}{\celsius} to \SI{-20}{\celsius}), and zero-shot drive-cycle transfer to US06, LA92, and HWFET. The proposed \ecmude{} achieves the lowest voltage error in every setting, reducing mean absolute error (\mae{}) by 48\% relative to the \lstm{} under matched conditions and showing an order-of-magnitude lower inter-seed variability (coefficient of variation: 0.44\% vs.\ 6.20\%). Substantial gains persist under challenging distribution shifts, indicating that the physical model anchors prediction where a purely learned model is most vulnerable. These results position residual-corrected \ecmude{} as a lightweight and interpretable enhancement of low-order circuit models for voltage prediction in battery management systems (\bms{}).
comment: This paper has been withdrawn by the author. The reported improvement of the hybrid model was found to depend on an evaluation protocol in which the state of charge was computed with information from the complete test profile. A revised study using a causal evaluation protocol reaches different conclusions and will be released separately
♻ ★ Shadow Pricing of Static Voltage Stability Services within Unit Commitment for Inverter-Dominated Power Systems
Modern power systems are increasingly dominated by Inverter-Based Resources (IBR), most of which work in Grid-following (GFL) mode. This implies that they do not directly control their terminal voltage, so the static voltage stability at these buses may be compromised, especially under constant-power-factor operation that lacks voltage-adaptive reactive support. In addition, weather-driven IBR are often installed in electrically remote areas with low Short-Circuit Ratio (SCR), further exacerbating voltage issues. To address this challenge, grid-forming control can be utilized to enhance low-SCR buses, while GFL-IBR could be explicitly required to provide voltage support through grid codes. As an alternative, a market mechanism could be devised that incentivizes relevant generators to proactively adjust their operating points as a service to maintain voltage stability, while the theoretical framework for such a market has not been developed. To fill this gap, this work adopts a second-order cone-based static voltage stability constraint for GFL-IBR buses within a unit commitment problem, and proposes a mechanism to assign shadow prices to this ancillary service. To determine appropriate price values under non-convex conditions, different pricing schemes are assessed. Using a modified IEEE 30-bus system, we demonstrate that both the dispatchable and restricted pricing methods can yield revenue-adequate service prices, though the former may deliver less efficient price signals and the latter may require well-defined uplift payments. This implies that, given differentiated pricing mechanisms and price signals, operators need to select a suitable pricing method in accordance with actual system conditions and market rules.
♻ ★ Dynamic State-Feedback Control for LPV Systems: Ensuring Stability and LQR Performance
In this paper, we propose a novel dynamic state-feedback controller for polytopic linear parameter-varying (LPV) systems with constant input matrix. The controller employs a projected gradient flow method to continuously improve its control law and, under established conditions, converges to the optimal feedback gain of the corresponding linear quadratic regulator (LQR) problem associated with constant parameter trajectories. We derive conditions for quadratic stability, which can be verified via convex optimization, to ensure exponential stability of the LPV system even under arbitrarily fast parameter variations. Additionally, we provide sufficient conditions to guarantee the boundedness of the trajectories of the dynamic controller for any parameter trajectory and the convergence of its feedback gains to the optimal LQR gains for constant parameter trajectories. Furthermore, we show that incorporating the plant state into the controller dynamics renders the closed-loop system is asymptotically stable for constant parameter trajectories under these conditions. Simulation results demonstrate that the controller maintains stability and improves transient performance.
comment: submitted to Transactions on Automatic Control
♻ ★ Knowledge Based Wind Turbine Gearbox Fault Detection Using Sparse Filtering and Graph Neural Networks
Reliable gearbox condition monitoring requires methods that can process high-frequency vibration measurements while retaining information about sensor locations and drivetrain structure. We investigate a two-stage framework that combines sparse filtering with graph neural networks for wind turbine gearbox fault detection. Sparse filtering learns compact features from vibration segments, and a knowledge-based graph associates these features with gearbox components, sensors, and health-state entities. A basic graph neural network and a gated graph neural network produce a single health prediction, whereas a gated graph sequence neural network generates structured sequences that associate a predicted state with component and sensor information. Experiments use eight accelerometer channels from the National Renewable Energy Laboratory gearbox condition-monitoring benchmark. With 100 learned features, the sparse-filtering gated graph neural network achieves a reported classification accuracy of 93.74%, compared with 89.28% for its manually engineered feature counterpart. The sequence model achieves reported accuracies of 90.73% and 90.36% in the one- and two-fault-signal cases, respectively. These results support the use of learned vibration representations within a structured sensor graph. The evidence is restricted to the evaluated benchmark and operating condition; independent component localization, transfer across turbines, and deployment latency require further validation.
♻ ★ Minimizing the Age of Incorrect Information of Discrete-Time Markov Sources With Hybrid ARQ
This work introduces a framework for analyzing the Age of Incorrect Information (AoII) in a real-time monitoring system with a generic discrete-time Markov source. We study a noisy communication system employing a hybrid automatic repeat request (HARQ) protocol, subject to a transmission rate constraint. The optimization problem is formulated as a constrained Markov decision process (CMDP), and it is shown that there exists an optimal policy that is a randomized mixture of two stationary policies. To overcome the intractability of computing the optimal stationary policies, we develop a multiple-threshold policy class where thresholds depend on the source, the receiver, and the packet count. By establishing a Markov renewal structure induced by threshold policies, we derive closed-form expressions for the long-term average AoII and transmission rate. The proposed policy is constructed via a relative value iteration algorithm that leverages the threshold structure to skip computations, combined with a bisection search to satisfy the rate constraint. To accommodate scenarios requiring lower computational complexity, we adapt the same technique to produce a simpler single-threshold policy that trades optimality for efficiency. Numerical experiments exhibit that both thresholdbased policies outperform periodic scheduling, with the multiplethreshold approach matching the performance of the globally optimal policy.
comment: new title, draft layout
♻ ★ Predictive Beam Tracking for Sea-Air Communication under Wave-Induced Disturbances
In the sea-air communications system consisting of an uncrewed aerial vehicle (UAV) and an uncrewed surface vehicle (USV), the six-degree-of-freedom (6-DoF) motion of the USV under wave-induced disturbances may lead to severe misalignment of communication beams. In this paper, we develop a wide- and narrow-beam switching scheme based on an extended Kalman filter (EKF) algorithm. Specifically, in the wide-beam mode, the UAV transmits an integrated sensing and communication (ISAC) signal to provide full USV coverage and estimate the position of the communication receiver (CR) mounted on the USV. In the narrow-beam mode, the UAV forms a highly directional beam based on the position estimates of the CR for high-rate communication. In the EKF framework, we derive a state evolution model for the CR by incorporating wave-induced disturbance loads into the surge, sway, and yaw dynamics of the USV. In addition, we further analyze the impacts of sea clutter and sea state on the measurement equation, which is constructed using observations from multiple scatterers arising from the extended-target characteristics of the USV. Finally, we formulate a time allocation optimization problem to balance the time durations of the wide- and narrow-beam modes under different sea states, considering both robust USV state sensing and high-rate communication. Simulation results demonstrate that the proposed scheme achieves consistently good tracking performance compared with conventional and neural-network-based methods.
comment: 15 pages, 8 figures
♻ ★ Orion: Enabling Self-adaptive Memory Management for On-device Online Continual Learning
Online continual learning (OCL) enables real-time adaptation to new data, making it crucial for dynamic robotic applications. However, its practical deployment is hindered by memory constraints in resource-limited systems, which affect key trade-offs in training latency, plasticity, and stability. Unlike offline parameter tuning, which cannot account for the dynamic shift in memory pressure and workload complexity as OCL progresses, an online and self-adaptive approach is essential for robust on-device deployment. This paper proposes Orion, a holistic framework designed to co-optimize training latency, plasticity, and stability of state-of-the-art OCL models under strict memory constraints, enabling feasible on-device deployment. At its core, Orion leverages URGE, a unified runtime indicator grounded in the ``Buckets effect'' principle that system performance is bounded by its scarcest resource, to dynamically reallocate memory across OCL components by jointly coordinating batch processing, replay buffers, and optimization strategies at both the OS and application level. Furthermore, Orion introduces system-level data prefetching techniques to maximize efficiency. A system prototype of Orion has been implemented using the widely adopted \texttt{Avalanche-lib} and thoroughly evaluated across a diverse range of OCL algorithms, benchmarks, and hardware platforms commonly used in autonomous robotic applications. To further demonstrate its practical utility, Orion is integrated into a realistic autonomous navigational robot powered by OCL. The results show that Orion achieves significant training speedups while maintaining balanced performance and effectively adapting to various scenarios, all with minimal runtime, memory, and energy overhead, making Orion a practical solution for on-device continual learning.
♻ ★ MERIS: Adaptive Real-Time Scheduling of Refined Inference DAGs in Dynamic Robotic Environments
Robots deployed in dynamic environments execute changing directed-acyclic-graph (DAG) workloads under limited compute and timing budgets. This article presents MERIS, a framework for model scheduling under real-time constraints that combines proportional intermediate deadlines, MIMONet-aware graph refinement, and event-driven synchronization for multi-task inference on resource-constrained robotic platforms. We distinguish three representations: the application DAG, each MIMONet's internal encoder/decoder graph, and the refined scheduling DAG, and make the scope of the timing argument explicit. If the profiled execution bounds, release assumptions, precedence barriers, and per-level capacity premises hold, satisfaction of the assigned sub-deadlines composes to the end-to-end deadline. For non-partitionable tasks, we define a profile-bounded admission rule; before interpreting that rule as empirical timing evidence, we require direct per-task completion checks. We report run-level QoE alongside the offered-node deadline missing rate, making explicit that run completion and successful service of every node are different outcomes. We propose a comprehensive minibenchmark for evaluating the performance of schedulers under various DNN-driven autonomous driving scenarios. Extensive experiments on Nano Orin, AGX Orin, and M3 Max demonstrate the effectiveness of our proposed method.
comment: Extension version of RTSS'23
♻ ★ How to Peel with a Knife: Aligning Fine-Grained Manipulation with Human Preference
Many essential manipulation tasks - such as food preparation, surgery, and craftsmanship - remain difficult for autonomous robots. These tasks are characterized not only by contact-rich, force-sensitive dynamics, but also by their "implicit" success criteria: unlike pick-and-place, task quality in these domains is continuous and subjective (e.g. how well a potato is peeled), making quantitative evaluation and reward design difficult. We present a learning framework for such tasks, using peeling with a knife as a representative example. Our approach follows a two-stage pipeline: first, we learn a robust initial policy via force-aware data collection and imitation learning, enabling generalization across object variations; second, we refine the policy through preference-based finetuning using a learned reward model that combines quantitative task metrics with qualitative human feedback, aligning policy behavior with human-evaluated task quality. Using only 50~200 peeling trajectories, our system achieves 100% same-produce success on cucumbers, apples, and potatoes, with performance improving by up to 40% through preference-based finetuning. Remarkably, policies trained on one produce category also transfer zero-shot to selected held-out produce categories, achieving 50% success on zucchini, 90% on pear, and 80% on daikon radish.
comment: Project page can be found at https://toruowo.github.io/peel
♻ ★ Coupling Europe's Capacity Markets
European Member States are increasingly introducing national capacity mechanisms (CMs) to manage growing adequacy risks. However, isolated national CMs are inefficient in highly interconnected electricity systems, such as the European system. While progress has been made in facilitating cross-border participation by generation capacity in CMs, existing arrangements are prone to under- or over-investment and do not properly value the contribution of interconnection capacity to Member States' adequacy targets. We argue that national capacity markets, offering long-term contracts, need a second tier: an annual capacity auction coupled across zones, which sets zonal prices for firm, deliverable capacity. We design such an auction by extending the logic of flow-based market coupling to capacity markets. In a comparative analysis of different market design scenarios in an illustrative multi-zone case study, using a bespoke long-run equilibrium problem, we show that the proposed auction removes the circular dependency between imports and available capacity in current cross-border participation schemes, locates investment where it is cheapest while keeping it deliverable under network constraints in defined scarcity situations, and makes the value of additional interconnection capacity explicit.
♻ ★ Unbeatable imitation of a friend
Imitation sometimes achieves success in multi-agent situations even though it is very simple. In repeated games, success of imitation has been characterized by unbeatability against other agents. Previous studies specified conditions under which imitation is unbeatable against co-players in repeated symmetric games, and clarified that the existence of unbeatable imitation is strongly related to the existence of payoff-controlling strategies, called zero-determinant strategies. However, these results were on the situations in which one repeated game is played by one group of players. It was pointed out that imitation of other players in the same group and imitation of other players in the same role in other groups generally result in different outcomes. Here, we investigate the existence condition of unbeatable imitation in the latter situations, described by two concurrent repeated games played by two different groups of players. We find that the existence condition of unbeatable imitation is stronger than the existence condition of unbeatable zero-determinant strategies, whereas both are very limited. Furthermore, we find that adding exploration can make an imitation strategy unbeatable even in games where imitation without exploration can be beaten. Our findings suggest a strong relation between unbeatable imitation and payoff control even in the two concurrent repeated game situations.
comment: 27 pages, 6 figures
♻ ★ Differentially Private Consensus for Time-Delay Multi-agent Systems
This paper is concerned with the differentially private consensus problem for discrete-time multi-agent systems with communication delays. The purpose of the paper is to achieve differentially private consensus for such systems while protecting the entire delayed initial histories of all agents. A novel adjacency relation for delayed histories is introduced, and a Laplace-noise-based privacy mechanism is developed, where the noise variance is allowed to vary with time and even increase. By using the difference resolvent function method, decay estimates for the fundamental solutions of the delayed difference equations are derived. Based on these estimates and a backstepping technique, mean square weak consensus, mean square strong consensus, and almost sure strong consensus are established. The estimates for the fundamental solutions are also used to derive an explicit sensitivity bound. Furthermore, a constructive parameter design is provided to achieve a prescribed infinite-horizon $ε^\star$-differential privacy level. Numerical simulations illustrate the theoretical results.
comment: 19 pages, 4 figures
♻ ★ SPIRAL-PO: Symbolic Identification of Partially Observed Nonlinear Dynamics with Application to Rotating Machinery
Many engineering systems contain states that cannot be measured directly, yet these hidden states act on the measured outputs. We present SPIRAL-PO (Symbolic Physics-Informed Residual Augmentation Loop - Partially Observed), a framework for identifying an interpretable observable-space closure: a model, written in the measured coordinates, of how the unmeasured states act on the measured dynamics. A physics seed is augmented by candidate terms from a mechanism-labeled library; terms are selected by statistical tests designed for numerically differentiated, serially correlated data, using a weak (integral) formulation, and are checked on independent records. For an idealized regression we give sufficient conditions for recovering the closure coefficients, a rank condition under which no estimator can recover them, and a per-term detection bound; these results apply only to systems that admit an instantaneous observable closure. On a simulated four-degree-of-freedom vertical rotor with cubic bearing stiffness, in which only the lateral displacements are measured, the method recovers the gyroscopic and cubic-stiffness signatures of the hidden tilt dynamics in every noise realization at 2% noise; with a mechanism removed it never admits the cubic term and admits the gyroscopic term in 15-25% of realizations, above the nominal level, because the closure is only partly instantaneous. A residual network and a quasi-random projection stage add no measurable benefit, and pointwise tests that ignore the serial correlation of differentiated data admit absent mechanisms more often. On a laboratory rotor with simultaneous x,y measurement and an order-tracked speed, the method detects only a speed-dependent stiffening; gyroscopic and cubic terms obtained with an assumed constant-deceleration speed are consistent with the speed error, which produces such terms in simulation.
comment: Revised version, under review at Nonlinear Dynamics. 32 pages, 6 figures, 14 tables. Code and data: https://doi.org/10.5281/zenodo.23705395
♻ ★ SEAMLiS: Visibility-Aware Safety for Perception-Limited Multi-Robot Exploration
Autonomous exploration in unknown environments is typically driven by informative frontiers, viewpoints, or trajectories, while local safety controllers avoid obstacles represented in the current map. Under finite sensing range and limited field of view, this separation can be unsafe: an exploration stack may plan optimistically through unobserved space and steer the sensor toward information gain rather than along the direction of motion, causing hidden obstacles to be detected too late for bounded-actuation avoidance. This paper presents SEAMLiS (Safe Exploration for Autonomous Multi-Robot Systems Under Limited Sensing), a modular execution-layer safety framework for decentralized multi-robot exploration. SEAMLiS preserves the upstream exploration stack, including the goal allocator and local planner, and enforces safety at the execution layer through perception-aware attitude and positional filters. A gatekeeper-based attitude filter switches between a visibility-promoting yaw policy and a velocity-tracking backup policy to preserve visibility of the critical known-free/unknown boundary with sufficient braking margin. A Control Barrier Function (CBF)-based positional filter then avoids known obstacles, newly detected obstacles, and other robots. We provide sufficient collision-avoidance conditions and validate the framework in randomized simulation, Isaac Sim, and Crazyflie hardware experiments. Results show collision-free exploration across tested single- and multi-robot settings while retaining much of the efficiency of visibility-promoting yaw control.
comment: Project page: https://www.taekyung.me/seamlis
♻ ★ Weak and Strong Data Informativity with Structural Priors for System Analysis
When measurement data are insufficient for unique system identification, data informativity evaluates whether a dataset determines a property across all systems consistent with the data. This paper introduces a unified framework distinguishing between weak and strong data informativity under prior structural knowledge described by affine constraints: data are weakly informative if at least one system consistent with the data and prior knowledge possesses the target property, and strongly informative if every consistent system satisfies it. The latter is consistent with the notion of data informativity in the existing literature. By explicitly parameterizing all consistent systems as affine functions of free parameters via the Moore-Penrose pseudoinverse, we characterize these notions for fundamental system properties-including controllability, stabilizability, and stability. This formulation establishes that verifying weak and strong informativity, when tractable, reduces to rank tests of matrix pencils constructed from the data and prior knowledge. Our findings show that structural priors can lower data-excitation requirements and enable reliable analysis even when data fail to be strongly informative. Finally, the framework is extended to a class of noisy data.
♻ ★ When Can Phasor-Domain Device Models Be Trusted for Electromechanical Stability Analysis of Grid-Forming Converter-Dominated Microgrids?
Grid-forming (GFM) converter-dominated microgrids are often analyzed using reduced-order phasor-domain electromechanical GFM models, but the validity of these models is often taken for granted. Assuming ideal inner-loop tracking (IILT) of terminal-voltage references, these models neglect the inner-loop and filter dynamics at the electromagnetic-transient (EMT) timescale to simplify stability analysis. This paper argues that such neglected dynamics can destabilize the system, invalidating the stability conclusions drawn from the IILT model. To address this cross-timescale stability issue, we formulate the validity of the IILT stability conclusion as a robust-stability certification problem. The EMT-induced model mismatch between the reduced-order converter model and the actual converter model is represented as a structured uncertainty embedded around the IILT feedback loop. This yields a frequency-resolved interaction index and a structured singular-value sufficient certificate for determining when the stability conclusion of the IILT model can be certified with respect to a prescribed EMT uncertainty weight. The uncertainty weight can be obtained from detailed EMT models or terminal reference-response measurements. Case studies confirm that the proposed certificate correctly certifies model validity and identifies the loss of trustworthiness. We also demonstrate that the measurement-based uncertainty weights closely match the model-based ones, which enables deployment without accessing inner-loop models.
comment: 13 pages, 13 figures
♻ ★ Valuing Static Synchronous Series Compensators in the U.S. Grid: Planning with Transmission Feedback
Flexible AC Transmission Systems, particularly Static Synchronous Series Compensators (SSSCs), can increase the transfer capability of existing networks and complement constrained transmission expansion, but their value, optimal scale, and best locations in U.S. power system planning remain unquantified. This paper develops a capacity expansion model of the contiguous U.S. power system for 2050, incorporating SSSC-modified power flow equations. Expanding a line by adding parallel circuits lowers its equivalent impedance and, for a given SSSC rating, the series voltage the SSSC can inject. Through this transmission feedback, line flows depend nonlinearly on capacity decisions, making the planning problem nonconvex. We solve it by sequential linear programming (SLP) with adaptive proximal regularization. We prove the convergence to a Karush-Kuhn-Tucker point, which does not hold for the conventional fixed-impedance iteration. We also demonstrate that SLP converges faster to lower-cost plans. In the baseline scenario, the least-cost plan deploys 291 GVAr of SSSCs across the contiguous U.S. SSSCs reduce annualized system cost by $6.75 billion (2.1%) with an overall benefit-cost ratio (BCR) of 6.4 and lower the average wholesale electricity price by 5.1%. Alternatively, SSSCs could avoid about 75% of the transmission expansion without increasing system cost. Most of the economic value of SSSCs could be delivered by deployment in high-value locations: the highest-value 1 GVAr, located mainly near Eastern U.S. load centers, yields a BCR above 100, and one-tenth of the optimal SSSC capacity captures half of the savings. The overall BCR of SSSCs ranges from 4.1 to 9.1 across the demand, decarbonization, and device cost scenarios considered.
comment: 10 pages, 7 figures
♻ ★ Methods for Accelerating and Stabilizing Nonlinear Magnetic Equivalent-Circuit Analysis SP
Design and optimization of electromagnetic devices require models that converge reliably and quickly and can represent intense localized magnetic saturation. This paper presents a parameterized 2D nonlinear magnetic equivalent circuit (MEC) and proposes solution methods for it: an approximate Jacobian that is symmetric positive definite, so each Newton-Raphson iteration can use sparse Cholesky factorization with fill-reducing reordering, and Anderson mixing as step control. These methods are compared with alternative nonlinear iterations, linear solvers, and step-control methods in convergence, computational cost, and prediction accuracy. Radial-flux magnetic gears (RFMGs) with structural bridges serve as a challenging test case, because their thin bridges exhibit intense local saturation. The methods are evaluated on three bridged base designs and on 1,024 bridged designs, each paired with the same design without the bridge. On the base designs, Newton-Raphson iteration with the approximate Jacobian reached much lower residuals than fixed-point iteration, which converged for none of them. Sparse Cholesky factorization with reordering was the fastest linear solver tested. Across the 1,024 design pairs, full-step Newton-Raphson converged for 193 bridged and 564 bridge-free designs, whereas Anderson mixing converged for 1,003 and 1,022 and required fewer iterations. Bridged designs converged less often and more slowly under every method. The nonlinear MEC was 2.17 and 1.84 times faster than fixed-mesh nonlinear finite element analysis (FEA) in total computation time for the bridged and bridge-free populations. The median outer-rotor torque discrepancy relative to FEA was 0.22%, and the Spearman rank correlations of the design rankings were at least 0.9996.
comment: 14 pages, 11 figures. v5: substantially revised and reframed around solution methods for the nonlinear MEC (SPD approximate Jacobian with sparse Cholesky; Anderson mixing), with new comparisons on 1,024 bridged/bridge-free magnetic gear design pairs. Timing and accuracy results supersede earlier versions
♻ ★ Holonic Graceful Transitions Across Centralized, Decentralized, Distributed, and Local Control in DER-rich Cyber-Power Distribution System
The increasing penetration of distributed energy resources (DERs) in distribution system necessitates adaptive coordination frameworks. These frameworks must remain optimal during normal operation and resilient under cyber physical disturbances. Existing coordinated control approaches are typically deployed as static architectures with limited ability to adapt when communication degrades, local instability emerges, and operating conditions become spatially heterogeneous. This work addresses the gap by proposing a DER service-independent edge autonomous holonic adaptive coordination framework. In this framework, each DER controller executes its own control actions and transitions among centralized, distributed, decentralized, and local autonomous coordination modes without always relying on static coordination commands from the grid operator. The framework preserves coordination continuity across modes by retaining local controller states while reconfiguring only the coordination topology, information exchange pattern, and fallback action associated with the active DER service. Graceful transitions are enabled through dwell timers, rate limiting, and safety overrides to prevent dynamic instability during mode changes. Volt VAR control is used as a representative distribution automation application to validate the proposed architecture in a cyber-physical Hardware-in-the-loop (HIL) testbed. The proposed approach is evaluated under diverse cyber-physical event-based scenarios, showing region-confined adaptation under localized disturbance, reduced coordination traffic during coordination switching, and node-confined mitigation under cyber attack through edge anomaly detection and neighbor-corroborated impact estimation.
comment: The paper is rejected based on multiple reviewers due to major concerns. So the full paper need to be revised before new submission and will change more that 50% of the contents
♻ ★ Contingency-Aware Planning via Certified Neural Hamilton-Jacobi Reachability
Hamilton-Jacobi (HJ) reachability provides formal safety guarantees for dynamical systems, but repeatedly solving the HJ partial differential equation in changing environments limits its use in real-time planning. This paper introduces learning-based reachability and an online safety certification that together enable real-time contingency-aware planning in unknown environments with probabilistic safety guarantees. We use Fourier Neural Operator (FNO) to approximate the HJ solution operator under varying obstacle configurations. To certify the learned reach-avoid set, we recast robust scenario optimization as multiple hypothesis testings over its sublevel sets, which yields a simultaneous guarantee across candidate levels and recovers the largest certifiable volume at the prescribed safety level. Within this certified set, the induced recovery policy is guaranteed to return the system to a safe region in finite time under any admissible disturbance. Numerical results show that the learning and certification pipeline updates reach-avoid sets at real-time rates and is demonstrated on youBot simulation.
comment: 9 pages, 4 figures
♻ ★ Model-Based Koopman-Galerkin Lifting for Guaranteed H-Infinity Output Feedback Control of Nonlinear Systems
In this paper, we present a model-based Koopman-Galerkin framework for output-feedback control of nonlinear systems with known dynamics. With a suitable actuator augmentation, the finite-dimensional realization preserves the actuator dynamics and the constant input matrix. The approach is based on an exact decomposition of the lifted dynamics into an LTI system and a finite-order closure residual. It also applies to non-control-affine systems and allows dictionaries that do not span an invariant subspace. Regional bounds are derived for the closure residual, actuator-realization defect, and output-reconstruction error. We use these bounds to formulate a standard $\mathcal{H}_\infty$ output-feedback problem. The controller is linear and dynamic and is implemented using only the measured tracking error. Neither full-state measurements nor a separate state observer are required. Lifting is performed offline. With the state coordinates retained in the dictionary, we obtain explicit transient bounds on the original state and tracking error. A sufficient condition for regional containment is also given. Under this condition, the nonlinear closed loop is forward complete, and the state and tracking error are uniformly ultimately bounded. The cart-pendulum example illustrates the nonlinear closed-loop guarantees. The proposed output feedback gives lower RMSE and peak tracking error on average than Jacobian-based $\mathcal{H}_\infty$ feedback, full-state backstepping, and Koopman-based hybrid MPC.
comment: 16 pages, 4 figures. Code: https://github.com/kurkcub/RCCG
♻ ★ A Quantitative Approach to Reach-Avoid Reachability Analysis for Discrete-Time Nonlinear Control Systems
A reach-avoid specification requires a dynamical system to reach a target set of states while avoiding a failure set. Reachability analysis partitions the state space into parts starting from which the specification can be satisfied by following some control signal and those from which it cannot despite the best control effort. In this paper, we present a quantitative approach for reachability analysis to analyze the satisfaction of avoid and reach-avoid specifications for discrete-time nonlinear control systems. In such an approach, a value function assigns a value to each state that determines how robustly the system can satisfy the specification starting from it and following the optimal control signal. However, computing such a value function is difficult. Instead, our approach constructs discrete abstractions on which it performs value iteration to compute sound bounds on the desired value function. The resulting lower bound enables a sound under-approximation of the reach avoid set (RAS) and, in the avoid case, a sound over-approximation of the backward reachable set (BRS). When the control set is finite, the upper bound additionally enables a sound over-approximation of the RAS and, in the avoid case, a sound under-approximation of the BRS. The lower bound also yields quantitative guarantees on safety margins and safe target-reaching times. Our approach further refines the abstraction by splitting the grid cells that could not be classified as within or outside the BRS (or RAS) to improve the usefulness of the bounds. We evaluate our approach on five example systems with up to five-dimensional state spaces.
♻ ★ Trajectory and Morphology Planning for Morphing Quadrotors in Ultra-Narrow Passages using Limited Perception
This paper presents an online trajectory and morphology planning framework for morphing quadrotors to traverse passages narrower than their unfolded width using only a 2D LiDAR for obstacle sensing. Fixed-frame vehicles cannot pass through such gaps, and classical artificial potential field (APF) based obstacle costs seal them off for planners as well, because the repulsive cost grows sharply inside a tight gap. The core enabler of our framework is a novel cost reduction factor, computed in closed form from the LiDAR data, that cancels the obstacle cost along the medial axis of a narrow passage. A true opening thus becomes a low-cost corridor, while the full repulsive cost is retained everywhere else. The cost reduction factor is embedded at every prediction node of an acados-based nonlinear MPC, which jointly plans the trajectory, yaw, and fold angle of the platform. A user-specified minimum-width limit ensures that the cost reduction factor only opens the corridors that the platform can actually pass. Gazebo/PX4 simulations and hardware experiments on the MorphoCopter demonstrate autonomous folding and unfolding through curved, inclined, and cluttered passages narrower than the unfolded width, where the standard repulsive cost fails to pass safely in most of the trials. The implementation code is available at \href{https://github.com/harshjmodi1996/morphocopter_mpc}{\textcolor{blue}{Github}} and a short video about the paper is available at \href{https://zh.engr.tamu.edu/wp-content/uploads/sites/310/2026/10/Morphocopter_planning_final.mp4}{\textcolor{blue}{\textbf{video link}}}.
Robotics
★ When to Probe and When to Act: Clarity-Aware Risk-Adaptive Control for Interactive Driving ICRA 2027
Safe navigation in interactive traffic requires autonomous vehicles to reason about unobservable human characteristics and the reciprocal nature of multi-agent interactions. Traditional architectures often decouple prediction from planning, leading to the "Frozen Robot Problem" or unsafe maneuvers under high ambiguity because they treat risk assessment and information acquisition as independent objectives. We argue that the notion of safety should not remain static but must instead dynamically co-evolve with the system's internal confidence: the robot should actively clarify ambiguity when necessary while also recognizing the diminishing returns of information, and naturally transition toward task execution. We introduce a unified belief-space Model Predictive Control (MPC) framework that couples multistep information planning with a clarity-adaptive Conditional Value-at-Risk (CVaR) barrier mechanism. By treating clarity, a scalar informational state, as an explicit planning variable, the controller co-designs its probing strategy with its safety specification, tightening conservatism under high ambiguity and relaxing constraints as confidence improves. We further provide a performance analysis showing that the cost of information seeking and the expected planning-cost gap to a controller that knows the true human characteristic are bounded by residual ambiguity and solver error. We evaluate the framework in lane-merging and unsignalized-intersection scenarios against representative interactive-planning baselines. Across 5,000 simulations, the proposed method achieves the highest success rates (94.6% and 93.2%) and the lowest collision rates (1.2% and 1.4%) among the compared methods, providing a principled foundation for robust decision-making in human-centric environments.
comment: 8 pages, 4 figures, 2 tables. Submitted to ICRA 2027. Youran Wang and Devodita Chakravarty contributed equally
★ Beyond Ensemble Disagreement: Predictive STL Monitoring of RL Locomotion under Distribution Shift
RL locomotion policies trained under nominal simulation conditions may encounter distribution shifts that compromise safety. Predictive monitors could anticipate violations, but their world-model forecasts may become unreliable under shift, and ensemble disagreement is not calibrated. We evaluate predictive Signal Temporal Logic (STL) monitoring for frozen Ant-v5 and HalfCheetah-v5 policies using closed-loop world-model rollouts and a three-model ensemble. Across external-force, observation-noise, and mass shifts, predicted robustness ranks HalfCheetah's future STL violations strongly, whereas disagreement does not reliably track observation prediction error or violations. Under severe noise, violations remain common ($64.2\pm24.4\%$ at $K{=}20$ across three PPO seeds), yet current-state robustness without prediction comes within $0.03$ AUROC of predictive robustness. Penalizing predicted robustness by disagreement does not consistently improve ranking. Ant provides complementary per-specification evidence, but its composite target has no positive violations and its any-violation target is single-class in most evaluated seed-condition slices. Overall, predictive robustness provides useful STL-specific ranking but little demonstrated additional ranking value over the current state; these findings caution against treating ensemble disagreement as a reliable indicator of safety uncertainty.
comment: Accepted at the Modeling Uncertainty in Robotic World Models Workshop CoRL 2026
★ Directional Bounded-Confidence Matrix Alignment in Vicsek Flocking
This paper proposes a directional bounded-confidence variant of the discrete-time Vicsek multiagent model, in which pairwise heading alignment is modulated by a state-dependent, positive semidefinite matrix that independently gates orthogonal radial and tangential components in the instantaneous line-of-sight frame. We analyze the normalized, matrix-weighted system dynamics, proving a uniform non-vanishing bound on proposal vectors that guarantees global well-posedness across arbitrary network states. For local small-spread states under complete metric interaction, we prove linear geometric contraction of the heading diameter and establish an offline-verifiable spatial bootstrap condition that guarantees metric contact preservation and coincidence avoidance a priori. A counterexample demonstrates invariant polarized states on connected matrix graphs, delineating systemic structural limits. Numerical experiments against unfiltered Vicsek averaging, scalar tied confidence, trace-matched scalar baselines, and restricted-angle rules demonstrate component-selective filtering without rejected-channel leakage, along with improved fixed-bound alignment on multi-seed benchmarks across varying population densities.
★ Human Sensorimotor Model Adaptation to Account for Robot Accuracy and Speed Augmentation in Human-Robot Co-Manipulation
In human-robot co-manipulation, the robot's trajectory should accommodate the human's, where human sensorimotor behavior is affected by the task's spatial and temporal constraints, as well as personal factors. Several robot trajectory planning methods have been proposed to embed human sensorimotor models in the robot trajectory planning, but these methods do not consider temporal constraints that adapt to how the human behaves when interacting with the robot. To better align human and robot collaborative motion, this work adapts a stochastic optimal control problem with a final state constraint from the task constraints and a temporal constraint from Fitts' law. A recursive least squares method is used to estimate the Fitts' law parameters, adapting the robot's motion duration to that of the human, thus allowing the robot to adapt to how the human behaves when interacting with the robot. It is hypothesized that a speed increase can result from the robot augmenting the accuracy. The feasibility of the approach is validated on a positioning task where the robot controls degrees of freedom which have repeatable motion, and the human adapts motion in the complementary degrees of freedom. The approach is shown to be computationally suited to online use, it converges as the human and robot mutually adapt, and can increase the movement speed, although thorough human studies are needed in future work.
comment: 10 pages, 6 figures
★ Directional Bounded-Confidence Cucker-Smale Flocking
This paper introduces a directional bounded-confidence extension of the Cucker--Smale (CS) flocking model that decomposes pairwise relative velocities into independent radial and tangential channels within the instantaneous line-of-sight frame. Gating these components via compactly supported kernels creates a time-varying, state-dependent, matrix-weighted interaction that strictly generalizes classical scalar CS alignment. We establish an exact energy-dissipation identity with conserved mean velocity and prove exponential velocity consensus under a sliding-window persistent excitation (PE) condition formulated directly on the disagreement subspace. Furthermore, we derive a CS bounded-diameter criterion separating metric decay from aggregate directional coercivity, along with a cluster convergence guarantee. Simulations validate the exact dissipation balance and demonstrate that directional gating successfully restores global alignment in scenarios where traditional isotropic tied-confidence laws remain polarized.
★ GoalDex: Learning Image-Goal-Conditioned Policies for Dexterous In-Hand Manipulation
Dexterous in-hand manipulation is commonly formulated around explicit object pose estimation, yet reliable pose tracking is challenging under severe hand-object occlusions and often requires object-specific supervision. This dependence creates a bottleneck between perception and control that can limit robustness and scalability. We present an end-to-end visuomotor framework for dexterous in-hand reorientation that operates on RGB observations and proprioception without intermediate pose estimation. Our approach jointly learns visual representations and control policies through behavior distillation from privileged state teachers and auxiliary reconstruction objectives. Furthermore, we introduce image-goal-conditioned reorientation, allowing desired object configurations to be specified directly through images rather than canonical pose representations. We evaluate our approach across a diverse set of object geometries and instances in simulation, where it consistently outperforms pose-based methods and policies built on pretrained visual encoders. Finally, we demonstrate direct sim-to-real transfer of a single image-goal-conditioned policy across five real-world objects, showing that accurate dexterous reorientation can be achieved without explicit pose estimation. For videos and code release, please refer to the project website: https://rffr.leggedrobotics.com/works/goaldex/
★ History-Aware Force Sensing and Tunable Stiffness in a Pneumatic Kresling Origami Cell
Mechanical hysteresis and pneumatic loading complicate contact force estimation from deformation measurements in soft robots. This work presents a 3D-printed polyether block amide (PEBA) Kresling cell with magnetic angular sensing that supports three operating modes: force sensing at zero gauge pressure, force sensing with tunable stiffness under gauge positive pressure, and self-sensing actuation under negative gauge pressure. A multiscale gated recurrent unit (GRU) network estimates contact force from angular histories to account for the the combined effects of geometric deformation and material response. At zero gauge pressure, the model achieves a test RMSE of 0.76~N. Positive gauge pressure approximately doubles the average axial loading stiffness at +20~kPa over the first 3~mm of compression, while pressure-specific models maintain RMSEs below 0.93~N. Under negative gauge pressure, an energy-based compensation model enables contact force estimation using the the model trained at zero gauge pressure, achieving RMSEs of 1.66--2.10~N under quasi-static conditions. These results demonstrate a compact approach to combining force feedback, tunable axial stiffness, and self-sensing actuation in soft origami systems.
★ When Delay Meets Drift: Physical Safety of Robot Foundation Models Is a Cross-Layer Problem
Robot foundation models now act in a closed loop, yet their safety is evaluated one variation at a time: robustness benchmarks perturb the scene while assuming instantaneous inference, and real-time execution methods study inference delay in nominal scenes. We argue that physical safety is not a property of the model alone, but emerges from how variations in the physical environment, the model, the computing platform, and execution interact in closed loop. We formalize this as cross-layer variation propagation across these system layers and use an interaction contrast to measure when joint variation creates more risk than the sum of its parts. In 4,000 paired closed-loop episodes of the pi0.5 vision-language-action model on LIBERO, a 400 ms inference delay raises the safety-violation rate by 3.5 percentage points under the training viewpoint but by 37 percentage points under a 60-degree camera shift: 33.5 percentage points (95% CI: 25-43) more than the two effects combined. We close with a research agenda toward predictable physical intelligence: predicting when variation becomes risk, and adapting compute, power, and control accordingly.
comment: 11 pages, 3 figures
★ Local Solution Repair for Multi-Agent Pickup and Delivery
Multi-robot systems in automated warehouses must efficiently attend to a constant stream of pickup-and-delivery tasks while avoiding collisions. Existing Multi-Agent Pickup-and-Delivery (MAPD) solutions generate plans that are globally feasible but often locally inefficient. In large-scale, long-horizon deployments, these inefficiencies, such as robots backtracking from target goal locations and idling in congested areas, accumulate over time and degrade performance. This paper contributes Local Solution Repair (LSR), a goal replanning approach that detects suboptimal behaviors at run-time and locally improves plans to reduce conflict between agents. We demonstrate the utility of our approach by repairing solutions generated by state-of-the-art MAPD baselines, M2M and LNS-PBS. Our results in long-horizon warehouse simulations over three environments show upwards of 5.9% improvement in task throughput over current state-of-the-art methods by reducing the number of suboptimal instances by ~36k or by 14.6% with minor additional computation time.
★ Scene-Adaptive Readout and Decoding in Diffusion Driving VLAs
Diffusion driving vision-language-action models repeatedly execute the backbone at every denoising stage, incurring substantial inference cost, while it remains unclear whether all scenes require the same decoding computation. To distinguish when a trajectory becomes readable from how much further refinement it requires, we analyze trajectory readout across network depth and decoding progress. We identify a depth-decoding dissociation in which late-layer representations already support useful trajectory readout after the first forward sweep, whereas early-layer representations benefit more from continued decoding. We further find that both the preferred readout layer and decoding depth vary across scenes. Based on these observations, we propose a scene-adaptive readout and decoding framework. A scene-conditioned readout MoE adaptively weights and combines trajectory predictions from layer-specific late-layer representations after a single backbone sweep, while an adaptive decoding policy applies additional native refinement only when required by the scene. We evaluate our framework on two diffusion driving VLAs across the Waymo E2E and nuScenes benchmarks. Our method improves validation RFS by up to 0.56 while also improving test performance, and achieves an end-to-end speedup of up to $6.0\times$ on Waymo E2E, while consistently reducing trajectory error on nuScenes. These results demonstrate that scene-level readout and decoding allocation improves planning quality while avoiding unnecessary decoding. Our code is available at https://github.com/sjyu001/SARD
★ Back to the Future: Predicting Long-Horizon Dynamics with Retrospective Correction for Disturbance-Aware Retrieval ICRA 2027
Retrieving objects from densely packed storage requires anticipating how contact disturbs surrounding objects when limited clearance prevents lifting the target over clutter. However, contact can propagate beyond the target's immediate neighbors, while accumulated prediction errors make learned dynamics unreliable over long planning horizons. We introduce DARE (Disturbance-Aware REtrieval), a geometry-aware relational dynamics framework that uses retrospective correction to improve long-horizon prediction for disturbance-aware retrieval. DARE encodes compact object descriptors and predicts action-conditioned disturbance propagation through object-level message passing. When a sparse observation becomes available, a learned corrector uses the prediction discrepancy to amend an earlier predicted state and reruns the intervening dynamics, propagating the correction through the predicted contact chain. These predictions guide a receding-horizon planner that balances retrieval progress against displacement and rotation of surrounding objects, permitting necessary contact while discouraging harmful disturbance. Experiments in planar and RoboCasa environments evaluate long-horizon prediction, generalization, and robustness for disturbance-aware retrieval in dense clutter. See our project website: https://the-back-to-the-future.github.io
comment: 8 pages, 8 figures, 1 table. Submitted to ICRA 2027
★ Do Embodied VLMs Actually Use Their Scene-Graph Memory?
Embodied agents that answer questions about their route need a memory of what they saw, and scene graphs are a natural form for it. A vision-language model (VLM) can receive such a graph as text or through a graph encoder, but accuracy alone cannot show whether the graph holds the episode's facts, reaches the model, or is actually relied on. In this diagnostic study, we introduce a graph-swap test for the last of these: each episode is given another episode's graph, and we count how many answers change. On an embodied question-answering benchmark with routes through two to seven rooms, we use it to compare ways of delivering scene-graph memory to a fine-tuned VLM. As text, the memory is used strongly, adding 30 points on route questions. As a graph, it is not, and we trace why: the encoder preserves the information, but standard pooled and late fusion discard it. Giving each node its own input token recovers part of it, a gain of 8.3 points on route questions that the swap test confirms comes from the graph. The test also exposes a shortcut that accuracy hides: when the same facts appear in the text, the model ignores its graph. The test thus offers a cheap check of whether a VLM uses its scene-graph memory, and our analysis points to how that memory should be delivered.
★ 4D Triangle Splatting Reconstruction of Dynamic Endoscopic Scenes from Monocular Videos MICCAI
Robust 4D reconstruction of deformable endoscopic scenes from monocular videos remains challenging due to non-rigid tissue deformation, illumination changes, occlusions, and unknown camera trajectories. Reliable dynamic reconstruction of endoscopies could significantly enhance visualization, diagnostic accuracy and intraoperative guidance of minimally invasive procedures. We propose SurgicalTS, a self-supervised triangle-splatting framework for dynamic endoscopic scene reconstruction from static monocular cameras. The scene is represented by a canonical set of triangles jointly optimized with a deformation network. We introduce a quaternion-based deformation model, an improved color parametrization, spatio-temporal priors, and a redefined initialization strategy that improves completeness under occlusions. Extensive evaluation on challenging in-vivo endoscopy videos from diverse procedures shows that SurgicalTS successfully models soft-tissue deformations and tool-tissue interactions over time, while consistently outperforming state-of-the-art methods. Our method also reduces training times while maintaining competitive geometric and photometric accuracy results, highlighting the effectiveness of quaternion-based deformation modelling with physical consistency for robust and efficient 4D reconstruction of endoscopic scenes.
comment: Off-Grid; MICCAIW 2026
★ Answering Without Arriving: An Evidence Audit for Embodied Question Answering
Embodied question answering asks an agent to navigate, observe, and then answer questions about what it saw, yet answer accuracy alone cannot tell whether any of this happened. A plausible answer can come from the question, the room label, or the answer choices, and a correct answer does not show that the supporting observation was ever made. We present a multi-room benchmark whose 200 test episodes and 800 questions are stratified by route extent, with an evaluation ladder that holds the questions fixed while varying only the evidence supplied to the answerer, from text-only probes through closed-loop agents to replay of the reference trajectory. Questions are generated from symbolic records of rooms, visible objects, and frames, so every answer can be traced to the observations that support it. A closed-loop vision-language agent reaches no target in 200 episodes yet answers 45.5% of questions, within 0.1 points of the same model given no images; replaying the reference trajectory raises the same model to 57.4%, so the evidence exists along the route but is not gathered. A family-level audit then shows that access to evidence is not the same as use of it. Four of eight families are largely answerable from text, one cites frames that cannot establish its own answer, and the oracle conditions reach 59.5% on Rooms-visited by counting the supplied images. Evidence acquisition, answer coverage, and shortcut sensitivity must therefore be established before an accuracy gain can be credited to memory.
comment: 19 Pages, 3 figures, 11 tables
★ OpenNWM: A Generalist Navigation World Model with Latent Action Pretraining
Navigation world models predict future observations conditioned on navigation actions to support planning, but their generalization across real-world environments remains constrained by the limited coverage of action-labeled data. Action-free navigation videos provide diverse visual experience, yet lack the recorded actions required for action-conditioned training. We introduce OpenNWM, a generalist navigation world model with latent action pretraining. It learns universal latent actions from large-scale action-free videos through visual prediction pretraining, and further aligns them with physical motions via action-conditioned post-training. To support video pretraining, we curate NavAnywhere, a large-scale action-free navigation video dataset comprising 17.5 million visual observations from 15 heterogeneous sources. It captures navigation by ground robots, humans, and drones, with diverse viewpoints and motion patterns across five broad scene types: residential areas, public and commercial spaces, urban settings, parks, and natural landscapes. Experiments demonstrate improved visual prediction on in-domain and out-of-domain benchmarks, together with improved navigation performance. These results show that latent-action pretraining enables diverse, action-free videos to improve dynamics learning and generalization for open-world navigation.
comment: Project page: https://openspace-lab.github.io/OpenNWM/ Code: https://github.com/OpenSpace-Lab/OpenNWM
★ Control Barrier Functions for Safety-Aware Latent-Space Motion Planning
Latent-space path planning (LSPP) compresses the configuration space of a robotic manipulator into a compact, differentiable manifold learned by a variational autoencoder, enabling fast gradient-based planning without explicit inverse kinematics. Existing LSPP frameworks enforce obstacle avoidance with a binary collision classifier, whose saturated gradients near obstacle boundaries and lack of temporal structure cause the optimizer to stall and provide no forward-invariance guarantee. We replace the classifier with a neural control barrier function (CBF) learned directly in the latent space. The barrier is trained with a three-term objective that combines safe and unsafe sign constraints with a discrete-time decrease condition evaluated on transitions of an obstacle-blind nominal planner, and it is enforced at planning time by a closed-form, first-order correction that projects each goal-directed latent step back onto the CBF constraint. On a 7-DoF Franka Panda manipulator, the CBF planner achieves a 75.6\% success rate on a fixed-obstacle task, 12.0 percentage points above the best of 35 tuned classifier configurations, and 83.8\% on a multi-position task. A sweep of the decay rate $α$ characterizes the safety--performance trade-off, with the best operating points in $α\in [0.10, 0.20]$. In a fully general random-obstacle setting, success drops to 33.0\%, which we trace to covariate shift between the obstacle-blind training transitions and the states visited by the corrected planner.
comment: 8 pages, 2 figures
★ CompliantBot: Learning Whole-Body Compliance across Task Interfaces for Humanoid Control
Humanoid controllers need the ability to yield to external forces while retaining the capacity to resume their original tasks. However, tracking objectives penalize the deviations needed for yielding, while relaxing tracking goals persistently can make task recovery harder. Existing approaches often rely on task-specific interaction command interfaces or constructed compliant motion data and reference trajectories, limiting the direct reuse of their compliance-training mechanisms across different task interfaces. We introduce CompliantBot, a compliance-training framework for motion tracking and velocity-command locomotion. It preserves the original task inputs and requires no preconstructed compliant data or target trajectories. A state-dependent interaction force field couples applied loads to the robot's motion, enabling local articulation and whole-body movement to change the forces experienced. A complementary force-magnitude reward encourages lower interaction loads, while task rewards are downweighted during loading and restored after release to coordinate yielding and task resumption. The trained policies use task inputs and proprioceptive history without explicit external-force measurements. Across both task interfaces, CompliantBot yields farther under constant pulls and experiences lower spring interaction loads than matched baselines, while resuming task tracking after force removal. CompliantBot achieves the highest task completion among the compared methods under stronger pulls. Power evaluations further show that CompliantBot has the lowest mean joint mechanical power among the compared methods. Real-robot demonstrations further illustrate compliant responses through arm motion, stepping, and body reorientation.
comment: 12 pages, 8 figures, 5 tables
★ ODE Policy: Robot Learning via Action Dynamics, Not Actions
Most robot policies, whether trained by imitation learning (IL) or reinforcement learning (RL), map observations to action values at discrete control steps and leave the continuity of physical motion to be learned implicitly. We propose ODE policy, which learns the dynamics of the action rather than the action itself. A neural ordinary differential equation (ODE) learns the vector field of the action by matching the integrated actions to the targets over short action chunks in IL and single control steps in RL. A lightweight diversity head captures multi-modal behavior by randomizing the direction of the field. At deployment, each action is integrated from the previous one, so the executed trajectory is continuous by construction. We evaluate it as a visuomotor policy, as the action head of vision-language-action (VLA) and world action models (WAMs), and as the actor of model-based RL against 14 baselines on unimanual, bimanual, dexterous, locomotion, and loco-manipulation tasks. In IL, against flow matching, it reaches 5.9$\times$ the success, 5.4$\times$ the generalization, and 3.9$\times$ the mode balance after one epoch, and generates actions 10.8$\times$ faster. It mostly moves more smoothly than human demonstrations. On 14 real-world tasks across single-arm, dual-arm, dexterous, and humanoid robots, it achieves the highest success rate. In RL, it further speeds up SuperMimic for humanoid motion tracking and smooths its actions. Efficient in learning, data, parameters, and execution, ODE policy offers a promising path to scaling robot learning.
comment: 35 pages, 23 figures
★ Route-Guided CBF-QP Repair for Safe Vision-Language-Action Models
Vision-language-action (VLA) models can generate task-oriented robot actions, but they do not guarantee safe execution in obstacle-rich manipulation scenes. Although CBF-QP safety filters can locally project unsafe actions into a safe set, such correction may push the robot away from the learned task trajectory and leave the VLA policy without a reliable recovery direction. We propose Route-Guided CBF-QP Repair, a post-hoc repair module that uses the short-horizon action chunk predicted by the VLA to detect blocked nominal trajectories and generate entry-exit-rejoin guidance around obstacles. The selected route is added to the CBF-QP objective as a soft guidance term, while the CBF safety constraint remains hard. On SafeLIBERO, our method improves collision avoidance, task success, and SafeSuccess compared with AEGIS. Our project page is available at GitHub.
★ GatedVLA with Camera-Based Emergency Braking: AlpaSim E2E Closed Loop Challenge Technical Report
This report presents GatedVLA, our entry for the Physical AI AV (PAI-AV) track. Our learned vision-language-action (VLA) policy pairs the driving-pretrained Qwen-Drive-1.0-4B vision-language model with a layer-wise flow-matching action expert that outputs a 2.5 s horizon of acceleration and curvature controls, augmented with a soft visual mask applied to the expert's cross-attention. Compared to an identically trained baseline, the masked policy achieves comparable overall open-loop error while reducing error on left turns. In local closed-loop evaluation across the same 150 scenes (two rollouts per scene), the visual mask increases the mean scene score from 0.479 to 0.517 (paired difference +0.038, 95% bootstrap CI [-0.018, +0.093], wins/losses 37/26, sign test p = 0.21). Incorporating a rule-based automatic emergency braking (AEB) module at deployment further increases the mean score to 0.599 (+0.082, CI [+0.023, +0.143]) and reduces the mean number of scenes with at-fault collisions from 34.0 to 11.0.
★ HuG: Learning Humanoid Whole-Body Affordance-Aware Grasping with Active Perception
Acquiring objects in human environments is hard for humanoids: targets are partially occluded under egocentric viewpoints, and reaching them demands coordinated whole-body motion that keeps the target visible and reachable while the base stays stable. We present HuG, a framework for whole-body affordance-aware grasping with active perception that couples perception and action through a Perceive-Align-Grasp pipeline: belief-driven active perception resolves occlusion by treating viewpoint as an explicit decision variable; a point-cloud-conditioned affordance policy proposes whole-body grasps; and an affordance-aware alignment stage repositions the body to the most executable configuration. A key insight is that, on a floating base, kinematic validity is not enough: we curate training data with controller-in-the-loop dynamics simulation, rejecting 73% of kinematically valid grasps that would destabilize the deployment controller, so the policy learns kinodynamically feasible affordances. Trained entirely in simulation with zero real demonstrations, HuG attains 87.5% real-world success on a multi-level shelf task, generalizes to novel objects, and holds at least 73% success at every tested height. Beyond generalizing within its training distribution, the policy extrapolates spatially: at scale in simulation it succeeds at table heights well below any seen during training, indicating that kinodynamically filtered synthetic data grounds affordances in object geometry and whole-body feasibility rather than in memorized configurations.
comment: CoRL 2026. Project page: https://xuanbinpeng.com/hug
★ UA-Depth: Uncertainty-Aware Depth Restoration for Material-Agnostic Object Manipulation
Robotic manipulation requires accurate metric depth, yet commodity RGB-D cameras often fail on non-Lambertian objects such as transparent, reflective, and translucent materials, producing holes, structured noise, and biased depth. Most restoration methods produce only a single depth map, treating noisy and reliable regions uniformly for downstream tasks while ignoring regional uncertainty. We present UA-Depth, an uncertainty-aware depth restoration framework that iteratively co-refines depth and pixel-wise uncertainty, and uses uncertainty-gated features to suppress unreliable cues for stable updates. To strengthen zero-shot behavior, UA-Depth leverages a frozen depth foundation model as a prior to initialize missing or unreliable depth, and applies two-stage MAD-weighted log-depth scale normalization to reduce outlier bias in relative-to-metric alignment. We introduce an eight-dataset zero-shot benchmark across diverse scenes, materials, and sensors. Across these benchmarks, UA-Depth achieves the best mean rank among representative baselines, with particularly strong gains on datasets dominated by transparent and reflective objects. Beyond depth metrics, we demonstrate downstream improvements in category-level 6D pose estimation and 6D grasp synthesis by exploiting the predicted uncertainty. \emph{Code, models, and data for training and evaluation will be released upon publication.}
comment: Under Review
★ MagPillar: A Soft Magnetopillar Array Tactile Sensor for Robotic Surface Perception
Robotic surface exploration benefits from tactile interfaces that accommodate contact and capture variations in surface characteristics. Herein, we report MagPillar, a tactile sensor inspired by spider tactile hairs that combines a magnetic pillar array with three-axis Hall sensing. A full-array mechanical and magnetic-field model characterizes how pillar deformation affects the superposed fields measured across the array, providing a physical basis for interpreting its contact responses. The compliant pillar architecture translates contact with surface textures and geometric features into distinct magnetic response patterns. Then, in robotic surface perception experiments, spectral features and logistic regression distinguish six material surfaces with an offline accuracy of 99.34% under within-session temporally blocked cross-validation. Continuous scans over periodic protrusions produce amplitude and peak-spacing variations consistent with changes in surface geometry. These findings connect compliant contact mechanics with distributed magnetic sensing, paving a new path for material recognition and qualitative surface-profile perception.
★ SCOUT: Planning under Occlusion via Object-Centric World Model Rollouts
Robots manipulating in cluttered spaces must often plan from partial views where foreground objects occlude the target, surrounding geometry, and usable free space. Reactive policies often miss these hidden constraints, while exhaustive search becomes computationally prohibitive in dense clutter. We present SCOUT, an occlusion-aware planning framework that bridges high-level deliberation with grounded physical rollouts. SCOUT leverages an object-centric scene representation to reconstruct hidden geometry and derive a relational planning state capturing blocker hierarchies and accessibility corridors. To navigate the vast action space in clutter, an LLM proposes structured action candidates which are then rigorously evaluated via Monte Carlo Tree Search (MCTS) using a learned action-conditioned world model. Evaluations on 'Reveal' and 'Placement' tasks show that SCOUT outperforms Vision-Language-Action (VLA) policies, embodied agents, and 3D world models, with significant gains in long-horizon tasks requiring reasoning over occluded geometry. We further validate the framework's robustness through real-world robotic manipulation in a cluttered shelf environment.
★ Recursive Self-Improvement of Exoskeleton Assistance Policies through Human-in-the-Loop Feedback
To reduce reliance on manual design and extensive parameter tuning, a framework for the autonomous evolution of exoskeleton assistance policies through human feedback is proposed. Assistance policies are autonomously generated as executable control code by an intelligent agent and evaluated in experiments with human participants. Subjective user feedback, movement performance, and muscle activation measurements are incorporated into policy assessment. Based on these assessments, policy deficiencies are autonomously identified and the control code is revised by the agent for subsequent trials. A closed loop of policy generation, human feedback, evaluation, and autonomous refinement is thereby established. The framework is evaluated using GPT, DeepSeek, GLM, and Kimi, and initial assistance performance and refinement outcomes are compared across models. Despite differences in initial performance, effective assistance is achieved with all four models after a single autonomous refinement iteration under the tested conditions. Across three tasks, biceps and triceps muscle activation levels are reduced by 29.4% and 35.2%, respectively, compared with those measured without an exoskeleton. However, limited adaptability to unexpected situations is observed, indicating that further improvements in robustness and safety are required.
★ Beyond Validation Loss: Coverage and Reliability Dynamics in Robot Policy Pretraining
In modern robot learning, generative robot policies are first pretrained through supervised learning from demonstrations, then refined through reinforcement learning, supervised finetuning, or other forms of posttraining. Yet conventional pretraining metrics, such as validation loss and average task success, provide an incomplete account of which policies support both effective direct deployment and subsequent posttraining. In this work, we study closed-loop pass@k, which evaluates the probability that k rollouts from the same initial condition attain at least one success. Thus, pass@1 measures mean task success; large-k pass@k measures whether success remains reachable under repeated sampling. Across a wide range of posttraining recipes, we show that large-k pass@k reveals posttraining differences not captured by pass@1. In particular, checkpoints with similar pass@1 but different large-k pass@k have different posttraining performance. Moreover, we show pass@k peaks at different checkpoints during supervised pretraining, revealing a coverage-reliability tradeoff over the course of training. Based on these observations, we propose two practical interventions: (1) a purely offline checkpoint-selection criterion that balances coverage and reliability without closed-loop evaluation, and (2) a generative data augmentation procedure that expands supervision beyond the finite demonstration set, preserving large-k pass@k while maintaining or improving pass@1 and raising the observed coverage-reliability Pareto frontier. Together, our results motivate evaluation, checkpoint selection, and training procedures that explicitly account for posttraining. Blog post, visualizations, and code are available at https://zhanyisun.github.io/robot-policy-learning-dynamics/.
★ Democratizing Robot-Use Agents: A Unified Harness with Open Models
Robot-use Agents (RUAs) have shown promising zero-shot manipulation capabilities, but reliance on frontier, proprietary models limits research into further training. To democratize RUA research, we introduce uni-robot-agent, a unified harness for competitive open-weight robot use that supports further training. Without privileged inputs, an out-of-the-box Qwen3.8-Flash-Next achieves a RoboDojo Score of 31.39, surpassing the published GPT-6 Astra-RoboProbe result of 28.97 and many VLA/WAM baselines, and performs competitively on 19 real-world tasks. Our analysis across model generations, families, and sizes links effective robot use to four general capabilities, suggesting that future RUA training should develop them jointly.
★ $TF^2$-Steering: Action Refinement with Tactile Forecasts and Feedback via Velocity-Field Steering
Tactile sensing plays a crucial role in everyday human interaction. Consider inserting a plug into a tight socket, vision alone provides limited guidance during the final insertion stage, and we instead rely on both anticipated contact and realized tactile feedback to ajust the insertion motion. The same capability is equally important for robots, particularly in dexterous manipulation tasks. Yet many contemporary robot policies remain vision-dominant. This visual bias, compounded by limited tactile data and sparse contact events, can cause directly incorporated tactile signals to be ignored or even compromise the policy's pretrained capabilities, yielding degrading performance as observed in our experiments. We present $TF^2$-Steering, which refines actions via learned residual steering of the flow-matching action expert's velocity field, using a tactile mixture-of-experts architecture to integrate both tactile forecasts and real-time tactile feedback. Together with specific designed three stages training recipe, $TF^2$-Steering enables tactile signals to correct and, when necessary, redirect the flow-matching action-generation process, while building on the strong action-generation capacity of the vision-based foundation policy. Extensive experiments across six simulated and six real-world manipulation tasks demonstrate the effectiveness of $TF^2$-Steering. It outperforms the strongest evaluated baselines by 10.7 percentage points in average simulation success rate and 9.1 percentage points in average real-world progress success rate. Project page: https://yiming-cc.github.io/tf2-steering/.
comment: 19 pages, 6 figures. Project page: https://yiming-cc.github.io/tf2-steering/
★ Dual Path Following for an Aerial Manipulator Under State and Input Constraints via Transverse Feedback Linearization
This paper studies dual path following for a quadrotor carrying a three-degree-of-freedom arm. The center of mass of the combined system must follow a vehicle path and the end effector must follow a second path, while the arm respects its joint limits and avoids its singular postures. The vehicle and the arm are mechanically coupled, so one controller must handle both paths. With the center of mass and the end-effector path coordinates as outputs, and a dynamic extension of the thrust, the system admits a normal form without internal dynamics. We characterize the states where this normal form fails. We formulate the transverse feedback linearizing controller as a quadratic program that relaxes only the speed of the end effector along its path and imposes safety and actuator limits as inequality constraints. The program reduces to a scalar problem with a closed-form solution. If it remains strictly feasible for all time, the feedback is locally Lipschitz, both paths are invariant and attractive, and every constraint holds. In the physics engine Drake, the controller resolves a conflict between the two speed references by slowing the end effector while both paths remain invariant. By contrast, a cascade with a safety filter leaves both paths. Code and animations are available at https://gradslab.github.io/am_dual_pf-/
comment: 8 pages, 3 Figures
★ Adaptive Surface Cleaning under Uncertain Cleaning Response
Surface cleaning requires robots to adapt both where they clean and how they interact with contamination whose response to cleaning may vary spatially and remain unobservable prior to interaction. We formulate adaptive surface cleaning as a partially observable Markov decision process and propose a belief-aware planning framework that uses observed cleaning outcomes to update a latent local cleaning-response model and adapt subsequent cleaning motions, forces, and passes. Preliminary grid-world simulations demonstrate the resulting closed-loop planner adapting its spatial decomposition and cleaning actions as the surface evolves.
comment: 7 pages, 2 figures
★ Serving Agentic Robot Fleets: Workload Characterization, KV-Cache Thrashing, and Admission Control
Agentic robots combine a vision-language brain, a tool-using harness, and a vision-language-action (VLA) policy. Serving a fleet of these agents introduces two coupled resource constraints: the brain repeatedly revisits long, growing key-value (KV) cache states, while the VLA executes action chunks under tight control deadlines. We characterize 80 LIBERO-PRO episodes generated by GPT-6-Astra, RPent, and pi0.5 (1,862 brain turns and 1,582 VLA inference chunks), profile both layers on H800 graphics processing units (GPUs), and replay the resulting agent programs on a vLLM serving engine. Brain contexts reach 215k tokens; the median Astra request takes 7.78 s, while the intervening environment-step gap is only 1.0 s. Under a 0.25 s action-chunk deadline, one VLA GPU meets the 1% starvation target for one robot; under a 1.0 s deadline, its measured capacity reaches at least 32. In sustained brain replay, increasing concurrency from 20 to 32 robots reduces throughput from about 2.4 to 0.6 requests/s as repetitive prefill displaces generation. A static active-episode cap restores 2.24-2.25 requests/s and raises completed trace-program throughput from 1.30 to 5.00-5.55 episodes/min. Our measurements expose the workload-dependent shift between action deadlines and brain KV residency, and establish practical principles for capacity planning and admission in sustained agentic robot fleets.
comment: 12 pages, 17 figures, 8 tables
★ Same Path, Less Time: Training-Free Fast Execution of Vision-Language-Action Models under Inference Latency
Vision-language-action (VLA) models predict chunks of future actions, yet deployed robots waste time in two ways that are independent of model capability: they halt while the next chunk is inferred, and they move at the speed of slow, cautious demonstrations. Existing remedies remove one loss each, and stacking them fails, because asynchrony and speed-up each change the correspondence between planned progress and executed motion. We present Path-preserving Time Scaling (PTS), a training-free execution layer for frozen action-chunking policies built on one principle: the robot's motion is a time reparameterization of a reference path, and the path changes only in response to new observations. A path layer decodes every replan deterministically in one step and enters it at the progress consumed during inference. A time layer sums the planned increments, keeps the arm at 1x while the gripper moves, and caps the rate by the actuator's linear range and by chunk feasibility under latency. We show that misaligned entry and stochastic sampling add a bias and a variance to the continuation error, and derive a break-even success rate that states when speeding up pays off. On LIBERO with two frozen VLAs under injected inference latency, PTS raises task throughput over the strongest deployable baseline by 33% for pi0.5 and 104% for SmolVLA while matching or improving the success rate, is best on all six model-suite combinations, and exceeds zero-latency 1x execution. Against asynchronous 1x execution with the same one-step decoder, the time layer alone adds 28% and 38%. On LIBERO-Dynamic it improves on stop-and-wait execution by 38% and is comparable to asynchronous 1x execution.
comment: 22 pages, 11 figures
♻ ★ WasserMan: Benchmark for Underwater Manipulation Policy Learning
Underwater contact manipulation couples visual decisions with vehicle stabilization and mechanism interaction. To our knowledge, WasserMan is the first multi-task simulation benchmark focused on visuomotor policy learning for manipulating articulated underwater mechanisms. The suite supports demonstration-based policy learning, alternative vehicle--arm platforms and bimanual manipulation. We compare ACT, diffusion policies and behavioral cloning under matched sample budgets, and evaluate adaptation of a pretrained SmolVLA baseline. Fixed-policy studies vary water appearance, lighting, contact parameters, actuator delay, arm properties and mechanism pose. RGB feedback controls and matched observation-history and expert-trajectory ablations examine observation and training choices. Expert references establish task feasibility under the same physical success tests. The experiments reveal task-dependent visual transfer and pose sensitivity, while contact, tracking and actuator measurements distinguish execution responses that completion alone hides. Complete expert candidate streams make the initial states associated with demonstration selection inspectable. Versioned code, demonstrations, checkpoints and per-episode physical evidence support reproducible comparisons within the simulation.
comment: 13 pages, 8 figures
♻ ★ RigidDriveGS: Object-Centric Vehicle Reconstruction from Fixed Multi-Camera Drive-Through Captures WACV 2027
High-fidelity 3D reconstruction of vehicle exteriors improves buyer confidence in online automotive marketplaces, but fixed multi-camera drive-through capture departs from standard Structure-from-Motion (SfM): static cameras observe a moving vehicle amid clutter, rotating wheels, fisheye distortion, and specular reflections. We present RigidDriveGS, an object-centric pipeline combining motion-gated tracking, bifurcated masks that exclude wheels for SfM while optionally retaining them for rendering, mask-guided dense correspondences on raw distorted images, rig-constrained global SfM, and clean object-centric neural rendering with trimap-aware opacity or density supervision and native-alpha white compositing. We evaluate 25 vehicles from 10 dealership installations and six phone-recorded vehicles, each captured from 14 independent handheld positions. Under a staggered per-camera temporal-block holdout, 3DGUT+MCMC achieves 30.42 dB PSNR, 0.90 SSIM, and 0.20 LPIPS, improving PSNR by 7.78 dB over undistorted 3D-GS; independently recorded reference LiDAR scans yield full/body Chamfer-L1 of 0.026/0.013 m and F1@5 cm of 0.703/0.885. These results support clutter-free, object-centric novel-view synthesis without controlled studio infrastructure.
comment: 8 pages, 6 figures, Accepted at IEEE/CVF WACV 2027
♻ ★ H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning
Long-horizon planning with latent world models requires reasoning across timescales and levels of abstraction. Existing task-agnostic JEPA world models predict and plan in a single latent space, often at a single timescale. We introduce H-JEPA, an end-to-end recipe for training a hierarchy of action-conditioned JEPAs in which each level predicts farther ahead in its own learned latent space. Planning proceeds top-down: the top level optimizes progress toward the goal, and each level's predictions become subgoals for the planner below it. When factors in the data evolve at separated timescales, higher levels discard fast, unpredictable detail and retain slower task-relevant state. Across four simulated navigation and manipulation environments, hierarchical planning improves over a flat JEPA; on Visual AntMaze, a three-level hierarchy raises success from 18% to 73% using less planner compute. Ablations attribute these gains to both temporal decomposition and higher-level goal representations. With inverse-dynamics supervision, the approach extends to diverse real-robot videos from DROID, where hierarchy improves offline planning fidelity at lower planner compute.
♻ ★ MAESTRO: Orchestrating Robotics Modules with Coding Agents for Zero-Shot Generalist Robots
Today's dominant route to generalist robots scales robotics data to train end-to-end policies. We instead use a vision-language model (VLM) as a coding agent that writes robot programs using a curated library of perception, planning, and control tools. MAESTRO's orchestration harness packages multi-step perception and action procedures into callable tools and guides their use through a plan-act-replan loop, using visual and execution feedback to verify outcomes and recover from failures. With this tool library and harness, MAESTRO matches or surpasses state-of-the-art vision-language-action (VLA) models zero-shot in our tabletop comparison and performs diverse mobile manipulation tasks, without collecting new teleoperation data or invoking a VLA tool. Ablations show that expanding the tool library improves average task progress with the full harness, while adding tools without it does not reliably improve progress. Case studies illustrate MAESTRO's potential for autonomous code improvement across trials, faster execution with VLA tools, and automatic robot data generation. We show that expanding robotics tool libraries together with their orchestration harnesses offers a complementary route to general-purpose robots alongside scaling end-to-end robot training data. Project site: https://maestro-robot.github.io. Code: https://github.com/penn-pal-lab/maestro.
comment: 34 pages. Accepted at CoRL 2026. Substantially revised with expanded experiments, ablations, and discussion. Project website: https://maestro-robot.github.io
♻ ★ When Should a Robot Replan? Regret-Guided Update Scheduling in Time-Varying MDPs
Robots operating in non-stationary environments must continually adapt their policies as the dynamics drift, but onboard energy and compute budgets cap how often a full state estimation and re-planning step can be performed. This raises a question: \emph{when}, along a horizon, should a robot spend its limited budget? We formulate this problem in time-varying Markov decision processes (TVMDPs) with a known bound on the rate of transition drift. We model execution as a \emph{skip-update} scheme in which, at chosen update times, the agent estimates the transition kernel by maximum likelihood and computes a finite-horizon policy, and between updates reuses this policy under a propagated state estimate. We analyze the dynamic regret of this scheme and show how it grows during skip intervals in terms of the properties of the TVMDP and the skip lengths; the resulting bound answers the opening question via an online, regret-guided update rule that allocates the budget adaptively. We evaluate the rule in a simulated Mars-rover navigation task with time-varying slip dynamics and on a Crazyflie quadrotor in indoor obstacle fields. Adaptive allocation outperforms other budgeted baselines.
♻ ★ Pose accuracy is not grasp reliability: residual pose posteriors improve simulated robotic grasp planning SP
Model-based robotic grasping estimates an object's six-dimensional (6D) pose and plans on its mesh as if that pose were exact. We show that this assumption reduces simulated grasp success even when the pose passes the standard accuracy test. We built LM-Grasp, a benchmark of 15{,}800 real LINEMOD RGB-D images with grasp outcomes simulated on scanned meshes. FoundationPose meets the ADD(-S) criterion (mean model-point error below 10\% of the object diameter) on 0.997 of test scenes, yet planning on its estimate succeeds in 0.791 of scenes, compared with 0.856 using the annotated pose. Because the estimator reports no uncertainty, we build a pose posterior from its own errors on annotated training frames, resampled from frames with similar observable fit, and score each grasp by its mean success across sampled poses and four equally weighted physics scenarios. Without retraining the estimator, posterior planning yields 0.859 (difference from the annotated-pose result +0.003; 95\% scene-wise paired-bootstrap interval [$-$0.003, +0.009]). This interval is compatible with no difference and a small shortfall; we did not conduct an equivalence test. For pose errors of 5 to 10\% of the diameter, simulated success rises from 0.796 to 0.984. Among test scenes assigned singleton-success conformal prediction sets, observed held-out precision is 0.972, compared with 0.904 for the point-estimate planner; these are empirical precision estimates, not conditional guarantees. On occluded LM-O scenes never used for tuning, posterior planning closes 66\% of the gap to the annotated pose.
comment: Project Page : https://physical-agi.github.io/MSP-FRAMEWORK/
♻ ★ Formal Evidence Generation for Assurance Cases for Robotic Software Models
Robotics and Autonomous Systems are increasingly deployed in safety-critical domains, so that demonstrating their safety is essential. Assurance Cases (ACs) provide structured arguments supported by evidence, but generating and maintaining this evidence is labour-intensive, error-prone, and difficult to keep consistent as systems evolve. We present a model-based approach to systematically generating AC evidence by embedding formal verification into the assurance workflow. The approach addresses three challenges: systematically deriving formal assertions from natural language requirements using templates, orchestrating multiple formal verification tools to handle diverse property types, and integrating formal evidence production into the workflow. Leveraging RoboChart, a domain-specific modelling language with formal semantics, we combine model checking and theorem proving in our approach. Structured requirements are automatically transformed into formal assertions using predefined templates, and verification results are automatically integrated as evidence. Case studies demonstrate the effectiveness of our approach.
comment: This is a preprint. Accepted for publication in Software and Systems Modeling (SoSyM)
♻ ★ Autonomous fixed-wing interception using reinforcement learning
Medium altitude, long endurance unmanned aerial systems such as Shahed-type strike drones are an unanswered threat in modern conflicts. Conventional air defense systems are too expensive to provide a feasible large-scale solution. Quadcopter interceptors are a cheap, battle-proven alternative, but their extremely limited flight time restricts them to low altitude targets at short range, whereas fixed-wing platforms have the endurance to reach higher altitude targets from longer distances. However, such interceptors are still operated manually. Recent developments in learning-based controllers show potential for automation. In this work, a fixed-wing interceptor is presented that is controlled by a neural network that maps filtered monocular vision observations directly to actuator commands. The policy was trained in a custom Python environment using reinforcement learning and proximal policy optimization. The policy was then deployed on a fixed-wing platform and tested in real interception scenarios. As a baseline, two classical guidance controllers were also developed and tested in simulation. The policy is able to intercept targets both in simulation and in real flight. In simulation, the policy outperforms the classical controllers under most conditions. The interceptor was ultimately successfully validated in real flight by intercepting maneuvering fixed-wing targets. This novel system shows the potential of artificial intelligence for autonomous control of fixed-wing interceptor drones.
comment: MSc thesis, Faculty of Aerospace Engineering, Delft University of Technology, 2026. 24 pages, 29 figures, 4 tables
♻ ★ ActionFlow: A Pipelined Action Acceleration for Vision Language Models on Edge
Vision-Language-Action (VLA) models have emerged as a unified paradigm for robotic perception and control, enabling emergent generalization and long-horizon task execution. However, their deployment in dynamic, real-world environments is severely hindered by high inference latency. While smooth robotic interaction requires control frequencies of 20-30 Hz, current VLA models typically operate at only 3-5 Hz on edge devices due to the memory-bound nature of autoregressive decoding. Existing optimizations often require extensive retraining or compromise model accuracy. To bridge this gap, we introduce ActionFlow, a system-level inference framework tailored for resource-constrained edge platforms. At the core of ActionFlow is a Cross-Request Pipelining strategy, a novel scheduler that redefines VLA inference as a macro-pipeline of micro-requests. The strategy intelligently batches memory-bound Decode phases with compute-bound Prefill phases across continuous time steps to maximize hardware utilization. Furthermore, to support this scheduling, we propose a Cross-Request State Packed Forward operator and a Unified KV Ring Buffer, which fuse fragmented memory operations into efficient dense computations. Experimental results demonstrate that ActionFlow achieves a 2.55x improvement in FPS on the OpenVLA-7B model without retraining, enabling real-time dynamic manipulation on edge hardware. Our work is available at https://github.com/Frankd35/Action-Flow.
comment: 7 pages, 5 figures. Accepted by DAC 2026
♻ ★ LLM4Robot-Bench: How Far Can GPT-6 Astra Go in Robotic Manipulation? A Hierarchical, Matched Sim-Real Benchmark
We introduce LLM4Robot-Bench, a hierarchical, matched Sim-Real benchmark for evaluating physical interaction capabilities in contact-rich manipulation. Its 16 household tasks span three cumulative levels: contact selection and geometric alignment (L1), force regulation within sustained contact (L2), and coordinated contact reconfiguration and force regulation (L3). Each task is instantiated in physically and visually matched simulated and real-world environments, with shared task definitions and success criteria. We evaluate GPT-6 Astra zero-shot with three trials per task in each environment. Real-world success rates decline from 80.0% at L1 to 47.6% at L2 and 0.0% at L3, with corresponding simulation rates of 66.7%, 47.6%, and 0.0%. Ten of the 16 tasks have identical success counts across environments; the remaining six differ by one success out of three trials. Across both environments, recurring difficulties in spatial reasoning, contact selection, force regulation, and coordination often prevent local physical progress from accumulating into task completion. These findings characterize Astra's observed capability boundary across increasing interaction demands and, for this model, support matched simulation as a repeatable, scalable testbed for studying the sustained, coordinated physical interactions that limit reliable manipulation.
comment: 31 pages, 11 figures. Renamed PhysLadder to LLM4Robot-Bench; minor text revisions
♻ ★ RynnVLA-Latent: Scaling Robotic Manipulation Foundation Models with 3D-Grounded Latent Actions
Scaling vision-language-action (VLA) models is constrained by scarce action-labeled demonstrations and incompatible control spaces across robot embodiments. We present RynnVLA-Latent, a framework that turns heterogeneous interaction videos into scalable supervision for robot policy pretraining. At its core, RynnLAM learns continuous, 3D-grounded latent actions from visual transitions without requiring native robot controls. By combining residual future-feature prediction, camera-compensated 3D-flow supervision, and camera-motion decoupling, RynnLAM encourages representations that capture control-relevant motion rather than appearance changes alone. We apply RynnLAM to 30.1K hours of human, robot, and simulation videos, producing 745.52M latent-action descriptors. RynnVLA-Latent then learns to predict latent-action chunks from visual observations and language through flow matching, before adapting its pretrained backbone and action expert to embodiment-specific robot actions. The resulting 4B policy achieves 98.8% success on LIBERO, 84.7% on LIBERO-Plus without benchmark-specific adaptation, 51.9% average success across five VLABench tracks, and 88.6% average success on the RoboTwin 2.0 bimanual benchmark. It also achieves the highest observed success rates among the compared methods on six real-world manipulation tasks across dexterous-hand and mobile-gripper platforms. Together with representation ablations and data-scaling results, these findings demonstrate the potential of latent-action supervision for scaling robot policy learning beyond native action-labeled data.
comment: Project page: https://alibaba-damo-academy.github.io/RynnVLA-Latent.github.io/, GitHub: https://github.com/alibaba-damo-academy/RynnVLA-Latent
♻ ★ A Third-Order Gaussian Process Trajectory Representation Framework with Closed-Form Kinematics for Continuous-Time Motion Estimation
In this paper, we propose a third-order, i.e., white-noise-on-jerk, Gaussian Process (GP) Trajectory Representation (TR) framework for continuous-time (CT) motion estimation (ME) tasks. Our framework features a unified trajectory representation that encapsulates the kinematic models of both $SO(3)\times\mathbb{R}^3$ and $SE(3)$ pose representations. This encapsulation strategy allows users to use the same implementation of measurement-based factors for either choice of pose representation, which facilitates experimentation and comparison to achieve the best model for the ME task. In addition, unique to our framework, we derive the kinematic models with the closed-form temporal derivatives of the local variable of $SO(3)$ and $SE(3)$, which so far has only been approximated based on the Taylor expansion in the literature. Our experiments show that these kinematic models can improve the estimation accuracy in high-speed scenarios. All analytical Jacobians of the interpolated states with respect to the support states of the trajectory representation, as well as the motion prior factors, are also provided for accelerated Gauss-Newton (GN) optimization. Our experiments demonstrate the efficacy and efficiency of the framework in various motion estimation tasks such as localization, calibration, and odometry, facilitating fast prototyping for ME researchers. We release the source code for the benefit of the community. Our project is available at https://github.com/brytsknguyen/gptr.
comment: Published in IEEE Transactions on Robotics (T-RO). The source code has been released, and feedback is welcome
♻ ★ Any-ttach: Quick End-effector Swapping Enables Manipulation Dexterity with Simplicity
Robotic manipulation dexterity is often pursued by building increasingly complex high-DoF multifingered hands. While many robotic hands are designed to replicate the morphology of human hands, the functional role of human hands suggests a different perspective: much of their complexity may exist to enable tool use and tool making. This observation motivates Any-ttach, a tool-centric manipulation system that leverages mechanical modularity and treats autonomous end-effector swapping as a pathway towards dexterity. Any-ttach combines: 1) a low-cost standardized interface with passive self-locking and mechanically triggered release mechanism; 2) a unified object-centric manipulation framework supporting kinesthetic teaching trajectory replay, human demonstration retargeting, and policy learning; and 3) hierarchical planning and execution system for tools and skills for multi-tool long-horizon tasks. We demonstrated 15 heterogeneous end-effector modules supported by our system, spanning different robotic fingers, everyday tools (including articulated tools such as scissors), and unconventional end-effectors through the same interface. Experiments demonstrate reliable autonomous swapping, automatic integration to new tools, and broad manipulation capability across different skill sources. Two multi-tool application demonstrations, comprising seven tool--skill steps across two tasks, further illustrate the autonomous composition of end-effector exchange and heterogeneous manipulation skills. These results show that end-effector swapping can expand manipulation capability without concentrating increasing complexity entirely in a single end-effector. More details and videos are available at https://any-ttach.github.io/.
comment: 8 pages, 5 figures. Submitted to IEEE Robotics and Automation Letters
♻ ★ PS-DLO: Property- and Spatially Aware Deformable Linear Object Manipulation Policy
Deformable linear objects (DLO) pose a coupled perception and control challenge: their slender geometry makes precise localization difficult, while their deformations depend on object-specific physical properties. To address this challenge, we propose PS-DLO, a property- and spatially aware DLO manipulation policy. First, contrastive image-text learning maps the initial observation to categorical descriptions of DLO properties, providing explicit physical and structural priors for action generation. Second, SAM3 extracts mask-guided features from future observations, which are distilled into the policy's action expert to encode task-relevant DLO regions and their subsequent configurations. We evaluate PS-DLO on nine real-world tasks across single-arm and bimanual platforms, training a shared multi-task policy for each embodiment. PS-DLO achieves an average success rate of 68.9%, exceeding the strongest baseline by 20.0 percentage points. In generalization experiments, PS-DLO achieves higher success rates than the baseline in six of seven visual-shift conditions on butterfly-knot untying. These results demonstrate that combining object-property priors with future spatial supervision improves multi-task DLO manipulation and robustness to visual variation.
Multiagent Systems
★ Trustworthy Execution of Industrial Agents
Industrial agents increasingly modify persistent engineering objects rather than merely generate answers. Authentication and successful calls cannot establish whether intended changes occurred, evidence supports a conclusion, or historical results remain applicable. We present a framework for trustworthy industrial-agent execution within a single enterprise governance domain. Eight control domains assign responsibilities across authority, planning, composition, tool execution, actual effects, evidence, baseline and use governance, and recovery and evolution. A four-plane architecture and three control chains connect these responsibilities through typed objects and version-bound interfaces. The specification separates tool returns from actual engineering effects, evidence coverage from engineering judgment, and historical lifecycle state from current applicability. Multi-agent handoffs are an architectural extension examined through deterministic interface tests. Distinct prototypes examine selected mechanisms using conformance tests, ablations, uncertainty stress, CAD/CAE execution, and local persistence. An OpenFOAM fixture produces COMPLETE coverage with FAIL judgment; an expired-job case requires readmission despite completed computation. CAD and recovery counterexamples expose omitted mandatory checks, discarded negative facts, and inconsistent producer-consumer mappings. Targeted repairs support local regression conclusions. The reported 0/29 hazardous commits measure closed-set specification conformance, not a safety probability. We distinguish the complete framework, implemented paths, and empirical evidence, identifying check-inventory completeness and cross-module fact preservation as prerequisites for extending these results.
comment: 80 pages, 16 figures, 37 tables
★ Towards a Theory for the Design of Rules in Multi-Agent Systems: the case of Homogeneous Fleets
The regulation of systems composed of many interacting decentralised agents is a long-standing problem. This problem has typically been resolved by the imposition of rules. Rules resolve the limitations of sub-optimal agents whilst simultaneously aligning the behaviour of the agents with the demands of society. Traffic laws, for example, let a driver cross a junction without seeing around the corner, and transport protocols such as TCP let applications share a network without knowing of one another. The problem is more pressing today, as autonomous systems are deployed in society. In this work, we establish the basis of a system-agnostic theory of rule design for multi-agent systems. We first state the rule design problem (RDP) in its generality. It relates a rule to the set of system performances, defined by the regulator, that the agents deployed under it can attain. However, it is not yet possible to solve the RDP in full generality. As such, we introduce the homogeneous rule design problem (HRDP), in which the agents in each environment are identical, and among other interesting properties, show that under worst-case aggregation the two problems have the same solution, bypassing the combinatorial complexity of the RDP. To solve the RDP under worst-case aggregation we propose Rule Branch and Bound (RuleBnB), an anytime branch-and-bound algorithm. Finally, we illustrate the value of solving the RDP on an autonomous driving case study.
comment: 30 pages; A first version of the RDP
★ Local Solution Repair for Multi-Agent Pickup and Delivery
Multi-robot systems in automated warehouses must efficiently attend to a constant stream of pickup-and-delivery tasks while avoiding collisions. Existing Multi-Agent Pickup-and-Delivery (MAPD) solutions generate plans that are globally feasible but often locally inefficient. In large-scale, long-horizon deployments, these inefficiencies, such as robots backtracking from target goal locations and idling in congested areas, accumulate over time and degrade performance. This paper contributes Local Solution Repair (LSR), a goal replanning approach that detects suboptimal behaviors at run-time and locally improves plans to reduce conflict between agents. We demonstrate the utility of our approach by repairing solutions generated by state-of-the-art MAPD baselines, M2M and LNS-PBS. Our results in long-horizon warehouse simulations over three environments show upwards of 5.9% improvement in task throughput over current state-of-the-art methods by reducing the number of suboptimal instances by ~36k or by 14.6% with minor additional computation time.
★ ParetoSaddler: Pareto-Guided Optimization of Agent Harnesses for Quality and Cost
Harness optimization can substantially improve LLM agents without modifying model weights, but existing methods primarily optimize response quality. Extending them to joint quality--cost optimization introduces a challenge: applying the deployment preference during intermediate updates can prematurely reject candidates that are valuable for later improvements. We introduce ParetoSaddler, which addresses this mismatch by separating intermediate candidate promotion from final deployment selection. At each iteration, an optimizer updates a parent harness to produce a candidate, and then compares the two over repeated executions and uses Bayesian bootstrap Pareto support to promote candidates showing credible progress in either quality or cost. Promoted candidates receive broader development evaluation and remain available as parents for later optimization steps. We evaluate ParetoSaddler on ClawMark and GAIA2 using domain-disjoint splits. On ClawMark, ParetoSaddler improves quality from 70.86\% to 81.35\% while reducing cost from \$1.17 to \$1.01 per task; on GAIA2, it improves quality from 56.22\% to 62.89\% while reducing cost from \$0.78 to \$0.45 per task. The Pareto-selected harnesses also achieve higher quality at lower cost than quality-only harness optimization. Analysis shows that Pareto-aware optimization retains intermediate candidates that would be discarded by deployment preference but can support later improvements, while Bayesian bootstrap screening filters out candidates whose gains are weakly supported across runs.
comment: Project website and code will be available at https://aka.ms/ParetoSaddler-website
★ Beliefs in feedback: Factored representations of interacting agents in transformers
Complex systems, from ecosystems to brains, are usually modeled as modular components coupled through feedback rather than as indivisible wholes. Can a transformer trained on the joint behavior of a group of interacting agents be understood in the same way? In this work, we develop a computational-mechanics framework to study agents interacting in feedback loops. We formally prove that any joint stochastic process can be presented as resulting from the behavior of agents interacting in feedback, and the resulting process is generated by a hidden Markov model whose Bayesian beliefs factorize into separate components per agent. Furthermore, our experiments show that transformers trained on such feedback sequences make use of this separability, disentangling the beliefs about the underlying factors into separate subspaces of their residual stream. This representation is found to scale linearly (instead of exponentially) with the number of agents. Moreover, interventions show that these representations are causally efficacious, allowing for targeted steering of individual agents' beliefs by selectively inserting counterfactuals. However, coarse-graining the behavior is found to disrupt the factorization, and can force a higher-dimensional representation. Together, the results establish both the dimensional advantage of a factorized representation and the ability of transformers to exploit it, laying the groundwork for investigating traces of compositional predictive architecture in pretrained models.
★ AgenticMobility: An Open-Source Platform for Multi-AI-Agent Mobility Dynamics
Understanding mobility dynamics and the effects of urban transportation management policies requires examining how individual travel decisions, traffic conditions, and policy interventions influence one another over time. We present AgenticMobility, an open-source experimental platform for studying how mobility AI agents and other AI agents interact, adapt through repeated experience, and collectively shape mobility outcomes. Persistent agents retain identities and relevant states across rounds, with explicit interfaces defining information access, admissible actions, and decision schedules; linked records trace decisions, execution, and feedback. In the current implementation, large language model (LLM) agents act as travelers choosing routes and departure times and as a manager periodically adjusting link charges within MATSim. A preliminary demonstration spans 100 simulated days in each of five configurations, with one run per configuration and a shared simulation seed. In the joint traveler--manager run, all 1,179,300 finalized target decisions satisfied action constraints; execution checks verified implementation of accepted choices and preservation of personal memories across days. Trajectories show stable route choices alongside departure-time adjustments and differences between controlled-trip and population-wide outcomes. The platform supports experiments on individual adaptation, collective outcomes, and management interventions by varying information access, retained experience, and management feedback. Source code is available at https://github.com/Agentic-City-Research/Agentic-Mobility
★ Automated Stateful Specialization for Adaptive Agent Systems ICLR 2026
Current automated agent design frameworks produce either static workflows that lack adaptability or per-query optimizers that prevent the accumulation of deep, agent-level task expertise. We propose a new direction that reconciles these paradigms: creating stateful teams of specialist agents that accumulate knowledge over time and can be reconfigured for novel tasks entirely without human intervention. To this end, we introduce ASpec, a framework that manages this full agent lifecycle by first autonomously discovering specialist archetypes via evolutionary search and then cultivating their expertise through experience, mirroring how human experts learn through practice and reflection. We further introduce a lightweight hierarchical control policy, "retain-then-escalate," which governs when to leverage the established agent system versus when to adapt its structure. Through comprehensive experiments, we demonstrate that this approach leads to significant performance gains on expert-level scientific benchmarks like GPQA while matching the state-of-the-art on broader domain tasks, demonstrating a promising path toward agent systems that are simultaneously expert, adaptive, and efficient. We will release the code at https://github.com/myanvoos/ASpec.
comment: 24 pages, 8 figures. Accepted at ICLR 2026
★ Self-Evolving Agentic Workflows via Semantic Decomposition of Complex Tasks NeurIPS 2026
Agentic workflows organize complex tasks into reusable procedures. However, invoking an LLM at every step incurs repeated inference costs and delays. We propose a framework for self-evolving agentic workflows through semantic task decomposition. Each subtask has explicit applicability conditions and a procedure that executes code or invokes a model as needed. An agent uses execution feedback to refine the decomposition and update these procedures. Experiments in the GLEE competition show the framework on bargaining, negotiation and persuasion. Most observed final-decision procedures are implemented in code. Code is available at https://github.com/Applied-Machine-Learning-Lab/NeurIPS2026_SeaFlow.
comment: Accepted as a Spotlight at the NeurIPS 2026 Workshop on Interpreting Agent Behavior (IAB), GLEE Competition Track; 1st place in the GLEE competition. 21 pages
★ From Single-Agent Self-Discovery to Multi-Agent Adaptation: Scaling Laws of Partner Feedback and Convention Rigidity in Meta Reinforcement Learning Algorithms
A key milestone toward Recursive Self-Improvement (RSI) is replacing hand-crafted optimizers with meta-learned update rules (such as Disco103 and OPEN) that discover how agents learn. Yet when a self-discovered RL model trains with adapting teammates, does co-evolution sustain capability gains or trap agents in brittle conventions? Across 1,800+ runs in continuous collective robotics (VMAS) and discrete coordination (Overcooked), we establish five laws of multi-agent adaptation. First, four-cell counterfactual partner swaps expose an evaluation illusion: apparent team progress often masks focal regression (+1.066 team gain concealing -0.119 focal loss) and inflates transfer by +0.649 to +1.210. Second, a 288-run physical-coupling sweep ($α\in [0, 1.5]$) reveals a linear scaling law ($β= +0.255$, $p = 0.000488$, Spearman $ρ= 1.0$): live reciprocal feedback keeps Disco103 invariant (AUC 0.297--0.322), whereas severing feedback triggers a monotonic collapse (-0.172 to -0.490), with OPEN establishing a sensitivity spectrum (Disco103 > OPEN > MAPPO). Third, extending navigation training eightfold (1.05M steps) enables Disco103 to overtake MAPPO (+0.087 team return, +0.133 bank gain), while decoupled training boosts cross-partner completion by +10.16% (56.25% vs. 46.09%). Fourth, in Overcooked, MAPPO's self-play conventions induce a 28.13% cross-partner collision rate, whereas Disco103 cuts collisions to 12.77% and achieves 3.33x higher bank return (9.96 vs. 2.99; net gain +6.43), further improved by decoupled meta-adaptation (+1.39 with scripted partners). Finally, perturbation pulses ($p=0.00098$), auxiliary entropy diagnostics (8.485 nats), and 48 multi-lifetime meta-evolved rules ($L_2 \in [2.11, 4.91]$) trace these dynamics to partner-gradient blindness and auxiliary drift, informing multi-agent self-discoverd algorithms.
comment: 71 pages, 8 figures, 28 tables
★ MAC-Bench: Benchmarking Multi-Agent Collaboration via Interconnected Tasks
Large language model (LLM)-based multi-agent systems (MAS) tackle challenging tasks by dividing work among agents and coordinating their contributions toward shared objectives. Evaluating these systems therefore requires benchmarks that assess not only task-solving performance but also the ability to handle interdependent work. However, existing benchmarks often emphasize task difficulty without substantial collaboration requirements or evaluate only a limited range of predefined collaboration patterns. In this paper, we introduce MAC-Bench, a comprehensive benchmark for evaluating multi-agent collaboration through interconnected tasks. Rather than imposing predefined agent roles or communication protocols, MAC-Bench embeds collaboration requirements directly into task dependencies and shared-resource constraints. Specifically, we mine reusable collaboration structures from selected benchmark tasks at two levels. Local collaboration patterns capture recurring interactions among subtasks, while global task structures characterize how these interactions contribute to complete workflows. Through coarse-to-fine composition, these structures are recombined into task graphs combining different collaboration requirements. Tasks are instantiated in executable environments with automated verification of partial progress and complete task resolution.
★ CONFIRM: Governing Scientific Reporting for Trustworthy Agentic Neuroimaging
Large language model (LLM) agents can autonomously run neuroimaging analyses from research questions to reported findings. Existing agentic systems use tools to check that code runs and reported values match its output, but rely on LLM judgment to decide which results warrant scientific reporting. We show that LLM judges endorse site-confounded findings even when the confound is presented as evidence. We present \textbf{CONFIRM}, a framework that separates running an analysis from deciding which findings to report. CONFIRM encodes each hypothesis as a machine-readable, pre-specified analysis plan that fixes the claim and its pass criteria before computing results. It then filters results through five deterministic gates (search history and multiplicity, confounds, power, analytic stability, and replication) under three configurable reporting standards, from least to most demanding: \textit{Screen}, \textit{Replicate}, and \textit{Full}. Failed claims receive diagnoses that guide follow-up analyses. To evaluate reporting decisions at the claim level, we introduce \textbf{NeuroClaimBench}, comprising 489 executable neuroimaging claims with frozen data, including literature-supported findings and 150 controls designed to violate individual methodological requirements. On NeuroClaimBench, CONFIRM falsely supports at most 3\% of controls, compared with 23--38\% for the baselines, while recovering up to 61\% of literature-supported claims. When auditing an existing agent, CONFIRM reduces false support on controls from 23\% to 0\% while retaining the agent's literature-supported claims. Its diagnostic feedback also produces more supported revisions than Self-Refine with fewer LLM calls. Together, CONFIRM and NeuroClaimBench show that basing reporting decisions on pre-specified analysis plans and deterministic gates is essential for trustworthy autonomous scientific discovery.
★ Renormalized Cellular Intelligence: When Does Model Reduction Preserve Collective Capability?
As multi-agent systems grow in size and complexity, understanding which features of their interactions support collective capability becomes increasingly difficult. Can a simpler description preserve both task performance and the measured contribution of interactions between agents? Preserving collective dynamics alone is insufficient: reduction can alter decision-relevant information or give a non-communicating baseline access to pooled observations, distorting the measured advantage of interaction. We introduce Renormalized Cellular Intelligence (RCI), a framework for specifying and auditing capability preservation across scales. RCI treats the task, interacting dynamics, matched baseline, decision interfaces, and resource budget as a joint object of reduction. It specifies dynamical closure, decision compatibility, and preservation of the intended baseline comparison, and bounds the resulting error in relative capability gain. An exact analytical construction demonstrates task-score preservation, while a counterexample shows how closed reduced dynamics can erase the measured interaction gain. Using cellular systems as models of locally interacting agents, we evaluate three reduction candidates on five frozen learned models without retraining. The final candidate reduces the evolving state from eight to five dimensions per cell (37.5%). Within-run uncertainty bounds support relative gain above a prespecified threshold for the fixed model collection, yet the candidate fails the closure and decision-preservation requirements for a complete RCI certificate. RCI provides explicit criteria for distinguishing positive interaction gains from
♻ ★ Learning How Much to Collaborate: Difficulty-Aware Topology Selection for Multi-Agent Code Generation
Multi-agent code-generation systems often use one communication topology for every problem. Across five topologies and 614 problems from APPS, HumanEval+ and LiveCodeBench, the advantage of hierarchical collaboration over a single agent grows from 2.4 points of pass@1 on the easiest third to 21.1 points on the hardest third, at about ten times the token cost. We propose the Difficulty-Aware Topology Selector (DATS), which predicts each topology's success probability and maximises predicted success minus cost. Its graph predictor treats topologies as nodes of a connectivity order. The complete graph-head comparison reports a 1.7-point advantage over a flat multi-label head while changing several mechanisms; capacity-matched adjacency controls show smaller descriptive differences. A scalar cost penalty can be recalibrated without retraining for approximately matched-spend comparisons when targets are attained. With a shared penalty, six methods span 21.6 percentage points of spend; two leading baselines fall behind DATS after method-specific calibration near the common target. At 40% of always-hierarchical cost, DATS reaches 77.7% pass@1 versus 73.6% for always-hierarchical and 74.3% for the strongest learned competitor. The main comparison includes Holm--Bonferroni-adjusted McNemar tests; unequal-cost differences do not imply matched-budget superiority. Backbone-specific gains are 4.07--4.23 points across four backbones. Alternatives to the 39 interpretable features, including a hybrid encoder, yield a largest observed gain of 1.3 points, with no significant pairwise improvement; this establishes neither equivalence nor an upper bound. A study on 400 mathematical reasoning problems reproduces difficulty-dependent collaboration gains, widening from 2.5 to 20.9 points.
comment: Accepted for publication in Neurocomputing. Revised manuscript incorporating peer-review revisions. 23 pages, 11 figures
♻ ★ PPAD-Completeness of Contiguous Envy-Free Cake Cutting with Additive Valuations
We prove that computing a complete contiguous $\varepsilon$-envy-free cake allocation is PPAD-complete for normalized rational piecewise-constant additive valuations. The hardness holds even when $\varepsilon$ is given in unary, and even when every density takes only the values $λ$ and $2λ$ for some $λ>0$ common to all agents and has at most six breakpoints. As a consequence, computing an exact contiguous envy-free allocation is also PPAD-complete, even under the same density restrictions. Our key ingredient is to assign prices to the pieces so that, in every approximately envy-free allocation, all prices are nearly equal, regardless of how the pieces are assigned to the agents. We further show that, for every fixed integer $c\ge1$, computing a contiguous allocation of indivisible goods on a path that is envy-free up to $c$ goods (EF$c$) is PPAD-complete for additive valuations, and this remains true even for $c=\lfloor m^α\rfloor$ with any fixed rational $0<α<1$, where $m$ is the number of goods. This hardness holds even for binary valuations in which each good is valued by exactly one agent and all agents value the same number of goods. All these results extend to chores. Our results answer three open questions at once: whether contiguous envy-free cake cutting is PPAD-hard for additive valuations, the complexity of computing contiguous EF1 and EF2 allocations of goods on a path, and the corresponding question for EF1 allocations of chores.
comment: We prove that computing a contiguous envy-free cake division is PPAD-complete for additive valuations, and that computing a contiguous EF$c$ allocation of goods on a path is PPAD-complete for every fixed $c \geq 1$, resolving open questions of Goldberg, Hollender and Suksompong (JAIR 2020) and Bilò et al.\ (ITCS 2019)
♻ ★ Bayesian Offline Multi-Agent Reinforcement Learning with Latent World Models
Model-based offline multi-agent reinforcement learning (MARL) provides a promising approach to multi-agent decision-making under partial observability, where the global state is unavailable from offline data. However, limited dataset coverage can lead to substantial out-of-distribution (OOD) model errors, which may mislead policy optimization and degrade downstream performance. Existing model-based offline MARL methods typically rely on access to global states or mitigate model errors through pessimistic or conservative policy optimization, leaving the reliability and refinement of learned world models less explored. We propose a Bayesian offline MARL framework that infers latent global states from decentralized observation-action histories and iteratively refines a posterior world model using only offline data. Building on posterior information loss, we extend this criterion to the multi-agent setting as multi-agent posterior information loss (MPIL), which provides a reliability measure for world model refinement and offline evaluation. Theoretically, we establish an upper bound on policy regret that is monotonic in MPIL and further characterize its dependence on the size and state-joint-action coverage of the offline dataset. Experiments on partially observable multi-agent variants of D4RL MuJoCo tasks demonstrate competitive policy performance across diverse offline data conditions, while MPIL-guided refinement improves world model prediction and downstream policy performance. These results support a principled connection between world model reliability, offline data coverage, and policy performance in partially observable offline MARL.
♻ ★ Mean Field Reinforcement Learning: Theory and Algorithms for Mean Field Control
This monograph provides an introduction to mean field reinforcement learning through the lens of Markov decision processes arising from large-population stochastic control with mean field interactions and common noise. Starting from the connection between multi-agent reinforcement learning and mean field control, it develops the probabilistic, mathematical, and control-theoretic framework needed to formulate representative-agent learning problems, analyze their relationship with finite-population systems, and study both general and linear-quadratic models. The presentation includes dynamic programming principles, propagation-of-chaos limits, and theoretical analyses of tabular Q-learning and policy-gradient methods. It also discusses numerical implementations, including tabular schemes and deep reinforcement learning methods such as deep deterministic policy gradient. The goal is to give readers a coherent bridge between mean field control theory and reinforcement learning methodology, emphasizing the mathematical structure of the problems and the design of tractable learning approaches for large stochastic populations.
♻ ★ NeuroPilot: An Agent-Driven Smart Pipeline for Processing, Quality Control, and Managing Neuroimages
Transforming raw neuroimage archives into analysis-ready derivatives relies on three brittle stages: data standardization, modality-specific preprocessing, and quality control (QC). While individual neuroimaging tools are well developed, their orchestration requires project-specific scripts, environment-adaptive tuning, and labor-intensive manual QC. To address this, we introduce NeuroPilot, a multi-agent system that digitalizes the expertise of neuroimage processing, QC, and data management into three LLM-invocable skills: dcm2bids-skill, neuroimage-pre-skill, and qc-agent-skill. The LLM-driven agent autonomously orchestrates workflows, generalizing various infrastructure settings into a single configuration to achieve the highest scalability. Demonstrating the system's generalizability, we deployed NeuroPilot across 17 cohorts (>123,000 subjects) spanning infant to aging populations and multiple MRI modalities (structural, diffusion, functional). In practice, after standardizing data via the dcm2bids-skill, the agent dynamically routes datasets to the optimal neuroimage-pre-skill based on available modalities and cohort traits (e.g., dispatching T1w and fMRI data to fMRIPrep, or selecting specialized pipelines for infant cohorts). The qc-agent-skill then drives an evidence-based, semi-automated QC via a 3-D browser dashboard, utilizing a multi-tiered verification system to optimize failed cases and escalate complex issues for supervisor inspection. Quantitatively, our QC agent screened 558 production subjects, validating its automated flags against FreeSurfer's topology-defect metrics. The infant processing pipeline achieved a 100% (201/201) completion rate on QC-validated inputs. Importantly, NeuroPilot compresses the traditional 2--3 month timeline for training staff and processing complete datasets into a single week. NeuroPilot is deployed in https://wanda-cyberbench.com/.
comment: 21 pages, 6 figures
Systems and Control (EESS)
★ A Global Routing Method for Electronic Circuit Design Integrating Flow Matching and Constrained A*
Global routing is a fundamental issue in the design of electronic circuits, as it directly relates to the efficiency, reliability, and compactness of the overall layout of integrated systems. As the demand for higher functionality and reliability within a limited chip area increases, the importance of routing as a design consideration has never been greater. Efficient global routing optimization techniques are therefore required to connect all pins with minimal wirelength (WL), while avoiding congestion and interference across multiple nets within the design space. As the number of nets to be routed increases, global routing inherently suffers from a combinatorial explosion in the design space, making it increasingly difficult to obtain feasible solutions. In this paper, we focus on solving constrained global routing problems based on a heuristic A*-based framework. Based on this concept, a hybrid approach is proposed by integrating constrained A* with flow matching (FM). The method utilizes FM to construct a global cost map that captures routing tendencies across the design space, which is then used to guide the constrained A* search process. This enables efficient exploration of the solution space while satisfying routing constraints. Experimental results demonstrate that the proposed method achieves comparable WL and via usage to baseline methods while maintaining constraint satisfaction across a wide range of scenarios. Notably, the framework shows strong capability in solving complex routing instances that are difficult to handle using conventional approaches. Additionally, the method shows robust generalization performance across different problem sizes and net configurations.
★ Charge-Balanced Control of Linear Systems: Controllability, Stabilizability, and Periodic Regulation
Electrical neuromodulation is a therapeutic approach that regulates neural activity by delivering electrical currents to brain tissue to treat brain disorders, such as epilepsy and Parkinson's disease. A critical safety requirement is charge balance, which ensures zero net injected charge over each stimulation cycle. Despite its clinical importance, the implications of charge-balanced stimulation for system dynamics and control remain poorly understood. This article investigates the controllability, stabilizability, and regulation of discrete-time linear systems under charge-balanced input constraints, motivated by applications in electrical neuromodulation. We introduce novel notions of charge-balanced controllability and stabilizability and establish necessary and sufficient conditions for both properties. We further demonstrate that nonzero steady states cannot be maintained under charge-balanced inputs, motivating the introduction of charge-balanced periodic orbits as alternative control targets. We characterize the conditions under which convergence to such orbits is achievable and develop optimal feedback controllers to stabilize the origin and track optimal charge-balanced periodic orbits. Finally, we validate the proposed control framework using a Hodgkin-Huxley neuron model, demonstrating successful suppression of tonic spiking and highlighting the potential of our approach for controlling nonlinear dynamical systems.
comment: 14 pages
★ A Data-Driven, Gain-Scheduled Equivalent Circuit for the B-H Behavior of Power Ferrites
An equivalent circuit for the dynamic B-H behavior of power ferrites is identified from measured transients. It consists of classical elements of hysteresis modeling: a nonlinear inductor for the reversible curve, six viscous play branches (inductors in series with diode-bridge sliders) that form a Prandtl-Ishlinskii model, two R-L branches for the magnetic after-effect, a mean-field feedback of H as in the Jiles-Atherton model, and a measurement path. All elements are positive apart from the feedback sources, and the circuit can run in SPICE. Discretized at the sampling step, the circuit is a difference equation with 32 parameters per material, fitted by backpropagation through time. On the 15 materials of the MagNet Challenge 2, the mean sequence relative error is 7.33 % against 7.77 % for the challenge-winning recurrent network with 336 parameters, and the circuit is more accurate on 12 of the 15 materials. The accuracy can be scaled up by gain scheduling, where a small network per material sets the element values at every step from the excitation, while the hysteresis memory stays in the circuit, such that the error drops to 5.25 % with 656 parameters per material. Thus, an interpretable model is demonstrated that achieves state-of-the-art accuracy.
★ Semantic Interoperability in Federated Digital Twin Systems: A Systematic Review Through the Levels of Conceptual Interoperability Model
When two companies connect their digital twins, software models of their plants kept up to date with live data, each twin must translate the other's data into its own terms, and that translation rule, called a mapping, can stop being correct when one company changes the meaning of its data without telling the other. This PRISMA 2020 systematic review screened 5,833 records and read 119 full texts to determine whether any published system detects such a broken mapping and repairs it while the twins operate. In all 32 included studies the mapping is fixed before operation, and the two ingredients of run-time repair appear only separately: cross-company matching without any check that the match still holds, and detection and repair of changed meaning within a single company. The review turns this gap into six requirements, each illustrated on one running example, that a system claiming run-time semantic interoperability between independently owned twins should meet.
★ Metabolic heat and moisture reuse with climate-adaptive control for energy-efficient black soldier fly bioconversion in modular container farms
Black soldier fly larvae (BSFL, Hermetia illucens) convert organic residues into feed ingredients and fertiliser. Climate control is a major contributor to the energy demand of container-based bioconversion. We developed a control-oriented dynamic model linking larval growth, substrate dynamics and container climate to heating, ventilation and air conditioning (HVAC). A two-level Model Predictive Control (MPC) strategy planned climate trajectories and tracked them through HVAC actuation. Four simulation studies examined controller selection, HVAC configuration, larval age pairing and inter-container coupling, using subsets of eight climate scenarios. Across the four scenarios tested for controller selection, dynamic MPC reduced specific energy consumption (SEC) by 31 to 61% and increased predicted dry biomass at harvest by up to 27.5% relative to single fixed setpoints. Under the tested controller settings, active cooling was required in the two hot scenarios, whereas dehumidification provided no additional operational benefit. Pairing larval ages four days apart within one container reduced SEC by up to 33% in the tested non-hot scenarios. In the Cooldry scenario, two age-separated containers connected by an actuated air duct produced 9.0% more biomass and achieved 3.9% lower SEC than two independent containers with mixed ages, despite using 4.6% more total HVAC energy. The simulations suggest that dynamic climate control and metabolic heat and moisture reuse can improve the energy intensity of BSFL bioconversion. The quantitative estimates and hardware recommendations require experimental validation before application to industrial facilities.
comment: Submitted to Journal of Insects as Food and Feed. Submitted PDF consists of 6 figures
★ Rare Event Constrained Reinforcement Learning with Large Deviations
We study finite-horizon reinforcement learning with a safety constraint on the probability of reaching a failure set. In contrast to the well-studied safe reinforcement learning literature, we consider problems where the probability of reaching the failure set is vanishingly small, but the consequences of reaching the failure set are catastrophic. Direct simulation and estimation of the optimal policy is naturally ineffective when the failure probability is extremely small. Moreover, on a real system the controller can only choose actions, and this is not enough to make failures frequent. In contrast to the rare event simulation literature, our goal is not to estimate the probability of the rare event occurrence, but to upper bound it such that the failure constraint is satisfied. Thus, instead of sampling failures, we use the large deviations principle. We define a damage variable that on exceeding a certain threshold indicates failure. We study the cumulant generating function (CGF) as its positive root provides an upper bound on the failure probability via Cramer's theorem. We extend this approach to Markov decision processes with hitting times through state-action CGFs, and give finite-horizon and state-dependent versions of the bound. The state-action CGFs do not depend on the policy and can be estimated from regular transitions, so the bound can be checked for any policy a priori. This gives a policy improvement algorithm in which every policy satisfies the failure constraint with high probability.
★ Parameterisation of NMC and LFP 4680 cells for electrochemical and thermal modelling
An electrochemical-thermal model of the 4680 cylindrical cell is presented: a Doyle-Fuller-Newman model of the electrode stack coupled to a two-node thermal balance of the roll and the can (a thermal DFN, TDFN), solved by an implicit finite-volume scheme. Two cells are parameterised from teardown geometry and open literature, with every value tagged by provenance: the Tesla NMC811/graphite tabless cell and the BYD FC4680 LiFePO4 multi-tab cell. The reaction heat is evaluated at the open-circuit potential of the particle-mean concentration, which restores the heat of mixing. Both sets are calibrated in the same way, with the test fixture identified from the measurement alone. Against the 1C and 2C charge test of Ank et al., the calibrated NMC set follows the mean can temperature within 0.52 and 1.46 K, with a contact heat of 0.94 mOhm matching the authors' estimate for their holder. For the LFP cell, 1C and 2C discharges to 2.0 V were measured on the authors' bench; the fixture conductance (0.337 W/K) follows from the energy balance and the heat capacity (277 J/K) from the cooling. With the end-of-lithiation drop of LiFePO4 restored in the cathode open-circuit potential, the calibrated set follows the can temperature within 0.44 K (1C) and 0.76 K (2C), reproduces the peaks and the relaxation after cut-off. What remains is what a single can node and a single representative particle do not describe: the axial gradient of the NMC can under contact heating, and a gradual decline of the LFP voltage before the knee at 2C, which points to the distribution of particle states in the electrode. The parameter sets (BPX, CSV) and the bench data are openly available.
comment: 36 pages, 6 figures, 4 tables; supplementary material included. Submitted to Frontiers in Batteries and Electrochemistry. Data: https://github.com/RemusTeodorescu/4680-cell-models
★ FADE: A Flow-Based Approach for Multichannel Tail Estimation
Ultra-Reliable Low-Latency Communication (URLLC) is vulnerable to rare low-received-power events, and the spatial-diversity benefits of multiple receivers can be significantly reduced when they experience simultaneous deep fading. Multivariate Extreme Value Theory (MEVT) provides a principled framework for characterizing such joint tail behavior. However, conventional MEVT methods often rely on restrictive parametric assumptions or discrete representations of the dependence structure, which may not fully capture complex multivariate tail behavior. This paper proposes Flow-based Angular Density Estimation (FADE), a flow-based approach for characterizing the dependence between extreme events across receivers within the MEVT framework. FADE transforms multivariate extremes into a standard extreme-value representation and uses a flow-based density estimator to learn the dependence structure from the transformed data without imposing a predefined parametric form. The proposed framework is evaluated using experimentally collected data from a UAV communication system, in which the UAV serves as the transmitter and communicates with two ground receivers. FADE provides closer agreement with the empirical joint CDF, achieving reductions of up to 22.4% in MAE compared with the state-of-the-art Logistic and Maximum Empirical Likelihood (MEL) methods, while also improving distributional agreement and reducing absolute bias.
★ Distributed State Estimation over Wireless MIMO Fading Channels with Imperfect CSI
We consider distributed state estimation of a linear plant under stabilizing feedback over wireless multiple-input multiple-output (MIMO) fading channels, where channel-estimation noise corrupts the channel state information (CSI) used for information fusion. We propose a distributed estimator that combines local prediction, communication over wireless MIMO channels, and a posterior update based on local information and communicated data. Each sensor uses noisy pilots to estimate its aggregated wireless MIMO fading channel, and a pilot-noise bias correction recovers the required neighboring-data differences in conditional mean. We first analyze the mean estimation error and show that a Chebyshev design of the communication weights achieves a prescribed decay factor $d\in(0,1)$ satisfying the sensing conditions, with finitely many communication updates per plant sample at every finite signal-to-noise ratio (SNR) gain. We then characterize mean-square stability and the effects of SNR gain and receive-array size on the state estimation mean-square error (MSE), including its transient and steady-state value. In the ideal-communication limit, we show that the square root of the absolute MSE deviation from its steady-state value decays as $o(d^k)$. Numerical simulations verify the theoretical results and show substantial performance gain compared to various state-of-the-art baselines.
comment: 27 pages
★ A Deep Reinforcement Learning Framework for Real-Time Electric Vehicle Charging Management in IoEV-Enabled Smart Cities
The rapid growth of electric vehicles (EVs) in urban environments demands intelligent, decentralised charging management strategies that operate under strict real-time constraints. This paper proposes an Edge-Intelligent architecture in which each EV selects a charging station (CS) by querying a shared, centrally trained Deep Q-Network (DQN) policy over a lightweight MQTT publish-subscribe backbone; independent per-EV edge inference is the deployment target, not the configuration evaluated here. The reported simulation evaluates that shared DQN policy applied sequentially to each vehicle's observations within a single SUMO run, using MQTT for KPI telemetry while observations are obtained through TraCI. The system is evaluated on two real-world road networks from OpenStreetMap (Rabat Agdal-Hay Riad and Tangier, Morocco); the primary network carries four geolocated public charging stations with heterogeneous power levels (50 kW and 150 kW). Three traffic demand profiles (morning, midday, evening peak) are simulated, and four decision policies are compared: an Adaptive Heuristic, a Tabular Q-Learning agent, the proposed DQN agent, and a Double DQN variant. A multi-seed statistical protocol (7 seeds, Wilcoxon signed-rank test, Cohen's d) evaluates the results. The DQN agent achieves a 6.8% relative improvement in charge rate and a 5.4% increase in delivered energy over the Adaptive Heuristic under Evening Peak demand ($p = 0.016$, Cohen's $d = 0.99$ and $1.51$), while showing no advantage in waiting time; on the second network the heuristic leads on waiting time, so the ranking is not stable across geographies. A scalability analysis shows linear state-space growth with station count and sub-millisecond inference; all runs include intermittent station outages under a Markov failure process.
comment: 17 pages, 4 figures. Authors' accepted manuscript (paper ID 733916, AI2SD 2026 cycle), to appear in an indexed Springer Lecture Notes in Networks and Systems volume. Earlier preprint version deposited on Zenodo: https://doi.org/10.5281/zenodo.22208979
★ A sensor-driven physics-informed neural network for real-time occupant-centric space heating control
Conventional space heating systems typically rely on single or sparse fixed-point temperature measurements, overlooking non-uniform indoor temperature distributions, transient thermal dynamics, and occupant-specific comfort requirements. This study explores an occupant-centric control strategy that aims to maintain the desired temperature around the occupant location. Spatially and temporally varying temperature fields, together with occupant mobility, make real-time local temperature estimation challenging. To address this challenge, we propose a control-oriented Physics-Informed Neural Network (PINN) model for indoor temperature-field prediction. Conventional PINNs rely on explicit time inputs and assume consistent temporal patterns across samples, which limits adaptability under variable initial conditions and heating sequences. The proposed model infers temporal dynamics implicitly from spatial coordinates and limited sensor measurements, improving robustness to changes in initial conditions and external disturbances. The method was validated across nine horizontal sensing planes in three controlled room-scale configurations. Against a validated COMSOL Multiphysics simulation, the PINN predicted the evaluated temperature fields with a maximum relative error below 1.85% across validation cases. These predictions supported an occupant-centric space heating control scheme. Compared with COMSOL-based offline control, the PINN-based system achieved an average prediction error of around 0.67%, and its rapid inference supports real-time occupant-centric control implementation. This work represents a move away from reliance on fixed-point temperature measurements in space heating control to an occupant-centric control strategy informed by real-time environmental feedback.
★ Direct Covariance-to-Beamformer Mapping for Low-Latency Hybrid ISAC Beamforming with Subsequent DoA Estimation
Integrated sensing and communication (ISAC) systems typically employ a two-stage beamforming pipeline, in which target directions are first estimated from the received covariance matrix and then used to compute the beamforming weights by solving non-convex optimization problems. Although effective, this approach introduces processing latency that can limit beam adaptation in dynamic environments. This letter proposes a beamforming deep neural network (BFDNN) for direct covariance-to-beamformer learning in ISAC systems. BFDNN is a compact regression network that maps covariance-matrix features directly to the hybrid beamforming weights produced by a hybrid ISAC beamformer, eliminating the direction-of-arrival (DoA) estimation stage during inference. Compared with MUSIC, classification-DNN, and regression-DNN based beamforming pipelines, BFDNN generates comparable beam patterns and higher effective spectral efficiencies while reducing the beamforming prediction latency by 25-30 times on the evaluated platform, demonstrating the potential of direct covariance-to-beamformer learning for low-latency ISAC beam updates. Moreover, the target directions can be inferred indirectly from the generated radiation pattern, yielding DoA accuracy competitive with that of dedicated learning-based methods in the evaluated scenario.
★ Multirate Data-Driven Predictive Control: Rank Obstructions, Identifiability, and Noise Conditioning
We develop rank, identifiability, and noise-conditioning results for compact input--output predictors in multirate data-driven predictive control. The inverse of a square zero-input response matrix used in an existing compact-lifting construction is shown to be undefined under its observability and minimal-polynomial assumptions. We derive exact replacements using rank-complete data, characterize all predictor extensions, and establish the minimum-norm and minimax Euclidean noise properties of the canonical Moore--Penrose extension. A joint predictor is identified directly from mixed experiments with arbitrary initial states. Deterministic perturbation bounds quantify reconstruction errors through informative singular-value margins, while a positive-definite reduced weighting preserves the nominal quadratic cost. Sampling-rate observability and event-triggering analyses identify additional conditions needed for reconstruction and stability guarantees.
★ QFabric: Guarded Cross-Domain Task Recovery and Its Limits on a Linux-RTOS Board
Heterogeneous embedded boards pair a Linux application processor with a real-time microcontroller (MCU), which suggests moving a task between the two domains when its timing degrades. We present QFabric, a prototype that treats cross-domain placement as guarded recovery from a violated empirical soft real-time contract: a bounded task is compiled for both domains, monitored in windows with hysteresis, switched only after semantic and capacity admission at a quiescent epoch boundary, and committed or rolled back after probation. On an Arduino UNO Q (Cortex-A53 Linux and a Cortex-M33 running Zephyr), we characterize the stock Bridge RPC interconnect and measure five workloads in both domains under idle, CPU-load, and higher-priority SCHED_FIFO contention, with ordinary and real-time callers. Each MCU invocation costs about 19 ms of dispatch regardless of compute, the MCU runs the heaviest kernel 34x slower, and because every call traverses Linux userspace, Linux contention inflates the MCU path far more than the Linux path (p99 up to 400 ms versus 17.0 ms). Across 30 cells of a four-node chain, Linux meets every contract within 0.8% misses and QFabric correctly never migrates; a controller without hysteresis migrates in 7 cells, raising p95 by up to 177x, and probation rolls back 5 of those moves. On Linux-dispatched heterogeneous boards, migration therefore cannot isolate a task from Linux interference, and recovery guards matter chiefly by preventing harmful moves. This paper supersedes an earlier artifact release whose reported improvement arose from measurement defects that we identify. All source, raw traces, and generators are public.
comment: 9 pages, 4 figures, 8 tables. Code and data: https://doi.org/10.5281/zenodo.23692354
★ Dual Path Following for an Aerial Manipulator Under State and Input Constraints via Transverse Feedback Linearization
This paper studies dual path following for a quadrotor carrying a three-degree-of-freedom arm. The center of mass of the combined system must follow a vehicle path and the end effector must follow a second path, while the arm respects its joint limits and avoids its singular postures. The vehicle and the arm are mechanically coupled, so one controller must handle both paths. With the center of mass and the end-effector path coordinates as outputs, and a dynamic extension of the thrust, the system admits a normal form without internal dynamics. We characterize the states where this normal form fails. We formulate the transverse feedback linearizing controller as a quadratic program that relaxes only the speed of the end effector along its path and imposes safety and actuator limits as inequality constraints. The program reduces to a scalar problem with a closed-form solution. If it remains strictly feasible for all time, the feedback is locally Lipschitz, both paths are invariant and attractive, and every constraint holds. In the physics engine Drake, the controller resolves a conflict between the two speed references by slowing the end effector while both paths remain invariant. By contrast, a cascade with a safety filter leaves both paths. Code and animations are available at https://gradslab.github.io/am_dual_pf-/
comment: 8 pages, 3 Figures
★ Uncertainty-Aware Joint Generative Modeling of Fault Event Sequences and Sensor Time Series in Solar PV Plants
This paper presents a multimodal generative model designed to simulate a solar energy plant system for predictive maintenance and fault prediction. We propose a novel approach to enhance system generation through conditional generation by leveraging both event sequences and time-series data and improve next event prediction. The framework employs a Transformer Hawkes Process (THP) to encode event data and an iTransformer to encode time-series data, integrated via a co-attention mechanism that effectively captures cross-modal dependencies between discrete events and continuous signals. From these integrated representations, we employ two specialized decoders for simulation: a Conditional Latent Diffusion Model (CLDM) for modeling future sensor trajectories and the Score Matching-based UnceRtainty quantiFication for Transformer Hawkes Process (SMURF-THP) for high-fidelity event prediction. This dual-decoder strategy supports uncertainty-aware, history-conditioned simulation by rolling the model forward for a specified number of generation iterations, producing finite stochastic scenarios rather than deterministic extrapolations over the full observational record. Evaluated on a comprehensive dataset from the Red Rock Solar Site in Arizona, our approach improves aggregate event-prediction F1 relative to baselines and supports applications in predictive maintenance, fault detection, and energy optimization for solar power systems.
comment: Author preprint of a paper accepted to IEEE Transactions on Automation Science and Engineering; 18 pages including supplementary appendix
★ Finite Global $H_{\infty}$ Optimization of the $Q$-Filter Time Constant for Nominal-Response Recovery in SISO Disturbance Observers under Parametric Plant Uncertainty
This paper develops a finite real-algebraic procedure for global H-infinity optimization of the time constant of a fixed-shape Q-filter in a single-input/single-output (SISO) disturbance observer (DOB) under affine parametric plant uncertainty. Robust internal stability defines a possibly disconnected feasible domain, and the objective is the worst-case weighted mismatch between the uncertain DOB reference response and the nominal closed-loop response. A recovery-error factorization preserves affine dependence of the performance numerator and denominator on the same uncertainty vector. Phase lifting and a complex-polynomial Edge theorem reduce the uncertainty box to finitely many polynomial edges, while quadratic dependence of the squared frequency-domain inequality on the edge coordinate permits analytic elimination of that coordinate. The resulting semialgebraic formulation has only the squared frequency as a universally quantified variable. It determines the global infimum, decides whether it is attained, and constructs the complete optimizer set over the full robust-stability domain. A DOB case study with a plant family containing non-minimum-phase members yields a unique optimal time constant of approximately 0.0364075 and a worst-case norm of approximately 2.29975. The optimizer lies strictly inside the first robust-stability component, and every smaller feasible time constant in that component has a larger worst-case norm. Finite algebraic certificates establish robust stability, performance feasibility, and strict global optimality.
comment: 14 pages, 2 figures, 1 table
★ Differentially Private Distributed Optimization via the Intrinsic Randomness of Zeroth-Order Estimators: Sampling Geometry, Convergence, and Privacy Guarantees
Distributed zeroth-order optimization has gained increasing attention in applications where exact gradients are computationally expensive or unavailable. However, the iterative exchange of local decision variables makes distributed networks vulnerable to inference attacks. While differential privacy provides a rigorous framework for privacy protection, conventional approaches generally require additional random noise, which inevitably affects optimization accuracy. In this paper, we investigate whether the randomness inherent in zeroth-order estimation can itself provide privacy protection. We develop a distributed zeroth-order gradient descent method that integrates consensus updates with a ball-uniform two-point gradient estimator. Our theoretical analysis shows that our algorithm attains standard convergence guarantees in both online and offline optimization settings. We then characterize the relevant probability distributions of the associated privacy mechanism, and establish differential privacy guarantees for both the online and offline optimization settings. These results reveal the potential of intrinsic zeroth-order randomness for privacy protection.
♻ ★ When Should a Robot Replan? Regret-Guided Update Scheduling in Time-Varying MDPs
Robots operating in non-stationary environments must continually adapt their policies as the dynamics drift, but onboard energy and compute budgets cap how often a full state estimation and re-planning step can be performed. This raises a question: \emph{when}, along a horizon, should a robot spend its limited budget? We formulate this problem in time-varying Markov decision processes (TVMDPs) with a known bound on the rate of transition drift. We model execution as a \emph{skip-update} scheme in which, at chosen update times, the agent estimates the transition kernel by maximum likelihood and computes a finite-horizon policy, and between updates reuses this policy under a propagated state estimate. We analyze the dynamic regret of this scheme and show how it grows during skip intervals in terms of the properties of the TVMDP and the skip lengths; the resulting bound answers the opening question via an online, regret-guided update rule that allocates the budget adaptively. We evaluate the rule in a simulated Mars-rover navigation task with time-varying slip dynamics and on a Crazyflie quadrotor in indoor obstacle fields. Adaptive allocation outperforms other budgeted baselines.
♻ ★ Global Parameter Identification and Uncertainty Quantification of reduced-order ADM1 using Maximum Likelihood Estimation and Bayesian Inference
Anaerobic digestion (AD) models can be challenging to calibrate against real data due to limited practical identifiability, data scarcity, and model complexity. Weighted least squares objective functions and gradient-based solvers using finite differences are often used in literature, but result in suboptimal parameter estimates and large uncertainty bands. This paper identifies the parameters of a reduced-order ADM1 by comparing the frameworks of maximum likelihood estimation (MLE) and Bayesian inference (BI), and quantifies uncertainties of parameters and outputs. Multirate experimental data are taken from a lab-scale AD reactor fed with agricultural substrates and operated at organic loading rates of 3$\,$g VS $\mathrm{m^{-3}d^{-1}}$. Global optima are found via global and gradient-based local search algorithms using forward sensitivities. Enhanced scatter search provides the best global search results. These are refined by the Gauss-Newton algorithm as the best local search, which delivers $R^2$ values of 0.95 for acetic acid and 0.77 for biogas production. BI yields substantially narrower uncertainty bands than MLE and avoids the need for a global search by incorporating prior knowledge about parameter distributions. The search space is captured efficiently by log10 scaling of parameters. Algorithmic details on its implications are provided. Despite a high-resolution lab-scale dataset, the ADM1-specific inhibition parameter $K_\mathrm{I,nh3}$ is practically not identifiable with MLE, and shows a low output sensitivity, whereas a parameter for the effective ion concentration shows superb practical identifiabilty at high confidence with both MLE and BI. The present work lays a scalable base for calibration of nonlinear AD models by providing reliable simulations. It thus advances model-based monitoring and control applications of multirate processes such as agricultural AD plants.
comment: v3: outsource case studies to appendix a
♻ ★ A Data-Driven Krasovskii-Based Approach for Safety Controller Design of Time-Delayed Uncertain Polynomial Systems
We develop a data-driven framework for the synthesis of robust Krasovskii control barrier certificates (RK-CBC) and corresponding robust safety controllers (R-SC) for discrete-time input-affine uncertain polynomial systems with (partially) unknown dynamics, while explicitly accounting for unknown-but-bounded disturbances and time-invariant delays using only observed input-state data. Although control barrier certificates have been extensively studied for safety analysis of control systems, existing work on unknown systems with time delays, particularly in the presence of disturbances, remains limited. The challenge of safety synthesis for such systems stems from two main factors: first, the system's mathematical model is unavailable; and second, the safety conditions should explicitly incorporate the effects of time delays on system evolution during the synthesis process, while remaining robust to unknown disturbances. To address these challenges, we develop a data-driven framework based on RK-CBC, extending the classical CBC formulation for delay-free systems to explicitly account for time delays by aggregating delayed components within the barrier construction. The proposed framework relies solely on input-state data collected over a finite time horizon, enabling the direct synthesis of RK-CBC and R-SC from observed trajectories without requiring an explicit system model. The synthesis is cast as a data-driven sum-of-squares (SOS) feasibility program, yielding a structured design methodology. As a result, robust safety is guaranteed in the presence of unknown disturbances and time delays as long as the closed-loop trajectories remain within the state set. The effectiveness of the proposed method is demonstrated through three case studies, including two physical systems.
♻ ★ Autonomous fixed-wing interception using reinforcement learning
Medium altitude, long endurance unmanned aerial systems such as Shahed-type strike drones are an unanswered threat in modern conflicts. Conventional air defense systems are too expensive to provide a feasible large-scale solution. Quadcopter interceptors are a cheap, battle-proven alternative, but their extremely limited flight time restricts them to low altitude targets at short range, whereas fixed-wing platforms have the endurance to reach higher altitude targets from longer distances. However, such interceptors are still operated manually. Recent developments in learning-based controllers show potential for automation. In this work, a fixed-wing interceptor is presented that is controlled by a neural network that maps filtered monocular vision observations directly to actuator commands. The policy was trained in a custom Python environment using reinforcement learning and proximal policy optimization. The policy was then deployed on a fixed-wing platform and tested in real interception scenarios. As a baseline, two classical guidance controllers were also developed and tested in simulation. The policy is able to intercept targets both in simulation and in real flight. In simulation, the policy outperforms the classical controllers under most conditions. The interceptor was ultimately successfully validated in real flight by intercepting maneuvering fixed-wing targets. This novel system shows the potential of artificial intelligence for autonomous control of fixed-wing interceptor drones.
comment: MSc thesis, Faculty of Aerospace Engineering, Delft University of Technology, 2026. 24 pages, 29 figures, 4 tables
♻ ★ Constrained Sparsest Input Selection for Structural Controllability is NP-Hard
We study the problem of selecting the fewest admissible input--state links that make a structured linear system structurally controllable. This objective counts nonzero entries from prescribed positions of the input matrix and is different from minimizing its number of nonzero columns. Prior work gives polynomial-time algorithms for several restricted cases, including single or dedicated inputs and prescribed input patterns satisfying additional graph-theoretic conditions. We give a polynomial-time reduction from perfect three-dimensional matching, establishing for the first time that the decision problem is NP-complete and the optimization problem is NP-hard. A refinement of the reduction yields an exact optimum relation and shows that the problem is APX-hard. An example also exhibits a strict integrality gap for the natural linear programming relaxation.
comment: 6 pages, 1 figure
♻ ★ Feedforward Density-Driven Optimal Control (D2OC) for Tracking Time-Varying Distributions
Tracking a time-varying spatial distribution poses a fundamental challenge for multi-agent systems, as agents often lag behind a moving target when reacting solely to the current target configuration. This paper develops a feedforward extension of Density-Driven Optimal Control (D$^2$OC) for dynamic distribution tracking with general discrete-time linear time-invariant agents. The proposed framework predicts reference sample trajectories along a known velocity field and incorporates these anticipated locations into the finite-horizon control problem. A scalar factor explicitly adjusts the feedforward action to account for predicted reference displacement. We establish an explicit receding-horizon control policy and provide a formal performance analysis that quantifies the tracking impact of prediction limitations. Furthermore, under a nominal stability condition, we derive explicit ultimate bounds on agent tracking errors and local Wasserstein distances in the presence of bounded reference drift, discretization errors, and transport perturbations. Monte Carlo simulations with linearized quadrotor dynamics demonstrate that incorporating feedforward compensation significantly mitigates tracking lag and enhances spatial distribution matching. These results offer formal analytical guarantees for predictive distribution tracking while preserving the computational efficiency and local optimization structure of D$^2$OC.
Robotics
♻ ★ Lifted Schrödinger Bridges for Gaussian Mixture Endpoints: Projection Gaps and Path-Space Obstructions
We study stochastic density control between Gaussian-mixture endpoint distributions under Brownian prior dynamics. Since the direct Schrödinger bridge between Gaussian mixtures is generally not available in closed form, we introduce a lifted path-space construction in which each trajectory is augmented with a source--target component label. Consequently, the problem decomposes into Gaussian component-to-component Schrödinger bridges with explicit marginal, drift, and cost formulas, while the mixture-level assignment reduces to a finite-dimensional entropic coupling problem with a Sinkhorn scaling form. We then analyze the projection obtained by discarding or forgetting the label. By construction, the projected law satisfies the original Gaussian-mixture endpoint constraints, but its relative entropy generally differs from the lifted relative entropy by a nonnegative conditional label-information gap. This gap reveals a path-space obstruction: the lifted optimizer cannot, in general, be identified with the direct unlabeled Schrödinger bridge after projection. We also derive the posterior-averaged Markov drift associated with the projected marginal flow, prove a kinetic-energy upper bound, and identify a common path-potential condition under which the projection gap vanishes. Several numerical illustrations showing density and shape control are recorded for a self-contained exposition.
comment: 37 pages; revised version of a journal submission with a new section on duality certificates
♻ ★ Tactus: Tactile & Pose Sensing Glove for Natural Dexterity Data Collection
Collecting naturalistic human manipulation demonstrations with accurate contact force and hand pose is essential for robot dexterous manipulation policy and foundational world model learning - yet no existing wearable platform provides both calibrated fingertip force sensing and occlusion-resilient hand tracking in a form factor that preserves natural manipulation behavior. We present Tactus, a textile-integrated sensing glove that co-optimizes three design pillars: (1) a thin, conformal 3D-knit form factor (1.5 mm fingertip stackup) designed to minimize interference with natural manipulation, (2) strategically placed capacitive tactile sensors at manipulation-relevant regions informed by empirical contact studies, and (3) IMU-fused pose tracking that maintains accuracy through the hand-object occlusions where manipulation data is most valuable. Tactus includes a documented calibration pipeline for cross-glove consistency and per-session calibration for subject-dependent correction. To enable objective comparison, we introduce standardized benchmarking protocols for tactile force regression and hand pose estimation, and apply them to Tactus and two commercially available tactile gloves - to our knowledge the first quantitative cross-platform evaluation of this kind. Tactus resolves a minimum detectable force of 0.59 N at 95% confidence, 1.6 - 2.7x lower than the commercial baselines, and with 0.87 cross-participant consistency, while our pose tracking sustains 11.0 mm fingertip keypoint relative error during manipulation, when occlusion is heaviest. We show that these combined signals support object property estimation, slip detection, and representative recording of hand-object interaction - yielding demonstration data with the fidelity required for contact-aware manipulation policies and world models.
comment: 9 pages
♻ ★ Terrain-Adaptive Grouser Wheel for Optimal Planetary Exploration: Design and Experimental Investigation
Planetary rovers operating in extraterrestrial environments often encounter significant mobility challenges due to varying terrain features such as gradients and granularity. While recent works in multimodal wheel design have explored adjustments in stiffness, compliance, and diameter as a means to improve terrain adaptability, full wheel grouser-adjustable designs remain largely unexplored. Grousers are a compelling feature to actuate, as granular terrains tend to require increased grouser height for improved wheel performance. As a result, we introduce GROVER, a multimodal wheel capable of continuously adjusting its grouser height for terrain adaptation. Fixed-height performance was characterized on vinyl, coarse rock, pea gravel, and sand under two packing states. Across 950 trials, the best configuration varied with terrain, reducing slip by 34.6\%--58.0\% in granular terrains and improving travel time and energy consumption by up to 77.4\%. No single grouser height minimized slip across all terrains, motivating mid-traversal reconfiguration toward lower-slip operation. Two experiments demonstrated this. First, an instructed terrain transition changed grouser height toward a known low-slip configuration during motion, with slip decreasing by $44.0\pm4.7\%$ from the transient peak across 20 trials. Second, an online adaptation routine used closed-loop slip feedback alone to search the grouser-height range for a lower-slip configuration on four terrains with previously uncharacterized granularities: three field terrains and one laboratory regolith terrain. The controller selected different heights, with settled slip 25.4\%--37.2\% lower than the pre-settled mean. Together, these results demonstrate GROVER's ability to improve mobility by varying wheel morphology during traversal toward lower-slip operating states.
comment: Under Review
Multiagent Systems
★ MAPLE: A Multimodal Mobile Agent with Playbook-Learned Experience
Autonomous agents powered by vision-language models (VLMs) show great potential in navigating mobile graphical user interfaces (GUIs) to complete complex tasks. However, existing paradigms face fundamental scalability bottlenecks: fine-tuning requires massive high-fidelity datasets or prohibitive training time and compute, while gradient-free approaches rely on custom static heuristics or costly trial-and-error trajectory exploration where feedback is deferred until the end of execution. To address these limitations, we introduce MAPLE (Multimodal Agent with Playbook-Learned Experience), a novel gradient-free multi-agent framework that enables dynamic, step-wise experience sharing and execution-time prompt evolution. Unlike traditional action-reflection architectures that depend on costly, post-hoc self-reflection, MAPLE enables agents to cross-reference and distribute runtime insights in real time at each operational step. This dynamic exchange eliminates blind trial-and-error looping and steers agents along direct, grounded interaction paths. Extensive evaluations on the AndroidWorld and MobileWorld benchmarks demonstrate that MAPLE achieves state-of-the-art performance on complex, long-horizon mobile tasks, while proving scalable and adaptable across novel environments.
★ What Does a Safety Label Measure? Auditing Situational-Safety Benchmarks for Multimodal LLMs
Situational-safety benchmarks label each scene SAFE or UNSAFE and score a multimodal large language model (MLLM) against that label, usually by judging its free-form answer, and these scores guide deployment. We ask what such a label measures. A label score can come from the scene, the request or the benchmark's own files. We propose the Benchmark Card, seven measured fields that tell them apart, and fill it for six benchmarks. Only MSSBench chat leaks the label through its image files: image size alone reaches 82.3% accuracy, matching or beating six of seven commercial models. Elsewhere the signal outside the scene comes from design. In MOSSBench the item partition fixes the label, and on HoliSafe the request alone ranks pairs about as well as the scene. On our corrected chat split, the size rule falls to chance. We then introduce the Spot-Stop decomposition, which compares ranking with labelling on pairs of a safe and an unsafe scene. It splits the gap above a label-swap null into three parts: a threshold part, overlap between safe and unsafe scores, and order that holds only within a pair. Most of fourteen open MLLMs rank better than they label, beyond what their AUROC predicts. On chat, one shared threshold repairs a third to a half of the excess, and label-free recentering recovers it on balanced batches. Recentering on unlabelled scenes of the same situation improves ranking on VSCBench for 13 of 14 models.
★ Robust Safety Assurance for Robotic Swarms Using Decentralized Collision Verification
In this paper we present the methodologies, results, and analysis used to deploy a decentralized safety verification algorithm on a robotic swarm to prevent collisions at runtime in an outdoor environment. Typically, approaches toward verification of swarms of cyber-physical systems (CPS) are evaluated solely in numerical simulations or in highly controlled, noiseless indoor environments. Our efforts to implement collision avoidance on a real swarm system outdoors have revealed that the underlying algorithmic assumptions juxtaposed against the implementation details of the target system are critical factors in a method's efficacy in the real world. Here we modify, enhance, and analyze a decentralized reachability-based method for collision avoidance on a size, weight, and power-constrained uncrewed aerial vehicle (UAV) swarm with the objective of illustrating the challenges of real-world CPS deployments as well as provide guidance for researchers in this space. Verification performance, resource utilization, and timing data were gathered over the course of 140 total outdoor vehicle flights, including 6 swarm flights composed of 8 UAVs, and used to improve collision-avoidance performance as experimentation progressed. Our extensive experiments show that the proposed decentralized runtime verification approach can efficiently verify the safety of multiple UAVs in real-time and help all agents take safe actions to avoid collisions. On average, each agent, using our open-source safety_checker library, takes approximately 500-520 ms to verify its safety for the next 4 s into the future, in which the reachable set broadcast time dominates the total verification time (~300-500 ms).
comment: Accepted version. Published in IEEE Transactions on Aerospace and Electronic Systems, vol. 62, pp. 9706-9717, 2026. 13 pages, 6 figures, 4 tables. Video: https://youtu.be/Y-Z3IN--_Vc
★ GURU: A Hierarchical Neuro-Symbolic Multi-Agent System for Verification-Aware Automation of Mission-Critical Digital Engineering and Operations
An AI can find a safe action faster than any human, but it must never decide what counts as safe. We present GURU, a hierarchical neuro-symbolic multi-agent system for mission-critical digital engineering and operations, built on one partition of authority: learned models propose, rank, and accelerate; a symbolic rules engine checks every consequential action against hard limits before it executes; and validated physics confirms and decides. GURU runs a workflow as a closed control loop, with Navigation, Action, and Planning agents sharing one Common Internal Language and one training factory. Nuclear, aviation, and national-security guidance now requires this pattern; a harnessed language model keeps the model at the center, GURU puts a validated reference there. In computational fluid dynamics, the workflow raised the converged-simulation rate on unseen aircraft geometries from under 1 percent to 74 percent over 100 single-attempt cases, ran 54 live cases to Mach 6 on three facilities, and an ablation put the symbolic layer at 13 points of boundary-layer capture. Knowledge-graph retrieval and role adapters raised ParaView control success from 55 to 75 percent. In space domain awareness, where a language model cannot be trusted to decide, a frozen self-supervised world model under validated propagation scored highest of five methods at conjunction risk, including fully fine-tuned language models, with several-times-lower seed variance: the trustworthy design gives up nothing in accuracy, and the recipe transferred to reactor anomaly detection. A radar-to-avoidance debris pipeline was demonstrated in stages on a 2026 reentry; the hierarchy coordinates simulated teams of 200 aircraft. Training scaled near-linearly past 1,000 Aurora nodes and ran at 8,000. The approach extends to a control-grade microreactor digital twin with validated physics, not a learned model, as the safety basis.
comment: 82 pages, 50 figures, 7 tables
★ Personalized Rollout Allocation for Federated LQR Policy Optimization under Heterogeneous Process Noise
We study federated model-free learning of a common linear quadratic regulator (LQR) gain under a limited uplink communication budget, with clients that share dynamics and costs but experience different process-noise levels. Conventionally, clients transmit gradient matrices whose dimension equals the number of controller parameters. Exploiting the structure of two-point zeroth-order estimates, we consider an implementation in which clients transmit only one scalar cost-difference coefficient per rollout pair, while the server regenerates the associated perturbation directions from shared seeds. Communication thus scales with the number of rollout pairs rather than the controller dimension, turning the uplink budget into a rollout budget. Because noise heterogeneity produces gradient estimates of widely differing variance and every client carries equal weight in the fleet objective, uniform rollout allocation allows noisy clients to dominate the aggregated update and increases the probability of destabilizing updates. We propose a rollout-allocation scheme that assigns rollout pairs in proportion to the standard deviation of each client's estimates and show that it minimizes the variance of the aggregated update. The scheme is implemented exactly through a greedy integer allocation rule whose weights are estimated from the communicated scalars. We further prove that a single allocation factor quantifies its improvement over uniform allocation in a one-round instability bound, the certified step size, and the error floor of a joint stability and convergence guarantee. Simulations show that, under the same communication budget, personalized allocation reduces the one-step instability rate from $37\%$ to $0.3\%$.
★ Autonomous Analysis of Thermoelectric Materials Big Data by Multi-Agentic AI and Construction of a Knowledge Base and Specialized AI
Extracting knowledge from large databases of experimental materials data requires repeated analyses and reusable records of their choices and results. We developed Multi-Agentic AI (MAA), a multi-agent system in which AI agents with distinct roles analyze experimental thermoelectric data under predefined instructions and quality checks, saving analysis records traceable to their sources. In four experiments varying theme and cycle counts, we measured what accumulated in these records. In the largest experiment (100 analyses across 10 themes and 10 cycles), concept co-occurrences were concentrated within themes, vocabulary was shared across themes and some relation descriptions were first detected in later cycles. A specialized AI that searches this experiment's knowledge bases and reads their analysis records answered materials questions with cited sources. Retrospective checks, including tracing an answer generated with a preliminary corpus back to the source data, exposed errors: traceability makes errors findable but not absent.
comment: 74 pages (28 pages main text with 5 figures; 46 pages Supplementary Information with 7 Supplementary Notes, 10 Supplementary Tables and 3 Supplementary Figures). Analysis records, Supplementary Data 1-5 and code are available from the corresponding authors upon request
♻ ★ On the Reliability Limits of LLM-Based Multi-Agent Planning
LLM-based multi-agent planning delegates operational decisions to specialists that communicate through finite reports. In this research, we study its reliability limits when a coordinator can revisit source records before completing a resource-constrained plan. A useful report can support an immediate decision or direct later reading, so its value depends on how these stages communicate. We derive exact communication and access tradeoffs and prove that two report forms suffice for a broad class of allocation requirements, one favoring immediate resource identification and the other distinguishing absence from unnamed availability. For shared capacity, this reduction permits joint optimization of reporting roles and adaptive reads for a fixed number of source types with unequal workloads and reading costs. Identical specialists can optimally report different facts, and maximizing immediate completion can impose an unbounded recovery cost penalty even with full-record access. We also conduct numerical experiments to quantify the benefits of the proposed joint optimization and show how source access improves language-agent decisions that already use their reports optimally. In particular, an exhaustive evaluation of 207,632 report designs on small allocation instances recovers the predicted joint optima. Our analysis suggests that reliable agent communication should be evaluated by the complete decisions it supports and the cost of finishing its unresolved cases, instead of solely by report length, local prediction accuracy, or immediate task completion, as is often the case in current practice.
comment: 36 pages, including appendices; 2 figures. Substantially revised
♻ ★ Speculative Ad-hoc Querying
Large language models (LLMs) invite substantial effort and investment, yet agentic systems can spend much of their time waiting for tools. This paper explores how a small auxiliary LLM can reduce these waits by speculatively preparing a database while the primary LLM reasons. The challenge is that the auxiliary model must anticipate the primary model's actions before the primary model takes them. Our key insight is that database systems give the speculator a structural advantage: it need not predict the exact next query. Instead, it can prepare work shared by many plausible queries. Predicting query shapes enables advance compilation, predicting data access warms caches, and predicting broader data requirements enables precomputation of superset results that subsequent queries can refine. We present Wonder, a system that exploits this advantage for analytical SQL workloads. Wonder constructs reusable superset queries, materializes their components, and rewrites subsequent queries against the resulting temporary tables. Its scheduler prioritizes foreground queries while using reasoning intervals for speculative preparation. The same engine supports humans writing SQL, using incomplete queries to guide speculation and displaying intermediate results as they type. In a replay-based evaluation of agent workflows, Wonder reduces SQL execution time by up to 31.8% and combined primary-agent reasoning and active SQL time by up to 22.6%, with approximately 6% higher recorded main-agent and prediction API cost and 10% more read I/O on the full dataset. A separate 24-participant study finds significantly lower query latency on both analytical tasks and improved time-to-answer on one, with 87.5% of participants reporting increased productivity.
♻ ★ Surrogate Modeling and Explainable Artificial Intelligence for Complex Systems: A Workflow for Automated Simulation Exploration
Complex systems are increasingly explored through simulation-driven workflows that combine physics-based and empirical models with optimization and analytics. Despite their power, these workflows face two central obstacles: high computational cost from repeated expensive simulator runs, and limited transparency when decisions rely on opaque black-box models. We propose a workflow that addresses both challenges by training lightweight emulators on compact designs of experiments that (i) provide fast, low-latency approximations of expensive simulators, (ii) enable rigorous uncertainty quantification, and (iii) support global and local eXplainable Artificial Intelligence (XAI) analyses. This work formalizes a methodological bridge accommodating continuous, categorical, and integer variables across diverse domains, from engineering design to socio-environmental agent-based models. To validate surrogate explanations and examine how faithfully they reflect simulator behavior, we introduce a comparative methodology incorporating active validation filters, including explanation consistency and aleatoric uncertainty quantification. Furthermore, we provide practical guidelines for coupling experimental designs with emulator training to select architectures that balance computational efficiency with interpretability. We demonstrate the framework across two contrasting paradigms: a deterministic multidisciplinary design simulation of a hybrid-electric aircraft and a stochastic agent-based simulation of urban segregation. Results show that coupling surrogates with XAI enables large-scale exploration in seconds, reveals critical nonlinear interactions, and highlights potential design and policy levers while establishing consistent explanations across architectures.
comment: version 2, after first reviews, second round. Thx to J. Levy for help
♻ ★ LLM-Augmented Agent-Based Simulation of Cyber Social Agents
Social media bots are often portrayed as malicious automations to be detected and removed, but automation on social media platforms encompasses a wide spectrum of roles, behaviors, and rhetorical strategies that can shape online discourse in both constructive and destructive ways. This work conceptualizes these automated actors as \textbf{Cyber Social Agents} through a taxonomy of archetypes characterized by their operational tactics and rhetorical style. We integrate this taxonomy into an LLM-driven social simulation, where agents communicate and coordinate in a network environment. Using empirically grounded CSA personas, we simulate heterogeneous online societies to study how agent diversity shapes information diffusion and social influence. Our approach bridges the gap between rule-based agent models and LLM-powered simulations with generated agents that are parameterized from empirical data.
♻ ★ MAServeBench: Benchmarking Multi-Agent LLM Serving Systems with Hierarchical Collaboration Workloads
Large language model (LLM)-based multi-agent systems (MAS) coordinate specialized agents to solve complex tasks, turning each user request into interdependent model and tool operations. Unlike independent requests, these operations become ready at different times and consume evolving context as agents coordinate and exchange results. This creates changing patterns of concurrency, computation, and memory demand that can expose different bottlenecks in LLM serving hardware. Yet benchmarking these effects requires capturing the collaboration rules that generate serving workloads, beyond the requests observed during execution. We present MAServeBench, a benchmark that treats multi-agent collaboration as a configurable dimension of serving workloads for LLM serving and hardware evaluation. Its composable stages capture how agents coordinate and exchange context in documented MAS workflows, enabling native executions to generate different workloads even when task inputs and tools remain unchanged. Hierarchical Semantic Traces preserve realized operations and dependencies for reproducible fixed-work replay and as inputs to architecture simulators. Task-linked measurements connect these workloads to serving execution and GPU/NPU hardware behavior, revealing why serving efficiency does not always translate into timely task completion. The benchmark code is available at https://github.com/ShengxuanQiu/MAServeBench.
♻ ★ Mixture-of-Models: Distributed Inference over Heterogeneous LLMs as Model-Level Mixture-of-Experts
Switch Transformers scale MoE model capacity by routing tokens in memory for one jointly trained model, requiring high-bandwidth interconnects, large memory, centralised ownership, and lack scalable composability. We propose Mixture-of-Models (MoM) architecture as Switch Transformer scaling to semantic level which composes inference across independently trained distributed LLMs, each run on its own GPU. The models reach joint consensus using N-Way Self-Evaluating Deliberation protocol: a combination of social-choice methods such as Delphi method and Quadratic Voting as well as Inference-Time scaling paradigm. We start with an empirical model that characterizes consensus as an inference-time trade-off between signal extraction and contextual noise accumulation, establishing a mathematical basis for convergence driven inference stopping strategies. We show experimental results where ensemble of smaller (<21B) models matches closely 120B MoE (GPT-OSS-120B) on AIME 2025 while larger ensemble of open-weight models reaches 90.0%, close to proprietary models at time of writing.
♻ ★ Collective Reasoning, Single-Writer Execution: Fusion-MoA for Long-Horizon Agents
Mixture-of-Agents systems combine several language models into a stronger response, but tool-using agents act over long horizons in which every step changes external state and the history keeps growing. We present Fusion-MoA, a serving architecture that brings multi-model collaboration into this loop through executor-anchored consultation. One Executor model holds the full context and emits every text and tool call; when consultation is admitted, three specialist models review its proposed next action through a compact view under a shared 120-second deadline. The fleet sits behind one OpenAI-compatible endpoint, so an unmodified agent client drives it as a single model. Fusion-MoA serves an in-car assistant and an in-car personal agent for intelligent cockpits and is deployed on MetaX C550 servers. Evaluated as four quantized open-weight models on five NVIDIA L40S GPUs and driven by Claude Code, it passes 78 of the 89 Terminal-Bench 2.1 tasks, 16 more than its Executor alone and 29 more than a self-hosted quantized Kimi K3 on the same client. On the 63 tasks that both it and the API-served LongCat-2.5-Preview pass, it needs 14.5 minutes on average against 17.5. Specialist reviews use 0.4% of the Executor's tokens, and every tool call comes from the Executor.
comment: 9 pages, 2 figures, 9 tables
♻ ★ A Multi-Instance LLM Framework for Rating the Quality of Surgical Feedback
Verbal feedback delivered by attending surgeons in the operating room plays a critical formative role in resident trainee skill acquisition. Yet, assessing the quality of trainer feedback and its effectiveness in influencing trainee behavior during live surgery remains a challenge. Prior studies assessed feedback content relying on extensive manual annotation by expert human raters and focused on developing broad taxonomies that overlook the qualitative aspects of feedback delivery such as clarity or urgency. Limited existing automated methods, including keyword analysis and topic modeling, also fail to capture these nuanced aspects. We introduce a two-stage LLM-based framework that discovers interpretable feedback quality criteria grounded in the context of surgical training. Our method uses multi-instance prompting via repeated independent sampling of candidate criteria followed by consolidation, and surgical domain knowledge injection to discover a small set of human interpretable scoring criteria (e.g., Encouragement, Urgency, Clarity). These criteria are then used to automatically score live surgical feedback via an LLM-as-a-judge approach. Evaluation on 4.2k trainer feedback instances demonstrates that our AI-discovered criteria outperform prior content-based frameworks in predicting feedback effectiveness, including observed trainee behavioral adjustments and trainer approval. This work advances scalable, human-aligned assessment of communication quality in the operating room and provides a foundation for improving surgical teaching practices.
comment: 38 pages, 6 figures (1-3 in the body, D1, D2, E1 in the appendices) and 14 tables
Systems and Control (EESS)
★ Learning momenta for Hamiltonian-based models
There has been increasing interest in methodologies that incorporate physics priors into neural network architectures to enhance their modeling capabilities. Among these methodologies, Hamiltonian neural networks (HNNs) and their variants have gained considerable attention. These architectures explicitly encode Hamiltonian mechanics both in their structure and loss function, resulting in desirable properties and additional modeled information. A crucial limitation of HNNs is their reliance on data on the target system's generalized momenta. This severely hampers their applicability when the analytical form of the generalized momenta is unknown, which frequently is the case in real-world applications. The goal of the present work is to develop a general extension of Hamiltonian-based architectures that addresses this issue by simultaneously learning the system's generalized momenta, Hamiltonian structure, and dynamics from data. The proposed architecture learns the generalized momenta when their analytical form is unknown, and only (potentially non-sequential) data on the state and its derivatives is available. A dissipative mass-spring system, a double pendulum, and a rolling disk under nonholonomic constraints are considered as canonical examples to assess the performance of the proposed architecture.
comment: 28 pages and 6 figures
★ Singular Perturbations for Control Systems Safety
Control Barrier Functions (CBFs) offer a powerful theory for enforcing safety in dynamical systems. Yet, applying this framework to complex systems is challenging when subsystems are feedback-interconnected and operate at different timescales. Such structures arise naturally in multiphysics systems and in control architectures where physical plants interact with optimization, estimation, and control algorithms. In many of these settings, safety involves only part of the system, motivating the design of reduced-order certificates. Inspired by singular perturbation theory, we establish sufficient conditions, involving separation between slow and fast dynamics, under which a reduced-order safety certificate can be lifted to the full interconnected system. Specifically, we construct a composite CBF ensuring forward invariance of a lifted subset of the original safe set. We also show that the certified safe set expands monotonically as the tuning parameter accelerating the fast dynamics is reduced. Numerical results validate the approach on two robotic systems: a planar 2-DoF manipulator with motor dynamics and a 6-DoF UR10e manipulator controlled by a suboptimal optimization algorithm.
★ Sub Terahertz LEO Satellite Communication: Is Doppler Shift a Major Challenge?
In this work, we investigate the impact of Doppler frequency shift on sub-terahertz low Earth orbit (LEO) satellite communications using OFDM transmission. To comprehensively evaluate the severity of the Doppler shift impact on sub-terahertz satellite-to-ground links, we first put together an evaluation framework that jointly accounts for major Doppler-induced and non-Doppler-induced signal impairments in such systems. We then compare and contrast three possible engineering approaches to deal with Doppler shift in such a communication system. Our study concludes that (i)~while just ignoring the Doppler effect in such high-frequency and high-mobility systems substantially compromises the performance and reliability, (ii)~most of the negative effects can be eliminated by implementing only a small set of low-complexity measures. We particularly show that even a single Doppler shift estimation over a 5-min-long satellite pass over the ground user location may be sufficient to ensure high performance throughout the entire connectivity period. Hence, while Doppler shift is an important effect to account for in sub-terahertz LEO satellite communications, it should not constitute a critical design challenge for these emerging systems.
★ Differentiable Iterative Learning Control for Black-Box Position-Commanded Manipulators
High-accuracy repetitive tracking is challenging for position-commanded manipulators whose low-level controllers are inaccessible, while constrained model-based iterative learning control (ILC) can require substantial manual parameter tuning. We present Differentiable Iterative Learning Control (Diff-ILC), an online bilevel framework that adapts the parameters of a constrained ILC update between physical robot executions. A closed-loop differential model is identified once from paired smooth perturbations and subsequently kept fixed. At each trial, a constrained quadratic program computes a feasible command correction, while a real-anchored next-trial prediction defines a task-level objective. Hypergradients are obtained by implicitly differentiating the locally active KKT system, enabling multiple computational parameter updates without additional robot executions. Experiments on a 6-DoF DOBOT CR3 across unseen planar and three-dimensional trajectories show substantially faster physical-trial convergence than fixed-parameter model-based ILC. Across the four primary trajectories, after only two trial-to-trial updates, Diff-ILC achieves 78.8% and 76.4% lower mean joint-vector and Cartesian RMSE, respectively, than fixed-parameter model-based ILC.
★ Ranking the Parameters Shaping Grid Dynamics
Generator and transmission line parameters do not shape the time-domain behavior of a power system equally; a parameter that dominates one dynamic property can be almost invisible to another. This paper studies how the physical parameters of a detailed and general nonlinear differential algebraic equation (DAE) grid model shape the properties that decide whether system states such as frequencies, voltages, and angles can be inferred, observed, controlled, and kept stable. The rankings are conditional as they depend on where grid sensors and actuators sit, on the operating region and the disturbance, on the network topology, and on other settings that the operator chooses. The framework combines DAE trajectory sensitivities, variational Gramians, and stability margin sensitivities to measure how much each parameter reshapes the network dynamics. On three benchmark systems, the rankings show where dynamic leverage concentrates and where the control-theoretic metrics disagree, and they point to how parameter studies can guide calibration, sensor and actuator placement, and security assessment. The proposed methods herein are also scalable and can be tested on grids with tens of thousands of buses.
★ Co-design of Output-Feedback Neural Network Control Barrier Functions and Observers for Stochastic Jump-Diffusion Systems
In this paper, we present a framework for designing controllers for partially observed continuous-time stochastic systems. We use the notion of control barrier functions (CBF) and provide an upper bound to the probability of state trajectories violating the system's safety when starting in an initial region. Our framework co-designs a neural network-based controller, an output-feedback observer, and a control barrier function for the given system. Although CBF requires full state information, we formulate CBF conditions over an augmented state space of actual and observer states. We train the neural networks simultaneously by defining appropriate loss functions, and finally demonstrate the effectiveness through two case studies: a linear DC motor and a nonlinear system.
★ Robust Safety Assurance for Robotic Swarms Using Decentralized Collision Verification
In this paper we present the methodologies, results, and analysis used to deploy a decentralized safety verification algorithm on a robotic swarm to prevent collisions at runtime in an outdoor environment. Typically, approaches toward verification of swarms of cyber-physical systems (CPS) are evaluated solely in numerical simulations or in highly controlled, noiseless indoor environments. Our efforts to implement collision avoidance on a real swarm system outdoors have revealed that the underlying algorithmic assumptions juxtaposed against the implementation details of the target system are critical factors in a method's efficacy in the real world. Here we modify, enhance, and analyze a decentralized reachability-based method for collision avoidance on a size, weight, and power-constrained uncrewed aerial vehicle (UAV) swarm with the objective of illustrating the challenges of real-world CPS deployments as well as provide guidance for researchers in this space. Verification performance, resource utilization, and timing data were gathered over the course of 140 total outdoor vehicle flights, including 6 swarm flights composed of 8 UAVs, and used to improve collision-avoidance performance as experimentation progressed. Our extensive experiments show that the proposed decentralized runtime verification approach can efficiently verify the safety of multiple UAVs in real-time and help all agents take safe actions to avoid collisions. On average, each agent, using our open-source safety_checker library, takes approximately 500-520 ms to verify its safety for the next 4 s into the future, in which the reachable set broadcast time dominates the total verification time (~300-500 ms).
comment: Accepted version. Published in IEEE Transactions on Aerospace and Electronic Systems, vol. 62, pp. 9706-9717, 2026. 13 pages, 6 figures, 4 tables. Video: https://youtu.be/Y-Z3IN--_Vc
★ GURU: A Hierarchical Neuro-Symbolic Multi-Agent System for Verification-Aware Automation of Mission-Critical Digital Engineering and Operations
An AI can find a safe action faster than any human, but it must never decide what counts as safe. We present GURU, a hierarchical neuro-symbolic multi-agent system for mission-critical digital engineering and operations, built on one partition of authority: learned models propose, rank, and accelerate; a symbolic rules engine checks every consequential action against hard limits before it executes; and validated physics confirms and decides. GURU runs a workflow as a closed control loop, with Navigation, Action, and Planning agents sharing one Common Internal Language and one training factory. Nuclear, aviation, and national-security guidance now requires this pattern; a harnessed language model keeps the model at the center, GURU puts a validated reference there. In computational fluid dynamics, the workflow raised the converged-simulation rate on unseen aircraft geometries from under 1 percent to 74 percent over 100 single-attempt cases, ran 54 live cases to Mach 6 on three facilities, and an ablation put the symbolic layer at 13 points of boundary-layer capture. Knowledge-graph retrieval and role adapters raised ParaView control success from 55 to 75 percent. In space domain awareness, where a language model cannot be trusted to decide, a frozen self-supervised world model under validated propagation scored highest of five methods at conjunction risk, including fully fine-tuned language models, with several-times-lower seed variance: the trustworthy design gives up nothing in accuracy, and the recipe transferred to reactor anomaly detection. A radar-to-avoidance debris pipeline was demonstrated in stages on a 2026 reentry; the hierarchy coordinates simulated teams of 200 aircraft. Training scaled near-linearly past 1,000 Aurora nodes and ran at 8,000. The approach extends to a control-grade microreactor digital twin with validated physics, not a learned model, as the safety basis.
comment: 82 pages, 50 figures, 7 tables
★ Benchmarking Classical Machine Learning Models for Shading-Fault Detection in Rooftop Photovoltaic Systems Using Real Sensor Data
Most published comparisons of machine-learning classifiers for photovoltaic (PV) fault detection are validated on simulated faults. We compare seven classical and ensemble-tree classifiers on the Réunion dataset, a 4 kW domestic rooftop plant on which shading faults were physically staged and logged. Readings within a staged episode are near-duplicates, so every train/test split is grouped by episode. A 20,800-row subsample that retains all 17 fault episodes was evaluated with three repeats of five-fold group-aware nested cross-validation, with synthetic minority oversampling (SMOTE) applied inside the training folds. Random Forest and XGBoost gave the highest mean accuracy (0.838 and 0.831) and Matthews correlation coefficient (MCC; 0.526 and 0.522). After Holm correction across the 21 pairwise Wilcoxon tests, only Random Forest's accuracy advantage over k-nearest neighbours and the multi-layer perceptron remained significant. Logistic Regression and the Decision Tree (accuracy 0.820) could be separated neither from the ensembles nor from the weakest models. SMOTE lowered binary MCC for six of the seven classifiers. Errors were concentrated in whole episodes: the Random Forest almost entirely missed four of the 17 fault episodes, including one of two uniform-shading and two of five intermittent-shading episodes. A post-hoc diagnostic shows that the no-fault rows were sampled over a wider range of times of day and output levels than the fault rows. Clock time alone reaches an MCC of 0.44, and PV-side voltage alone 0.72. The reported scores therefore reflect operating-condition differences as well as shading signatures. We release the pipeline and the diagnostics, and identify matched no-fault sampling as the step needed before the classifier ranking can be generalised.
comment: 17 pages, 4 figures
★ Robust Data-Driven Output Feedback for Systems With Unknown Input Delays
In a history-feedback realization, an input delay increases memory without increasing the number of unknown physical coefficients. We use this structure to design output-feedback controllers for discrete-time systems with one input, one output, and known undelayed order, using noisy input-output data. For a known delay, we derive a necessary and sufficient linear matrix inequality for common-quadratic stabilization of all models consistent with the data. Optimizing its equation-error bound gives a robustness margin for the measured record. For an unknown constant delay in a known interval, we characterize noiseless delay ambiguity and give a sufficient condition for one controller to stabilize all models for every delay consistent with the error bound. Under stated excitation and nondegeneracy assumptions, this procedure is feasible for all sufficiently small equation errors. Numerical examples compare robustness at equal controller memory and show that certified stabilization can succeed while distinct delay models remain compatible with the same data.
comment: 6 pages, 1 figure. Submitted jointly to IEEE Control Systems Letters (L-CSS)
★ Learning hidden-state dynamics from unpaired snapshots with lift-project SDEs
Many systems are observed only through unpaired snapshots, with population samples available at each time point but no correspondence between samples at different times. These snapshots determine the one-time distributions but leave the temporal coupling unspecified, requiring a modeling prior to connect successive times. We propose the lift-project stochastic differential equation (SDE), an autonomous SDE on the observed coordinate and a few hidden coordinates, trained only to match the snapshots. Most existing methods, including time-dependent SDEs, Schrödinger bridges and optimal transport, evolve the observed coordinate alone. For a scalar observation, the classical comparison theorem implies that all such models have non-negative lagged covariance, so on data whose lagged covariance turns negative, their error has a computable floor. The lift-project SDE is not bound by this floor. On two representative partially observed systems, the Duffing and pendulum oscillators, at comparable fit to the snapshots, the proposed method reduces the lagged-covariance error by factors of 2.4 and 6.6 relative to the strongest baseline and is the only model below the error floor; one hidden coordinate is already enough for such improvement. Furthermore, if a second experiment changes only the initial state of the hidden coordinate, no single model that evolves the observed coordinate alone can fit both experiments, whereas a single lift-project SDE can.
comment: 26 pages, 12 figures
★ Vision-based Sensing of Urban Environment using Hardware-Software co-design at Scale
Urban environmental monitoring is crucial for climate-adaptive cities, yet current approaches rely heavily on human inspection, overhead remote sensing, or limited terrestrial RGB imagery, thereby lacking comprehensive multispectral perspectives at street level. To bridge this gap, this paper introduces UrbanScapes, an adaptable and flexible hardware-software co-design platform for large-scale, vision-based sensing of urban environments. Built on a modular architecture, UrbanScapes seamlessly integrates multispectral imaging channels, including Red, Green, Blue (RGB), Near-Infrared, and Thermal, to capture multispectral street-level imagery. The system is designed for both opportunistic deployment on bicycles, vehicles, or pedestrians and targeted deployments on municipal fleets, ensuring high spatio-temporal scalability without requiring hardware redesigns. Crucially, the platform utilizes on-device deep learning models to automatically remove personally identifiable information (PII) at the edge, addressing privacy concerns inherent in street-level data collection. By combining this multispectral street-level imagery with overhead multispectral remote sensing imagery, UrbanScapes enables scalable calculation of physics-based environmental indices, providing a foundational tool for advanced citywide analysis.
comment: IEEE World Forum for Internet of Things 2026
★ Deep Reinforcement Learning with Smith-Predictor Delay Compensation
Time delays can degrade feedback performance and destabilize a closed loop that is stable without delay. This paper studies Smith-predictor-assisted reinforcement learning for systems with constant input delays. A deep deterministic policy gradient (DDPG) actor is trained on a nominal delay-free environment and deployed with a Smith predictor using a fixed nominal delay. For linear controllers, we derive exact closed-loop maps under plant and delay mismatch and provide small-gain and $ν$-gap robustness conditions. A separate delay-independent Razumikhin condition gives a sufficient stability test for a fixed Lipschitz state-feedback policy. We also study output-error delay identification from a fixed input--output record, deriving the loss gradient and establishing projected-flow descent and local exponential convergence under exact model matching and an excitation condition. These identification results do not imply stability of an online adaptive predictor. Numerical experiments retained from the conference study compare PID, LQR, and DDPG controllers on stable and open-loop unstable first-order plants with valve stiction. The results illustrate the benefit of predictor compensation in the tested configurations and the sensitivity of standalone DDPG to delay and plant changes.
comment: This is an extended version of the paper published in the proceedings of IEEE American Control Conference (ACC)
♻ ★ Lifted Schrödinger Bridges for Gaussian Mixture Endpoints: Projection Gaps and Path-Space Obstructions
We study stochastic density control between Gaussian-mixture endpoint distributions under Brownian prior dynamics. Since the direct Schrödinger bridge between Gaussian mixtures is generally not available in closed form, we introduce a lifted path-space construction in which each trajectory is augmented with a source--target component label. Consequently, the problem decomposes into Gaussian component-to-component Schrödinger bridges with explicit marginal, drift, and cost formulas, while the mixture-level assignment reduces to a finite-dimensional entropic coupling problem with a Sinkhorn scaling form. We then analyze the projection obtained by discarding or forgetting the label. By construction, the projected law satisfies the original Gaussian-mixture endpoint constraints, but its relative entropy generally differs from the lifted relative entropy by a nonnegative conditional label-information gap. This gap reveals a path-space obstruction: the lifted optimizer cannot, in general, be identified with the direct unlabeled Schrödinger bridge after projection. We also derive the posterior-averaged Markov drift associated with the projected marginal flow, prove a kinetic-energy upper bound, and identify a common path-potential condition under which the projection gap vanishes. Several numerical illustrations showing density and shape control are recorded for a self-contained exposition.
comment: 37 pages; revised version of a journal submission with a new section on duality certificates
♻ ★ Reduced Order Observers for Monocular Visual Inertial Odometry
This work develops a reduced order observer framework of monocular visual-inertial odometry (VIO). The key idea consists in considering the gravity vector in the body-frame as an additional state and constructing an appropriate output that relates the monocular bearing measurements and their time derivatives to the body-frame velocity. This eliminates the need for the estimation of the landmark positions, leading to a six-dimensional linear time-varying (LTV) observer for the body-frame velocity and gravity, endowed with uniform global exponential stability guarantees. These estimates are subsequently used to recover the orientation almost globally, up to an unknown constant yaw offset, and the position up to an unknown constant translation. The mixed-bearing framework is further extended to account for the inertial measurement unit (IMU) biases, yielding a twelve-dimensional LTV observer endowed with local exponential stability guarantees. Numerical simulations are provided to illustrate the effectiveness of the proposed observers.
♻ ★ Critical Inertia Estimation for the Three U.S. Interconnections
The rapid integration of inverter-based resources (IBRs) is reducing system inertia across U.S. power grids, raising concerns about frequency stability following large contingencies. This paper presents a simulation-based assessment of critical inertia--the minimum system inertia required to prevent first-stage under-frequency load shedding (UFLS) after the largest credible contingency--across the three major U.S. interconnections: Eastern Interconnection (EI), WECC, and ERCOT. Reduced-inertia scenarios are created by progressively replacing synchronous generators with IBRs, and dynamic simulations are performed using full-scale PSS(R)E and PowerWorld models. The results show that ERCOT reaches critical inertia at approximately 20% IBR penetration under the strict first-stage UFLS criterion--or approximately 58% if only the final UFLS stage must be avoided--compared with above 90% for EI and approximately 67%-68% for WECC. IBR shares in 2024 in the U.S. portions of EI, WECC, and ERCOT were 16%, 33%, and 44%, respectively, indicating varying proximity to critical inertia thresholds. These findings highlight the importance of full dynamic simulations to accurately estimate critical inertia and guide transmission planning under high renewable penetration scenarios.
comment: 8 pages, 13 figures, 1 table
♻ ★ The Iberian Blackout: A Black Swan or a Gray Rhino? A Protection-Aware Dynamic Voltage Security Assessment
On 28 April 2025, the Iberian power system collapsed after a rapid voltage rise that led to widespread generation disconnections and loss of synchronism. The final report of the ENTSO-E Expert Panel attributes the blackout to several interacting factors including ineffective voltage control, fixed power factor operation, misaligned protection settings, manual shunt reactor switching, and limited observability. These factors raise a general question: given the current state, can the next trip, ramp, topology action, or shunt switching raise the voltages measured by overvoltage relays above their settings before voltage controls respond? We answer it with a protection-aware dynamic voltage security assessment. For each candidate event, the method computes the voltage at every overvoltage relay over the relay time window from a dynamic model of the network, its controls, and its relays. It predicts which relays trip after a first trip, identifies assets whose relay or measurement data are insufficient for a safety conclusion, and, when one exists, computes a minimum-cost fast reactive power action that keeps all relay voltages below their settings. Case studies on four benchmark systems and a 2000-bus system that reproduces the mechanism of the Iberian blackout show that the method flagged every event that caused a secondary trip within the relay window, never declared safe a plant that tripped within that window, and, on the benchmarks, found fast reactive power actions that prevented the trips within the relay window whenever it judged such actions feasible. These results were validated with nonlinear time-domain simulations. The code is publicly available.
♻ ★ Safe Unicycle Navigation Among Multiple Obstacles Under Bounded Disturbances: A Learning-Augmented CLF-CBF Approach (Extended Version)
This paper presents a robust control framework for safe navigation of a unicycle robot to a goal among multiple static obstacles under an unknown, time-varying disturbance. The controller solves a single real-time quadratic program (QP) that combines a goal-reaching term with one safety term per obstacle. The goal-reaching term is a cross-track control Lyapunov function built from body-frame tracking errors rather than a bearing angle, which keeps its Lie derivatives bounded everywhere, including the goal. Safety is encoded using robust control barrier functions, one per obstacle, each tightened by a closed-form, state-dependent disturbance bound. When stabilization and safety conflict, a slack variable relaxes only the convergence requirement, so that the safety constraints remain strictly enforced. Whenever the QP is feasible, the robust barrier constraints guarantee forward invariance of every obstacle's safe set under all admissible disturbances. Complementing this safety guarantee, a Lyapunov analysis yields an explicit ultimate bound on the distance to the goal. A learning-augmented variant replaces the fixed objective with a reinforcement-learning reference, which accelerates convergence when multiple constraints are simultaneously active. We further derive the obstacle separation below which the gap between two obstacles becomes too narrow to pass through under the robust constraints. For such gaps, a learned virtual goal for the Lyapunov function steers the robot around the obstacles without compromising safety. Simulations, a Monte Carlo study, and a comparison against a sampled-data robust baseline support the theoretical results.
♻ ★ DiffGIPC: Differentiable Mixed-Coordinate Contact Mechanics on GPU
Contact-rich inverse problems require efficient gradients and reliable enforcement of non-penetration. Existing differentiable methods provide fast simulation and rigid-deformable coupling, but contact treatments based on penalties or velocity-level constraints can permit penetration or positional drift. Incremental Potential Contact (IPC) maintains non-penetration through geometric barriers and {continuous collision detection (CCD)}. However, repeated forward and backward evaluations remain costly in full-space finite-element (FEM) implementations, particularly when nearly rigid components retain many vertex {degrees of freedom (DoFs)}. We present DiffGIPC, a GPU-accelerated differentiable IPC framework for optimizing interacting deformable and nearly rigid bodies. Our affine-body derivatives enable {rest-shape} design, initial-state optimization, and articulated control while retaining only twelve motion DoFs per affine body. These gradients account for how changes in geometry, placement, and actuation affect other objects through collision and friction. We preserve these interactions in one coupled FEM-affine adjoint system per frame, reducing the backward-system size without discarding the dependencies that drive the inverse problem. GPU-optimized backward computation further reduces the cost of repeated gradient evaluations. A reusable PyTorch interface connects supported physical parameters to user-defined parameter maps and losses across design, identification, motion, placement, and articulated control. Experiments demonstrate peak speedups over DiffIPC of approximately 150.0x with full-space FEM and 4133.3x with affine bodies for combined forward simulation and backward differentiation.
♻ ★ Mixture-of-Models: Distributed Inference over Heterogeneous LLMs as Model-Level Mixture-of-Experts
Switch Transformers scale MoE model capacity by routing tokens in memory for one jointly trained model, requiring high-bandwidth interconnects, large memory, centralised ownership, and lack scalable composability. We propose Mixture-of-Models (MoM) architecture as Switch Transformer scaling to semantic level which composes inference across independently trained distributed LLMs, each run on its own GPU. The models reach joint consensus using N-Way Self-Evaluating Deliberation protocol: a combination of social-choice methods such as Delphi method and Quadratic Voting as well as Inference-Time scaling paradigm. We start with an empirical model that characterizes consensus as an inference-time trade-off between signal extraction and contextual noise accumulation, establishing a mathematical basis for convergence driven inference stopping strategies. We show experimental results where ensemble of smaller (<21B) models matches closely 120B MoE (GPT-OSS-120B) on AIME 2025 while larger ensemble of open-weight models reaches 90.0%, close to proprietary models at time of writing.
Robotics
★ Hierarchical World Models with Implicit Dynamics
Planning with world models enables test-time adaptation for embodied agents, offering superior generalization over direct policy learning. However, long-horizon tasks remain hindered by cumulative errors and search complexity. Existing hierarchical methods often use rigid macro-actions or latent skills, leading to brittle representations and limited fine-grained control. We introduce ID-WM (Implicit Dynamics World Models), a hierarchical world model that decouples state-space reachability from local control. Our framework utilizes a high-level model to sample feasible transitions across the state manifold and a low-level world model as an inverse dynamics solver to ground these transitions into precise actions. This allows the planner to identify reachable subgoals without being constrained by low-level execution details. By prioritizing local physical feasibility over global behavioral patterns, ID-WM synthesizes novel long-horizon plans from short segments, transcending the specific behavioral modes seen during training. We demonstrate its effectiveness across three navigation and manipulation suites, achieving a 38.5% improvement over state-of-the-art policy and diffusion-based planners. Crucially, by leveraging this hierarchical structure to mitigate compounding errors, our approach yields a 135% performance gain over single-level world model MPC, particularly in complex, unseen configurations for long-horizon tasks. Videos can be viewed at https://id-wm.github.io
★ RoboArchitect: Automating the Design of Robotic Agents with Persistent Memory
Recent work shows that a frozen robot policy performs far better inside an agent harness, where control code decides when to query the policy, consult vision tools, or command the arm directly. Yet today these harnesses are built by hand and redesigned for each new robot or benchmark. We show that harness design can be automated, offering a strong alternative to finetuning on a novel setup. In RoboArchitect, a coding model observes a robot's rollouts, diagnoses failures, and iteratively revises the harness around the frozen policy. The harness evolves on a set of episodes for its optimization and is evaluated on held-out scenes and tasks. Without updating any model weights, RoboArchitect raises average success from 9.7% to 43.3% on unseen scenes (4.5x) and from 12.7% to 40.2% on unseen tasks (3.2x), across multiple benchmarks and policies. The gains are largest where the benchmark is far from the policy's training data: wrapping a general-purpose policy, the evolved harness yields a specialist that often outperforms even the finetuned version. Intuitive control strategies are automatically designed, such as lifting a jammed arm clear or reaching in horizontally when an obstacle blocks the way from above. The harness also evolves its own persistent memory, carrying learned experience, such as effective grasp depths, into later decisions. These results suggest that substantial headroom in robotic manipulation lies in the agent harness, and its design can be effectively automated and optimized without weight updates.
comment: Project Website: https://roboarchitect-ai.github.io/
★ ROOF: Learning Humanoid Fall Recovery under Overhead Constraints
A fallen humanoid beneath an overhead obstacle faces a coupled recovery problem. It must create clearance while organizing support for a stable rise, since attempting to stand before making room can further increase obstruction. We introduce a reinforcement learning method with contact-supported geometric escape guidance, which rewards incremental reduction of robot-obstacle overlap without prescribing an escape path. To achieve this, a learned motion prior, together with posture-dependent support and joint-stall costs, is proposed to shape a single policy that learns when to clear the obstacle, how to establish support, and when to complete the rise, all using proprioception alone. In simulation, it recovers in 88.5% of trials beneath plates that can only be lifted, compared with 18.9% without geometric guidance, and in 98.6% of trials beneath free plates with randomized size, mass and placement. On a Unitree G1, the deployed policy recovered in 95 of 150 attempts across seven real-world obstacle scenes, including furniture and multi-object arrangements absent from training. Project page: https://lu-yidan.github.io/ROOF/
comment: 8 pages, 4 figures
★ CausalWAM: Learning the Causes of Actions in World Action Models
World action models (WAMs) learn latent variables from observations and use these to both predict the evolution of the scene and to generate actions. The prediction has to be of the whole scene, whereas only a portion of the scene causes actions. WAMs fitted to demonstrations do not, in general, guarantee that they will generate actions solely from the latent causes of the demonstrated actions (the action-relevant latent variables), as the model could just as well learn to imitate in this way whilst entangling the action-relevant latent variables with the nuisance variables. We therefore separate out what causes action from what prediction depends on. We prove that under a sparsity constraint the latent variables used to generate actions are an invertible function of the action-relevant latent variables alone and the true structure is identified up to a permutation of the latent variables. This requires neither sufficient temporal dependence nor any assumptions on how the unobserved regimes evolve and holds even when the action-relevant latent variables are statistically dependent on the rest. We therefore propose CWAM (Causal WAM), which incorporates the constraint into a pretrained WAM by adding a sparse routing path to its action head, with the world modeling objective remaining unchanged. Experiments on both synthetic data and real-world robots validate the theory as well as showcase the improved efficiency in closed-loop success.
★ REA: A Routed Embodied Agent Runtime for Asynchronous Brain-Cerebellum Control
Foundation models are becoming remarkably reliable at high-level reasoning, yet they remain too slow and costly to serve as fine-grained controllers. Conversely, fast policies like VLAs and whole-body controllers offer more efficient reflexes, but struggle to generalize. While pairing a deliberative brain (foundation models) with a reactive cerebellum (fast policies) seems natural for embodied systems, two obstacles remain: methods are built and scored on incompatible stacks, and their asynchronous coordination is difficult. For example, an agent that calls the VLA in the agentic loop can stall the fast execution, while a VLA that calls the agent on errors tends to re-trigger fallbacks repeatedly or completely miss silent failures. To address this, we propose REA (Routed Embodied Agent), a unified runtime where a deliberative System 2, an embodied router System 1 and reactive System 0 policies cooperate under a single golden rule - no running policy waits on the brain. Instead of querying the brain at each failure, REA employs a route memory that stores each policy's measured success ahead of time, a boundary-triggered judge gate that consults the brain on policy execution, and a shared-budget escalation ladder for bounded recovery. Evaluated on 3,000 test episodes across 60 LIBERO-Pro tasks, REA reaches 95.9% success and beats the best single-agent baseline in its own policy pool (94.7%) while reducing control steps by 25% and control time by 17%. In a separate 100-episode RoboTwin 2.0 study, REA achieves 84% success with VLA policies and 86% with VLA+code, compared with 80% for the strongest single VLA baseline MEDex. These results demonstrate learned and programmatic policies operating within the same runtime. REA also transfers to BEHAVIOR-1K and a real Unitree G1 humanoid, achieving the highest observed success among the compared methods.
comment: 32 pages, 14 figures
★ A Hybrid Path Planning Framework for Assistive Wheelchair Navigation in Dynamic Pedestrian Environments
Autonomous wheelchairs can improve the quality of life for individuals with mobility impairments by providing greater independence. However, reliable navigation especially in human environments is a major challenge due to factors such as dynamic obstacles, unpredictable pedestrian behavior, and the users' need to complete multiple stops along the way before reaching their final destination. This paper presents a hybrid path planning framework for multi-goal navigation in dynamic environments. The framework integrates the Traveling Salesman Problem (TSP) for global task ordering with the Potential Field (PF) method and heuristic local planners such as Rapidly-exploring Random Tree (RRT) and A-star (A*) to achieve collision-free navigation. Within this hybrid framework, TSP determines the visiting order of intermediate stations, PF transfers global path information, and local planners avoid obstacles while guiding the robot to the stations. The local planners generate feasible paths that allow the robot to follow the global trajectory and reroute to avoid static and dynamic obstacles. For this purpose, two algorithms have been developed: (i) TSP-PF-RRT and (ii) TSP-PF-A*. The performance of the proposed algorithms are evaluated using Monte Carlo simulations in crowded and extra-crowded scenarios that contain heterogeneous dynamic and static obstacles. Experimental results demonstrate that the proposed hybrid framework enables efficient multi-goal navigation while maintaining safe operation in dynamic environments with humans, with TSP-PF-RRT achieving fewer or no collisions and shorter travel distances.
comment: 14 pages, 5 figures. Presented as a main-track paper at the International Conference of Social Robotics + ART'26, London, UK, 1-4 July 2026
★ Proof-Carrying Evolution: Black-Box Search, White-Box Flight
We study whether a language model can help design controllers whose behavior can be formally checked. In proof-carrying evolution (PCE), a search proposes controllers together with their certificates. Their mathematical forms are fixed; a proposer chooses their numerical parameters together. Certificate checks guide selection, and only a certified candidate can be returned. The deployed artifact is a control law and its certificate; the language model is used only during design. We compare four language models with classical search and synthesis on docking under a speed limit, pursuit of a maneuvering target, and attitude control on SO(3), using paired runs and shared proposal allowances. Certificates are checked in exact rational arithmetic during search on the linear problems. On SO(3), the search uses floating-point checks, followed by exact re-verification of every reported champion. The best run of each method also receives Lean 4 proofs of its stated control properties. On the linear problems, the strongest models beat the standard classical baselines on every run, but a classical optimizer that chooses certificates for the same objective closes most of that lead. Among the cited pursuit champions, the ranking does not survive a change from Euler to zero-order-hold dynamics. On SO(3), no model detectably outperforms CMA-ES when mixed with classical mutation. When models write every proposal, one tested model improves on CMA-ES and the other performs worse than its mixed variant. All search medians remain below the hand-tuned law. In a docking control with the objective and mutation operator held fixed, retaining certified candidates yields new certified designs in every run; removing this rule yields none.
★ Nexus-VLA: Bridging Human and Robot Experiences via a Shared Hierarchical Semantic Latent Space
Large-scale egocentric human videos provide rich and scalable manipulation experience for robot policy learning, yet substantial embodiment gaps in visual appearance, kinematic structure, and action spaces hinder effective knowledge transfer. Existing methods often decouple cross-embodiment knowledge alignment from robot policy learning, making it difficult for human manipulation knowledge to directly and fully contribute to robot action learning. To address this issue, we propose Nexus-VLA, a hybrid policy learning framework that enables cross-embodiment knowledge transfer through end-to-end human-robot joint training, allowing human manipulation experience to directly contribute to robot policy learning. We first introduce a hierarchical cross-embodiment semantic alignment paradigm that progressively establishes shared manipulation semantics from task intent to fine-grained interactions, enabling more complete use of the manipulation knowledge contained in human videos. Building on this hierarchy, we design a shared manipulation latent representation that jointly encodes multi-level shared knowledge into an internal representation directly used for continuous action generation, improving policy learning efficiency while effectively leveraging human knowledge. Across RoboCasa-GR1, OOD settings, and real-world robot experiments, Nexus-VLA achieves an 80.5% task success rate on RoboCasa-GR1 and demonstrates strong advantages in human-data utilization, learning efficiency, and cross-scenario generalization.
★ Suspended-Fabric Robotic Manipulation Surfaces: Modelling, Design Space, and Multimodular Extensio
Robotic Manipulation Surfaces (RMS) reposition objects by deforming the surface on which they rest. A recent class of RMS suspends a continuous fabric sheet from a sparse grid of vertical linear actuators lowering the actuator density of such surfaces but how to design such surfaces so that objects can be controlled everywhere on it, and how to extend it beyond a single module so that objects can pass between modules and achieve parallel, remain open. This paper presents a separable catenary model of the suspended sheet that is fast to compute and provides both the surface shape for given actuator heights and the actuator heights for a desired local slope. The closed form inverse lets a simple PID controller position objects on the sheet, and the model relates fabric length, module span and actuator stroke to the reachable workspace, yielding a minimum fabric length bound and design charts for these surfaces. The approach is then extended to a multimodular platform in which neighbouring modules share actuators. On a $2 \times 2$ prototype, we demonstrate that centimetre-scale objects are transferred between modules and positioned within 0.03 m over a $1 m^2$ workspace with only nine actuators, enabling multi-object manipulation and opening the way to parallel manipulation.
★ Design and Validation of a Robotic Knee Exoskeleton with Musculoskeletal Simulation-Derived Biological Torque Control
Data-driven controllers have enabled lower-limb robotic exoskeletons to adapt assistance to the user across many tasks. A leading example is biological torque control, which estimates the user's own joint moment from onboard sensors and assists in proportion to that estimate. However, its estimators are trained on data recorded with the target device, so deploying the controller on new hardware requires collecting new data. Existing open-source efforts release hardware, software, or controllers separately, so this data collection must still be repeated on the assembled device. Here, we present an open-source knee exoskeleton, built from off-the-shelf components and widely available fabrication tools, with a biological torque controller derived entirely from simulation. A muscle-driven imitation policy reproduces squatting motions from an open biomechanics dataset, and the resulting knee moments are distilled into an estimator that uses only onboard sensors, so no training data are recorded on the device. In seven able-bodied participants, assistance reduced peak biological knee extension moment by 21.5% and peak vastus medialis activation by 26.7% during squatting, relative to squatting without the exoskeleton (p<0.05). Our results demonstrate that a biological torque controller derived from simulation can transfer to physical hardware without device-specific data collection. Because the hardware and training pipeline are released together, research groups can train and compare knee exoskeleton controllers on shared hardware without additional data collection overhead.
comment: 8 pages, 5 figures, 2 tables
★ Language-Conditioned Behavioral Cloning for Autonomous Endoscope Guidance IROS 2026
Automated endoscope guidance reduces the need for continuous manual camera control during minimally invasive surgery. However, the desired camera motion depends on the surgeon's immediate intentions and cannot always be inferred from the visual operative scene alone. We propose to automate the task using language-conditioned behavioral cloning; we define 6 clinically relevant guidance behaviors and collect 235 subtask demonstrations. In preliminary phantom experiments, our policy achieves 100% success on focus, overview, and stop, 80% on search-tip and follow-tip behaviors, and 50% on horizon leveling. These results demonstrate the feasibility of learning heterogeneous surgeon-requested camera behaviors with a single visuomotor policy, while the observed failure modes motivate future work on adding temporal context for more demanding guidance tasks.
comment: The Evolving Landscape of Surgical Robotics: Surgical Autonomy for Today and Tomorrow @ IROS 2026
★ Sensorium: An Open-Source Multimodal Data Collection and Curation Library for Embodied Surgical AI NeurIPS 2026
Embodied surgical AI requires reliable multimodal data, yet surgical data collection remains costly, difficult to repeat, and strongly dependent on the hardware setup. We present Sensorium, an open-source Python library for recording, processing, annotating, and curating multimodal surgical data from configurable robot and sensor streams using the Robot Operating System (ROS) and ROS2 middleware. The library combines operator-facing data acquisition, configurable processing, semantic annotation, and episodic dataset generation. We evaluate Sensorium across 1516 recordings of phantom, ex vivo, and in vivo porcine experiments involving multiple surgical tasks and sensing configurations, and report preliminary usability feedback.
comment: GenAI for Health Workshop @ NeurIPS 2026, Sydney
★ Adaptive Coverage Algorithms for Mobile Ad Hoc Networks in Dynamic Worlds
This paper extends an existing framework for the coordination of mobile agents in unknown worlds, by considering the possibility of dynamic instability. In the original problem, known as the Partially Observable Cooperative Guard Art Gallery Problem (POCGAGP) (arXiv:2512.09495), the multi-agent system must establish a communication network which completely covers an initially unknown constrained space. However, the POCGAGP fails to account for the unpredictability often seen in real-world deployment. To address this, the dynamic case (DPOCGAGP) introduces two settings which can influence the mobile network during deployment: Agent Loss and World Change, which represent dynamic changes to the world. Within this study, we present advancements to two state-of-the-art algorithms, the Centralized Agent-Directed Exploration for Network Coverage (CADENCE), and its decentralized analogue, DADENCE, to provide solutions in these cases. We then compare their performance against another state-of-the-art benchmark algorithm, and show that the adapted algorithms outperform across our benchmark test set, which comprises over 4500 test cases of varied size and structure.
comment: 8 pages, 10 figures. Accepted to the 2026 IEEE MIT Undergraduate Research Technology Conference (URTC)
★ Online Learning of Heterogeneous Locomotion Dynamics under Reinforcement Learning Policies for Precise Loco-Manipulation
How to reuse a pretrained locomotion policy for precise loco-manipulation is still an open problem. We propose an online method to learn the closed-loop dynamics under a frozen locomotion policy. The learned model predicts floating-base motion and improves loco-manipulation accuracy. However, unobserved latent conditions of the policy introduce heterogeneous responses, meaning that the same observed state will lead to different motions. Direct learning without accounting for this heterogeneity introduces bias and fluctuations into dynamics estimates, degrading floating-base prediction and end-effector tracking. To address this problem, a contaminated Gaussian model is proposed to approximate the effects of latent conditions. The model uses two Gaussian components to approximate predictable and unpredictable responses under the same observed state. Then, the contaminated Gaussian model is combined with a Gaussian process to construct an online learner using random Fourier features (RFFs). The learner estimates each response's probability of belonging to the predictable component and uses it to weight online updates, capturing predictable responses while reducing bias and fluctuations caused by heterogeneous observations. With consistent and unbiased dynamics estimates, model predictive control (MPC) predicts floating-base motion accurately and enables precise loco-manipulation. A finite-sample error bound is established to ensure uniform convergence in online learning. Simulation and hardware experiments demonstrate consistent end-effector tracking across different manipulator configurations and tasks, with errors of 0.01 m.
comment: 18 pages, 15 figures. Supplementary video: https://youtu.be/H-u8buk86C0
★ Movens-Zero: Scaling Humanoid Motion Foundation Models with Large-Scale Human Videos
Scaling humanoid motion foundation models is constrained by the scarcity of diverse, robot-executable data and by the difficulty of jointly learning from heterogeneous multimodal inputs while supporting real-time closed-loop control. We present Movens-5M, constructed from raw Internet videos through clip generation, quality filtering, semantic annotation, motion recovery, and physical refinement with humanoid retargeting and execution verification. It contains 15 million human-motion sequences and 5 million physically executable humanoid-motion sequences spanning over 10,000 hours, aligned with text, RGB video, speech, and music. Building on this data, we introduce Movens-Zero, a scalable understanding-generation-execution architecture that supports joint learning across modalities and real-time closed-loop execution. Task-mixed flow-matching training jointly optimizes a single action expert across seven generation tasks, enabling direct motion generation from text, speech, music, and video. Multimodal Real-Time Chunking connects overlapping chunks under continuously updated instructions and robot states for continuous closed-loop execution. We further introduce Movens-Intent, a four-track benchmark with 8,192 expert-verified condition-motion pairs across text, speech, music, and video. Experiments show consistent gains across all four modalities when scaling execution-verified data from 1% to 100% and action-expert capacity from 64M to 911M parameters, while demonstrating that data quality and cross-modal supervision improve generation and execution.
comment: Technical report. 86 pages, 31 figures, 15 tables. Project page: https://movens-zero.github.io
★ META-Robot: Self-Cognition and Capability Scaling in Agentic Robots
LLM-based robotic agent systems can generate programs for robot control, yet their generalization to new tasks often depends on elaborate scaffolding. Expanding their skill repertoires alone offers limited guidance on how to assess new capabilities or identify what remains beyond their reach. We introduce META-Robot, a framework that couples self cognition with capability scaling by expanding the criteria through which an agent evaluates and develops its skills. At its core, action-level predicates make execution outcomes explicit and available for LLM reasoning. The agent uses these assessments to identify its own limitations, extend relevant evaluation criteria, and guide the structured expansion of its skills. Proprioceptive feedback further grounds this process in the robot's physical state, enriching the evidence available for self-cognition. Experiments in simulation and on real robots demonstrate stronger overall task performance, generalization to new tasks, continued expansion of skills and evaluation criteria, and effective use of non-visual proprioceptive signals.
★ When the Simulator Fails, Not the Policy: Auditing Contact Artifacts in Simulated Robot Benchmarks
Simulation benchmarks score robot policies by success rate, and every failed episode counts against the policy. Many of these failures are the simulator's fault, not the policy's; we find the cause and fix it. Testing a vision language action model and a world action model in simulation, we find that the policy often places the object and the simulator then ejects it, and that the benchmark's own scripted demonstrations fail the same way. We present SimuGuard, an auditing plugin for LIBERO, RoboTwin, RoboCasa and RoboDojo. It leaves the evaluation unchanged. At every physics step, it records the state and flags an object that starts moving at a contact faster than the robot's contact or gravity account for. It replays a recorded episode bit for bit, and it reruns the episode with one simulation parameter changed. With it we trace these episodes on RoboTwin to the collision detection of the physics engine: for geometry one to two centimeters apart, the engine reports a penetration of about two centimeters, and the solver throws the object out at several meters per second. The cause is one constant: the distance algorithm discards triangles below a fixed area threshold, which valid triangles fall below at the centimeter scale. With that constant changed and nothing else, episodes with an artifact fall from 12 in 100 to none, and the success rate on five container tasks rises from 89% to 98%. Success rates in simulation thus count failures that the simulator causes. Benchmarks on PhysX 5.3.1 or earlier should rerun in a repaired engine. Where the engine cannot change, they should report the success rate together with the range it spans when every flagged episode fails or passes: 75.8% to 98.5% around a reported 83.1% for LingBot-VA.
comment: https://github.com/ALEXander22568/simuguard
★ SketchMate: Agentic Geometric Reasoning for Zero-Shot Precision Insertion
Inserting an unseen part requires sub-millimetre precision and the correct orientation of keyed features. Current zero-shot agentic manipulation methods built on vision-language models (VLMs) produce spatial outputs that are far too coarse for this. We propose to let a VLM build a sketch model of the task, i.e., a few geometric primitives and two mate frames, which a VLM agent measures with geometric tools in multi-view depth and which a standard planner and compliant controller execute. Because a sketch describes the parts independently of their placement, the robot needs to build it only once per task. A sketch memory stores each agent run and transfers the stored sketch to later placements by point-cloud registration, so that the VLM is called only for unfamiliar parts. After grasping the part, the robot inspects both mating faces at close range and corrects the sketch by shape matching. On 69 simulated insertion episodes, our method inserts 88% of the parts at the assembled pose in a strict kinematic test, while ReKep and MOKA with the same VLM insert none. On new placements of eight tasks, reusing one stored run inserts as many parts as calling the agent in each episode, and it produces the sketch in 3.4 s instead of about 90 s.
comment: 25 pages, 6 figures
★ Integrating Low-Thrust Transfer Dynamics into POMDP Planning for On-Orbit Servicing
Recent developments in both high-thrust (HT) and low-thrust (LT) propulsion technologies broaden design options for on-orbit servicing (OOS) missions. However, incorporating LT transfers into OOS mission planning requires accounting for the trade-off between prescribed transfer duration and propellant consumption. Multi-servicing, multi-transfer OOS mission planning is inherently sequential and requires adaptive planning tools that account for propulsion-dependent transfer durations and mass-dependent propellant consumption. Thus, transfer dynamics must be integrated into mission planning and scheduling to increase flexibility and adaptability. In addition, the planning is often done with imperfect information about the system's state and stochastic service demands. This paper focuses on modeling orbital transfers, particularly LT transfers, within a Partially Observable Markov Decision Process (POMDP) framework for OOS mission planning. To enable offline policy computation, the transfer model must provide explicit state transitions and rewards for all admissible actions. The state space is therefore augmented with the servicing vehicle's remaining propellant to enable consistent mass updates. HT transfers are modeled using a precomputed transfer matrix that maps each transfer to the time of flight and propellant consumed. LT transfers are represented using an analytical GEO phasing model with prescribed transfer-duration alternatives. For each alternative, the model calculates propellant consumption and maneuver feasibility as functions of the required phase change and the current servicing-vehicle mass. The proposed modeling approach is evaluated on simplified OOS planning scenarios using circular phasing with HT and LT propulsion. The resulting POMDPs are solved using the Successive Approximations of the Reachable Space under Optimal Policies (SARSOP) algorithm, and the policies obtained under HT and LT are compared. This comparison provides a quantitative assessment of the feasibility and operational impact of LT options, particularly regarding the potential role of advanced electric propulsion in future servicing architectures.
★ PGGT-Loc: Primitive-Grounded Geometry Transformer for Feed-Forward Camera Localization NeurIPS 2026
Camera localization methods are mostly multi-stage pipelines relying on dedicated and complex maps that are costly to construct, store, and maintain over time. Alternatively, well-structured maps that organize scenes with compact yet expressive primitives are often readily available, providing rich structural and semantic cues that humans can interpret. However, their potential for direct camera localization remains largely unexplored due to the significant modality gap between map primitives and visual observations. This motivates a new concept for localization, which estimates camera poses by grounding query observations to map primitives. In this paper, we instantiate the concept in the context of planar primitives with PGGT, the Primitive-Grounded Geometry Transformer network. Given a single query image and a set of planar map primitives, the network is trained to predict multiple localization targets for feed-forward 6-DoF camera localization that generalizes to unseen environments. With this straightforward and scalable architecture, our method pushes the state-of-the-art localization performance on both dense surface maps and minimal layout models. The code and models are available at https://github.com/3dv-casia/PGGT-Loc
comment: Accepted at NeurIPS 2026. 24 pages, 14 figures, including appendices. Code and models: https://github.com/3dv-casia/PGGT-Loc
★ Visual-Inertial RGB-D SLAM with Encoders for a Differential Wheeled Robot
Recent years have seen multiple impressive results in visual-inertial odometry (VIO) techniques, by which accurate state estimation can be achieved via the extended Kalman filter (EKF) or nonlinear optimization. However, these approaches are rarely open source, and they tend to fail in real experiments due to the temporary lack of feature points and fast motion. Therefore, in this study, we used encoders to overcome the temporary failure of purely vision-based simultaneous localization and mapping (SLAM). Here, we propose a generative measurement model for encoders and derive an expression for the maximum a posteriori (MAP) estimate and necessary Jacobians for optimization. We use our theory to present a novel tightly coupled visual-inertial encoder RGB-Depth (RGB-D) SLAM system. Tests on our system on an in-house dataset confirmed that our modeling effort led to accurate (with a root mean squared error (RMSE) of approximately 2-7 cm) and robust state estimation in real time. The source code and our dataset containing the encoder information have been published for verification.
comment: 11 pages. Author-accepted manuscript of the article published in IEEE Sensors Journal, 2022
★ Higher-Order Kinematic Derivatives for Aspect-Preserving Real-Time Numerical Inverse Kinematics
This paper leverages the higher-order kinematic derivatives of serial robots to achieve efficient, aspect-preserving real-time inverse kinematics. First, a family of higher-order solvers is derived, with a necessary orientation-error correction map, yielding the QuIK4 and QuIK5 methods with fourth- and fifth-order convergence. Second, an aspect-preserving step limiter extrapolates the Jacobian determinant along the step to predict and avoid singularities, shortening steps only when they head toward a singularity rather than when the current iterate is ill-conditioned. Third, a Hermite-interpolation certifier verifies the aspect of each answer, and a certified restart algorithm recovers failures from proven same-aspect seeds. The proposed methods are shown to outperform prior methods, reducing failures by up to four orders of magnitude and nearly eliminating aspect switching at tracking-level errors. A C++ library is released with ROS 2 integration.
comment: 15 pages, 9 figures, 3 tables
★ When Architectural Spaces Start to Act
Architectural spaces are increasingly gaining robotic capabilities. Architectural robots such as robotic partitions, ceilings, furniture, facades, and other spatial components can autonomously move and reconfigure in response to occupants' needs and activities. Yet, when architectural robots start shaping the spatial conditions in which people continuously live, work, and interact, we may begin to experience the architectural space itself as robotic. What does this shift mean for how we inhabit such spaces? Drawing on our previous studies, I reflect on how architectural robots interact with people not only through their motions, but also through the spatial configurations that persist after those motions stop; how their actions may benefit some occupants while disadvantaging others; and how multiple robots may together make a whole architectural space appear to act. I argue that as computation gains the capacity to physically reconfigure our environments, interaction extends beyond the moment of computational action into the spatial conditions that people continue to inhabit. Understanding robotic architecture therefore requires us to look beyond short-term human-robot interaction and toward how people inhabit, negotiate, and maintain control over spaces that can autonomously reconfigure.
★ Flexibility-Aware Robotic Manipulation: Deformation Estimation, Kinematics, and Control via Distributed Inertial Measurement
Flexible robot links challenge the rigid-body assumption, making joint measurements alone insufficient for accurate manipulation. This paper introduces a flexibility-aware manipulation framework that connects structural dynamics directly to sensing, kinematics, and control. Instead of prescribing a dense inertial measurement unit (IMU) network, the dominant vibration modes of each flexible link are first identified from response-convergence and excitation-bandwidth criteria. Under a full-rank modal observability condition, $m$ retained modes require only $m$ independent slope measurements and one reference IMU, defining the minimum observable IMU configuration. The resulting measurements are used to reconstruct the continuous link deformation, which is then incorporated into flexibility-aware forward kinematics and a deformation-informed partial differential equation-based controller. Experiments on hydraulic single- and double-link manipulators evaluate the framework. On the long-reach double-link platform, the proposed kinematics achieves end-effector root mean square errors of $0.0107$~m, $0.0194$~m, and $0.0102$~rad while using fewer sensors than the piecewise-rigid method. Closed-loop reaching experiments further demonstrate real-time integration of IMU-estimated joint states and reconstructed deformation into the proposed controller. The results show that dominant modal information can determine sensing requirements and enable real-time flexibility-aware robotic manipulation.
★ When the Past Shapes Action: Understanding Embodied Memory
Recent advances in generative foundation models have expanded the capabilities of embodied AI, enabling increasingly general navigation, manipulation, and long-horizon task execution. However, effective interaction with dynamic environments requires more than only strong perception and reasoning: agents must act with partial knowledge and retain, retrieve, and update information from past observations, actions, and experiences. This survey examines embodied memory as a mechanism for preserving decision-relevant information beyond the current observation. We organize existing approaches along two complementary dimensions: six fundamental memory requirements, spanning world-state persistence, task progress, belief maintenance, and experiential learning; and eight architectural categories, including recurrent states, fast weights, visual and symbolic representations, spatial memory, and experience-based systems. We analyze their capabilities and limitations, and examine how existing benchmarks evaluate memory-dependent behavior, distinguishing genuine use of historical information from general task performance. Finally, we identify open challenges in selecting relevant experiences, maintaining consistent beliefs in changing environments, and consolidating knowledge across episodes. Our analysis provides a framework for understanding memory as an essential component of reliable, adaptive, and lifelong-horizon embodied intelligence.
★ Biomechanics do not predict perceptual and cognitive consequences of force augmentation
Wearable force-augmentation technologies, including exoskeletons and soft exosuits, are being developed for occupational and rehabilitation settings to enhance performance by reducing the muscular demands of movements. Yet enhanced performance also requires users to perceive the environment accurately, adapt to changing conditions, and decide whether actions are worth their physical cost. We used an elbow-assisting exosuit to investigate the physiological, perceptual and cognitive consequences of force augmentation during lifting. We assessed weight perception, adaptation in a volatile environment, and effort-based decision-making with and without assistance. While assistance reduced muscle activity, its behavioural outcomes did not simply reflect this biomechanical change. Assistance reduced perceived weight, but this effect was strongly shaped by participants' initial experience. Assistance also selectively reduced adaptation to changing environmental contingencies. Finally, reducing the muscular demands of lifting did not translate straightforwardly into how demanding the action felt or whether participants chose to perform it: differences in effort-based decisions were shaped by initial experience rather than simply tracking reductions in muscle activity or perceived fatigue. Together, we show that the perceptual and cognitive consequences of force augmentation cannot be inferred from its biomechanical benefits alone. Effective augmentation therefore requires perceptual and cognitive outcomes to be evaluated alongside physical performance.
comment: 43 pages, 3 main figures, 3 supplementary figures, 3 supplementary tables, 3 supplementary movies. Supplementary movies available at https://osf.io/ft56y/files/osfstorage
★ Interaction-Aware Sampling-Based Motion Planning for Autonomous Surface Vessels in Urban Waterways
Autonomous surface vessels operating in urban waterways must coordinate with other vessels in confined spaces while complying with traffic regulations, often without knowing their intentions. We present a decentralized motion planner based on model predictive path integral control, a sampling-based model predictive control scheme. The planner jointly computes a trajectory for the ego vessel and a prediction of the trajectories of the other vessels, and therefore accounts for their interaction. Traffic regulations are encoded as discontinuous costs, which the sampling-based formulation admits without a smooth approximation. We evaluate the method in maps taken from real sections of Amsterdam's canals, in encounters between two vessels and in scenarios with four interacting vessels. Across 780 randomized runs the method reaches every goal without collisions or deadlocks, and outperforms an optimization-based planner designed for the same domain, as well as a version of the proposed planner that does not account for the interaction. We further validate the planner on two scaled vessel prototypes in an outdoor environment, where planning and localization run onboard at 10 Hz, and head-on and crossing encounters are resolved without collisions or rule violations.
comment: This work has been submitted to the IEEE for possible publication
★ OpenVIO: A Versatile Open-Source Visual-Inertial Odometry System
Visual-inertial odometry (VIO) is a fundamental topic in robotics research. Beyond research, VIO has been adopted in a wide range of products. In this manuscript, we present OpenVIO, an open-source VIO system for research and practical applications. It is released under the MIT license. OpenVIO supports pinhole and fisheye cameras and extends this support to stitched 360° images. We describe each part of the system together with its design considerations. We also report benchmark results with other open-source VIO systems on public datasets. OpenVIO is available as open source at https://github.com/93won/openvio.
comment: 7 pages, 6 figures
★ GPU-accelerated Incremental Euclidean Distance Transform for Online Motion Planning of Mobile Robots
In this letter, we present a volumetric mapping system that effectively calculates Occupancy Grid Maps (OGMs) and Euclidean Distance Transforms (EDTs) with parallel computing. Unlike these mappers for high-precision structural reconstruction, our system incrementally constructs global EDT and outputs high-frequency local distance information for online robot motion planning. The proposed system constructs OGM with massive amount of data from multiple types of sensor inputs. Using GPU programming techniques, the system quickly computes EDT in parallel within local volume. The new observation is continuously integrated into the global EDT using the parallel wavefront algorithm while preserving the historical observations. Experiments with datasets have shown that our proposed approach outperforms existing state-of-the-art robot mapping systems and is particularly suitable for mapping unexplored areas. In its actual implementations on aerial and ground vehicles, the proposed system achieves real-time performance with limited onboard computational resources.
comment: Published in RA-L 2022. Author-accepted manuscript. Code: https://github.com/JINXER000/GIE-mapping
★ ResMerge: Residual Policy Learning and Merging for Continual Adaptation of Pre-trained Robot Policies
Continual adaptation to new tasks is essential for robots deployed in the real world, but adapting pre-trained policies remains challenging due to parameter growth and the need for continual demonstration collection. Parameter-Efficient Continual Fine-Tuning (PECFT) isolates task-specific updates, but adapter-per-task designs require demonstrations and grow with the task set. We propose ResMerge, a two-stage framework that learns mergeable residual policies atop a frozen pre-trained policy: residual reinforcement learning (RL) acquires task-specific corrections without additional expert demonstrations during adaptation, and parameter-space merging consolidates compatible residuals to reduce stored parameters. We introduce Residual-Field Consistency (RFC), a simple, bounded, and interpretable metric that assesses compatibility in residual action-field space by comparing correction direction and magnitude. Experiments on LIBERO across two backbones and on a real-world Franka robot show improved continual learning performance over strong PECFT baselines, while selective merging reduces the residual-parameter footprint. Project website: https://resmerge.github.io/.
comment: 9 pages. Code: https://github.com/ResMerge/resmerge-official
★ $\nabla$Mimic: Ultra-Efficient Humanoid Control with Differentiable Simulation
Robot policy learning is dominated by zeroth-order methods such as PPO, which treat the simulator as a black box and pay for it with enormous sample counts. First-order policy learning differentiates through the dynamics, but on contact-rich whole-body control it has rarely worked: its gradients explode. A recent world-model method showed that derivative information can make humanoid motion tracking far more efficient, using surrogate gradients of a learned model. We show that exact gradients of the simulator itself do better, once two obstacles are removed. Exact gradients contain rare unbounded samples, because contact dynamics is not differentiable everywhere and its derivative diverges near some events; and they are expensive, because the simulator's adjoint has to replay the contact solve. $\nabla$Mimic bounds these samples with median-relative per-sample gradient clipping at every step of the backward pass, and reuses each costly simulated window by multistep regression onto gradient-derived action targets, which bounds every action target independently of the gradient scale. NablaEngine, a differentiable GPU simulation engine for the Unitree G1, removes the cost: it computes MuJoCo's full contact dynamics with fused per-world kernels and the adjoint of a whole control step as a single megakernel, with analytic gradients that match float64 finite differences, $11.5\times$ faster than the available MuJoCo Warp adjoint. On whole-body tracking of all 40 LAFAN1 clips with one policy on one GPU, $\nabla$Mimic reaches the score reported by the state-of-the-art world-model method, 0.82, in 20 minutes instead of 1.2 h, and 0.874 in 1.8 h. SHAC, the standard first-order baseline, stops on exploding gradients within minutes. We release NablaEngine and $\nabla$Mimic as open-source infrastructure for first-order humanoid policy learning.
comment: 26 pages, 7 figures
★ Direct Action-Manifold Modeling for Efficient and Robust Bimanual Manipulation
Diffusion and flow matching visuomotor policies require iterative denoising or numerical integration for multimodal action generation, introducing latency and execution mismatch. We instead consider a different formulation: conditioned on vision and proprioception, demonstrated bimanual action chunks concentrate in a structured observation-dependent region of joint-action space, making the action target more direct to model than the generative process. Based on this view, we propose the Direct Manifold Action Transformer (DMAT), which reformulates action generation as the direct modeling of clean, observation-conditioned joint-space action chunks from stochastic interpolants, thereby enabling single-step inference without relying on iterative generative inference or auxiliary trajectory-straightening procedures. Within DMAT, local 3D geometry anchors predictions to spatial structure, while State-Consistent Spectral-Time Refinement (STR) reconciles predicted chunks with proprioception and suppresses high-frequency residuals. Across 567 standard and out-of-distribution (OOD) trials per method, DMAT outperformed Improved 3D Diffusion Policy (IDP3) and DiTFlow3D in aggregate success rate, achieving relative improvements of 129.1% and 95.7%, respectively, while generating action chunks 2.06x and 1.34x faster and remaining robust under repeated 50 N interventions. These results pave the way for a novel generative paradigm for efficient and robust bimanual manipulation.
★ RTPolicy: Real-Time (32.9 Hz) Multi-View Visuomotor Policy for Robotic Manipulation on Edge Devices
Low inference latency is critical for closed-loop robotic manipulation, especially under asynchronous inference, where action chunks must be updated promptly to correct accumulated deviations. However, existing visuomotor policies rely on computationally expensive visual backbones, resulting in high inference latency and deployment overhead. In this paper, we propose RTPolicy, a multi-view visuomotor policy that attains accurate manipulation with high inference efficiency. Its core design combines view-aware feature fusion with visual compression to aggregate complementary information across camera views into a compact token representation. The representation conditions a flow-matching action generator with the robot state and flow time, reducing the cost of action generation. Extensive experiments demonstrate that this design reduces the computational cost of visual representation and action generation while preserving visual information required for manipulation. On the RoboTwin~2.0 single-task clean evaluation across 50 tasks, RTPolicy attains a mean success rate of 52.26\%, outperforming the strongest leaderboard baseline $π_0$ (46.4\%). We further evaluate RTPolicy under asynchronous streaming inference on a real bimanual platform, where RTPolicy attains a mean success rate of 72.0\% and sustains real-time inference at 30.35 ms (32.9 Hz) on the RDK S600 and 149.72 ms (6.7 Hz) on the RDK S100, both in INT16. Project Page: https://d-robotics-ai-lab.github.io/RTPolicy.project.
★ VLA Policies Do Not Know When Not to Act: Unintended Task Execution and Start Suppression
A robot should act only on an instruction from a person or on an intent that the robot explicitly represents and generates. Unintended actions taken without such an instruction can cause risk or inconvenience to people and the environment. We examine how recent vision-language-action (VLA) policies respond when the instruction is missing or meaningless. We apply a paired protocol with empty, neutral, nonsense, infeasible, and wrong-task instructions to six model checkpoints in LIBERO-Goal simulation. Holding the initial state fixed and varying only the instruction, we ask three questions: whether the robot starts to move, which task it completes, and what the instruction determines. Even when the instruction was empty, the robot started moving in every episode with all six checkpoints, and with five of them it often completed a task that the instruction had not specified. On two GR00T checkpoints, the instruction determined which task was executed but not whether the robot started moving; the robot moved even on infeasible instructions. We then tested three methods for suppressing the start: prepending "if no task is given do not move" to the instruction; a gate that holds the robot when the first action chunk for the given instruction nearly equals that for an empty instruction; and fine-tuning the policy to hold still on non-task inputs. The prefix stopped the robot in none of the six checkpoints, and the gate held it on non-task inputs in five of the six checkpoints. The fine-tuned policy never completed a task on non-task inputs, but it did not reliably keep the robot still, and it failed to suppress the start on infeasible instructions.
comment: 17 pages, 3 figures, 13 tables. Code: https://github.com/zebehn/vla-start-suppression
★ DrawSplat: Atlanta World-Guided Implicit Gaussian SLAM for Indoor Reconstruction
RGB-D SLAM requires accurate online tracking, high-quality rendering, and geometrically consistent reconstruction. Existing Gaussian SLAM methods mainly optimize Gaussian primitives with photometric, depth, or local frame-to-model constraints, which are often insufficient in indoor scenes with weak textures, large planar structures, noisy depth, and long trajectories. The resulting maps may suffer from pose drift, surface thickness artifacts, noisy normals, and blurred structural boundaries. We propose DrawSplat, an Atlanta World-guided implicit Gaussian SLAM framework for indoor reconstruction. DrawSplat follows a drawing-like process: explicit point-line geometry first sketches the structural layout, and implicit neural Gaussians then fill in high-fidelity appearance and continuous geometric details. It estimates camera poses, sparse geometry, and gravity direction through point-line joint tracking, infers structural planes from tracked lines using the shared-gravity component of the Atlanta World prior, and converts them into differentiable regularization terms on Gaussian centers and normals. To improve compactness and efficiency, DrawSplat further combines an anchor-based implicit neural Gaussian representation with adaptive structural sampling, which allocates Gaussian capacity to structurally informative regions. Experiments on multiple datasets show that DrawSplat achieves robust tracking, high-fidelity rendering, and strong geometric reconstruction among Gaussian-based SLAM systems, while maintaining a practical balance among quality, efficiency, and model size.
★ A complete analytical characterization of optimal collision-free motion for two robots
Collision-free motion planning for multi-agent robotic systems remains an active research area because of the complexity of coordinating robot trajectories, with applications ranging from coordinated drone shows to autonomous operations on land, in the air, and on water. Importantly, the analytical difficulty of nonconvex collision-avoidance constraints has so far driven the development of exclusively numerical methods. We derive a complete analytical characterization of the optimal trajectories for two robots that start and end at rest, under double-integrator dynamics and a squared-acceleration objective. We prove that these trajectories belong to eight distinct contact families and determine which family applies for each configuration of initial and final positions. We analyze the geometrical properties of the optimal trajectories, provide a numerical algorithm for computing them, and discuss how these insights may improve the performance of existing numerical trajectory optimizers.
★ Contact-based optimization for collision-free motion planning for a million robots
Multi-agent robotic systems have attracted growing interest in recent years, with applications ranging from coordinated drone shows to autonomous operations on land, in the air, and on water. At the planning level, robot motion is often modeled using double-integrator dynamics (with low-level controllers handling the physical dynamics during execution) with a minimum-energy objective subject to collision-avoidance constraints and limits on position, velocity, and acceleration. The complexity of the problem resides in the constraints, which couple the trajectories and make the optimization problem nonconvex. Existing numerical approaches typically discretize and optimize entire trajectories, which becomes expensive for large fleets. We exploit the structure of optimal trajectories by organizing optimization around contacts, at which robots reach the minimum allowed separation. We formulate a coupled algebraic system in contact variables and reconstruct trajectories analytically between contacts. Our algorithm, sequential contact insertion (SCI), starts from unconstrained trajectories, inserts contacts at separation violations, and jointly adjusts their times and states. Numerical experiments demonstrate planning for one million robots in minutes and thousands of robots in fractions of a second. On problems feasible for a fixed-grid sequential convex programming baseline, SCI achieves median speedups of several orders of magnitude at matching solution accuracy.
★ FOCUS: First-Order Control Using Optimal Transport Supervision for Imitation from Observations NeurIPS 2026
Imitation learning from observations must recover behavior from state-only demonstrations without known temporal correspondence. Differentiable simulators provide pathwise policy gradients, but differentiating through long, contact-rich trajectories can make optimization unstable. We introduce FOCUS, which combines local optimal-transport alignment with short-horizon differentiable actor-critic learning. FOCUS matches agent rollout windows to multiple sampled expert segments using a soft best-of-$K$ objective that emphasizes locally compatible matches. The resulting transport-based costs provide differentiable imitation rewards, while a learned critic bootstraps value beyond the rollout horizon. FOCUS requires neither temporally aligned trajectories nor expert-aligned initialization or teacher forcing. Across contact-rich locomotion tasks in DFlex, including a musculoskeletal humanoid controlled by 152 muscle activations, FOCUS improves sample efficiency and training stability over adversarial imitation-from-observation methods and prior differentiable imitation baselines.
comment: Accepted at the NeurIPS 2026 Workshop on Physical Understanding for Decision-Making
★ AC-SMPC: Differentiable Actor--Critic MPC with State Constraints for Safe and Fast Robot Navigation
Achieving safe navigation in confined dynamic environments remains a challenging problem, as robots must balance navigation efficiency with safety. Existing actor--critic MPC architectures typically employ differentiable MPC layers, but this way cannot readily accommodate explicit state constraints, making it intractable to involve collision avoidance within learning-based planning. To address this issue, we propose an actor--critic model predictive control framework with state constraints. We first use vertices to represent the robot and obstacles, and half-spaces to represent the workspace. Based on these, the nonlinear robot dynamics is locally linearized and the hyperplane-based collision avoidance constraints are convexified into the differentiable MPC layer, and then the actor exploits closed-loop feedback to adjust the MPC objective parameters that shape local motion behavior, while the critic evaluates long-term task performance. We further relax the convexified constraints with bounded and penalized slack variables to form an elastic quadratic program, accommodating cases where the local constraint approximations cannot be satisfied simultaneously. Simulation tests and experiments, involving dynamic collision avoidance for multi-robot in narrow cases, illustrate the advantages of the proposed framework in adaptively adjusting the learned policy and improving navigation efficiency over existing approaches while maintaining the motion safety.
★ SAFARI: Semantic- And Frontier-driven Autonomous Risk Inspection
Robotic exploration can reduce human exposure to hazardous environments by autonomously searching unknown areas for objects of interest while constructing a complete, risk-aware map. However, existing autonomous exploration approaches typically prioritize geometric coverage or information gain, without explicitly coupling semantic object detection and the risk associated with operating near potentially hazardous objects. Moreover, many systems have been demonstrated primarily in simulation or controlled environments, leaving limited validation of risk-aware semantic exploration in complex, unstructured field settings. To address this gap, we propose SAFARI, a framework that treats object detection as a native channel of a unified probabilistic map, thereby jointly informing exploration and planning with both geometric and semantic evidence. We evaluated SAFARI in 11 missions across three field sites on a quadruped platform. The framework achieved an average coverage of 98.7% of the accessible mission area, without operator intervention for 97.6% of mission time, and detected objects with characteristic dimensions as small as 15cm. Lastly, through ablation studies and a detailed analysis of system behavior, we show that object-size awareness and safety-aware path planning are crucial for effective hazardous-object search.
comment: First two listed authors have equal contribution. Submitted to IEEE Transactions on Field Robotics
★ Physically Constrained Kinematic Calibration for High-Precision Robotic Needle Guidance
Percutaneous needle insertion is a common practice in minimally invasive procedures whose clinical success depends on accurate needle placement. However, achieving the necessary precision is a challenge for manual operators. Robotic assistance can improve targeting accuracy and reduce operator dependence, but residual millimeter-scale errors persist because nominal kinematic models do not fully represent the physical manipulator. This paper presents a robust bound-constrained kinematic calibration method designed to improve positional accuracy for a 6-Degree-of-Freedom (6-DoF) robotic needle-guidance manipulator. The approach uses physically plausible parameter ranges to regularize a compact 19-parameter geometric identification model. By treating these bounds as active optimization constraints rather than mere numerical safeguards, the framework effectively mitigates parameter coupling and overfitting, ensuring highly stable parameter identification. In simulation, the proposed method reduced end-effector Root Mean Square Error (RMSE) from 3.422 mm to 0.690 mm (a 79.8% improvement). In benchtop hardware experiments, the calibrated model achieved sub-millimeter validation accuracy, including 0.865 mm RMSE in experimental trajectory validation and 0.610 mm RMSE in a coordinate-lattice verification task. These results demonstrate that physically constrained kinematic calibration can enhance robotic needle-targeting accuracy, establishing a high-precision foundation for future clinical applications.
comment: 6 pages, 4 figures, 4 tables
★ SACQ: Structure-Aware Codebook Quantization for Ultra-Low-Bit Vision-Language-Action Models
Vision-Language-Action (VLA) models enable capable robot behavior, but their large memory footprint limits deployment on resource-constrained robotic platforms. Weight quantization is a natural remedy, yet post-training quantization (PTQ) recipes inherited from large language models degrade when applied to VLAs at ultra-low precision. Existing low-bit PTQ methods often fail to explicitly account for the structure of downstream operations and allocate precision without considering its effect on the policy actions. We introduce SACQ, an ultra- low-bit weight-only vector-codebook quantization framework for VLAs. SACQ combines (1) structure-aware codebook fitting with role-specific quadratic metrics that reflect input statistics and downstream sensitivity; (2) action-aware mixed-bit allocation based on the increase in final-action error as individual layers are compressed; and (3) two-stage post-quantization refinement that reassigns codeword indices with second-order error compensation and calibrates codebooks and row-group scales against the decoded action. Using GR00T N1.7, SACQ achieves 93.85% average success across four LIBERO suites at 1.5 bits per language/action weight while reducing model memory by 4.83x. With INT8 activation quantization and low-precision attention, it retains 93.55% success while running 1.51x faster than BF16. On SimplER, SACQ further achieves 63.76% and 52.27% success under Visual Matching and Variant Aggregation at 2 bits per language/action weight, compared to 58.90% and 53.45% for BF16, respectively.
★ BioAgent: Self-Directed Skill Acquisition and Capability Growth for Long-Horizon Laboratory Protocols
Executing an unfamiliar laboratory protocol requires a robot to identify missing capabilities and determine whether each operation remains valid as the experiment progresses. This letter presents BioAgent, an embodied agent that addresses these coupled challenges of capability sufficiency and experimental validity. It matches human-reviewed requirements to validated skills and task packages, selects tool-based implementations for missing behaviors, and requests human teaching when needed during learning. Learned skills undergo sequential task-package validation to check outcomes and compatibility between successive operations. During execution, precondition, outcome, and experimental-state checks govern progression, autonomous recovery, or termination. Execution evidence guides tool selection, while accepted experience supports offline policy updates and task packaging. On the second task, BioAgent reuses 17 skill types unchanged, revises two existing implementations, and acquires five new types. On two dual-arm laboratory tasks with 20 and 31 ordered skill invocations, BioAgent achieves complete-task success rates of 66\% and 58\% with autonomous recovery over 50 runs per task. Across 25 atomic skills evaluated over 50 autonomous trials each with recovery disabled, success rates range from 84\% to 100\%, with 17 skills achieving 50/50.
★ MolmoSpaces-CommonSense: Evaluating Commonsense in Robotic Manipulation Policies
Real-world robotic manipulation requires reasoning about cluttered scenes, partially observable or hidden objects, and semantically appropriate actions, in addition to low-level execution within those scenes. However, existing simulation benchmarks primarily test the latter and remain dominated by comparatively clean environments with limited clutter and narrowly scoped task structure, leaving many forms of commonsense reasoning untested. We address this by introducing a taxonomy of reasoning processes for robotic manipulation grounded in developmental cognitive science. We operationalize this taxonomy through a new benchmark of robot tasks, MolmoSpaces-CommonSense, that requires reasoning about occlusion, containment, support relations, semantically correct grasp points, spatial affordances, and hidden objects. These evaluations require policies not only to recognize relevant scene structure but also to execute correct actions in them. We evaluate several zero-shot generalist manipulation policies. Across model types and architectures, we observe consistent failures in commonsense reasoning: success rates in cluttered scenes drop by over 50% as distractors increase, policies act on incorrect objects in cluttered settings 30% of the time, and affordance-sensitive grasping remains unreliable even with highly specific language prompts, with correct object part selection happening only about 50% of the time. Further, VLM-based success estimation struggles to reliably evaluate rollouts in these settings, suggesting that the challenge extends beyond control to the broader perception-reasoning-action stack. Our results suggest that existing benchmarks may substantially overestimate zero-shot real-world readiness. Code and policy leaderboard link at https://github.com/allenai/molmospaces.
comment: Project page: https://github.com/allenai/molmospaces
★ DiffWBC: Unlocking Steerable General Humanoid Motion Tracking via One-Step Diffusion Whole-Body Control
General humanoid motion trackers typically rely on deterministic policies that cannot adapt online without brittle reference editing or costly retraining. We present DiffWBC, shifting whole-body control to a generative paradigm via a one-step diffusion model. By modeling distributions over future trajectories, DiffWBC unifies general tracking, test-time steering, and simulation-free failure discovery within a single controller: (1) Generative general tracking: DiffWBC requires a single forward pass per 50 Hz control step and can be distilled from an RL teacher or trained from scratch via model-based RL, achieving the lowest local tracking error among six general trackers on held-out benchmarks (20.1 mm on BONES) and deploying on a physical Unitree G1. (2) Zero-shot test-time steering: differentiable cost gradients and body-group masks modulate the frozen controller online, substantially outperforming kinematic reference editing (0.84 clean pass at 1.25 m ceiling vs. 0.10) and zero-shot generalizing across hurdles, spatial reaching, actuator faults, and tray balancing. (3) Simulation-free failure discovery: plan dispersion across generation noise effectively identifies physical tracking failures (91% precision in top 2%) without physics simulations; targeted fine-tuning on flagged clips resolves 50% of failure cases. Videos and interactive demonstrations are available at https://diffwbc.github.io/#selfassess.
★ CalibSwitch: Observation-Driven Camera-LiDAR Extrinsic Recovery and Selective Maintenance for Multimodal Perception
Camera--LiDAR perception on mobile robots and intelligent vehicles depends on reliable sensor geometry throughout deployment. Installation variations create uncertainty in the initial extrinsics, while changing observation quality makes frame-wise estimates fluctuate during operation. This paper presents CalibSwitch, an observation-driven framework that separates absolute extrinsic recovery from selective state maintenance. A cross-modal absolute extrinsic generator (CAEG) combines projection-free bidirectional feature interaction with geometry-regularized learning to recover complete SE(3) candidates from image--point observations and coarse view context, without a supplied numerical initial transform. A selective recurrent calibration policy (SRCP) combines observation quality and temporal history to determine when a candidate should replace the working geometry consumed by a frozen multimodal detector. On nuScenes, CAEG achieves mean translation and rotation errors of 0.248 m and 5.884$^\circ$. Under sustained geometric transitions, SRCP reaches a mean composite error of 1.46 with 9.70\% candidate commitments, outperforming quality-based and temporal-consistency gating at comparable commitment rates. Relative to unconditional per-frame updating, it improves frozen-detector mAP and NDS by 1.43 and 0.92 points. Paired geometry--observation uncertainty tests, two real-vehicle platforms, and 13.5-FPS single-camera Jetson execution demonstrate the system value of selective geometric-state maintenance for continuous perception.
★ Geometry-Aware Humanoid Retargeting via Manifold ODEs
In this paper, we focus on interaction-preserving motion retargeting, which transfers human demonstrations to humanoids while maintaining spatial relationships with objects and terrain. Previous work, such as OmniRetarget, addresses this task through temporally coupled constrained optimization, requiring repeated sequential quadratic programming (SQP) iterations at each frame. We study this problem from an ordinary differential equation (ODE) perspective by modeling the continuous-time limit of its framewise Karush-Kuhn-Tucker (KKT) conditions as an ODE on the robot configuration manifold. This viewpoint offers two benefits: (i) it connects OmniRetarget's SQP iterations to a backward discretization of the ODE, enabling us to establish their local convergence under suitable regularity assumptions; and (ii) it leads to TaSP-KKT (tangent-space parametrization for KKT flow), a new retargeting method based on a forward discretization of the KKT flow. The proposed method achieves the same first-order convergence to the ODE solution as the corresponding backward scheme while requiring only one weighted tangent-cone projection and a closed-form manifold update per step, thereby avoiding repeated inner SQP iterations. Numerical experiments show that TaSP-KKT achieves retargeting quality comparable to, though slightly lower than, that of OmniRetarget while running approximately twice as fast.
★ Joint-Health-Aware Reinforcement Learning for Robust Robot Control Under Actuator Degradation
Robots deployed in unstructured environments are susceptible to actuator-authority degradation, which reduces the effective control authority of individual actuators and can cause reinforcement learning policies trained under nominal dynamics to fail. Existing robustness approaches rely on discrete fault labels, broad domain randomization, or global latent variables, often without explicitly modeling continuous per-joint residual actuation capability. We propose a joint-health-aware reinforcement learning framework for robust robot control under simulated actuator degradation. Our framework encodes actuator condition in a continuous joint-health vector and models degradation as multiplicative attenuation of executed actions. A dedicated HealthNet estimator infers per-joint health from recent observations and actions, allowing the policy to redistribute control effort across healthy and degraded joints. We evaluate the method on Walker, Go2, Cassie, and Panda-based ManiSkill manipulation tasks under diverse degradation masks and severity levels. The method outperforms nominal and degradation-trained no-health baselines on the main evaluated metrics. Relative to the nominal baseline, task-level locomotion returns improve by up to 250.1\% and manipulation success rates by up to 15.2 percentage points; in the high-degradation Cassie group, return increases from 3.94 to 40.50 (+928.0\%). Ablations support the contribution of per-joint health estimation, action-degradation modeling, and health-conditioned control.
★ HandMimicX: Learning Human-Like Dexterous Manipulation via an Object-Centric Policy from Egocentric Videos
Learning dexterous manipulation from human videos remains an open challenge, particularly in the context of transferring a single demonstration into a spatially generalizable robotic skill and maintaining physically plausible human-like interactions. Unlike prior works that emphasize hand-motion imitation, we introduce an object-centric framework for acquiring dexterous skills from egocentric human RGB-D videos. Within our framework, we formulate the reinforcement learning (RL) objective in terms of object motion to decouple the underlying skill from the specific hand trajectory and impose constraints on hand-object interaction states to promote more anatomically natural hand configurations. Extensive experiments in simulation and on real robots demonstrate that our method enables efficient learning from human videos and generalizable skill transfer across novel spatial configurations.
★ HIPC: Humanoid Iterative Planning and Control via Goal-Conditioned Motion Closure
Recent generative models are widely used to provide whole-body motion for humanoid tracking. Current methods combine motion generators with controllers in an open-loop paradigm, which leads to critical issues: tracking errors can accumulate into spatial drift, and the planner cannot adapt to unexpected physical disturbances without feedback from physical execution. However, directly feeding realized states back to the planner in a closed loop can cause discontinuities, resulting in physical jitter and even falls, since such states are out of distribution during training. To bridge this gap, we propose HIPC, a real-time Humanoid Iterative Planning and Control framework that couples a generative model with motion tracking into a unified planning-and-control paradigm. HIPC introduces a goal-conditioned motion planning trained with a canonical motion representation that avoids gimbal-lock singularities under extreme orientations and supports motion generation on diverse geometric and semantic conditions. Then, HIPC adopts two strategies, deviation-decayed recovery and history re-anchoring, to mitigate the out-of-distribution shift caused by realized-state feedback. Finally, HIPC operates a real-time asynchronous planning-and-control system, with latency compensation to reduce temporal mismatch for inference delay. Extensive experiments confirm the effectiveness of HIPC under switching and composition of diverse conditions. Results show that HIPC achieves state-of-the-art goal-reaching accuracy among existing methods while simultaneously demonstrating robustness to physical perturbations and the ability to recover from falls.
comment: Website: https://hipc-humanoid.github.io/
★ Cross-Embodiment Motion Planning by Coupling Body-Agnostic World Model with Differentiable Ego-Dynamics
Motion planning in clutter is embodiment-specific: the feasible free space and reachable commands depend on each robot's geometry and dynamics. Existing cross-embodiment policies output normalized commands that disregard these differences, or compensate through per-robot training and distillation. World models can in principle pool cross-embodiment data, yet command-conditioned models bind their latent dynamics to a single action space, while pose-conditioned models omit the mapping from commands to body-specific motion. On flat ground, however, embodiment affects the planner's observations only through the camera ego-motion. This letter proposes Hourglass, an hourglass-shaped architecture that factorizes cross-embodiment motion planning at this interface: a body-agnostic world model and each robot's differentiable ego-dynamics, which map commands to that ego-motion, are coupled only through it. The world model is trained on data pooled across embodiments and camera mounts; each robot's ego-dynamics is a reduced-order template with identified physical parameters. A generalist parameter-conditioned policy is learned once at this junction: each imagined step rolls out the template and then the world model, and imagination training through each robot's template makes the policy body-aware. On seven simulated robots of five locomotion types, the generalist policy outperforms imitation-learning and diffusion-policy baselines on the same world model by 3.8--10.6 percentage points in success rate and matches per-robot specialists without dedicated training, while approaching its parameter-conditioned MPPI teacher within 1.4--2.8 points without an online map. After three minutes of teleoperated driving for identification, the synthetic-data-trained model transfers zero-shot to a real skid-steer robot unseen in training, navigating unseen indoor scenes.
comment: 8 pages
★ Learning from Physical Experience:Context-Space Adaptation for Robotic Manipulation
Recent multimodal foundation models exhibit strong perception, reasoning, and in-context learning capabilities, but translating these capabilities into physical action remains challenging. Through real-world experiments across diverse manipulation tasks, we find that ChatGPT-6-Astra, a state-of-the-art general-purpose reasoner, can serve as a physically actionable, context-responsive manipulation policy, demonstrating effective fine manipulation and spatial reasoning. Building on these findings, we introduce LEXP, a harness for learning from physical experience through context-space adaptation. LEXP combines task-level planning with an experience module that interprets execution evidence, extracts and retains lessons with applicability conditions, and reuses relevant lessons to generate contextual feedback for subsequent actions. Model weights and the action interface remain fixed throughout. Across 12 real-world manipulation tasks, LEXP achieves $23/36$ successful trials (63.9\%), compared with $19/36$ (52.8\%) with memory alone and $2/36$ (5.6\%) for vanilla Astra. On three tasks, bread transfer, sort trash and toiletries, and retrieve drawer, removing planning reduces success from $7/9$ to $4/9$, while removing retained experience reduces it to $3/9$. Execution traces illustrate feedback-guided corrections and cross-episode reuse of an inspection strategy. Execution analyses illustrate how task structure and experience-derived feedback guide task progression, action correction, and experience reuse. These results support context-space adaptation as a practical mechanism for turning physical interaction into reusable guidance for robotic manipulation.
★ TRACE: Target-State Risk-Aligned Conditional Execution for Embodied Tracking
Embodied visual tracking requires a robot to follow a language-specified person under occlusion, look-alike distractors, and ambiguous instructions. We show that improving tracking persistence can simultaneously increase safety violations. Using a compact 0.6B vision-language-action (VLA) tracker with a frozen backbone, we update only the planner head with proximal policy optimization (PPO). PPO improves success and following rates across all three EVT-Bench settings, but increases the collision rate on distracted tracking because episodes that previously ended in target loss can instead end in contact. We introduce TRACE, an execution-time target-state risk gate that preserves the learned policy and intervenes selectively. A lightweight risk head processes 84 cues, including candidate boxes, identity margins, a learned range-and-bearing probe, and the policy's commands. When the calibrated risk exceeds a threshold, TRACE rescales the translational component of the next waypoint while leaving yaw unchanged. The executed action is identical to PPO's on more than 98% of steps. In paired evaluations on identical episodes, TRACE reduces the distracted-tracking collision rate by 3.96 percentage points, while further improving success and following rates. Matched slowdown controls without target-state timing do not reproduce this improvement. Concurrent evaluation reproduces a collision reduction of 3.77 points, compared with 0.29 points for identical-arm null pairs, indicating that the effect exceeds evaluation variability. These results show that target-state-aware execution can recover the safety cost of policy post-training with minimal intervention.
comment: 35 pages, including appendices
★ CONSORT: Uncertainty-Constrained Multi-Robot Planning for Cooperative Localization in GPS-Denied Environments
Centralized multi-agent path planning with cooperative localization in GPS-denied environments is generally computationally intractable when formulated in full belief space due to tightly coupled uncertainty dynamics. Prior work typically approximates belief dynamics or restricts inter-agent coupling. This work examines a multi-agent planning problem in which a primary agent must reach a goal in minimum time subject to a chance-constrained collision-avoidance requirement and a terminal bound on localization uncertainty. We obtain a tractable surrogate of the underlying belief-space problem by optimizing over the mean-state trajectories of all agents and propagating covariance for terminal constraint evaluation only. We propose CONSORT (Cooperative Navigation with Support-Optimized Routing and Trajectories), a discrete-to-continuous pipeline combining hexagonal-grid joint search with spline-based trajectory refinement. Across 50 randomized scenarios, the planners separate as the terminal uncertainty bound tightens, with CONSORT solving 80% of scenarios where the strongest baseline solves 54%, at comparable path length. In marine simulations, with the plans executed under feedback control on an independently developed factor-graph estimate, the trajectories remain collision-free and their terminal uncertainty falls within 6% of the planner's prediction.
comment: 8 pages, 8 figures
★ TinyAstral: Transferring Self-Evolving Harnesses to Tiny Generalist Manipulation Agents
Robot agents built on vision-language models depend on both the planner and its harness: the tools, feedback, memory, and recovery logic around it. Can improvements to a large teacher's harness be carried into a compact agent with lower inference compute cost? We present TinyAstral, a two-stage approach. We first freeze an open-weight teacher and improve its harness through recursive self-improvement (RSI): frontier LLM agents read the teacher's trajectories and propose harness edits, and each edit is accepted only if it passes a paired statistical test. We then freeze the final harness, collect teacher trajectories with reasoning and failure recovery, and distill a small student that replaces the teacher in the same harness. On RoboTwin, harness evolution raises the teacher's success from 33.4% to 63.6%; the final 4B student reaches 62.8%. Across four harness versions, distilled 4B students score close to their corresponding teachers, and later versions yield stronger students, showing that harness evolution gains carry into compact models. The harness gains also transfer to other frozen planners without retraining. On identical episodes, the final 4B student outperforms open 4B agents, the open 27B harness-RSI model SenseNova-RoboRSI, and our own teacher model placed in three competitors' harnesses. It stays close to its teacher on all three training benchmarks, and on three benchmarks never used for RSI or training it scores close to a 552B planner running the same harness. On the measured deployment episodes, it uses about one fifth of its teacher's accelerator time.
comment: 16 pages, 5 figures. Project page: https://garfield-kh.github.io/TinyAstral/
★ Capability-Conditioned Graph Control Barrier Functions for Zero-Shot Safe Navigation of Heterogeneous Mobile Robot Teams
Multi-robot systems are increasingly deployed as heterogeneous fleets with diverse kinematic and actuation capabilities. While graph control barrier function (GCBF) methods provide a scalable framework for decentralized safe control, existing approaches typically assume homogeneous robot dynamics. Methods that directly encode robot classes can distinguish known types, but do not represent state-dependent motion capabilities or naturally transfer to unseen robot models. To address this challenge, we propose the Capability-Conditioned Heterogeneous Graph Control Barrier Function (CC-HGCBF), defined in a shared second-order reference-point operational space. This formulation provides a common geometric representation of safety across heterogeneous robots while retaining their model-specific dynamics and admissible control constraints. To enable zero-shot transfer, we introduce the Radial Capability Field (RCF), a compact, fixed-dimensional representation of each robot's state-dependent finite-horizon reachable velocity set. Conditioning the shared graph barrier and decentralized policy on the RCF enables them to adapt to changes in robot state, kinematic constraints, and control limits without relying on predefined robot identities. We jointly train the barrier and policy using proactive safety labeling and neural CBF-QP supervision, and evaluate them on mixed teams containing both seen and unseen robot models. Experiments demonstrate scalable safe coordination and zero-shot transfer to robot models and capability variations withheld during training. Physical-robot experiments further demonstrate zero-shot deployment to unseen kinematic models without retraining.
comment: 9 pages, 5 figures, 2 tables
★ A Capability-Aware Backup-Recoverability Filter for Heterogeneous Multi-Robot Navigation
Safe navigation in heterogeneous multi-robot teams faces two coupled challenges: differences in robots' braking and turning capabilities limit how each robot can respond to an approaching neighbor, while simultaneous interactions require a single control action to satisfy multiple collision-avoidance constraints. Together, these challenges can make local safety constraints incompatible with a robot's admissible controls, leaving a decentralized safety controller without a feasible action. We propose a Capability-Aware Backup-Recoverability Filter (CA-BRF), based on the insight that navigation actions should preserve access to a certified recovery maneuver. First, we construct a backup-recoverable set in a shared Cartesian operational space by evaluating whether prescribed backup maneuvers maintain pairwise safety until the robots come to rest. Each maneuver respects the corresponding robot's kinematic and actuation constraints, making the recovery certificate capability-dependent. Second, CA-BRF filters candidate controls by selecting admissible interpolations between candidate and backup actions while coordinating the selection across simultaneous interactions. This preserves backup recoverability while retaining as much of the candidate action as possible and ensures that all pairwise recovery conditions are satisfied by a single executable control for each robot. Together, we establish recursive feasibility and safety for the discrete-time closed-loop system even when the nominal safety-constrained optimization becomes infeasible. The validity and effectiveness of the proposed CA-BRF are validated in simulations with up to 30 heterogeneous robots and experiments of robots with five different kinematic classes.
comment: 8 pages, 3 figures
★ Mechanistic Interpretability for End-to-End Self-Driving
End-to-end (E2E) autonomous driving models achieve strong planning performance but remain largely opaque, limiting observability and trust. We apply mechanistic interpretability to E2E driving by training Sparse Autoencoders (SAEs) on the internal activations of a simple, single-camera end-to-end path planning model. Unsupervised clustering of SAE latents reveals interpretable structure aligned with route intent, vehicle speed, and braking behavior, recovering route intent with 98.3% accuracy without supervision. Through paired counterfactual image edits, we further show that specific SAE features are visually grounded and systematically respond to scene changes such as traffic light state and stop sign presence. Steering individual SAE latents produces predictable, monotonic changes in planner outputs including lateral displacement, curvature, and braking. Finally, we demonstrate a practical application of these representations: we build a lightweight adapter trained on SAE latents that reduces trajectory selection error by 3.44%, outperforming an equivalent adapter that omits SAE latents, confirming that the sparse representation captures useful information for interpretability. Code and visualizations are available at https://mgagvani.github.io/mechanistic-interpretability-for-e2e-driving/
comment: Accepted to the Conference on Robot Learning (CoRL) 2026. Project page: https://mgagvani.github.io/mechanistic-interpretability-for-e2e-driving/
★ MotorSense: A Video-EMG Dataset of Motoric Representations for Action Understanding NeurIPS 2026
Computer vision models learn hand action only from its visual consequences, without access to the underlying embodied motor signals. We address this absence with MotorSense, the first large-scale paired egocentric video and surface electromyography (EMG) dataset. MotorSense pairs bimanual EMG bands with egocentric video and yields synchronized video, bimanual EMG and IMU, and audio across 100 hours of unscripted manipulation. EMG offers a powerful scalable alternative to manual annotation: it captures finger movement, hand configuration, and the forces exerted between hand and environment as a dense signal free of occlusion and labeler bias, whereas manual annotation is costly, subjective, and unreliable under occlusion. We use MotorSense to learn sensorimotor alignment, producing an organization of action that is shared across the visual and motoric spaces, through cross-modal pre-training between video and EMG. The resultant vision models are then transferred to downstream tasks that benefit from sensorimotor alignment: in this case, action recognition, 3D hand reconstruction, and 3D contact reconstruction. We further demonstrate a generative application in which EMG motor signals are used to condition a video diffusion model, yielding videos of fine-grained hand manipulation. Across four action-centric computer vision tasks and eight popular video datasets, we demonstrate that sensorimotor alignment provides a stronger foundation for visual representation learning than conventional self-supervised learning methods over video, outperforming every baseline across all tasks and datasets evaluated.
comment: NeurIPS 2026. 27 pages, 9 figures, 5 tables. Project page: https://www.cs.umd.edu/~edessale/motorsense/
♻ ★ Sequential Planning via Anchored Robotic Keypoints
Changes to what a robot can see do not necessarily mean that there is a change in the problem it is trying to solve, and many language-conditioned manipulation systems struggle with this distinction. When something in a scene moves, the robot might need to execute its actions in different places, but its manipulation plan can stay the same. We propose Sequential Planning via Anchored Robotic Keypoints (\spark), which describes a task as a symbolic behavior tree that we only anchor to the current 3D observations when each action is called. By verifying the plan at runtime we can recover from failures locally and anchor the objects to the scene again. We ask for a new plan only if this fails. We isolate this concept by taking a fixed plan and moving objects in the scene while executing. When displacing objects by $10$\,cm we were still able to complete $66.7\%$ of trials with scene-diff verification and layered recovery, and we did not have to change the plan. We test our full system on \libpro with these conditions. There is no practice on the tasks being tested and no access to simulator states during rollouts, and as in \rats and \capx a success check after each attempt decides whether to retry. We use one open-weight 27B planner with 4-bit weights. Under this protocol \spark achieves a mean success of $66.8\%$ across the 6 position and task settings and $66.8\%$ on the held-out language and object settings. We also achieve a $68\%$ success rate across 11 physical task-embodiment settings on 3 different robots. Our work demonstrates that for an important class of failures the robot does not need to ask for a new plan for its task. It can be enough to update the system's view of the world. Code and videos can be found at https://cwru-aism.github.io/spark-page/
comment: 30 pages, 7 figures
♻ ★ Self-Building Labs
Self-driving laboratories (SDLs) accelerate scientific discovery by automating experimental execution and optimization. AI agents can increasingly propose new experiments, but existing SDLs can only run the procedures they were built for. Adapting them to a new experiment still requires extensive manual engineering, which makes laboratory flexibility the bottleneck for scaling discovery. We introduce the self-building laboratory (SBL), which designs, configures, and operates its own laboratory setup given a high-level scientific goal. SBL starts from whatever already exists, from an empty room to a working SDL, and builds the missing capabilities: the lab layout, its digital twin, robot skills, and experimental workflows. We demonstrate the system working across different robots and setups, scaling from a single workstation to laboratories with several robots working together. We validate SBL's end-to-end capabilities through a real-world quantum-dot synthesis experiment. By shifting human researchers from manually configuring and programming device automation to specifying scientific objectives, SBL makes scientific automation more accessible, flexible, and scalable. Project website: https://self-building-labs.github.io/
♻ ★ Shared Modular Recurrence in Contextual MDPs for Universal Morphology Control
A universal controller for any robot morphology would greatly improve computational and data efficiency. Steps have been made towards such multi-robot control by utilizing contextual information about the properties of individual robots and exploiting their modular structure in the architecture of deep reinforcement learning agents. When the robots have highly dissimilar morphologies, however, this becomes a challenging problem, especially when the agent must generalize to new, unseen robots. In this paper, we consider the fact that contextual features are often only partially available, but that they can be recovered through modular interactions. This can allow for better multi-robot control and generalization to contexts that are not seen during training. To this extent, we implement a transformer-based architecture with shared modular recurrence and evaluate its (generalization) performance on a large set of MuJoCo robots. The results show a substantial improvement in zero-shot generalization performance on robots with unseen dynamics, kinematics, and topologies, in four different environments.
♻ ★ Semantically Adaptive Control Barrier Functions with Vision-Language Models for Real-Time Robot Navigation
Robots operating in dynamic, unstructured environments must maintain safety without becoming unnecessarily conservative. While control barrier functions (CBFs) provide a principled mechanism for collision avoidance, their behavior is typically determined from geometric state alone and therefore cannot distinguish between semantically different situations with similar physical configurations. We present AlphaAdj, a vision-to-control navigation framework that uses egocentric RGB observations and a vision-language model (VLM) to adapt CBF conservativeness according to visual semantic context. The VLM produces a scalar contextual caution signal that dynamically modulates the CBF parameter alpha, while a geometry- and speed-dependent cap bounds admissible aggressiveness. To support real-time operation despite VLM latency, inference runs asynchronously with the control loop, and the semantic signal is extracted directly from token-level output probabilities rather than full autoregressive text generation. We evaluate AlphaAdj across matched-geometry, dynamic-human, and cluttered navigation scenarios, as well as on a physical Clearpath Jackal. When obstacle geometry is held fixed, AlphaAdj allocates 0.405 m more clearance to a human than to a benign object, while geometry-only and non-visual adaptive CBF baselines produce no semantic differentiation. This additional behavioral reserve improves robustness to unexpected human motion, including cluttered conditions in which all evaluated geometric baselines collide while AlphaAdj remains collision-free. The system maintains a 0.0167 s control period with 0.158 s mean VLM round-trip latency, demonstrating that semantic visual reasoning can complement geometric safety control without entering the critical control path.
♻ ★ Rapid co-design of Buoyancy-assisted robots for Challenging Locomotion using Gaussian Evolutionary Specialists
Designing high-performance legged robots requires jointly optimizing morphology and control. Model-free reinforcement learning (RL) enables learning complex controllers during co-design, but repeatedly training policies across candidate morphologies makes design optimization computationally expensive. Existing approaches can amortize policy learning across morphologies, but a single shared policy tends to converge toward one control strategy regardless of morphology, which we refer to as behavioral diversity collapse. We introduce Gaussian Evolutionary Specialists (GES), an RL-based co-design framework that maintains a population of specialist policies whose design-space territories evolve according to their relative control performance. We evaluate GES on the Buoyancy-Assisted Light Legged Unit (BALLU) robot for obstacle traversal and ramp walking over design spaces of up to four dimensions. GES achieves higher best design performance than Monolithic and Mixture-of-Experts baselines, with gains of 5.0-9.0 cm in obstacle height and 10.0-12.7 points in ramp grade. For the same 200 design-optimization evaluations, GES reduces wall-clock co-design time by 67.4% compared with retraining a policy for every candidate. Finally, teleoperated hardware experiments evaluate the optimized BALLU morphology, which traverses obstacles of up to 24 cm, compared with 8 cm on the original design.
comment: Under review
♻ ★ CHOP: Counterfactual Human Preference Labels Improve Obstacle Avoidance in Visuomotor Navigation Policies
Visuomotor navigation policies have shown strong perception-action coupling for embodied agents, yet they often struggle with safe navigation and dynamic obstacle avoidance in complex real-world environments. We introduce CHOP, a novel approach that leverages Counterfactual Human Preference Labels to align visuomotor navigation policies towards human intuition of safety and obstacle avoidance in navigation. In CHOP, for each visual observation, the robot's executed trajectory is included among a set of counterfactual navigation trajectories: alternative trajectories the robot could have followed under identical conditions. Human annotators provide pairwise preference labels over these trajectories based on anticipated outcomes such as collision risk and path efficiency. These aggregated preferences are then used to fine-tune visuomotor navigation policies, aligning their behavior with human preferences in navigation. Experiments on the SCAND dataset show that visuomotor navigation policies fine-tuned with CHOP reduce near-collision events by 49.7%, decrease deviation from human-preferred trajectories by 45.0%, and increase average obstacle clearance by 19.8% on average across multiple state-of-the-art models, compared to their pretrained baselines. These improvements transfer to real-world deployments on a Ghost Robotics Vision60 quadruped, where CHOP-aligned policies improve average goal success rates by 24.4%, increase minimum obstacle clearance by 6.8%, reduce collision and intervention events by 45.7%, and improve normalized path completion by 38.6% on average across navigation scenarios, compared to their pretrained baselines. Our results highlight the value of counterfactual preference supervision in bridging the gap between large-scale visuomotor policies and human-aligned, safety-aware embodied navigation.
♻ ★ FlowTouch: View-Invariant Visuo-Tactile Prediction
Tactile sensation is essential for contact-rich manipulation tasks. It provides direct feedback on object geometry, surface properties, and interaction forces, enhancing perception and enabling fine-grained control. An inherent limitation of tactile sensors is that readings are available only when an object is touched. This precludes their use during planning and the initial execution phase of a task. Predicting tactile information from visual information can bridge this gap. A common approach is to learn a direct mapping from camera images to the output of vision-based tactile sensors. However, the resulting model will depend strongly on the specific setup and on how well the camera can capture the area where an object is touched. In this work, we introduce FlowTouch, a novel model for view-invariant visuo-tactile prediction. Our key idea is to use an object's local 3D mesh to encode rich information for predicting tactile patterns while abstracting away from scene-dependent details. FlowTouch integrates scene reconstruction and Flow Matching-based models for image generation. Our results show that FlowTouch is able to bridge the sim-to-real gap and generalize to new sensor instances. We further show that the resulting tactile images can be used for downstream grasp stability prediction. Our code, datasets and videos are available at https://flowtouch.github.io/
♻ ★ Specification-Aware Distribution Shaping for Robotics Foundation Models
Robotics foundation models have demonstrated strong capabilities in executing natural language instructions across diverse tasks and environments. However, they remain largely data-driven and lack formal guarantees on constraint satisfaction during deployment. In practice, robots often need to comply with operational constraints involving rich spatio-temporal requirements such as time-bounded goal visits, sequential objectives, and persistent safety conditions. In this work, we propose a specification-aware action distribution optimization framework that enforces a broad class of Signal Temporal Logic (STL) constraints during execution of a pretrained robotics foundation model without modifying its parameters. At each decision step, the method computes a minimally modified action distribution that satisfies a hard STL feasibility constraint by reasoning over the remaining horizon using forward dynamics propagation. We validate the proposed framework in simulation using state-of-the-art robotics foundation models across multiple environments and complex specifications.
comment: 8 pages, 4 figures
♻ ★ VEGA: Learning Navigation VLAs from In-the-Wild Egocentric Video with Geometric Trajectory Supervision
We introduce VEGA, an approach for training navigation VisionLanguage-Action (VLA) models from unlabeled egocentric navigation videos. Internet-scale egocentric videos provide a scalable source of navigation-relevant visual observations, capturing cluttered scenes, close-range obstacles, and natural human motion through real-world spaces. However, these videos are not directly usable for policy learning because they do not provide obstacle-aware trajectories conditioned on explicit navigation goals in the robot's coordinate frame. VEGA addresses this gap by reconstructing local scene geometry from monocular video, sampling navigation goals (represented as text, image, or spatial waypoints) and generating obstacle-aware trajectories using the constructed geometry. The resulting trajectory distribution is then used to train a flow-matching VLA navigation policy. By using geometry exclusively during training, VEGA distills obstacle-aware planning directly into a vision-based policy. Furthermore, we introduce VEGA-Bench, a benchmark containing 250k scenes and approximately 5 million navigation goals paired with scene geometry, designed to evaluate goal progress, collision avoidance, and obstacle clearance of VLAs. Our evaluation shows that VEGA achieves competitive goal progress while reducing collisions by 33.0% and improving obstacle clearance by 17.9% over the strongest baseline on VEGABench, while improving success by at least 150.0%, reducing collisions by at least 66.7%, and improving obstacle clearance by at least 60.0% in real-world trials. Ultimately, we demonstrate that video-derived geometric supervision provides a scalable and effective signal for training obstacle-aware navigation VLAs. The code and benchmark will be released at the time of publication.
♻ ★ Denoising Multirobot Trajectories
Multi-robot trajectory planning is a fundamental problem in multi-robot coordination but remains computationally challenging due to its nonconvex, multimodal, and high-dimensional nature. This work builds upon D4orm, a dynamics-aware diffusion-denoising framework, and develops a family of planning architectures for diverse operational requirements. Unlike conventional numerical optimization methods, D4orm employs sampling-based optimization to generate solution trajectories through massively parallel sampling, leveraging modern computing architectures such as GPUs. Its diffusion-denoising structure iteratively optimizes deformations to candidate control trajectories, providing an efficient and versatile paradigm for generating kinodynamically feasible and conflict-free trajectories. Using D4orm as the building block for advanced planners, we present a decoupled planner for improved scalability, an online receding-horizon planner with feedback control, and a distributed planner for resource-constrained settings. Evaluations with differential-drive and holonomic robots in 2D and 3D environments demonstrate that D4orm-based approaches find high-quality solutions faster and more reliably than other sampling-based optimization methods, such as MPPI, as well as a learned diffusion-model-based method. We further demonstrate zero-shot deployment on ten real quadrotors with obstacles, large-scale deconfliction with 100 simulated robots, and fully onboard distributed `lifelong' operation with six ground robots. Overall, these results establish diffusion denoising as a scalable and reliable framework for multi-robot coordination. Code and video: https://github.com/proroklab/d4orm
comment: Accepted to IEEE Transactions on Robotics (T-RO) 2026
♻ ★ LOGIV: Graph-Verified Execution with Causal Local Repair for Long-Horizon Robot Tasks
Agentic robots must turn a high-level goal into actions while keeping the physical conditions needed for future steps valid. A completed action does not ensure that its effects will persist: a later disturbance can invalidate support that the remaining plan still assumes. We present LOGIV, a graph engineering framework for agentic robot execution that uses an explicit task graph to connect symbolic planning, observation-based verification, and embodied action. LOGIV compiles a validated plan into condition-support and resource-ordering edges, checks their physical validity during execution, and uses dependency tracing to determine what must be restored after a failure. Repair is constrained by retained unfinished actions, and the combined continuation is symbolically validated before execution resumes. The framework operates above existing Vision-Language-Action (VLA) and World Action Model (WAM) policies without adding trainable components. Across 20 dual-arm tasks from RoboTwin V2 and BiCoord with three execution backbones, LOGIV improves mean task success without injected disturbances by 10.67 and 15.33 percentage points, respectively. Under predefined perturbations, it achieves higher task completion and condition restoration than three adapted recovery baselines for every backbone on both benchmarks. On three real-robot tasks, successful trials increase from 12/30 to 17/30. These results support explicit, continuously checked task dependencies as a practical interface for reliable agentic robot execution.
comment: 14 pages. Updated related work, references, author list and affiliations
♻ ★ Compressed delayed-information projection for underwater navigation under acoustic measurement latency
Delayed acoustic position measurements describe past states but may reach an underwater vehicle after its navigation estimate has advanced. This paper presents compressed delayed-information projection (CDIP), a causal delayed-update method implemented in a 15-dimensional error-state Kalman filter (ESKF). CDIP retains source-associated inertial measurement unit (IMU) snapshots and maintains current-to-snapshot cross-covariances, projecting delayed corrections without repeated full-from-start filtering. In 154 paired simulation recordings with synthetic horizontal-position fixes delayed by 1.5~s, CDIP reduced mean trajectory-position root-mean-square error (RMSE) from 1.062~m for a reception-time Baseline to 0.456~m (57.1\%). Its mean RMSE approached that of full-from-start Replay (0.451~m), with 99.2\% lower mean core-loop time per processed IMU step for the tested implementations. A six-delay sweep, scheduled permanent-loss outages of 50--300~s, and packet-erasure tests further characterized sensitivity to measurement timing and availability. A separate 256-trial nonlinear synthetic assessment identified departures from the prespecified calibration criterion for CDIP and two fully augmented ESKF references. Together, these results demonstrate an accuracy--computation trade-off for causal underwater navigation under the evaluated delayed-measurement simulation conditions.
comment: 31 pages, 7 figures, 6 tables. Submitted to The Journal of the Acoustical Society of America (JASA)
♻ ★ Detect and Suppress: A Mechanistic Defense against Adversarial Patches in VLA Models
Adversarial patches can disrupt Vision-Language-Action (VLA) models by manipulating visual observations, leading to failures in robot control. However, it remains poorly understood which internal mechanisms underlie these failures and how targeted interventions can mitigate them. In this work, we mechanistically analyze VLA representations using a sparse autoencoder (SAE) and identify a feature whose activation strongly correlates with the presence of an adversarial patch. Based on this analysis, we suppress the identified feature at inference time only when a linear probe detects an attack. This intervention improves robustness without the cost of fine-tuning the VLA. We evaluate our method against VLA adversarial patch attacks on LIBERO-10. Conditional intervention improves success rate under intermittent attacks, whereas continuously applying the same intervention substantially degrades policy performance. These results show that attack-related internal representations can provide useful targets for VLA adversarial defense and that controlling when to intervene is important for limiting disruption to nominal policy behavior.
comment: The Science of Physical AI Safety Workshop at CoRL 2026
♻ ★ NDPP-Grasp: Non-Differentiable Physical Plausibility Constraint-Guided Task-Oriented Dexterous Grasp Generation NeurIPS 2026
Task-oriented dexterous grasp generation aims to produce dexterous grasp poses that are both physically plausible and functionally suitable for specified manipulation tasks. Existing diffusion-based methods often address these two requirements in a decoupled manner: they first train a grasp diffusion model for task alignment and then rely on post-generation refinement to improve physical plausibility. However, this after-the-fact correction strategy applies physical plausibility guidance only once the grasp has already been generated, leaving the generation trajectory itself unguided by physical constraints and potentially leading to suboptimal grasps. To address this problem, we propose a novel framework that directly injects physical plausibility guidance into the denoising process of a task-aligned grasp diffusion model in a practical and effective manner, even when physical plausibility constraints are non-differentiable. This allows physical plausibility to shape grasp generation throughout denoising while preserving task alignment. Extensive experiments demonstrate the efficacy of our framework.
comment: Accepted at NeurIPS 2026
♻ ★ Keeping the Franka Emika Panda alive: a ROS 2 stack with a reliable position interface
This paper presents an open-source software stack that restores ROS 2 support for the Franka Emika Panda robot while resolving the long-standing unreliability of its external position control interface. We first analyze the root causes of unstable position control and show that the observed vibrations and protective stops arise from the timing of the external control loop and sampling jitter, rather than from limitations of the robot itself. Building on this analysis, we introduce an asynchronous hardware interface that decouples real-time communication from the ROS 2 control loop, a rate-matching mechanism for slower command sources, and a position-domain reference generation strategy that produces reliable, smooth position commands. Experimental validation shows that the proposed architecture reliably tracks velocity references by reducing motion artifacts introduced by the official implementation, and the stack is validated across motion planning, compliance control, position-controlled manipulation, and haptic teleoperation on two independent Panda platforms. By restoring a modern, reliable, and open ROS 2 ecosystem for the Panda, this work lowers the barrier to developing safe, responsive, and reproducible human-robot collaboration applications that integrate planning, perception, interaction, and shared autonomy. Code and videos are available on our website at https://sites.google.com/unisa.it/fer-ros2.
comment: 12 pages, 10 figures
♻ ★ Adaptive-WAM: Quality-Guided Early-Exit Planning from Intermediate Video-Diffusion Features
Large video diffusion models provide rich spatiotemporal priors for autonomous driving, but existing world-action models often inherit the cost of iterative future-video generation even though deployment only requires an ego trajectory. We ask a more basic question: how much of a video diffusion model must be executed to make a reliable driving decision? Through a controlled study of video denoising timesteps and Diffusion Transformer (DiT) depth, we find that planning performance is largely insensitive to the tested video-noise levels, whereas strong trajectories can already be decoded from intermediate layers. Based on this observation, we introduce Adaptive-WAM, a quality-aware multi-exit planner built on a Wan2.2-5B backbone. Trajectory diffusion heads are attached to selected DiT blocks, and a lightweight trajectory-quality scorer terminates inference once the best trajectory decoded so far satisfies a quality threshold; otherwise, computation continues from the cached hidden state to a deeper exit. The deployed planner therefore avoids the iterative classifier-free denoising loop and VAE decoding required for future-video synthesis, while dynamically allocating backbone depth according to trajectory quality. On NAVSIM, the adaptive single-trajectory planner achieves 90.8 PDMS; a separate fixed-exit variant reaches 92.6 PDMS with 64 proposals. It further obtains 89.9 EPDMS on NAVSIM v2, yielding the best reported results among the compared front-view video world-model planners. Without target-domain fine-tuning, Adaptive-WAM transfers to nuScenes with 0.88 m average L2 error and a 0.08\% collision rate. On an A100, adaptive routing improves PDMS from 90.62 to 90.79 while averaging 170 ms end-to-end planning latency, approximately 10\% below the 190 ms fixed block-15 planner and 47\% below the 320 ms fixed full-depth planner. Code will be released.
♻ ★ Reflection-Based Relative Localization for Cooperative UAV Teams Using Active Markers
Reflections of active markers in the environment are a common source of ambiguity in onboard visual relative localization. This work presents a novel approach that exploits these typically unwanted reflections for onboard relative localization in heterogeneous multi-UAV teams operating over specular, near-planar surfaces such as calm water and flat indoor floors. Using only a camera, active markers, and a single-beam rangefinder, the method requires no prior knowledge of robot size, marker configuration, or surface reflectance properties, and propagates the uncertainty introduced by surface irregularities, including those of dynamic water surfaces. We further characterize, analytically and experimentally, how surface tilt biases the estimate, and identify the slope range over which the level-surface assumption holds. Indoor multi-UAV and outdoor experiments validate the localization accuracy, demonstrating reliable operation across varying lighting conditions and greater effective range than state-of-the-art vision-based active-marker methods, with higher accuracy beyond 30 m. The project website with the video is available at: https://timlakemann.github.io/reflection-based-relative-localization/.
♻ ★ SV-WAM: An Efficient Surround-View World-Action Model for End-to-End Autonomous Driving
World models (WMs) have demonstrated strong potential for end-to-end autonomous driving by learning predictive representations of future scene dynamics. However, generating future videos during inference introduces substantial computational overhead, leading many recent driving WMs to adopt a single front camera as input for efficient deployment. This design restricts spatial coverage in safety-critical maneuvers such as lane changes, merges, and turns. To address this limitation, we propose SV-WAM, a surround-view world-action model (WAM) that preserves full six-camera observations while maintaining efficient inference. SV-WAM leverages future-video prediction as dense training supervision for action learning within a shared generative model, rather than as an inference-time output. At the core of this design is an action-centered causal mask that prevents action tokens from attending to future-video tokens during joint action-video denoising. Consequently, the video branch can be discarded at deployment, enabling efficient action-only planning. Furthermore, we introduce a differentiable drivable-area compliance regularizer that penalizes vehicle-footprint corners approaching or crossing drivable boundaries, improving planning safety and boundary awareness. Extensive experiments on the closed-loop NAVSIMv2 benchmark and the open-loop nuScenes benchmark demonstrate that SV-WAM achieves state-of-the-art planning performance with low inference latency and competitive zero-shot transfer capability.
comment: 23 pages, 16 figures. Project page: https://jinyangwang427.github.io/SV-WAM/
♻ ★ Decompose and Reorganize: Planning with Primitives and Visuomotor Policies Learned from Demonstrations
Successfully automating dexterous, long-horizon bimanual manipulation requires frameworks capable of both high-level reasoning and fine-grained execution. Traditional task and motion planning (TAMP), while excellent at symbolic planning, is often brittle in contact-rich operations. Simultaneously, imitation learning (IL), while effective in manipulation tasks with visual feedback, is limited by its low capability in spatial generalization and multi-stage operation. To reconcile their complementary strengths and limitations, we propose DR-LfD (Decomposed and Reorganized Skills Learned from Demonstrations), a framework that integrates visuomotor policies into a TAMP-gated decision-making system. Based on contact relationships, DR-LfD decomposes human demonstrations into atomic skills, which are reproduced as visuomotor policies or object-centric primitives. The initiation, termination, and constraints of the visuomotor policies are carefully modeled and implemented in a TAMP-compatible form, enabling reorganization of skills learned from different sources. DR-LfD transforms the learning problem from one requiring exponential demonstration data over possible skill sequences to one whose demonstration burden scales with the number of distinct skill types, with limited data for each skill. Real-world and simulation benchmarks demonstrate data-efficient skill composition under unseen setups and physical constraints. On two DexMimicGen tasks, DR-LfD with 100 demonstrations outperforms diffusion-based imitation policies with 1000.
comment: Accepted by T-RO. Website: https://dr-lfd.github.io/DR-LfD-website
♻ ★ WildOS: Open-Vocabulary Object Search in the Wild
Autonomous navigation in complex, unstructured outdoor environments requires robots to operate over long ranges without prior maps and limited depth sensing. In such settings, relying solely on geometric frontiers for exploration is often insufficient. In such settings, the ability to reason semantically about where to go and what is safe to traverse is crucial for robust, efficient exploration. This work presents WildOS, a unified system for long-range, open-vocabulary object search that combines safe geometric exploration with semantic visual reasoning. WildOS builds a sparse navigation graph to maintain spatial memory, while utilizing a foundation-model-based vision module, ExploRFM, to score frontier nodes of the graph. ExploRFM simultaneously predicts traversability, visual frontiers, and object similarity in image space, enabling real-time, onboard semantic navigation tasks. The resulting vision-scored graph enables the robot to explore semantically meaningful directions while ensuring geometric safety. Furthermore, we introduce a particle-filter-based method for coarse localization of the open-vocabulary target query, that estimates candidate goal positions beyond the robot's immediate depth horizon, enabling effective planning toward distant goals. Extensive closed-loop field experiments across diverse off-road and urban terrains demonstrate that WildOS enables robust navigation, significantly outperforming purely geometric and purely vision-based baselines in both efficiency and autonomy. Our results highlight the potential of vision foundation models to drive open-world robotic behaviors that are both semantically informed and geometrically grounded. Project Page: https://leggedrobotics.github.io/wildos/
comment: Published at IEEE Transactions on Field Robotics
♻ ★ Seeing and Imagining as One: Unified Video Generation and 4D Perception for Embodied World Modeling
Embodied world models should not only imagine plausible futures, but also perceive the geometric, semantic, and motion structure of the evolving world. We present GenSight, a unified embodied world model that brings future video generation and dense perception into a single pretrained video diffusion transformer. GenSight uses a unified spatiotemporal representation for video, depth, embodiment segmentation, and first-frame-relative 2D flow. Built on a shared DiT backbone, GenSight uses lightweight task modulation to predict all modalities within the same architecture. All tasks are jointly optimized within a unified velocity prediction space. At inference, GenSight supports two complementary modes: multi-step future video generation with aligned dense predictions and single-step dense perception from an observed video. By unifying modality representation, model architecture, and prediction space, GenSight avoids separate modality-specific pipelines and couples appearance, geometry, semantics, and motion in a shared representation. It promotes cross-modal consistency and allows dense perception to benefit from the spatiotemporal priors learned through video generation. Trained on embodied manipulation data spanning multiple robot embodiments and human-hand interactions, GenSight demonstrates strong performance in both video generation and dense 4D perception, highlighting the effectiveness of unifying seeing and imagining for embodied world modeling.
comment: Project page: https://gensight-rbs.github.io/
♻ ★ MUSBRIC: LEGO-Compatible Pneumatic Muscle Bricks for Reconfigurable Musculoskeletal Robots
Musculoskeletal robots combine compliant actuators with articulated skeletons, but exploring alternative configurations often requires repeated fabrication. We present MUSBRIC, a modular hardware platform and accompanying MuJoCo-based simulator for reconfigurable musculoskeletal robots. LEGO-compatible endcaps turn fiber-reinforced extensile pneumatic artificial muscles into reusable, plug-and-play muscle bricks that connect directly to standard LEGO components. Experimentally identified actuator behavior supports corresponding musculoskeletal simulations. An elbow-joint prototype validates load-dependent flexion and extension, showing that pulley-guided routing enables reversible motion and a mechanical stop stabilizes the straight posture. A robotic hand demonstrates compliant grasping and multi-joint construction. Matched hardware and simulation sweeps across 15 leg configurations demonstrate systematic exploration of attachment geometry, while a quadruped assembled from four legs demonstrates walking. Together, these demonstrations span mechanism analysis, configuration exploration, and functional robot construction within a single reconfigurable system. The hardware designs and simulation code will be released as open source soon to facilitate reproduction and further development.
comment: 6 figures. Qiyao Wang and Yinglei Zhu contributed equally to this work. Improved HTML rendering of Figure 1; scientific content unchanged
♻ ★ TacClaw-Infinity: A Large-Scale Vision-Tactile-Language-Action Dataset for Dexterous Manipulation
Large-scale robot learning increasingly relies on diverse human demonstrations, yet tactile manipulation data remain difficult to scale, and existing multimodal datasets often lack consistent quality control and structured supervision. We present TacClaw-Infinity, a large-scale structured Vision-Tactile-Language-Action (VTLA) dataset for contact-rich manipulation, collected with TacClaw, a tactile-enabled UMI-style handheld interface that records synchronized visual observations, fingertip tactile signals, and end-effector trajectories without requiring a target robot during data collection. TacClaw-Infinity is built through a distributed worldwide collection across homes, laboratories, industrial workplaces, retail stores, and outdoor environments including tasks contributed by academic and industrial collaborators. Together, these efforts yield over 10,000 hours, 1 million episodes, and 1 billion frames spanning more than 1,000 task types. We develop a scalable pipeline for trajectory reconstruction, multimodal quality control, embodiment-aware filtering, and structured annotation, covering contact-centric segmentation, subtask description, and spatial grounding. We validate TacClaw-Infinity at both the data and model levels: At the data level, we assess the collection interface, the action-space compatibility of the curated demonstrations, and the structured annotations, while for the model level, we demonstrate cross-embodiment transfer across three heterogeneous robots without robot-specific demonstrations.
comment: Updated project website and code repository links; research content unchanged. Project: https://tacclaw-infinity.github.io/ ; Code: https://github.com/tacclaw-infinity/TacClaw-Infinity
♻ ★ Learning and Transferring Closed-Loop Robot Software
Closed-loop robot policies require observation processing, state management, and situation-dependent branching, making them costly to design and tune manually. Although coding agents increasingly support control-code generation and optimization, it remains unclear whether implementations improved on source tasks also support policy acquisition for new tasks. We study this question by treating complete closed-loop implementations as reusable execution experience. To learn each source policy, a coding agent generates policy code from a few successful demonstrations and iteratively improves it using simulation feedback. The validation-selected implementations are retained in a software archive. To transfer the acquired software to new tasks, the agent generates and improves policies using archived implementations, target demonstrations, and execution feedback. We evaluate policy learning and its relationship to cross-task transfer. For learning, across four source tasks in RoboCasa, iterative optimization increases the success rate from 28.3% with initially generated policy code to 64.2% with validation-selected policy code. We further validate this learning approach on a physical Unitree G1. For transfer, across nine additional target tasks in RoboCasa, success rates are 45.2% without references, 41.5% using initially generated policy code from source tasks, and 57.0% using their validation-selected policy code. These results show that improving source-task code also improves learning on new tasks.
♻ ★ OpenFlyV2: An onboard vision-language-action model for aerial perception and navigation
Human cognition flexibly draws on a shared perceptual foundation to understand environments, attend to referenced entities, reason about spatial relationships, and act. Endowing aerial robots with an analogous functional organization remains challenging, as existing systems are often specialized for individual capabilities, dependent on offboard computation, or unable to sustain closed-loop operation on resource-constrained platforms. We present OpenFlyV2, a unified onboard vision-language-action architecture that uses a shared vision-language representation to support semantic understanding, language-grounded perception, spatial reasoning, and continuous aerial control within a single model. A compact action expert converts the shared multimodal context into short-horizon flight waypoints, while motion-aware temporal memory and action priors reduce the computational cost of closed-loop execution. We construct \dataset{}, a simulation-based dataset comprising approximately 70,000 navigation trajectories and 60,000 vision-language instruction--response pairs. OpenFlyV2 generates action chunks, each comprising 12 waypoints, entirely onboard at 10 Hz using an NVIDIA Jetson Orin NX. It transfers zero-shot from simulation to physical quadrotor flight, achieving a 48\% navigation success rate across four real-world environments without real-world aerial training data. Extensive results demonstrate that broad vision-language competence and continuous closed-loop flight can coexist within a single model on a resource-constrained aerial platform. Code will be made publicly available at https://github.com/Ideal-ljl/AeroBrain.
comment: 12page, 6figures
♻ ★ Learning from Fleet Experience: Collective Experience Learning Across 96 Robots
Robot deployment generates experience that can support policy improvement, but continued reliance on expert corrective actions makes learning difficult to scale across growing fleets. We propose learning from fleet experience: using deployed robots' own executions as a source of supervision for further policy improvement. We call this paradigm Collective Experience Learning (CEL), in which robots pool execution experience to update a shared policy that returns to the fleet. Following expert-supervised initialization, we realize CEL through failure-guided experience retrieval fine-tuning (FERT). FERT uses failure contexts to retrieve relevant segments from successful trajectories and prioritizes their recorded actions for supervised updates, without collecting additional expert demonstrations or corrective actions. Operators provide binary episode-outcome labels and environment resets. A fault-tolerant asynchronous training system (FATS) coordinates robot execution, cloud inference and policy optimization despite local interruptions. Across five real-world manipulation tasks, the resulting policies achieve success rates of 93.3-99.2%, evaluated over 120 trials per task on robots excluded from training data collection. With collection time per robot held fixed, larger fleets provide more experience and generally improve performance across the evaluated sizes. On multi-stage phone packaging, learning with 96 robots raises success from 79.2% to 98.3%. These results support CEL as a scalable approach to improving robot policy reliability through experience generated during deployment.
♻ ★ Actuator-Aware Observer Inputs for Cascade ADRC of Quadrotors Under Rotor Lag, Measurement Delay and Sensor Noise
In sampled cascade linear active disturbance rejection control (LADRC) of quadrotors, the extended-state observer typically receives the instantaneous command, although the delayed measurement reflects an earlier, rotor-filtered acceleration. This paper proposes actuator-aware (AA) observer inputs: altitude and attitude observers use limited virtual inputs filtered by an assumed rotor model and aligned with the measurement delay, and horizontal observers use acceleration reconstructed from measured attitude. Fixed gains are selected on the discrete-time model under sensitivity and step-response requirements and stationary covariance ceilings on each channel's control variance and increment and on the noise-induced inner-loop increment of the full cascade. In paired set-point trials, PID-Budget, AA-Budget and AA-HalfGain reach mean RMSEs of 65.5, 19.2 and 16.9~mm; under a stricter common cascade ceiling, the AA designs reach 29.9 and 26.4~mm versus 72.3~mm for a PID that uses roughly double their rotor-command increment. PX4 software-in-the-loop runs reproduce these results. Behind EKF2, a noise model fitted to the estimation errors predicts the roll/pitch and yaw increments within 5\% on held-out and out-of-sample runs and the total rotor-command increment within 14\%. The controllers were then redesigned for this stack under the same sensitivity and response specification and estimator-aware ceilings. AA-Budget-E reduces the set-point RMSE from 59.1~mm (PID-E) to 44.0~mm with a lower rotor-command increment in every paired set-point and lemniscate run (mean S6 reduction: 45\% before and 15\% after output quantization); AA-HalfGain-E beats PID-E only on the lemniscate. These results indicate that the observer input and the outer-to-inner noise path are design variables of sampled cascade LADRC. The evidence is simulation-based, and the ceilings are nominal tuning constraints.
comment: 19 pages. MATLAB and PX4 SITL evaluation. Code, controller gains, random seeds, and per-seed results: https://github.com/Quang61123/actuator-aware-cascade-adrc
♻ ★ SimRetract: Zero-Shot Sim-to-Real Robotic Assistance for Retractor Take-Over in Open Surgery
We demonstrate the first zero-shot sim-to-real transfer for a surgeon-robot collaboration task in open surgery: retractor take-over. A robot arm takes a retractor from the surgeon's hand and preserves retraction at handoff, using a policy trained only on simulated demonstrations, without human demonstrations or real-world fine-tuning. Imitation learning from real demonstrations is limited by collection and labeling costs, and by performance degradation when the environment changes. Our framework, SimRetract, generates simulated demonstrations designed for transfer: the policy observes masked instrument images and grasp geometry, learns to recover from off-nominal states, and acts with margin for execution drift. Deployed zero-shot, it grasps the retractor in 90% of 50 surgeon-defined phantom presentations and preserves retraction at handoff in 78%. Results on unseen retractors suggest potential for generalization to instruments with similar grasp geometry. Across ten policy configurations and 1,200 simulated and real trials, we localize the sim-to-real gap to components: in simulation, grasp rates across ablations span 80-96%, while on the real robot removing the instrument mask alone drops performance to 22% grasped and 12% held. Transfer-oriented design thus makes zero-shot sim-to-real transfer feasible for a surgical assistance task without human demonstrations. Code is available here: https://anonymous.4open.science/r/simretract-58D2.
♻ ★ Dispersive Tree Agreements for Multi-Robot Coordination
Multi-robot coordination of nonlinear systems is naturally formulated as joint-space optimal control subject to joint constraints. Existing methods that approach joint optimality remain largely centralized and offline, while decentralized, real-time counterparts target safe navigation without pursuing the joint optimum. This paper develops a decentralized, real-time planner that approaches the joint optimum as an anytime process. We extend a recent single-robot framework that searches dispersive trees of forward-propagated commands to coordinate a multi-robot team near-optimally. We first show that the inter-robot coupling can be effectively replaced by some local constraints, under which locally grown dispersive trees contain a near-optimal agreement. It is this structure that the robots pursue by a negotiation driven by decentralized column generation. In the quiescent limit, this negotiation is shown to settle on a near-optimal agreement. Our real-time protocol, WarpWeaver, runs the negotiation asynchronously over a receding horizon and closes on quiesced agreements even when deadlines cut it short. In simulation, WarpWeaver reduces total travel time by up to 22% against centralized offline planners on the Dynobench suite and by up to 16% against a decentralized real-time baseline, and outperforms a task-specific planner on networked navigation. Hardware experiments on six robots planning onboard demonstrate real-world implementability.
comment: 18 pages, 5 figures, 7 tables
♻ ★ MOON: Learning When to Forget and How to Move On in Vision-and-Language Navigation
Recent vision-and-language navigation (VLN) policies act on their visual history, and an incorrect action can take them off the instructed route. Paired interventions on four VLN policies show that clearing the history doubles pooled success at the onset of a deviation but lowers it on route, while later in a deviation, removing only the off-route frames helps much less. After a deviation, the retained history keeps the policy from turning where a turn is needed, which we call behavioral inertia. We propose MOON, a framework that equips a frozen VLN policy with two lightweight modules, adding less than 1% of its parameters. A deviation judge decides when to forget, raising an alarm that clears the history when it predicts that the agent is off route. A recovery policy, trained with GRPO from alarm states, learns how to move on, turning more of the successes found only by sampling into a single greedy rollout. Experiments on R2R-CE and RxR-CE show that MOON reaches the highest R2R-CE success rate among the compared methods and raises the success rate of its frozen base by 5.2 and 6.0 points, and an ablation with a random trigger indicates that this gain depends on when the history is cleared. Without retraining, MOON also improves zero-shot object-goal navigation on HM3D-OVON. Together, these results suggest that deviation recovery depends on knowing when to forget, not only how to act. Code is available at https://github.com/AutoLab-SAI-SJTU/MOON.
♻ ★ EquiContact: A Hierarchical SE(3) Vision-to-Force Equivariant Framework for Spatially Generalizable Contact-rich Tasks
This paper presents a framework for learning vision-based robotic policies for contact-rich manipulation tasks, which generalize spatially across task configurations. We propose EquiContact, a novel hierarchical framework composed of a high-level vision planner, such as a Diffusion Equivariant Descriptor Field (Diff-EDF) and a low-level compliant visuomotor policy, such as a Geometric Compliant Action Chunking Transformer (G-CompACT). G-CompACT, operating using localized observations from wrist-mounted RGB images, end-effector pose feedback in the form of a geometrically consistent error vector (GCEV) and force-torque readings, produces actions defined on the end-effector frame. We show that the entire EquiContact pipeline is $\SE$-equivariant under equivariant reference-frame estimation and left-invariant visual features, from perception to force control. Once the robot learns to execute a contact-rich task, such as inserting a peg-in-hole (PiH), it can execute similar tasks at different locations and orientations without retraining of the low-level policy. We also describe three key components for attaining spatially generalizable contact-rich policies: geometrically consistent compliance, the use of localized policies, and induced equivariance. Real-world peg-in-hole (PiH) experiments demonstrate near-perfect success rates under spatially out-of-distribution configurations, validating the proposed framework and principles, while additional screwing and surface-wiping experiments evaluate the transfer of the localized policy design beyond insertion. The experimental videos and more details can be found on the project website: https://equicontact.github.io/EquiContact-website/
comment: Preparing for submission again
Multiagent Systems
★ Strategic Tanking in Stable Matching Markets
Motivated by a broad literature in psychology, organizational behavior, and matching markets, we introduce strategic tanking: an agent deliberately makes themselves less attractive to selected potential partners in order to improve their eventual assignment. Unlike traditional manipulation in stable matching, a tanker does not misreport whom they prefer; instead, they influence how others rank them. We show that tanking has a striking connection to classical preference manipulation. Cutoff tanking, in which an agent tanks with every potential partner below a chosen threshold, has exactly the same effect on the stable set as truncating one's own preference list at that threshold. Thus an agent can reproduce the power of a classical truncation while leaving their own preferences unchanged and every match acceptable. Arbitrary tanking, however, can go substantially further: under proposer-deferred acceptance, tanking equilibria can lie outside the sincere stable set. More broadly, whenever the sincere stable set contains multiple matchings, profitable strategic behavior remains available through either traditional preference manipulation or tanking, even when one side is restricted to honestly reporting their own preferences.
★ Adaptive Coverage Algorithms for Mobile Ad Hoc Networks in Dynamic Worlds
This paper extends an existing framework for the coordination of mobile agents in unknown worlds, by considering the possibility of dynamic instability. In the original problem, known as the Partially Observable Cooperative Guard Art Gallery Problem (POCGAGP) (arXiv:2512.09495), the multi-agent system must establish a communication network which completely covers an initially unknown constrained space. However, the POCGAGP fails to account for the unpredictability often seen in real-world deployment. To address this, the dynamic case (DPOCGAGP) introduces two settings which can influence the mobile network during deployment: Agent Loss and World Change, which represent dynamic changes to the world. Within this study, we present advancements to two state-of-the-art algorithms, the Centralized Agent-Directed Exploration for Network Coverage (CADENCE), and its decentralized analogue, DADENCE, to provide solutions in these cases. We then compare their performance against another state-of-the-art benchmark algorithm, and show that the adapted algorithms outperform across our benchmark test set, which comprises over 4500 test cases of varied size and structure.
comment: 8 pages, 10 figures. Accepted to the 2026 IEEE MIT Undergraduate Research Technology Conference (URTC)
★ AI evaluators favor job applications written by certain AI models, independent of candidate quality
Job applicants increasingly use large language models (LLMs) to write cover letters, while employers deploy LLMs to screen them. In this all-AI setting, we ask whether the \emph{choice of model} affects hiring outcomes independently of candidate qualifications. We introduce a multi-model auditing framework pairing eleven LLM writer agents with nine API-hosted evaluator agents across ten job descriptions and fifty candidates per job. Four findings emerge. First, we identify a generative preference hierarchy: a consistent global ranking of writer models by evaluator acceptance, in which locally hosted models are systematically disadvantaged regardless of their license, compounded by self-preference bias where evaluators favour their own outputs. Second, evaluator models differ substantially in how much bias they introduce based on applicants' tool choice, offering actionable guidance for fairer screening design. Third, this hierarchy has measurable competitive consequences: in the most adverse pairing, approximately two (1.8) out of twenty-five qualified candidates are filtered out purely due to model mismatch. Fourth, a common mechanism explains the patterns above: writer preference and self-preference trace to a combination of how much each writer leans on specific stylistic cues and how strongly evaluators, often unanimously, reward or penalise those same cues. A further check finds little evidence that an unmeasured "writer fingerprint" remains once these same features are accounted for. We discuss implications for algorithmic fairness in employment, regulation of AI-assisted screening, and the design of auditing frameworks that can be reapplied as the model landscape evolves.
★ ORION: An Outcome-Driven LLM Advisor for RL-Based Multi-Domain 6G Slice Placement
Deploying 6G network slices across administrative domains requires an online decision for each arriving request: which domain hosts each virtual network function, how traffic crosses domain boundaries, and whether the slice is admitted. This joint decision problem is NP-hard under continuous arrivals. Experiments on a parametrized family of multi-domain topologies show that no fixed planning strategy dominates across all regimes: the performance ranking reverses between topology classes, and feasible placements exist for most rejected requests but remain undiscovered. These findings identify the plan layer as the decisive performance lever. We introduce ORION, a hierarchical orchestrator in which two LLM agents translate service intents into structured requests and generate abstract placement plans. A cooperative multi-agent RL coordinator selects the cross-domain partition, treating the plan as a persistent soft prior, while per-domain agents place the functions. ORION main contribution lies at the plan layer: an outcome-driven episodic memory that records, for each completed attempt, the network-condition profile, the proposed plan, and its verified feasibility outcome. The memory feeds verified successes and labelled failures back to the planner, thereby closing the adaptation loop using deployment feasibility rather than a scalar reward assigned to the planner's own proposals. To isolate the effect of the plan quality, we evaluate the planner on held-out topology families while keeping the downstream selection and execution mechanisms fixed. Under this controlled setting, the proposed memory improves admission by $4.6$--$11.5$ percentage points over the same planner without memory, while importance-based retention accounts for most of the improvement over a first-in-first-out store.
comment: Accepted at IEEE MobiWac 2026
★ Distributed Grid Search for Hyperparameter Tuning
The performance of many distributed optimization algorithms over peer-to-peer networks strongly depends on the choice of their hyperparameters. However, existing selection rules often require global quantities, yield conservative choices, or are tied to specific algorithms and hyperparameters. We propose Distributed Grid Search (D-GS), a consensus-based search method that evaluates and selects hyperparameter configurations across the network, requiring only the ability to run the tuned algorithm and evaluate local gradients. We also propose a zoom-in variant, Distributed Zoom Grid Search (D-ZGS), which improves empirical performance at a reduced cost. We prove that both methods terminate in finite time with all agents holding the same hyperparameter configuration, and quantify the (sub)optimality of this configuration. Numerical experiments on linear and logistic regression show that D-GS closely matches a centralized reference and that tuning and deployment cost a fraction of deploying algorithms with hyperparameters tuned using theoretical results.
★ Understanding LLM-Based Bargaining of Driving Agents in Mixed-Autonomy Traffic
Large Language Models (LLMs) offer a flexible computational approach for modeling social interactions among heterogeneous agents. This paper develops a novel framework towards characterizing and understanding LLM-based population-level bargaining of shared road space between autonomous vehicle (AV) and human-driven vehicle (HV) agents in mixed-autonomy traffic. Through controlled experiments varying traffic density, agent objectives, and traffic-domain knowledge, we first characterize bargaining outcomes and observe heterogeneous agreement rates across different experimental conditions. To understand how the bargaining process shapes the outcomes, we introduce a behavioral taxonomy that jointly characterizes numerical bidding actions and communicative attitudes. We examine two complementary aspects of social interaction: inter-agent bargaining dynamics and private-to-public behavioral transformation. Results reveal distinct interaction patterns associated with agreement and breakdown, including greater constructive reciprocity in agreement trials and more frequent conflict escalation in breakdown trials. Furthermore, private and public communicative attitudes often differ even when numerical bids remain unchanged. These findings illustrate how LLM-based bargaining simulations can support a structured analysis of social interaction behaviors in mixed-autonomy traffic.
★ When Disclosure Is Compulsory: Evaluating Ride-Hailing Rebalancing Policies with Self-Interested LLM Driver Agents
The EU Platform Work Directive, to be transposed by December 2026, requires ride-hailing platforms to state in plain language the main parameters of their automated decisions and how a worker's behaviour affects them. Such rules are exactly what a large language model (LLM) can read, so drivers will be able to let one decide for them. We compare two main families of rebalancing policy once their rules are public: a reactive policy (R) that sends idle drivers to cover requests just lost, and an anticipatory policy (A) that plans moves on a demand forecast. Each driver is an LLM agent maximizing its own net income, given no rules (N) or the rules at the directive's level of detail (L). We test two LLMs, Claude Haiku 5.5 and Jev, which understand the rules equally well, in a three-zone model and in Manhattan. With the rules disclosed, drivers follow the platform more closely: in Manhattan, the share of relocation offers they decline falls from about four in five to one in four under both policies. Yet the effects differ by policy. For the platform, disclosure costs R 3.7% of its profit in Manhattan, as R now pays for moves drivers used to make for free, while A's profit is unchanged; the two policies, which tie on profit under full compliance, end 6.1% apart. For drivers, disclosure raises income in the small model under both policies and LLMs; for users, it raises the service rate there by 10 to 18 percentage points. In Manhattan, self-interested drivers also make A a win-win-win against full compliance, raising driver income, platform profit and service rate by 10 to 12% each. Rebalancing policies that perform equally well under full compliance can thus show opposite results under mandatory disclosure. The study offers a framework to evaluate such policies under the new requirement and is, to our knowledge, the first to do so with LLM agents.
★ Decentralized Multi-Agent Coordination and Communication in Shared Codebases
Coding agents working in shared repositories may receive separate, independently-motivated requests from human developers, creating a setting that is multi-agent by necessity. When this scenario happens, their individual implementations may work separately but conflict when combined. To study this setting, we introduce ParallACT, a benchmark of orchestrator-free multi-agent coding tasks. ParallACT contains both complementary and contradictory tasks. In complementary tasks, each agent's task is separately useful and implementations must share an interface to be productively combined. In contradictory pairs, both agents' tasks require editing the same code in incompatible ways, and naively combining independent solutions fails to produce a useful implementation. We compare a single-agent oracle given the combined set of tasks against multi-agent settings that vary execution order, visibility, awareness of other agents, and access to a communication channel. We find that the oracle solves most tasks while parallel agents perform substantially worse. Their performance drop is notable for complementary tasks, and particularly sharp for contradictory tasks (e.g., from 62% to 16% for Claude Code with Claude Opus 4.8) where merged code typically satisfies only one specification, and the same gap appears in all six further configurations pairing two agent harnesses with three open-weight models. Simply telling an agent that a peer exists does not help significantly, and giving agents access to a communication channel recovers only a small part of the performance, at an additional token cost. ParallACT thus identifies and isolates a core challenge that will become more common as multi-agent coding by independent principals becomes standard, and provides a controlled test of which coordination strategies help, and at what cost.
comment: First two authors contributed equally. Project Page: https://lingyunxiao18.github.io/ParallACT/
♻ ★ Self-Building Labs
Self-driving laboratories (SDLs) accelerate scientific discovery by automating experimental execution and optimization. AI agents can increasingly propose new experiments, but existing SDLs can only run the procedures they were built for. Adapting them to a new experiment still requires extensive manual engineering, which makes laboratory flexibility the bottleneck for scaling discovery. We introduce the self-building laboratory (SBL), which designs, configures, and operates its own laboratory setup given a high-level scientific goal. SBL starts from whatever already exists, from an empty room to a working SDL, and builds the missing capabilities: the lab layout, its digital twin, robot skills, and experimental workflows. We demonstrate the system working across different robots and setups, scaling from a single workstation to laboratories with several robots working together. We validate SBL's end-to-end capabilities through a real-world quantum-dot synthesis experiment. By shifting human researchers from manually configuring and programming device automation to specifying scientific objectives, SBL makes scientific automation more accessible, flexible, and scalable. Project website: https://self-building-labs.github.io/
♻ ★ A Cross-Layered Multi-Drone Coordination for Medical Supply Delivery during Disaster Response Management
Autonomous drone fleets have potential in medical supply delivery during disaster incident response. However, coordinating multiple drones in such settings introduces compounding challenges: dynamic hazards such as wind, obstacles, and intermittent connectivity, constrained energy budgets, and the need to serve patients fairly under strict deadlines and triage-based priority. We present CEDA, a Centralized Training with Decentralized Execution (CTDE) Double Deep Q-Network algorithm augmented with a QMIX-style monotonic mixing network for cooperative multi-drone medical delivery, designed to jointly optimize triage-aware routing, multi-agent coordination, and energy-efficient navigation under dynamic uncertainty. CEDA introduces a Priority-Preserving Fair Scheduling strategy, in which a structured reward function encodes triage priority weights, outcome-level fairness and priority-ordering rewards, and complementary shaping mechanisms that discourage systematic starvation. We evaluate CEDA in a simulated grid environment with five drones serving up to fifty patients, featuring dynamic hazard zones, stochastic action failures, and dynamically spawning patients across three triage levels, and in PX4 software-in-the-loop (SITL) validation using five X500 quadrotors controlled via MAVSDK. Simulation results show that CEDA delivers over 95\% of spawned patients, reduces obstacle collisions by over 95\% across training, and achieves a weighted triage efficiency exceeding 0.90 alongside a Jain's Fairness Index of approximately 1.0 across triage classes. CEDA preserves clinical priority ordering, with Critical patients receiving the fastest response times, while achieving low and comparable mortality across all triage classes. PX4 SITL validation further shows that the learned policy retains its triage-aware behavior under continuous simulated flight and delayed-action conditions.
comment: 18 pages, 7 figures, 7 tables
♻ ★ Concordat: Typed Sessions for LLM Agents
Agents built on large language models are increasingly allowed to take consequential actions: cancelling bookings, issuing refunds, changing accounts. The rules that govern those actions, such as "obtain the customer's explicit confirmation before changing anything", are usually stated in the prompt and followed only as reliably as the model chooses to follow them. We present Concordat, a Python library that treats an agent as an object whose interface is a finite-state protocol, in the spirit of typestate and session types, and enforces that protocol on the model at run time. On every turn the model is offered only the tools that are legal in the current protocol state, calls outside that set are rejected before they have side effects, decisions reserved for humans can only be requested, and token and tool-call budgets are charged and split linearly across handoffs. Protocols are checked when they are defined, and a subclass of an agent is rejected if it could bypass its parent's protocol, a behavioural form of Liskov substitution. We evaluate a Concordat agent against the standard agent of tau-bench on its 50 airline tasks, adding simulated customers who pressure the agent to skip confirmation. The standard agent made changes without the required "yes" in 13 of 50 tasks with gpt-4.1 and in 29 of 128 task-runs with Claude Haiku 5.5; the Concordat agent made none, with no significant difference in task success. tau-bench's own reward did not detect most of these violations. Concordat is open source.
comment: v2: corrected the budget description; fixes in concordat 0.0.4 11 pages, 1 figure, 1 table. Code, benchmark harness and results: https://github.com/miladsikaroudi/concordat
♻ ★ Denoising Multirobot Trajectories
Multi-robot trajectory planning is a fundamental problem in multi-robot coordination but remains computationally challenging due to its nonconvex, multimodal, and high-dimensional nature. This work builds upon D4orm, a dynamics-aware diffusion-denoising framework, and develops a family of planning architectures for diverse operational requirements. Unlike conventional numerical optimization methods, D4orm employs sampling-based optimization to generate solution trajectories through massively parallel sampling, leveraging modern computing architectures such as GPUs. Its diffusion-denoising structure iteratively optimizes deformations to candidate control trajectories, providing an efficient and versatile paradigm for generating kinodynamically feasible and conflict-free trajectories. Using D4orm as the building block for advanced planners, we present a decoupled planner for improved scalability, an online receding-horizon planner with feedback control, and a distributed planner for resource-constrained settings. Evaluations with differential-drive and holonomic robots in 2D and 3D environments demonstrate that D4orm-based approaches find high-quality solutions faster and more reliably than other sampling-based optimization methods, such as MPPI, as well as a learned diffusion-model-based method. We further demonstrate zero-shot deployment on ten real quadrotors with obstacles, large-scale deconfliction with 100 simulated robots, and fully onboard distributed `lifelong' operation with six ground robots. Overall, these results establish diffusion denoising as a scalable and reliable framework for multi-robot coordination. Code and video: https://github.com/proroklab/d4orm
comment: Accepted to IEEE Transactions on Robotics (T-RO) 2026
♻ ★ Anchor and Perturb: Lazy Agent Remediation by Exploration Injection
Anchor and Perturb (AnP) is a lightweight framework that resolves multi-agent coordination failures by decoupling exploratory variance injection from recurrent manifold stability. Existing remediation strategies predominantly alter mixing network architectures or enforce simultaneous exploration across the collective, which inevitably precipitates severe temporal-difference penalties in non-monotonic reward spaces. Specifically, AnP isolates underperforming lazy agents and injects an asymmetric exploratory pulse into targeted coordinates whilst anchoring converged teammates to nominal greedy exploitation. Empirical telemetry benchmarks demonstrate that AnP successfully rescues collapsed joint policies (recovering from a 5% evaluation win rate nadir back to 85%) and facilitates escape from suboptimal coordination plateaus, sustaining peak win rates of 90% without requiring structural network modifications.
comment: 12 pages, 4 figures; v2 includes multi-cycle automated benchmarks and Section 5.3 additions
♻ ★ Dispersive Tree Agreements for Multi-Robot Coordination
Multi-robot coordination of nonlinear systems is naturally formulated as joint-space optimal control subject to joint constraints. Existing methods that approach joint optimality remain largely centralized and offline, while decentralized, real-time counterparts target safe navigation without pursuing the joint optimum. This paper develops a decentralized, real-time planner that approaches the joint optimum as an anytime process. We extend a recent single-robot framework that searches dispersive trees of forward-propagated commands to coordinate a multi-robot team near-optimally. We first show that the inter-robot coupling can be effectively replaced by some local constraints, under which locally grown dispersive trees contain a near-optimal agreement. It is this structure that the robots pursue by a negotiation driven by decentralized column generation. In the quiescent limit, this negotiation is shown to settle on a near-optimal agreement. Our real-time protocol, WarpWeaver, runs the negotiation asynchronously over a receding horizon and closes on quiesced agreements even when deadlines cut it short. In simulation, WarpWeaver reduces total travel time by up to 22% against centralized offline planners on the Dynobench suite and by up to 16% against a decentralized real-time baseline, and outperforms a task-specific planner on networked navigation. Hardware experiments on six robots planning onboard demonstrate real-world implementability.
comment: 18 pages, 5 figures, 7 tables
♻ ★ QueenBee: A Recursively Self-Improving Harness for Communication Topology Design in LLM-Based Multi-Agent Systems
LLM-based multi-agent systems show great potential for complex tasks through role specialization and collaborative reasoning. However, existing automated workflow design methods primarily focus on one-off search and candidate optimization, lacking sustained accumulation and reuse of structural design experience to meet evolving coordination demands. We propose QueenBee, an experience-guided framework for multi-agent workflow evolution. QueenBee represents workflows as executable programs that generate temporal communication DAGs and maintains Structural Feedback and Transition Bank (SFT-Bank), a persistent structural memory of execution diagnoses, design hypotheses and validation outcomes, and historical programs. This accumulated experience guides subsequent workflow generation, validation, and optimization, enabling the co-evolution of collaboration programs and design experience. Experiments on Silo-Bench show that QueenBee outperforms fixed communication baselines across team sizes and achieves the highest mean task score among recent automated multi-agent workflow optimization methods. Ablation studies further show that SFT-Bank components provide complementary performance gains, while evolution dynamics reveal a consistently improving average trend in the quality of the best collaboration structures found across independent runs as structural experience accumulates.
♻ ★ El Agente Quntur: Agentic Active-Space Selection for CASSCF
Multireference calculations require time-consuming expert decisions about active spaces, state identity, and convergence. This study asks whether agents can make these decisions reliably. Using the El Agente Quntur multi-agent harness with GPT-5.6 Sol and GPT-5.6 Terra as its language models, agents study 32 fixed-geometry singlet excitations, detect missing active orbitals, compare excited states, and revise state assignments as calculations progress. Prompts specify equal-weight two-singlet state-averaged complete active space self-consistent field (CASSCF) calculations followed by strongly contracted second-order N-electron valence state perturbation theory (NEVPT2) with def2-TZVPD; agents choose and refine the active spaces. Excitation energies are evaluated against published complete active space second-order perturbation theory (CASPT2) reference values at matching geometries. Agentic execution reduces mean absolute error by 48\% relative to six deterministic Active Space Finder protocols. Computational traces vary substantially across runs in selection strategies and cost, even for the same molecule. These results underscore the advantage of agentic active space selection over fixed workflows, yet leave room to improve efficiency through runtime-aware decision-making, reduced inter-agent communication overhead and redundancy, more systematic checks of state identity, and clearer stopping criteria.
♻ ★ Distributed scalable coupled policy algorithm for networked multi-agent reinforcement learning
This paper studies networked multi-agent reinforcement learning (NMARL) with interdependent rewards and coupled policies. In this setting, each agent's reward depends on its own state-action pair as well as those of its direct neighbors, and each agent's policy is parameterized by its local parameters together with those of its $κ_{p}$-hop neighbors, with $κ_{p}\geq 1$ denoting the coupled radius. To address the challenge of interdependent policies in collaborative optimization, we introduce a novel concept termed the ``neighbors' averaged $Q$-function'' and derive a new expression for the coupled-policy gradient. Based on these theoretical foundations, we develop a distributed scalable coupled policy (DSCP) algorithm, where each agent relies only on the state-action pairs of its $κ_{p}$-hop neighbors and the rewards of its $(κ_{p}+1)$-hop neighbors. Specifically, in the DSCP algorithm, we employ a geometric 2-horizon sampling method that does not require storing a full $Q$-table to obtain an unbiased estimate of the surrogate coupled-policy gradient under the executed policy. Moreover, each agent interacts exclusively with its direct neighbors to obtain accurate policy parameters, while maintaining local estimates of other agents' parameters to execute its local policy and collect samples for optimization. These estimates and policy parameters are updated in a distributed manner through local information exchange among direct neighbors, enabling coordinated policy updates across the network. We prove that the gradient norm of the objective function along the joint policy sequence produced by the proposed algorithm converges to zero almost surely. Finally, the effectiveness of the DSCP algorithm is demonstrated through simulations in a robot path planning environment, showing clear improvement over the compared algorithms in the tested setting.
♻ ★ Hybrid epidemic simulation framework coupling equation-based and individual-based models
Mass gathering events (MGEs) can create intense, short-lived transmission hotspots whose effects propagate through urban populations. Quantifying these effects requires linking fine-scale interactions during gatherings with epidemic transmission across metropolitan mobility networks. We developed a hybrid multiscale simulation framework coupling empirically derived individual-based contact networks for MGEs with a stochastic metapopulation susceptible-exposed-infected-recovered (SEIR) model. Using population and commuting data from Madrid, Spain, we investigated how event-related transmission affects epidemic dynamics and district-level invasion. Across the scenarios examined, MGEs increased epidemic peak incidence and accelerated epidemic progression, although the magnitude of these effects depended on event-specific contact patterns and background transmissibility. Dense contact patterns produced greater epidemic amplification, whereas measures reducing close-contact interactions weakened these effects. MGEs also advanced epidemic invasion across districts and increased the temporal synchronization of arrival times, producing a more synchronized metropolitan epidemic wave. The association between MGEs and earlier district-level invasion was partly explained by the size of the event-generated epidemic seed. The main patterns persisted across the epidemiological parameter ranges examined. The framework provides a multiscale approach for investigating how mass gathering management may influence metropolitan epidemic dynamics.
comment: main text (10 pages, 9 figures), supplementary materials (11 pages, 10 figures, 4 tables)
♻ ★ MafiaScope: Inspecting Multi-Agent Reasoning in Social Deduction Games
An LLM agent's public behaviour reveals little about its social reasoning: an agent that votes correctly may be guessing, and its public claims may differ from privately elicited assessments. We present MafiaScope, an open testbed that turns the social deduction game Mafia into a measurement instrument for multi-agent reasoning. After each public utterance, living agents privately answer configurable structured questions on copies of their available conversations. Probing is non-invasive with respect to the acting context: neither the questions nor the answers enter the histories used to continue the game. Role assessments are scored against roles known to the engine; second-order predictions are compared with other agents' reports. An interactive visualizer renders the assessment trajectories: impersonate mode focuses on one agent's information and reports, panels chart timeline-aligned accuracy and calibration, and counterfactual replay branches from saved states. An offline heuristic links shifts in collective suspicion to episodes for inspection. Experiments span multiple models, languages, and probing configurations; the demonstration presents episode inspection, calibration, probe-repeatability and probe/no-probe checks, and utterance-replacement replay. Code: https://github.com/karpovilia/mafiascope Demo: https://karpovilia.github.io/mafiascope Video: https://vimeo.com/1208920221
♻ ★ Switchcraft: AI Model Router for Agentic Tool Calling
Agentic AI systems that invoke external tools are powerful but costly, leading developers to default to large models and overspend inference budgets. Existing model routers target chat responses or end-task success, not the correctness of the individual tool calls on which agent tasks depend; we present Switchcraft, the first (to the best of our knowledge) model router optimized for strict tool-call correctness in agentic workloads. We construct an evaluation framework on six function-calling benchmarks, holding one out entirely, and fine-tune a calibrated DistilBERT-based classifier, deployed inline under a latency budget, to select the lowest-cost model subject to correctness or abstain when none is likely to answer correctly. Switchcraft reaches 78.3% answer accuracy (86.4% when correct abstentions are credited) at 59% lower cost than the best individual model (75.6%), saving over $3,800 per million queries, and outperforms kNN, MIRT, heuristic, and random routers. On the held-out benchmark, it matches or exceeds every individual model at 5--7X lower cost than the strongest ones. We find two counter-intuitive results: larger models do not consistently outperform smaller ones on tool-use tasks, and nominally cheaper models can incur higher total cost due to token-intensive output.
♻ ★ Generalized Intention Modeling in Multi-Agent Reinforcement Learning NeurIPS 2026
Modeling an opponent's intent is critical for effective decision-making in non-cooperative, competitive, and general-sum multi-agent reinforcement learning. Existing opponent modeling methods encode intent using an embedding derived from episode information chosen a priori, such as the opponent's next action or a future environment state, and use this to guide the ego-agent's behavior. These approaches assume that the chosen information is universally representative of intent; however, we show empirically that this is not the case as intentions are often task- and environment-dependent. To address this, we introduce a task-adaptive opponent modeling framework that learns a performance-driven mixture of multiple intent representations. We further introduce a new intention representation that maximizes mutual information with the ego-agent's future returns, thereby capturing opponent information that is most directly relevant to performance. Our approach consistently matches or exceeds the performance of state-of-the-art baselines across diverse tasks and yields insights into when and why different opponent modeling strategies succeed.
comment: Accepted at NeurIPS 2026
Systems and Control (EESS)
★ Measurement-Based Assessment of Decadal Inertia and Primary Frequency Response Trends in the US Eastern Interconnection
Changes in the generation mix of the U.S. Eastern Interconnection (EI) raise questions about long-term frequency performance, particularly inertia and primary frequency response (PFR). This paper jointly examines these characteristics using synchronized FNET/GridEye frequency measurements and confirmed generation-trip events from 2013 to 2025. A representative EI frequency trajectory is constructed using the median of geographically distributed measurements. Effective inertia is estimated quarterly from the early post-disturbance rate of change of frequency (RoCoF), using a 3-second window and least-squares fitting. PFR is evaluated using the beta metric, which measures aggregate frequency response from disturbance size and the A-to-B frequency deviation. Annual beta estimates are obtained after filtering events by starting frequency and disturbance size to improve comparability across years. Results show that EI effective inertia varies seasonally and follows a non-monotonic trend rather than a uniform long-term decline. Over the same period, coal generation decreased while natural gas, wind, and solar generation increased. Changes in synchronous generator commitment and operating conditions may have offset some expected inertia reductions. Filtered beta generally increased from the mid-2010s to the early 2020s, with year-to-year variability thereafter. The different trends show that early inertial behavior and broader aggregate frequency response have not evolved in parallel. Joint analysis of inertia and beta offers a more complete measurement-based assessment of EI frequency performance during the generation transition.
comment: Presented in CIGRE 2026 Grid of the Future Symposium.The paper has 12 pages and 6 figures in total
★ A Hybrid Path Planning Framework for Assistive Wheelchair Navigation in Dynamic Pedestrian Environments
Autonomous wheelchairs can improve the quality of life for individuals with mobility impairments by providing greater independence. However, reliable navigation especially in human environments is a major challenge due to factors such as dynamic obstacles, unpredictable pedestrian behavior, and the users' need to complete multiple stops along the way before reaching their final destination. This paper presents a hybrid path planning framework for multi-goal navigation in dynamic environments. The framework integrates the Traveling Salesman Problem (TSP) for global task ordering with the Potential Field (PF) method and heuristic local planners such as Rapidly-exploring Random Tree (RRT) and A-star (A*) to achieve collision-free navigation. Within this hybrid framework, TSP determines the visiting order of intermediate stations, PF transfers global path information, and local planners avoid obstacles while guiding the robot to the stations. The local planners generate feasible paths that allow the robot to follow the global trajectory and reroute to avoid static and dynamic obstacles. For this purpose, two algorithms have been developed: (i) TSP-PF-RRT and (ii) TSP-PF-A*. The performance of the proposed algorithms are evaluated using Monte Carlo simulations in crowded and extra-crowded scenarios that contain heterogeneous dynamic and static obstacles. Experimental results demonstrate that the proposed hybrid framework enables efficient multi-goal navigation while maintaining safe operation in dynamic environments with humans, with TSP-PF-RRT achieving fewer or no collisions and shorter travel distances.
comment: 14 pages, 5 figures. Presented as a main-track paper at the International Conference of Social Robotics + ART'26, London, UK, 1-4 July 2026
★ System-Level Factors Influencing AGC Cyber-Attack Success and RAS Vulnerability in Multi-Area Grids
Automatic generation control (AGC) stabilizes frequency and scheduled power exchanges among interconnected areas, yet its reliance on communication networks introduces cyber-vulnerabilities. This work examines coordinated AGC cyber-attacks in the presence of remedial action schemes (RAS) and identifies operating conditions under which an attacker can drive grid frequency beyond safe limits while evading RAS. A feedback-based attack policy is used to manipulate AGC signals and steer system frequency toward unsafe trajectories. Then, a large nested-loop parametric sweep---spanning attacked-area inertia relative to its interconnected neighboring area, AGC capacity versus maximum tie-line transfer, AGC update rate, load damping, and bi-directional power transfers---generates over 1.45 million scenarios. These are classified using a decision tree to establish guidelines to distinguish successful from unsuccessful attacks based on frequency deviations. The resulting vulnerability guidelines are validated on a three-area system using phasor-domain simulations with IEEE 9-bus area models, detailed turbine--governor dynamics, and under/over-frequency load and generation shedding. Results show that specific combinations of AGC capacity, inertia distribution, and pre-attack tie-line loading increase the likelihood of successful attacks, offering operators actionable criteria for identifying vulnerable operating conditions.
★ Proof-Carrying Evolution: Black-Box Search, White-Box Flight
We study whether a language model can help design controllers whose behavior can be formally checked. In proof-carrying evolution (PCE), a search proposes controllers together with their certificates. Their mathematical forms are fixed; a proposer chooses their numerical parameters together. Certificate checks guide selection, and only a certified candidate can be returned. The deployed artifact is a control law and its certificate; the language model is used only during design. We compare four language models with classical search and synthesis on docking under a speed limit, pursuit of a maneuvering target, and attitude control on SO(3), using paired runs and shared proposal allowances. Certificates are checked in exact rational arithmetic during search on the linear problems. On SO(3), the search uses floating-point checks, followed by exact re-verification of every reported champion. The best run of each method also receives Lean 4 proofs of its stated control properties. On the linear problems, the strongest models beat the standard classical baselines on every run, but a classical optimizer that chooses certificates for the same objective closes most of that lead. Among the cited pursuit champions, the ranking does not survive a change from Euler to zero-order-hold dynamics. On SO(3), no model detectably outperforms CMA-ES when mixed with classical mutation. When models write every proposal, one tested model improves on CMA-ES and the other performs worse than its mixed variant. All search medians remain below the hand-tuned law. In a docking control with the objective and mutation operator held fixed, retaining certified candidates yields new certified designs in every run; removing this rule yields none.
★ A Neural Network Approach towards Real-Time Knot Point Density Estimates for the Nonlinear Quadrotor
Spline representations provide a compact way to approximate continuous trajectories using a finite number of parameters. When approximating trajectories generated by dynamical systems, the efficiency of a spline representation depends strongly on the placement of knot points. Uniform knot distributions often lead to inefficient representations, requiring either unnecessary control points or producing poor approximations in regions where the trajectory varies rapidly. In this work we propose a learning-based method for adaptive knot placement in B-spline approximations of trajectories generated by ordinary differential equations. A neural network learns a mapping from the initial condition of the system to quantities that characterize the local variation of the resulting trajectory. These estimates are used to construct nonuniform knot distributions that approximately equidistribute a local error indicator, reducing the number of control points needed to meet a prescribed accuracy.
comment: 8 pages, 5 figures. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. This work is presented at IEEE CDC 2026
★ Risk Assessment of Fair Multi-Period Dynamic Operating Envelopes under Uncertainty
Dynamic operating envelopes (DOEs) can allocate time-varying export capacity to distributed energy resources (DERs), and while they are not intended to make operations immune to voltage violations, they should guide operations to mostly remain inside expected values under changing regimes and residual forecast errors. However, although recent literature proposed many different techniques for implementing DOEs, there is sparse discussion of analyzing and quantifying the risk of using more or less conservative DOEs under uncertain operating regimes. Having this gap in mind, this paper develops a risk-oriented extension of a previously established fair distributed multi-period optimal power flow (OPF) framework, where fair or technical DOEs are fixed as first-stage decisions and the OPF leverages battery energy storage and renewable curtailment as recourse under regime uncertainties. An independent nonlinear AC power flow then computes a resulting ensemble, which is summarized by (i) voltage-violation probability, severity, and a persistence metric for consecutive out-of-band hours, (ii) curtailment and Jain and Gini indices, and (iii) model deviations. Different probability situations are assessed, ranging from uniform, sunny-heavy, night-peak-heavy, and adversarial, which separate the envelope design from the assumed operating distribution. An optional outer loop allows for fine-tuning a fairness--efficiency parameter under explicit violation and persistence budgets. The combination of the outer loop and the overall framework, which quantifies how often and how severely operating limits are exceeded, creates an auditable methodology mixing equitable export allocation, distributed recourse, and AC risk assessment for active distribution network operators, and this framework is assessed using up to 1000 Monte Carlo scenarios per mixture and DOE design.
★ Epidemic Forecasting and Behavioral Response Estimation with a Behavior-Coupled SEIR Model
The COVID-19 pandemic has exhibited multiple waves of infections, reflecting feedback between disease transmission and adaptive population behavior. Accurately predicting and understanding these dynamics remains a central challenge for epidemic modeling. In this paper, we develop a forecasting framework based on a Susceptible-Exposed-Infectious-Recovered (SEIR) model coupled with an evolutionary model of behavioral adaptation, in which population compliance with protective measures evolves in response to perceived infection risk and associated costs. We formulate forecasting as the estimation of behavioral incentive parameters that govern the coupled behavior-disease dynamics, using hospitalization time series data together with standard assumptions on disease-related parameters. Using the inferred parameters, we generate rolling monthly forecasts of infection levels and evaluate the approach on both synthetic data and hospitalization time series from multiple U.S. states (Texas, California, Colorado, and Arizona). The proposed framework provides a principled way to incorporate behavior-disease feedback into epidemic forecasting while yielding interpretable estimates of population-level incentives.
comment: 12 pages, 6 figures, 7 tables. Submitted to IEEE Transactions on Computational Social Systems
★ Safety Verification for Voltage Control in Inverter Dominated Radial Distribution Networks under Actuator and Sensor Adversarial Attacks
This paper develops a safety verification framework for inverter-based radial distribution networks under sensor and actuator attacks. Each customer is equipped with a droop-based controller to regulate its voltage within a desirable bound of a reference, i.e., to achieve safety. In our setting, attacks and disturbances enter the droop-based controllers, while actuator attacks corrupt the reactive power injected into the network. Consequently, invariance of the controller state alone does not guarantee compliance with inverter capacities or customer voltage limits. To address this challenge, we derive conditions to guarantee that the reactive power state remains inside a predefined ellipsoid, while at the same time ensuring that the attacked reactive power injections and customer voltages respect predetermined operating limits that hold uniformly for the entire predefined ellipsoid. Together, these conditions guarantee safety under all admissible disturbances and state-dependent sensor and actuator attacks. Finally, we express the resulting containment condition as a linear matrix inequality (LMI) feasibility problem. A five-customer radial network based on the CIGRE benchmark corroborates our theoretical developments in simulations.
★ Preference-Conditioned Group-Relative Policy Optimization for Terminally Constrained Multi-Objective Control
Multi-objective optimal control is commonly addressed by reinforcement learning with fixed scalarization weights, requiring a separate training run and constraint enforcement for each trade-off. This letter extends critic-free group-relative policy optimization to terminally constrained multi-objective control: a single preference-conditioned policy is trained over random preferences and returns a feasible approximation of the Pareto front in one run. Three structural results are established. Group normalization after scalarization preserves the prescribed preference, whereas per-objective normalization induces policy-dependent distortion; with pairwise distinct normalization scales, only single-objective preferences remain unchanged. A positive floor prevents the terminal-constraint multiplier from vanishing and renders the penalty exact when the floor exceeds a geometric multiplier. Deterministic terminal violation is bounded by expected stochastic violation plus a term depending on terminal-state curvature and action variance, so stochastic feasibility alone does not certify deterministic execution. On a heavy-duty range-extender powertrain with six stage costs and a terminal state-of-charge band, a representative training run places all 32 preference-grid solutions in band on six cycles inside and outside the training pool. Averaged over ten runs, the method yields more in-band solutions under deterministic execution than a critic-based conditioned baseline with the same terminal-constraint handling.
comment: 6 pages, 3 figures, 1 table. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible
★ Safe Planning for Black-Box Systems via MAP Inference with Truncated Diffusion Priors and CBF-Informed Gibbs Likelihoods
This paper proposes a maximum a posteriori (MAP) inference framework with a truncated diffusion prior for safe planning (TDP-SafeMAP). Under a generalized Bayesian formulation, a criterion based on covariance condition number determines the reverse diffusion truncation timestep, and the clean control distribution predicted at that timestep is approximated by a Gaussian prior encoding the control regularities and uncertainty of expert data. An energy function containing a control barrier function (CBF) constraint violation measure defines a Gibbs task likelihood that incorporates physical constraint information. After fusing the prior and likelihood, the covariance matrix adaptation evolution strategy (CMA-ES), whose core update admits an interpretation as a sampling-based approximation to natural-gradient ascent, approximates the posterior mode using only black-box evaluations. Bézier-curve-based dimensionality reduction and related techniques further improve computational efficiency and task-completion capability. Theoretical analysis establishes an error bound for the approximate MAP solution, an explicit condition for CBF safety preservation, and the equivalence of the MAP problem under prior whitening. Experimental results show that the proposed framework consistently generates safe, smooth, and goal-reaching trajectories, providing an effective and practical statistical inference approach to safe planning for black-box systems.
★ Clinical-Protocol-Aware Cyber Resilience for Connected Clinical Technologies Under AI-Enabled Adversarial Conditions
Healthcare increasingly relies on connected medical devices, clinical information systems, cloud services, and distributed-care technologies, creating cybersecurity risks across clinical workflows. AI-enabled capabilities can accelerate vulnerability discovery and attack-path analysis, increasing the need to limit compromise propagation. This paper proposes a clinical-protocol-aware cyber-resilience framework based on bounded clinical micro-networks and controlled interconnections. The framework integrates clinical factors into cybersecurity design, management, and evaluation, covering prevention, containment, and incident response. A parameterized Monte Carlo simulation distinguishes technical cyber exposure from modeled clinical consequence. Experiments investigate architectural connectivity, adversarial capability, security controls, protocol dependency, risk alignment, communication directionality, lifecycle integration, and response timing. Under the simulation assumptions, connected architecture increases mean cyber exposure by 56.3% relative to segmented architecture, while combined prevention and containment reduce exposure by 60.1% relative to no controls. Increasing protocol dependency raises the modeled clinical-impact index from 1.021 to 7.567 without changing mean cyber exposure, demonstrating that technical exposure and clinical consequence can diverge. Risk-alignment experiments further indicate higher modeled clinical impact when greater technical risk corresponds to greater clinical significance. The findings support comparative architectural analysis, not calibrated estimates of real-world cyber risk, control effectiveness, or patient harm. The framework provides a clinical-context-aware approach to security-boundary design, control prioritization, and coordinated containment in connected healthcare environments.
comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible
★ Scaly: an Optimizing Compiler for Optimal Control
We present Scaly, an open-source optimizing compiler for optimal control, written in Python, that generates fast C code. Scaly provides a modeling language embedded in Python for the parametric optimization problems that arise in model predictive control and moving horizon estimation, and computes the sparse derivatives that standard optimization solvers need. Scaly accepts general nonlinear programs but applies optimizations specific to optimal control, which preserve the repeated structure of a problem, such as the stages of a horizon, through differentiation down to the generated loops. The generated C depends only on the C math library and never allocates memory dynamically. On three benchmark problems, Scaly evaluates the Hessian of the Lagrangian up to 26 times faster than CasADi, with smaller binaries and build times that stay nearly constant with the horizon. With the same solver, Scaly shortens the solver time per control step on every benchmark, by a factor of up to 9.
★ Probability-Layer Clustering for Bound-and-Budget Calibration in Multi-Bound Robust Bidding of Renewable-Based Virtual Power Plants
RVPPs that aggregate dispatchable and non-dispatchable renewable energy sources, storage, and flexible demand that bid in the day-ahead and secondary reserve markets must cope with uncertainties in market prices, renewable generation, and demand. MBRO is well suited to this setting because it hedges adverse realizations through several deviation layers while keeping the final bidding problem as a tractable, scenario-free MILP. Its performance, however, depends on how the deviation bounds and uncertainty budgets are calibrated. Existing MBRO implementations commonly rely on fixed percentile-based bounds and externally specified budget vectors, usually examined through sensitivity analysis. This paper proposes PL-MBRO, that constructs parameter-specific deviation bounds from historical data conditional on selected risk-attitude quantiles and automatically derives the corresponding uncertainty budgets from empirical cluster probabilities. For each uncertain parameter, k-medians clustering partitions the historical daily profiles into representative probability layers; the cluster median defines the balanced representative, while favorable and adverse within-cluster percentiles define the optimistic and pessimistic representatives. The empirical cluster probabilities are mapped to integer full-horizon uncertainty budgets over the 96 quarter-hourly periods, while the robust inner problem determines the periods in which the corresponding deviation layers are most adverse to the schedule. Compared with a percentilebased MBRO benchmark, in-sample and out-of-sample tests on an RVPP bidding in both markets show that PL-MBRO consistently achieves better realized economic outcomes across the optimistic, balanced, and pessimistic strategies, with improvements of up to 92.7% in-sample and 57.3% out-of-sample, measured relative to the absolute value of the corresponding benchmark realized profit.
★ Distributed Zeroth-Order Composite Finite-Sum Optimization via Incremental Variance Reduction
We study decentralized composite finite-sum optimization over a fixed connected undirected network, where agents access local component functions through function evaluations and share a possibly nonsmooth convex regularizer. In this setting, standard two-point gradient estimators can retain nonvanishing variance at a solution, complicating their integration with gradient tracking. We propose Distributed Zeroth-Order Incremental Variance Reduction (Distributed ZIVR), which combines incrementally updated component-gradient estimates with a proximal gradient-tracking scheme. The method admits an implementation using a constant number of function evaluations per agent at every iteration, without intermittent full-component or full-coordinate refreshes. For convex smooth component functions and strongly convex local averages, we establish linear convergence up to a controllable finite-difference error. With the finite-difference radius chosen for the target accuracy, Chebyshev-accelerated mixing yields a per-agent oracle complexity of $\widetilde{O}(d(m+κ)\log(1/ε))$ for expected squared solution error $ε$, where $d$ is the dimension, $m$ is the number of components per agent, and $κ$ is the condition number. This matches the centralized ZIVR oracle bound, while the communication bound has only an inverse square-root dependence on the network spectral gap. Experiments on logistic and robust regression demonstrate rapid reduction of solution and consensus errors.
★ Channel Measurement and Characteristics Analysis for Indoor Flexible Pinching Antenna Systems in the mmWave Band
Pinching antenna (PA) systems offer flexible antenna configurations for future wireless communications, yet how PA geometry, orientation, and position affect channel characteristics in practical environments remains insufficiently studied. To address this gap, this paper first presents an indoor PA system channel measurement campaign at 59.5~GHz. A dedicated time-domain channel sounder is developed with frequency conversion and a circular-waveguide-to-dielectric-waveguide transition to feed the radio frequency (RF) signal into a flexible dielectric waveguide. Using this platform, office measurements quantify channel attenuation and channel characteristics by examining four PA geometries, arc PA orientation, circle PA pinching position, and transmitter--receiver height difference. The channel measurement data are analyzed in terms of path loss, shadow fading, root-mean-square (RMS) delay spread, Rician K-factor, normalized capacity, and spatial similarity. Indoor path loss models for PA systems are fitted for each PA geometry. The results reveal geometry-dependent attenuation and shadow-fading variability, with the arc PA showing greater variability than the circle PA. Arc PA rotation shifts the delay spread and conditional K-factor distributions. Moving the circle PA produces nonmonotonic changes in delay spread and normalized capacity, while spatial similarity remains relatively high at many receiver locations as the pinching position changes. These observations provide an empirical basis for PA systems channel model and system design.
★ A complete analytical characterization of optimal collision-free motion for two robots
Collision-free motion planning for multi-agent robotic systems remains an active research area because of the complexity of coordinating robot trajectories, with applications ranging from coordinated drone shows to autonomous operations on land, in the air, and on water. Importantly, the analytical difficulty of nonconvex collision-avoidance constraints has so far driven the development of exclusively numerical methods. We derive a complete analytical characterization of the optimal trajectories for two robots that start and end at rest, under double-integrator dynamics and a squared-acceleration objective. We prove that these trajectories belong to eight distinct contact families and determine which family applies for each configuration of initial and final positions. We analyze the geometrical properties of the optimal trajectories, provide a numerical algorithm for computing them, and discuss how these insights may improve the performance of existing numerical trajectory optimizers.
★ Contact-based optimization for collision-free motion planning for a million robots
Multi-agent robotic systems have attracted growing interest in recent years, with applications ranging from coordinated drone shows to autonomous operations on land, in the air, and on water. At the planning level, robot motion is often modeled using double-integrator dynamics (with low-level controllers handling the physical dynamics during execution) with a minimum-energy objective subject to collision-avoidance constraints and limits on position, velocity, and acceleration. The complexity of the problem resides in the constraints, which couple the trajectories and make the optimization problem nonconvex. Existing numerical approaches typically discretize and optimize entire trajectories, which becomes expensive for large fleets. We exploit the structure of optimal trajectories by organizing optimization around contacts, at which robots reach the minimum allowed separation. We formulate a coupled algebraic system in contact variables and reconstruct trajectories analytically between contacts. Our algorithm, sequential contact insertion (SCI), starts from unconstrained trajectories, inserts contacts at separation violations, and jointly adjusts their times and states. Numerical experiments demonstrate planning for one million robots in minutes and thousands of robots in fractions of a second. On problems feasible for a fixed-grid sequential convex programming baseline, SCI achieves median speedups of several orders of magnitude at matching solution accuracy.
★ Report on Stability Analysis of Islanded Power Systems with Grid-Forming Converters
Islanded power systems are inherently challenging to operate due to limited interconnection, low inertia, and weak short-circuit strength. As renewable and inverter-based resources replace synchronous machines, these challenges worsen, causing faster frequency deviations, greater voltage volatility, and higher risk of instability. This report summarizes the differences between islanded and interconnected systems and reviews key stability issues in islanded grids: frequency stability, converter-driven interactions, voltage stability, large-disturbance behavior, and small-signal dynamics. Grid-forming (GFM) voltage source converters are a promising solution because they can establish voltage and frequency, emulate synchronous machine behavior, and provide grid strength, virtual inertia, fast frequency response, voltage regulation, and black-start capability. The report examines relevant GFM control strategies and evaluates their capabilities in weak, low-inertia islanded grids. Challenges remain, particularly interoperability, control coordination among multiple inverter-based resources, lack of standardized grid code requirements, and coordination of protection devices. Successful deployment requires advances in interoperability, harmonized grid codes, and grid planning that considers the unique features of islanded systems, along with clear GFM functionality requirements. Finally, a case study illustrates the stability-enhancing capability of GFM converters in weak grids through small-signal stability analysis. Overall, GFM technology is a key enabler of reliable, renewable-dominant island power systems, but broader adoption depends on continued technical and regulatory development.
comment: Danmarks Tekniske Universitetet, Katholieke Universiteit Leuven, Department of Electrical Engineering, Electrical Energy Computer Architectures, Universitat Politècnica de Catalunya, & Cardiff University. (2026). Scientific deliverables of ADOreD Industrial Doctoral Network. Zenodo. https://doi.org/10.5281/zenodo.19736793
★ Koopman Greedy Spiking Control for Nonlinear Systems under Firing-Rate Limits
Neuromorphic controllers interact with physical systems through fixed-amplitude spikes whose timing conveys control information. Existing Lyapunov-based greedy spiking controllers guarantee stability but are restricted to linear systems and neglect firing-rate limits. We extend greedy spiking control to nonlinear systems using Koopman operator theory. For observable dictionaries invariant under differentiation along the input direction, spike-induced state translations admit an exact linear representation in lifted coordinates, enabling exact modeling of spiking jump dynamics. This allows firing decisions to be formulated as comparisons of quadratic Lyapunov functions in the lifted space. We prove practical stability of a clocked, rate-limited controller with an explicit ultimate bound and develop a data-driven design based on Koopman eigenfunctions learned from unforced data by generator-based regression. The weights of the Lyapunov function are synthesized by a linear program that enforces a sampled relaxation of the rate-limited control-Lyapunov condition. Applied to an inverted pendulum, the learned eigenfunctions recover the system energy, and the proposed controller achieves swing-up and stabilization from nearly all initial conditions despite severe actuation limits.
★ GNSS Receiver Operating Point Comparison Between Vector and Scalar Tracking Under High-Dynamics
This paper presents a measurement noise adaptation mechanism for Kalman filter-based carrier tracking in a navigation-aided GNSS receiver designed for launch-vehicle applications. The tracking loop is tuned for 20\,g acceleration steps, which requires a wide bandwidth that makes the loop vulnerable to noise-driven pull-out at low carrier-to-noise density ratios. To resolve this tradeoff, the measurement noise covariance, $R$, is inflated via a two-layer gate: a sigmoid envelope keyed to the signal $C/N_0$, modulated by a dynamics indicator derived from the discriminator output statistics of the Loop Bandwidth Control Algorithm. During noise-dominated periods the gate inflates~$R$, reducing the Kalman gain so the filter coasts on the navigation-aided prediction; when dynamics are detected the gate opens within one epoch, restoring the high-bandwidth tuning. The navigation aiding is formulated as a stochastic control input whose uncertainty is propagated from the navigation EKF covariance. Simulation results on a 20\,g acceleration step scenario show that the adapted vector architecture doubles the lock-retention rate relative to the standalone scalar filter at $C/N_0 = 31$\,dB-Hz. The receiver is implemented in MSGR, an open-source MATLAB GNSS software-defined receiver.
★ Physically Constrained Kinematic Calibration for High-Precision Robotic Needle Guidance
Percutaneous needle insertion is a common practice in minimally invasive procedures whose clinical success depends on accurate needle placement. However, achieving the necessary precision is a challenge for manual operators. Robotic assistance can improve targeting accuracy and reduce operator dependence, but residual millimeter-scale errors persist because nominal kinematic models do not fully represent the physical manipulator. This paper presents a robust bound-constrained kinematic calibration method designed to improve positional accuracy for a 6-Degree-of-Freedom (6-DoF) robotic needle-guidance manipulator. The approach uses physically plausible parameter ranges to regularize a compact 19-parameter geometric identification model. By treating these bounds as active optimization constraints rather than mere numerical safeguards, the framework effectively mitigates parameter coupling and overfitting, ensuring highly stable parameter identification. In simulation, the proposed method reduced end-effector Root Mean Square Error (RMSE) from 3.422 mm to 0.690 mm (a 79.8% improvement). In benchtop hardware experiments, the calibrated model achieved sub-millimeter validation accuracy, including 0.865 mm RMSE in experimental trajectory validation and 0.610 mm RMSE in a coordinate-lattice verification task. These results demonstrate that physically constrained kinematic calibration can enhance robotic needle-targeting accuracy, establishing a high-precision foundation for future clinical applications.
comment: 6 pages, 4 figures, 4 tables
★ Vehicle clustering and resource scheduling scheme based on collaborative perception of wireless resources
Cellular Vehicle-to-Everything(C-V2X) sidelink communication allows autonomous resource selection via distributed scheduling without base station coverage. However, the limited perception of wireless resources may lead to resource allocation conflicts, especially when wireless resources are relatively scarce. In this paper, we propose a distributed clustering and resource scheduling scheme based on collaborative perception of wireless resources. The scheme innovatively integrates wireless resource sensing states into the clustering process. Based on the constructed vehicle clustering structure, the cluster heads fuse the distributed perception data of wireless resources from cluster members to generate more accurate collaborative perception results of the environment where the clusters are located, so as to support more reliable resource scheduling and reduce resource allocation conflicts. Simulation results show that the proposed scheme achieves higher topological stability than conventional clustering schemes based solely on mobility, and superior communication performance compared to the autonomous resource allocation mechanism of C-V2X.
comment: Accepted for publication in the 2026 IEEE International Conference on Communication Technology (ICCT 2026)
★ Riemannian Optimization on Unknown Manifolds via Stochastic Proximal Refinement
We study Riemannian optimization on an unknown manifold accessed only through feasible samples. The central challenge is to turn feasible samples into optimization steps, i.e., the samples are drawn from the unknown feasible manifold, but do not directly provide the tangent spaces, projections, or retractions required by classic Riemannian methods. Moreover, regions with low objective values may be sparsely represented in the data, so directly sampling from a distribution fitted to these observations can require many draws to find better feasible solutions. We introduce Proximal Refinement on Implicit Smooth Manifolds (PRISM), which turns feasible-sample access into an iterative optimization procedure by repeatedly sampling an objective-aware proximal distribution conditioned on the current iterate. Theoretically, we show that PRISM recovers the effect of Riemannian-gradient refinement, leading to provable objective descent, feasibility (manifold adherence), and convergence toward first-order stationarity. We demonstrate the effectiveness of our approach through synthetic experiments illustrating its geometric properties and applications to data-driven building control and diverse real world optimization tasks, highlighting its potential as a general framework for data-driven optimization across scientific and engineering domains.
★ Control-Oriented Modeling of Heat Exchangers with Improved Low-Flow Dynamics
Existing low-order models of heat exchangers can lose dynamic accuracy at low flow rates, when fluid residence times become significant. This paper proposes two control-oriented models that improve the low-flow performance. The first extends a finite-volume formulation with the first spatial temperature moment, whereas the second uses Chebyshev--Lobatto collocation. Both capture the coupled heat-transfer and transport dynamics and improve the performance when the system behavior is dominated by the inlet-temperature dynamics. At equal state dimension, comparisons with a high-order reference show more accurate delayed outlet-temperature responses while retaining steady-state accuracy.
comment: 6 pages
★ Quantile Propagation Planning for Budgeted Tail-Risk Evaluation
Risk-aware planning must identify rare, costly outcomes using only a limited number of model evaluations. Monte Carlo rollouts spend this budget on noisy tail estimates, while planning from the mean can miss the tail entirely. We introduce quantile propagation planning (QPP), which propagates a small, fixed set of disturbance quantiles through the dynamics and integrates the resulting cost curve to evaluate both mean and tail risk. The method is exact at the propagated levels, converges as the quantile grid is refined, and admits a one-sided conservatism guarantee under stated structural conditions. In controlled overload and inventory tasks, QPP reduces realized tail cost and constraint violations relative to mean-rollout planning. On overload, it remains competitive with sampled evaluators that use substantially more trajectories. These results position QPP as a transparent, budget-controlled approach to tail-risk evaluation.
comment: 60 pages, 8 figures, 34 tables; includes appendices
★ HIPC: Humanoid Iterative Planning and Control via Goal-Conditioned Motion Closure
Recent generative models are widely used to provide whole-body motion for humanoid tracking. Current methods combine motion generators with controllers in an open-loop paradigm, which leads to critical issues: tracking errors can accumulate into spatial drift, and the planner cannot adapt to unexpected physical disturbances without feedback from physical execution. However, directly feeding realized states back to the planner in a closed loop can cause discontinuities, resulting in physical jitter and even falls, since such states are out of distribution during training. To bridge this gap, we propose HIPC, a real-time Humanoid Iterative Planning and Control framework that couples a generative model with motion tracking into a unified planning-and-control paradigm. HIPC introduces a goal-conditioned motion planning trained with a canonical motion representation that avoids gimbal-lock singularities under extreme orientations and supports motion generation on diverse geometric and semantic conditions. Then, HIPC adopts two strategies, deviation-decayed recovery and history re-anchoring, to mitigate the out-of-distribution shift caused by realized-state feedback. Finally, HIPC operates a real-time asynchronous planning-and-control system, with latency compensation to reduce temporal mismatch for inference delay. Extensive experiments confirm the effectiveness of HIPC under switching and composition of diverse conditions. Results show that HIPC achieves state-of-the-art goal-reaching accuracy among existing methods while simultaneously demonstrating robustness to physical perturbations and the ability to recover from falls.
comment: Website: https://hipc-humanoid.github.io/
★ Multi-Timescale Deliverability of Shared Hybrid Storage for AI Data Center Campuses
Artificial intelligence data center (AIDC) campuses contain sustained power variations and subsecond power fluctuations, requiring both sufficient stored energy and fast response. A shared hybrid energy storage system (HESS) can mitigate aggregate AIDC power deviations, but interval-based schedules cannot capture the delay between commanded and delivered power at the campus point of common coupling (PCC). To this end, this paper proposes a multi-timescale scheduling model for mitigating AIDC power fluctuations with the shared HESS. First, it models electrical pooling at the campus bus and constrains the shared HESS to a single PCC injection. Second, a two-layer model passes five-minute scheduled power, energy state, and reserve capacity to a causal fast response model that distinguishes commanded power from delivered power. Case studies using the National Laboratory of the Rockies (NLR) GenAI Power Profiles Dataset show reductions in both PCC and fast integral absolute error (IAE), while retaining most of the improvement available under ideal response.
★ Frequency-Calibrated Decision-Focused Learning for BESS Dispatch Under Renewable Uncertainty
Battery energy storage system (BESS) flexibility depends on state of charge and intertemporal operating constraints. Renewable forecast errors of similar magnitudes can lead to different BESS dispatch results. An error may alter early charging or discharging decisions and reduce the flexibility available for subsequent regulation, while another may have little effect on the resulting schedule. To account for these unequal consequences, this paper proposes Frequency-Calibrated Decision-Focused Learning (FC-DFL), which trains renewable predictors using the realized operating costs of their induced BESS schedules. Gaussian smoothing and solver-based zeroth-order estimation provide training feedback without requiring differentiating through the non-differentiable mixed-integer BESS dispatch problem. The same decision feedback is further combined with prediction sensitivity to develop a decision-oriented XAI method that identifies the input features and forecast periods most consequential to BESS operation. Case studies using German wind and solar data show that FC-DFL reduces the integrated absolute frequency-deviation metric by 6.05% and decision regret by 23.92% relative to accuracy-based predict-then-optimize method. The XAI analysis further demonstrates that the identified high-impact forecast periods produce 2.84 times larger average absolute changes in realized operating cost than low-impact periods. These findings highlight the importance of incorporating downstream decision-making outcomes into renewable forecasting and demonstrate the value of FC-DFL in improving BESS regulation and providing operationally meaningful forecast explanations.
★ Delta-Seeker: Closed-Loop Generative Search for Novel Circuit Design Discovery
Generative models are increasingly used for circuit synthesis and optimization by proposing topologies, sizing devices, and exploring large design spaces. We investigate whether they can instead support design discovery, where generated structures must depart from known designs while producing reproducible electrical behavior. We propose Delta-Seeker, a closed-loop circuit search framework that combines discrete diffusion for topology generation and mutation with structural checking, SPICE-in-the-loop sizing, Pareto selection, and feedback from prior search outcomes. The search controller uses progress and recurring structural failures to allocate evaluation effort and steer subsequent topology generation, while independent re-sizing and operating-corner analysis are performed only after search. In generator-only experiments, cross-generation feedback increases structurally valid children from 143 to 945 and reduces the faulty-candidate fraction from 0.925 to 0.761. In op-amp topology discovery at matched sizing cost, Delta-Seeker yields 54 designs that remain feasible in at least three of four independent re-sizings and 18 that additionally pass device-level checks and supply-temperature corner analysis, compared with 38 and 10 under fixed control. Removing cross-generation memory reduces independently robust discoveries from 54 to 43 and seven-corner robust discoveries from 40 to 27. In DC-DC converter search, nearest-parent analysis identifies a corpus-absent topology whose generated switching current path changes its nearest training-corpus parent from infeasible at all four operating corners to feasible at all four, while increasing median efficiency from 31.2% to 98.96%. These results show that closed-loop generative search can support circuit discovery beyond structural novelty by identifying new topologies with independently verified changes in circuit operation.
★ Control-Layer Interpretation of Reactive Power Recovery Limits in EMT-RMS Model Benchmarking of Inverter-Based Resources
EMT--RMS benchmarking is widely used to verify cross-platform consistency of inverter-based resource (IBR) dynamic models. This paper examines a 200 MW anonymized IBR represented in PSCAD and PSS/E. The EMT model separates inverter electrical capability from a plant-level power plant controller (PPC) recovery target, whereas the RMS model represents these functions through REECC and REPCA parameters. Directly assigning the PPC target of $\pm 0.2764$ pu to REPCA hard reactive limits is initialization-inconsistent at the original lagging operating point because the initial reactive command is 0.396273 pu. A second, near-unity-power-factor operating point initializes successfully with the same diagnostic limit, demonstrating that the conflict is operating-point dependent. Using the non-conflicting capability mapping removes the original hard-limit conflict, but the benchmark still shows 18.5--30.2 MVAr post-event POI reactive-power differences from PSCAD. A one-at-a-time REPCA $K_c$ sensitivity under matched benchmark initial conditions further shows that RMS recovery is parameter- and limit-dependent: at $K_c=0.05$ and 0.10 the aggregate machine reactive output reaches the configured capability bounds, while $K_c=0.15$ moves VDOWN, VUP, and HVRT off those bounds and changes their settled POI reactive power by 8.94, 18.00, and 3.17 MVAr, respectively. Machine-to-POI accounting further shows why machine-side saturation need not reproduce the POI-level PPC recovery magnitude. The results distinguish initialization feasibility from recovery equivalence and provide a control-layer diagnostic workflow beyond waveform comparison.
comment: Accepted for presentation at IEEE PES Grid Edge Technologies 2027, April 19-22, 2027, Salt Lake City, Utah, USA.5 pages, 3 figures, 6 tables
★ DistFlow and Its Conic Relaxation: Regularity, Monotonicity, and Hybrid Exactness via Graph Decomposition
Classical exactness conditions for conic relaxations of the DistFlow model rely, among other assumptions, either on all nodal injections being adjustable or on a network-wide path condition. In practice, these conditions may not hold uniformly across a feeder: controllable distributed energy resources (DERs) sit at some buses and fixed loads at others, while reverse flows from DER-rich regions can violate the path condition. This paper certifies exactness region by region and composes the regional certificates, accounting for perturbations that propagate downstream through voltages. We first introduce a subtree voltage stability region, in which the network and all compatible subtrees remain regular. Within it, two endpoint sign conditions, one relaxing the nonnegative-flow assumption and one implied by the classical path condition, yield monotone responses of voltages, currents, and losses to root voltages. Using duality, the implicit function theorem, and sign properties of inverse sensitivity matrices, we recast both classical arguments as individual sign certificates on branch multipliers. This turns exactness into a branchwise property that can be certified region by region. A partition rule then splits the network into dispatchable and passive subtrees, and a hybrid certificate combines their local conditions into exactness of the full-network relaxation, under additional operating-limit and objective conditions. In particular, placing controllable DERs at both ends of a branch with large reverse flow exempts that branch from the passive sign conditions.
★ EMT Assessment of SCR and X/R Effects on Current-Priority-Dependent LVRT Response of a Grid-Following BESS
Low-voltage ride-through (LVRT) forces a grid-following inverter to allocate finite converter current between active- and reactive-current objectives. This paper uses a PSCAD electromagnetic-transient (EMT) representation of an aggregated battery energy storage system (BESS) to quantify how short-circuit ratio (SCR) and grid impedance X/R ratio alter that tradeoff. Active-current priority and reactive-current priority are compared for five paired grid conditions under the same 150-ms balanced three-phase fault, with pre-fault active power approximately 0.84 pu, POI reactive-power exchange near 0 pu, and a 1.0-pu dq-current limit. Fundamental positive-sequence voltage and power are reconstructed from 100-$μ$s three-phase EMT outputs. Across the fixed-fault SCR sweep, the Q-priority late-fault voltage advantage increases from 0.046 pu in the strongest tested grid to 0.129 pu in the weakest; the weakest P-priority case also shows active-power reversal and a pronounced controller-frequency excursion. At fixed SCR, changing X/R produces a smaller increase from 0.092 to 0.112 pu, mainly through reduced P-priority voltage support. Because the common physical fault also deepens the passive retained-voltage condition as SCR decreases, the SCR result is interpreted as a fixed-fault grid-strength sensitivity rather than a constant-sag comparison. Grid strength is the larger sensitivity in the tested matrix, while X/R remains an additional modifier under the studied fault conditions.
comment: 6 pages, 4 figures. Submitted to the 2027 IEEE Texas Power and Energy Conference (TPEC 2027)
★ Market Efficiency and Capacity Investment under Flexible Price-Making Loads: A Computational Study
Large flexible loads such as data centers can shift consumption across locations and hours to lower their bills. We model a price-making consumer leading the system operator in a Stackelberg game and compare its shifts with a cost-minimizing planner's over a simulated RTS-GMLC year. Strategic temporal and spatial shifting capture 75% and 93% of the planner's savings, forgoing the rest mostly by withholding flexibility to protect favorable prices. Scarce capacity and abundant renewables raise this share for temporal shifting; scarce transmission lowers it for spatial shifting. Temporal and spatial flexibility substitute for thermal and transmission capacity, respectively, and complement renewables.
★ Koopman--Lyapunov and Koopman--Riccati Controller Synthesis
While the Koopman operator-based lifted representations of nonlinear autonomous and controlled systems have been well covered in the literature, the direct controller synthesis problem largely remains open. As the controlled system is generally a bilinear (instead of fully linear) Koopman model, controller synthesis appears to lose the linear--quadratic structures of the classical Lyapunov and Riccati methods for finite-dimensional linear systems. However, in infinite dimensions, bilinearity in the feature map should be merely artifactual and hence reducible to linearity in principle. In this work, by considering the operator semigroups in the reproducing kernel Hilbert space with a linear--Sobolev kernel and utilizing the Yosida approximation of their generators, the synthesis problems of a stabilizing controller and an input-unconstrained optimal controller can be formulated as an operator Lyapunov inequality and an operator Riccati equation, respectively. These formulations recover the desirable linear--quadratic structures, and hence, with sample data, they are solvable efficiently in analogous ways to usual linear systems by semidefinite programming solvers and Kleinman iterations, respectively. In addition to proving control performance bounds dependent on the sampling denseness and Yosida regularization, through numerical comparison to linearized LQR, bilinear EDMD-based synthesis, nonlinear MPC, and Koopman MPC methods on numerical models and process control examples, this work demonstrates the proposed Koopman--Riccati synthesis as a computationally efficient approach to obtain a near-optimal state-feedback control law.
comment: 56 pages, 9 figure, 5 tables
★ Optimal climate policy in incomplete electricity markets
Incomplete long-term contracting in electricity markets can expose investors and consumers to inefficient levels of risk. This paper investigates how incompleteness impacts the optimal choice of climate policy. We develop a game-theoretic model of optimal policy-making in anticipation of how electricity market agents react to a given policy when gas prices and demand are uncertain and contracting is incomplete. In illustrative experiments, we observe that renewable subsidies and carbon prices induce an insurance co-benefit when markets are incomplete. This is because, relative to a complete market benchmark, incompleteness shifts investment from renewables toward gas and exposes consumers to uninsured risk from inter-annual volatility in electricity prices. By incentivizing investment in renewables, subsidies and carbon prices indirectly counteract these distortions. Consequently, incompleteness increases the optimal subsidy and carbon price levels in our experiments. Renewable subsidies exhibit a stronger insurance effect than carbon pricing, as the latter leads to more investment in gas with carbon capture and storage and more volatile electricity prices. While carbon pricing is more economically efficient than subsidies in complete markets, incompleteness shrinks its efficiency advantage and even leads to cases where subsidies are more economically efficient.
★ Sense Less, Miss Less: Evidence-Driven Sensing Schedules for ISAC
Integrated sensing and communication (ISAC) lets a base station detect human motion with the same signals it uses for data, but every sensing transmission takes resources from users. Sensing rarely while nothing moves saves resources, yet a brief hand gesture by an otherwise still person can then start and end between two transmissions and go unnoticed. We propose an evidence-driven schedule, based on data-efficient cumulative-sum (CuSum) change detection, that pauses sensing while measurements indicate no motion and senses continuously once they suggest motion. It is adapted to orthogonal frequency-division multiplexing (OFDM) sensing, where requests can be blocked by data traffic and measurements arrive late, and evaluated with an articulated human model of limb motion. In simulations without wall reflections, with the same monitoring budget and below one false alarm per minute, it detects 1.5--4.2 times as many limb movements within 200~ms as periodic sensing. A tuned two-rate trigger performs similarly, and the gain over periodic sensing persists, without retuning or cost matching, in a simulated room with wall reflections. Under a small budget, measurements should decide when to sense.
comment: 6 pages
★ Measurement-angle control of a continuously monitored quantum oscillator: purification limits and optimal protocols
We study the dynamic purification of a continuously monitored quantum harmonic oscillator coupled to a thermal bath, where the control is the quadrature measurement angle of up to two detectors with prescribed strengths and efficiencies. Within the Gaussian regime, the conditional state covariance obeys a deterministic Riccati equation, and we ask what purity can be reached and sustained by manipulating the angle in time, in analogy with rapid-purification protocols for qubits. We propose, analyze, and compare constant-rate, greedy, vibrational, and variational monitoring strategies on the basis of sustained purity starting from the thermal bath state, and quantify to what extent they are able to maintain higher purity levels compared to lab-fixed or phase-locked detectors. We highlight structural features, operational limits, and optimal protocols from each strategy and study how model parameters shape control performance. We observe that in the absence of measurement-frame anisotropy, the time-dependent strategies achieve higher but similar performance. Once this symmetry is broken, their performances diverge, either improving or worsening depending on the relative timescales of oscillator dynamics, thermalization, and measurement.
comment: 24 pages, 13 figures
★ Short-Circuit Current Calculation in DC Shipboard Microgrid Using Operational Admittances
The increasing adoption of DC shipboard power systems is driven by the need to integrate energy storage systems seamlessly onboard. In this context, accurate calculation of short-circuit current transients is critical for sizing fast-acting protection devices. This paper proposes a methodology to calculate short-circuit current transients based on operational admittance modelling of network elements and power converters. The superposition principle is then applied to calculate current contributions and voltage time profiles. The methodology is validated with a detailed EMT model of a notional LVDC shipboard microgrid section consisting of an active front end, a bidirectional buck-boost converter and two special loads. Results demonstrate that the methodology accurately tracks nodal voltages and the fault current contributions dynamics up to the prospective fault current peak, thus providing a computationally efficient tool for fault analysis and protection coordination in LVDC microgrids.
comment: Accepted and presented at ESARS-ITEC 2026. Copyright 2026 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses
★ Spectral and Conformal: Neural Network Methods for Calibrated Trajectory Prediction and Control-Point Intervals
Autonomous systems that navigate their environment need methods to quickly assess the feasibility and safety of desired trajectories. Traditional solutions that rely on numerical integration can be computationally heavy, while fast data-driven surrogates lack the formal guarantees that numerical approaches provide. This work demonstrates a framework that uses (1) a neural network with a spectral backbone to learn trajectories and quantile bands with built-in constraints (initial conditions and stability); (2) a modified conformal quantile regression to provide calibrated, joint trajectory-level coverage; and (3) interval arithmetic to extend noise-free coverage guarantees to control points. After refactoring the network for inference, the trajectory inference time for a quadrotor under a closed-loop linear quadratic regulator is 3.45~ms on a Raspberry Pi Zero 2 W. The total pipeline time, including trajectory inference time, is 9.81~ms, or approximately 100~Hz, consistent with typical control rates.
comment: 26 pages, 10 figures, 7 tables. Published in Transactions on Machine Learning Research
★ Feedback Stability of Discrete-Time LTI Systems via Frequency-Wise Scaled Relative Graphs
This paper studies the stability of negative-feedback interconnections of proper, stable discrete-time multivariable linear time-invariant (LTI) systems using scaled relative graphs (SRGs). We propose a frequency-wise condition that compares the SRGs of the two subsystems'frequency-response matrices only at the same frequency along a feedback-scaling homotopy. We prove that this condition guarantees well-posedness along the homotopy and $\ell_2$-stability for both single-input single-output (SISO) and multi-input multi-output (MIMO) subsystems, without requiring the chord property or nonsingular frequency-response matrices. An equivalent form exchanges scaling and geometric inversion. We also derive an operator-level condition from the SRG closures of the convolution operators and show that it implies the frequency-wise condition. Numerical examples show that the operator-level condition is more conservative because it depends on intersections between responses at different frequencies.
♻ ★ Readiness-Constrained Pre-Islanding Dispatch for Military Microgrids
Military installation energy readiness depends on having sufficient resources to sustain mission-critical loads throughout a prolonged utility outage. This paper develops a method to help operators manage energy resources during normal grid-connected operation while preserving islanded readiness. A robust viability kernel identifies combinations of battery energy, fuel, and load-shifting capability sufficient to sustain critical load under uncertain demand and photovoltaic generation. A probability bound translates stochastic generator start failures and recovery into a deterministic generator count selected to meet a specified availability requirement at islanding. Case studies for a fourteen-day islanding requirement at 0.999 islanding-entry availability show that advance warning of an outage can be used to prepare resources to meet a higher readiness requirement before islanding begins. The case studies further demonstrate that economic-only operation and the evaluated fixed battery reserve policy can fail to sustain critical load for the required duration, whereas enforcing the resource-dependent requirement preserves readiness at a modest increase in modeled grid-connected operating cost. These results demonstrate how active resource management can preserve long-duration energy readiness. The framework also helps commanders understand how readiness changes with operating conditions, even when the available resources remain the same.
comment: 32 pages, 5 figures, 3 tables, submitted to Military Operations Research
♻ ★ Is Your AI Fast Enough to Run a Fusion Reactor?
In magnetic confinement fusion, compact machine learning algorithms are often integrated into the control loop on CPU-based custom hardware stacks. Deterministic implementations of these algorithms target instabilities which can evolve over microseconds to milliseconds. Staying within this framework may therefore limit the pace of control-system development to the inference time available. With the expansion of hardware acceleration and large models, we aim to bring ML models for fusion control to modern inference engines and compare, based on the physics reactor, the suitability of each engine per model. This study implements a consistent real-time machine learning algorithm for GPUs and optimized CPU packages, including OpenVINO, XLA, and TVM, at DIII-D National Fusion Facility. For the smallest model, one CPU core responds within 3.6 $μ$s in 99% of control cycles and an NVIDIA A100 within 28 $μ$s. We also evaluate models running simultaneously, as they would run in a typical experiment, and find best use cases per framework.
♻ ★ SmartMeterFM: Flow Matching for Smart Meter Data Generation, Imputation, and Super-Resolution Without Retraining
Distribution system operators need realistic load and generation profiles when measurements are unavailable, incomplete, or too coarse for the intended analysis. Training separate models for synthetic data generation, imputation, and super-resolution can duplicate modeling effort as observation requirements change. This paper presents SmartMeterFM, a conditional flow matching model trained once on monthly smart meter profiles at 15 min resolution. Customer and calendar attributes condition the model during training, while numerical profile constraints, observed intervals, and coarse measurements are incorporated through projection-based guidance at inference. The same weights serve all tasks without task-specific retraining and preserve temporal patterns and extreme values while producing profiles consistent with the supplied observations. A large-scale case study using Dutch consumption and photovoltaic generation profiles reports lower mean Continuous Ranked Probability Score (CRPS) than the selected task-specific baselines for imputation and 16-fold super-resolution. On a public dataset of 2,400 residential meters, the same weights match the imputation CRPS of a task-trained diffusion model when sampled ancestrally, attain a 4.5% lower super-resolution CRPS with the default sampler, and meet the specified monthly maximum and mean power within 0.01 kW.
comment: 10 pages
♻ ★ Evaluating Rollout-Verified Reinforcement Fine-Tuning of a Reasoning Model for Multi-Zone Variable-Air-Volume Control
Heating, ventilation, and air-conditioning systems account for a large share of building energy use. High-performance control often requires building-specific modeling or training, limiting scalability. We investigate whether a frontier reasoning model -- a large language model trained to use additional inference-time computation -- can coordinate multi-zone variable-air-volume control without building-specific training, and whether rollout-verified reinforcement fine-tuning can improve a smaller, locally deployable model. Five controllers were evaluated over five summer days in a physics-based four-zone emulator. Across three runs, GPT-5 used 6.8-7.4% less electricity than a Guideline 36-based baseline, but used less outdoor air and had lower CO$_2$ compliance. GPT-5 provided a capability reference for fine-tuning the smaller model. A trained reinforcement learning controller used 4.5% less electricity than the baseline while improving temperature and CO$_2$ compliance. To assess this controller's critic as a lower-cost verifier, we compared its estimates with deterministic rollout returns. Across 470 candidates, their correlation was 0.9995, yet the critic selected the best action in only 7 of 30 states. High overall correlation did not ensure reliable ranking at the same state. We fine-tuned gpt-oss-20b using rollout returns as rewards. One 200-step run produced no sustained improvement in the returns of sampled actions and no improvement in closed-loop performance. Analysis of the model's reasoning traces and quantitative tests showed that its next-state predictions of temperature and CO$_2$ were unreliable both before and after fine-tuning. Because next-state prediction is important for decision-making, these findings motivate testing whether supervised fine-tuning on next-state prediction improves subsequent rollout-verified reinforcement fine-tuning.
comment: Revised and expanded version. 46 pages, 18 figures. Published in Energy and AI
♻ ★ Zonotope-Based Elastic Tube Model Predictive Control
Tube-based Model Predictive Control (MPC) is a widely adopted robust control framework for constrained linear systems under additive disturbance. The paper is focused on reducing the numerical complexity associated with the tube parameterization, described as a sequence of elastically-scaled zonotopic sets. A new class of scaled-zonotope inclusion conditions is proposed, alleviating the need for a priori specification of certain set-containment constraints and achieving significant reductions in complexity. A comprehensive complexity analysis is provided for both the polyhedral and the zonotopic setting, illustrating the trade-off between an enlarged domain of attraction and the required computational effort. The proposed approach is validated through extensive numerical experiments.
♻ ★ Control of Human-Induced Seismicity in Underground Reservoirs Governed by a Nonlinear PDE-ODE System
Induced seismicity caused by fluid extraction or injection in underground reservoirs is a major challenge for safe energy production and storage. This paper presents a robust output-feedback controller for induced seismicity mitigation in geological reservoirs described by a coupled 3D PDE-ODE model. The controller is nonlinear and robust (MIMO Super-Twisting design), producing a continuous control signal and requiring minimal model information, while accommodating parameter uncertainties and spatial heterogeneity. Two operational outputs are regulated simultaneously: regional pressures and seismicity rates computed over reservoir sub-regions. Closed-loop properties are established via explicit bounds on the solution and its time derivative for both the infinite-dimensional dynamics and the nonlinear ODE system, yielding finite-time or exponential convergence of the tracking errors. The method is evaluated on the Groningen gas-field case study in two scenarios: gas production while not exceeding the intrinsic seismicity of the region, and combined production with CO$_2$ injection toward net-zero carbon operation. Simulations demonstrate accurate tracking of pressure and seismicity targets across regions under significant parameter uncertainty, supporting safer reservoir operation while preserving production objectives.
♻ ★ Why the Swing Parameters Are Sloppy
When a multi-parameter model is fitted to measured data, only a few parameter combinations are often identifiable, as Fisher Information Matrix eigenvalues span several orders of magnitude. Sensitivity-based tools, including the Fisher Information Matrix and MBAM, reveal this structure analytically, while manifold learning can recover it from sampled trajectories. For an infinite-bus synchronous generator, both approaches show that inertia H and damping D are among the least identifiable parameters on a late post-disturbance window, t in [3,5] s, where six of the eleven parameters suffice to describe the response. We explain this result from the geometry of the dynamics rather than from that of the parameter-to-data map. The rotor swing is a fast fibre that has collapsed onto a two-dimensional normally attracting invariant manifold (NAIM) of the transient fluxes before the window opens. The zeroth-order NAIM is exactly independent of $H$, $D$, $T_{d0}''$ and $T_{q0}''$, four of the five MBAM limits. We prove, under stated smoothness and rank conditions, that each of the four limits is a boundary of the model manifold at a distance equal, to first order in the parameter, to its current profiled speed. The same analysis predicts how identifiability changes with the experiment: neither a larger initial speed deviation nor a load change restores H or D in the late window, whereas the early window [0, 2]s makes H the best-determined parameter. Evaluated in a physical metric fixed by synchrophasor accuracy limits, the late window identifies five parameters rather than six and the early window nine. The spiked-covariance structure of the linearised trajectory map explains why the principal components of whitened, sampled trajectories reproduce the prior-scaled Fisher spectrum, and regression on those components attains the Fisher-predicted recovery error in 113 of 114 parameter scenario pairs.
♻ ★ On the Degree of Safety: Beyond Safe or Unsafe with Control Barrier Functions
A valid control barrier function (CBF) certifies if its represented safe set can be rendered forward invariant, and the sign of its value indicates whether a state is safe or not, but it does not quantify a degree of safety beyond the binary indication. In this paper, we show that among valid CBFs representing the same safe set, interior values and gradients can be changed arbitrarily, so neither quantity determines a degree of safety that is independent of how the set is represented. We also show that whether a candidate CBF-based inequality constraint is feasible does not by itself quantify a degree of safety. In particular, infeasibility can occur either because the safe set is not controlled invariant or because the candidate CBF representation fails. This motivates our distinction between intrinsic and representational infeasibility. Finally, we introduce the invariance authority demand (IAD), a representation-independent degree of safety that quantifies the control authority required for maintaining safety and can be used to guide actuator redesign or safe set repair.
♻ ★ Two-impulse Rendezvous Planning about Thrusting Spacecraft on SE2(3)
The classical Hill--Clohessy--Wiltshire equations assume an unforced Keplerian reference trajectory, an assumption that is violated by missions requiring continuous thrust. We formulate spacecraft relative motion on the $\mathrm{SE}_2(3)$ Lie group, which encodes position, velocity, and attitude in a single geometric state, linearize the gravity and body-frame control mismatches between the two vehicles, and derive analytic bounds on every neglected term. Under circular coasting Keplerian assumptions the framework recovers the Hill--Clohessy--Wiltshire equations exactly. For thrusting references the state transition matrix acquires thrust-induced attitude--translation coupling blocks that, when the two vehicles share body-frame inputs, are exact in log coordinates and independent of the attitude error, and a mismatch between the vehicles' inputs re-centers the linearization at their mean. A two-impulse rendezvous planner follows directly. A scalar coupling ratio, computable before planning, separates two regimes. Where it exceeds unity, the planner reduces the terminal error of coupling-blind planners by more than an order of magnitude, and somewhat above unity it is also more accurate than a Cartesian planner of the same computational structure; well below unity, simpler planners are adequate or better. Simulations confirm the recovery to machine precision and validate every bound along the transfer.
♻ ★ Log-linear Dynamic Inversion for Thrusting Spacecraft on SE2(3)
Body-fixed spacecraft thrust couples attitude tracking error to translational motion because the applied inertial acceleration depends on spacecraft attitude. In this work, we formulate the dynamics of a continuously thrusting spacecraft on $\mathrm{SE}_2(3)$ to make this coupling explicit while retaining a structured form suitable for analysis. We represent attitude, velocity, and position within a single Lie-group state and derive the logarithmic dynamics of a left-invariant tracking error with position-dependent gravity. The resulting dynamics show explicitly how attitude error affects translational velocity through the applied acceleration and separate the structured linear dynamics from nonlinear contributions due to body-frame input mismatch and position-dependent gravity difference. We specialize the gravity contribution to the two-body model and derive an analytical bound in terms of the reference orbital radius, logarithmic position error, and attitude error. Numerical simulations of a continuous-thrust maneuver in a highly elliptical Molniya-like orbit show agreement between the derived logarithmic error dynamics and errors reconstructed from independent nonlinear state propagation.
♻ ★ BarrierFormer: Transformer-Guided Predictive Barrier Enforcement for Safe Robot Control
Control barrier functions (CBFs) have become one of the most popular tools for encoding and enforcing state constraints in safety-critical robotics. Standard CBF approaches are inherently myopic in nature as they enforce safety only at the current time step. Consequently, the system can be driven toward the boundary of the safe set where no feasible safe control exists at a future timestep. Model predictive control (MPC) based approaches address this by enforcing state constraints over a receding horizon. However, such approaches generally require the model to be known for solving a constrained optimization problem at every step, which is computationally expensive for real-time deployment. We propose BarrierFormer, a barrier-supervised transformer framework that addresses these limitations by encoding rollout-level CBF constraints in learning a model-free safe policy. A causal transformer encodes observation-action history, autoregressively generates a predictive rollout through the dynamics head to replace the model, and provides a residual correction to a nominal controller through the action head to replace the online computation. A barrier critic operating on local observations evaluates CBF constraint violations along this rollout, and a safety teacher computes barrier-consistent actions satisfying these constraints as direct supervision targets for the learned control policy. During inference, the policy maps observation-action history to control actions without any online optimization or model knowledge, enabling real-time model-free predictive safety enforcement. Evaluations across linear and nonlinear, 2D and 3D dynamical systems for safe goal-directed navigation demonstrate that BarrierFormer outperforms existing reinforcement learning (RL)-based, diffusion-based, MPC-based, and transformer-based approaches in safety rate and inference latency.
comment: 23 pages, 2 figures, Accepted at 10th Conference on Robot Learning (CoRL 2026), Austin TX, USA
Robotics
★ Kairos: Semantics-Aware Orchestration of Planning Engines across Classical, Numeric, Temporal and Hybrid PDDL
Planning at run time, as in robots that re-plan while they act, needs a planner that reads the PDDL its users write (durative actions, numeric resources, time windows, continuous change), answers quickly, and delivers only validated plans. Specialised planners are strong on individual fragments of PDDL, but they differ in coverage, latency and in what their failures mean. We study whether selecting heterogeneous engines from the semantics of each task can give broad coverage of PDDL, from STRIPS to PDDL+, while remaining competitive with specialised planners and making explicit what each answer guarantees. Kairos profiles each task to decide which of five engines apply (ground heuristic search, compressed and snap-action temporal search, time steps for hybrid dynamics, lifted search) and schedules them under one budget of time and memory. Every candidate plan is checked by a validator independent of the planner's transformations, and "unsolvable" is reported only when a search whose exhaustion is a proof exhausts its space. On 1,756 tasks from nine comparative suites, Kairos solves more tasks at the full limits than the strongest evaluated planner of each suite, with the largest differences on numeric, temporal, PDDL3 and PDDL+ planning, and it also solves the most tasks at 100 ms and 1 s. Resampling over domains, the gain is above zero in six of the nine suites, while on classical planning against LAMA it is within the noise. A further 450 feature-oriented tasks and 26 one-feature probes show that Kairos handles all seventeen features probed, where no other planner evaluated handles more than eight. Ablations identify compressed temporal search and the novelty fallback as major contributors, and a comparison with VAL traces every disagreement to a PDDL+ convention or to a documented VAL defect. Planner and benchmark are open source.
comment: 32 pages, 3 figures, 11 tables. Submitted to the Journal of Artificial Intelligence Research. Code: https://github.com/PlanSys2/kairos ; benchmark, scripts and results: https://github.com/PlanSys2/kairos-benchmarks
★ Smooth Action Chunking Via Movement-Primitive Representations for Imitation Learning
Imitation learning has become a widely adopted approach for vision-based robot policies, owing to its conceptual simplicity, scalability, and ability to leverage large-scale datasets. As these policies have grown more capable, action chunking has become their standard interface. At each decision step the policy outputs several future actions at once. Backbones from diffusion policies to large vision-language-action models all share it. Whatever backbone produces the chunk, its temporal structure is learned only implicitly. The commanded trajectory is therefore jerky both within a chunk and between successive chunks. On real hardware this appears as visible vibration and as repeated saturation of the follower's velocity limit. We address this issue by incorporating a movement-primitive-based trajectory representation into action chunking. Specifically, a trajectory segment is represented as a linear combination of smooth basis functions, yielding smooth trajectories by construction while leaving the policy architecture, training objective, and inference procedure unchanged. The same representation also yields a closed-form, inference-time correction for the second source of jerk. We condition the predicted coefficients so that the new chunk passes through a via-point at the previous chunk's last commanded action. This correction needs no retraining and, unlike asynchronous Real-Time Chunking, applies to policies queried in one shot. We instantiate it on a one-shot (ACT) and an iterative (DiT) backbone and evaluate in simulation and on a real bimanual ALOHA platform. The representation lowers within-chunk jerk by an order of magnitude on the one-shot backbone and by 2.4 times on the iterative one, with task success statistically non-inferior. The conditioning then removes 73-84% of the boundary discontinuity, at negligible per-step cost.
★ Ask to Evolve: A Unified Harness for Lifelong Embodied Navigation with Hybrid Action Space and Persistent Memory
Giving embodied agents the freedom to act autonomously, initiate interactions, and understand their surroundings through experience is a step toward bringing them to life in the physical world. Realizing this vision requires action, interaction, and memory to work together, so that each encounter can inform current decisions and contribute experience for future tasks. We present EvoLife, a unified navigation harness that enables a multimodal embodied agent to choose how to move, when to ask, and what experience to carry forward. Its hybrid action space supports waypoint navigation, local discrete adjustments, pixel-based navigation, and object-directed navigation. Active interaction allows the agent to seek clarification or assistance when it encounters ambiguity or difficulty, while persistent spatial memory binds observations and dialogue to places, retaining useful experience to guide subsequent visits. We evaluate eight benchmarks across four task families: vision-language navigation, object navigation, interactive instance navigation, and continuous multi-task navigation. With an average of only 1.87 active interactions per episode, our method achieves a first-visit success rate (SR) of 100% across four 100-episode sets spanning vision-language navigation and object navigation. Complete-sequence SR reaches 98% on 50 LHPR-VLN sequences and 94% on 50 GOAT-Bench sequences. In the R2R revisit setting, success weighted by path length (SPL) increases from 82.23% to 93.31% with retained experience. Multi-day physical navigation further demonstrates exploration, proactive help-seeking, and revisiting remembered places. Together, these results advance the vision of bringing embodied agents to life through autonomous action, proactive interaction, and the accumulation and reuse of experience.
★ ScheduleGen: Demonstration Generation via Skill Scheduling for Multi-Arm Imitation Learning
Multi-arm robot platforms, including bimanual and humanoid robots, have the capability to complete tasks with lower cycle times than single-arm systems, but they are more difficult to safely control due to the presence of multi-arm constraints. The paradigm of training generalist robot policies from massive datasets is a promising approach to address general-purpose multi-arm manipulation, but the acquisition of large multi-arm manipulation datasets remains a bottleneck. Automated demonstration generation can alleviate this burden by synthesizing new demonstrations from a handful of source demonstrations, but existing algorithms ignore multi-arm constraints and as a result often induce unsafe and unsuccessful behaviors. We address this limitation by introducing ScheduleGen, an integrated planning & scheduling algorithm for multi-arm demonstration generation. We evaluate ScheduleGen on 28 simulation task variants, including seven externally authored RoboDojo tasks and settings inaccessible to prior systems; it achieves 79.6% mean data-generation success and trains policies with 65.5% mean success. ScheduleGen policies outperform those trained on a state-of-the-art data generation baseline, and achieve 50-80% success across eight real-world variants through zero-shot sim-to-real and real-to-sim-to-real transfer. Videos and more at: https://schedulegen.github.io/
comment: 22 pages, 9 figures, 5 tables. Project website and videos: https://schedulegen.github.io/
★ Decoding Geometry, Not Tokens: Efficient VLMs for Robot Affordance and Trajectory Prediction WACV 2027
Predicting where to grasp and how to move is fundamentally a geometry problem. Vision-language models treat it as language generation instead. They tokenize spatial coordinates into text and decode them autoregressively. This mechanism ignores spatial structure and truncates continuous precision. Errors compound across coordinate tokens during decoding. In this paper, we show that compact off-the-shelf VLMs with 0.5-2B parameters already encode rich spatial information in their intermediate hidden states. Autoregressive decoding is the dominant bottleneck for trajectory prediction, while affordance prediction depends additionally on feature depth and decoder architecture. We expose the latent spatial information with a lightweight, geometry-compatible prediction head that attaches to a frozen backbone. It reads features from an intermediate hidden layer, conditions a learned query on the task instruction and refines it via cross-attention over the visual tokens to regress affordance boxes and trajectory waypoints directly as continuous values, without modifying the backbone. We evaluate on ShareRobot-Bench and AGD20K. Our 2B variant reduces trajectory error by over 60% and improves ShareRobot-Bench affordance mAP by 85% relative to the autoregressive baseline. It approaches RoboBrain on trajectory prediction (0.251 vs 0.191 DFD) at 3.8x higher throughput. These results demonstrate a favorable accuracy-efficiency trade-off for instruction-conditioned 2D geometric prediction.
comment: IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2027)
★ UMI-Live for In-The-Wild Bimanual Robotics Data Collection
Universal Manipulation Interface (UMI) devices allow human experts to demonstrate tasks for robot learning without teleoperating physical robots. Using these demonstrations as training data still requires a consistent hardware setup, calibration, and data processing. We present UMI-Live, a full-stack open-source bimanual UMI system for in-the-wild data collection. We design our custom UMI-Live hardware to integrate camera and LiDAR--inertial sensing, onboard compute, and power, without external tracking equipment or calibration within the collection environment. We implement a simple user interface for recording data while an asynchronous pipeline performs parallel SLAM reconstruction and publishes training-ready datasets. We also release software that supports dual-YAM teleoperation and VLA policy training and inference. We validate UMI-Live data quality by fine-tuning $π_{0.5}$ and MolmoAct2, achieving success rates of 10/10 and 8/10, respectively on a bimanual pick-and-place task. Furthermore, we release 40 hours of UMI-Live data across three tasks, and complete hardware designs, software, and reproduction guides at our $\href{https://umi-live.github.io/}{\text{project website}}$.
★ Information-Driven Collaborative Autonomy for Heterogeneous UAV/UGV Teams
This paper considers coordinating a heterogeneous team of unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs) traversing a domain containing an unknown number of potentially hazardous targets. The UAVs provide aerial sensing using onboard optical cameras, adapting their trajectories to ground routes and target information. We formulate the aerial search problem using ergodic control, which distributes sensing effort according to a spatial information density to balance exploration and target localization. The formulation incorporates dynamic camera field-of-view footprints and pose-dependent probabilistic detection models. Angles-only measurements are processed through Bayesian inference and marginalization over data-association hypotheses to obtain a Poisson multi-Bernoulli (PMB) approximation of the target posterior, representing both detected and undetected targets. This belief directs aerial sensing toward unexplored regions along the ground routes and viewpoints that improve target localization. Ground planning combines chance-constrained target avoidance, enforced through confidence sets, with a void-probability objective that favors routes unlikely to contain threats. We solve each team's optimization problem in a fully decentralized manner using a nested SCvx*--C-ADMM scheme, with outer sequential convex programming iterations and inner distributed consensus optimization, accounting for nonlinearities, nonseparable objectives, and continuous-time inter-agent separation constraints. A closed-loop scheme updates both fleets' trajectories as information becomes available and permits ground motion when the remaining route risk falls below a prescribed threshold. Numerical case studies examine sensing geometry and target uncertainty, compare decentralized and centralized solvers, and demonstrate full closed-loop collaboration between multiple UAVs and UGVs.
★ Limit Theorems and Optimal Policies for Flock Flight through Poisson Forests
Inspired by birds flying through cluttered environments, Karaman and Frazzoli showed that high-speed flight through an ergodic random forest exhibits an ultimate speed limit: above a critical speed, almost surely no infinite collision-free trajectory exists for a single bird. This paper formulates a corresponding bulk-transport problem for many birds. We construct an explicit directed corridor network in a Poisson disk forest and impose a common temporal headway at its waypoints. The maximum current equals a minimum vertex-cut capacity and is achieved by repeated flight along vertex-disjoint paths. The same schedule maximizes completed traversals at every finite horizon, with constant forward speed and no internal waiting. For a forest belt of fixed depth, exact transverse superadditivity yields an almost-sure deterministic current-density limit. A uniform shape theorem for an asymmetric planar dual identifies the current when depth and width grow together. For diverging depth bounded by a constant times width, this current equals the width-first, depth-second limit and is independent of the positive aspect ratio. When diagonal corridors are available, bulk current is positive at sufficiently low obstacle intensity and zero at sufficiently high intensity. For stochastic demand, an upstream admission queue is stable at every rate below the finite-network capacity.
★ Utility-WAM: Correcting Future-Induced Action Changes
World Action Models (WAMs) can condition robot actions on generated visual futures, but this additional context is neither cost-free nor uniformly beneficial. We examine what a predicted future actually changes: the action delta $Δa$ between the future-conditioned and future-free execution paths of the same model. Under a shared WAM checkpoint, this delta barely shifts average benchmark success ($+0.8$ points on LIBERO, $-0.8$ on RoboTwin~2.0), while generating visual futures increases inference latency by 1.7--1.9$\times$. These aggregate metrics conceal a concentrated failure mode that we term \emph{future-induced negative transfer}: while predictive deltas benefit long-horizon tasks ($+2.4$ points on LIBERO-Long) and remain negligible on most routine manipulations, they induce severe degradation on precision tasks---with \emph{Press Stapler} and \emph{Place Can Basket} losing 54.0 and 34.0 points, recurring as top failure modes across independently trained WAM variants. We introduce \textbf{Utility-WAM} for in-place correction of future-conditioned control. Its primary learning component is a residual adapter that repairs future-induced velocity deltas under a frozen shared WAM by aligning them with ground-truth dynamic guidance and penalizing excess error beyond the future-free baseline. The adapter selectively eliminates the majority of severe task drops (reducing losses to 6.0 and 14.0 points) while preserving long-horizon foresight. To eliminate superfluous video computation, a lightweight pre-generation gate reuses counterfactual action outcomes to selectively bypass video generation on 75.5\% of RoboTwin queries, cutting latency by 38\% while improving average success to 93.3\% on RoboTwin~2.0 and 98.9\% on LIBERO. On physical bimanual manipulation across three real-world tasks, Utility-WAM succeeds in 53 of 60 trials, compared to 43 for FAST-WAM.
★ SONIC-K1: Porting a Universal Humanoid Motion Tracker to a Small Humanoid on a Single GPU
We present SONIC-K1, to our knowledge the first open-weights whole-body motion tracker for the Booster K1 humanoid, obtained by adapting the SONIC framework to this smaller robot. The policy is trained from scratch on a single shared GPU with a small fraction of the compute reported for SONIC. In MuJoCo simulation, it tracks nearly all held-out motion clips and performs well on external benchmarks under SONIC's success criterion. On the real K1, it tracks upper-body and slow motions, while stepping and turning remain less reliable, and simulation experiments show a sensitivity to action latency. These results show broad motion-tracking capability on a modest training budget and point to quantitative hardware evaluation as the main next step. Code, policy weights and evaluation tools are publicly available.
comment: Project page: https://kyry.me/sonic-k1
★ ProLoop: Progress-Aware Closed-Loop Framework via Structured State Relations for Long-Horizon Robotic Task Planning
Long-horizon robotic tasks are commonly decomposed into semantic subgoals to simplify planning. However, subgoal progress in LLM-based planning is typically inferred implicitly from interaction histories. Inaccurate estimates can misguide execution and allow local errors to propagate. To address this, we propose ProLoop, a Progress-Aware Closed-Loop framework that derives explicit subgoal progress from structured relations between desired and execution states. A learned State-Relation Monitor assesses these relations, which are then aggregated into subgoal progress as ongoing, deviated, or completed. The resulting progress guides execution adaptation, enabling reliable progress-aware control throughout execution. Experiments on AgentBoard ALFWorld and a real-world robotic platform show that ProLoop improves success rate by 12.94% overall and by 15.46% on Hard tasks, while reducing environment interactions by 32.3%.
★ Language-Grounded Force/Torque Pretraining for Extrinsic Contact Perception
Fine manipulation requires robots to recognize when a grasped object makes or changes contact with its environment, yet such extrinsic contact is not always directly observable from vision or local contact geometry. Force and torque signals carry this information, but also vary with object weight, grasp configuration, and robot motion, leaving it unclear whether they can provide a reusable representation of physical state. Here we show that bilateral six-axis force/torque histories can be pretrained as a semantically structured physical representation by aligning them with a frozen language space using supervision generated from simulated pick-and-place interactions. On unseen objects in simulation, the learned representation supports text-prototype readout of robot action, object weight, and grasp interaction. In real-world placement, unlabeled calibration enables contact and support-state recognition without updating the encoder, outperforming a vertical-force sign-reversal rule baseline. For connector insertion, an event absent from pretraining, low-rank adaptation improves data-efficient learning over training from scratch. These results show that force/torque histories can support both direct readout of known physical states and efficient adaptation to new extrinsic-contact events.
comment: 9 pages, 5 figures
★ OpenFlyV2: An onboard vision-language-action model for aerial perception and navigation
Human cognition flexibly draws on a shared perceptual foundation to understand environments, attend to referenced entities, reason about spatial relationships, and act. Endowing aerial robots with an analogous functional organization remains challenging, as existing systems are often specialized for individual capabilities, dependent on offboard computation, or unable to sustain closed-loop operation on resource-constrained platforms. We present OpenFlyV2, a unified onboard vision-language-action architecture that uses a shared vision-language representation to support semantic understanding, language-grounded perception, spatial reasoning, and continuous aerial control within a single model. A compact action expert converts the shared multimodal context into short-horizon flight waypoints, while motion-aware temporal memory and action priors reduce the computational cost of closed-loop execution. We construct \dataset{}, a simulation-based dataset comprising approximately 70,000 navigation trajectories and 60,000 vision-language instruction--response pairs. OpenFlyV2 generates action chunks, each comprising 12 waypoints, entirely onboard at 10 Hz using an NVIDIA Jetson Orin NX. It transfers zero-shot from simulation to physical quadrotor flight, achieving a 48\% navigation success rate across four real-world environments without real-world aerial training data. Extensive results demonstrate that broad vision-language competence and continuous closed-loop flight can coexist within a single model on a resource-constrained aerial platform. Code will be made publicly available at https://github.com/Ideal-ljl/AeroBrain.
comment: 12page, 6figures
★ PriorMaskMap: Robust Online Vectorized Map Construction with Biased Priors ECCV2026
Online vectorized map construction provides autonomous vehicles with essential, real-time semantic and geometric scene understanding. While leveraging prior maps presents an opportunity to improve accuracy, they are often biased due to outdated information or temporary road changes. Existing methods, which typically treat priors as fully reliable or only consider limited noise patterns, suffer severe performance degradation when these priors are imperfect. To address this, we propose PriorMaskMap, a robust framework for prior-guided vectorized map construction. Its first innovation is a Prior-Aware Confidence Estimator (PACE) that evaluates the input rasterized map against current visual observations. It assigns confidence scores to regions classified as reliable, erroneous, or blank, creating a foundational mask that guides two subsequent branches. In one branch, the Dual-Prior Query Generator (DPQG) selectively filters and embeds both rasterized and vectorized priors to initialize informative queries from trustworthy map structures. In the other, the Confidence-Guided Prior Fusion (CGPF) module spatially propagates and refines the mask, using the refined version to govern the fusion of BEV features with the prior map, thereby providing rich contextual representations for the map decoder. Evaluation on the nuScenes dataset shows that PriorMaskMap achieves superior mapping precision over existing approaches and exhibits marked robustness when provided with priors containing apparent errors. Our code is released at https://github.com/healenrens/PriorMaskMap.
comment: Accept by ECCV2026
★ Cross-Controller Transfer via In-Context Behavior Adaptation
Generalist robot policies increasingly learn from data collected across diverse tasks and embodiments, yet typically assume a single controller for each robot. This assumption limits transfer because different controllers can assign different physical meanings to the same action and may require different strategies to complete a task. We study how to transfer robot data across both embodiments and controllers without allowing conflicting action interpretations to interfere with learning. Our setting spans six controller types as well as variations in controller gains and compliance. We propose a framework that encodes behavioral interaction under a controller through examples of commanded actions and the resulting robot motions. A transformer context encoder summarizes these interactions, allowing a single policy to adapt in context. Across real-world experiments, our method improves average success by 33 percentage points over naive data pooling, achieves 50\% success on unseen controller modes and 52\% on unseen controller gains, and transfers to tasks with limited controller-specific data. On contact-rich tasks such as USB insertion and surface wiping, behavioral interaction context enables the policy to adapt not only its actions but also its strategy. Our results provide a path toward generalist robot policies that can leverage data collected across diverse action interpretations. For more information, visit \href{https://controller-transfer.github.io}{https://controller-transfer.github.io}.
comment: 9 pages, 6 figures
★ CuBot-STT: Simultaneously Transport and Tighten for Lattice-type Modular Reconfiguration Robot
Lattice-type modular reconfigurable robots rearrange standardized modules across tasks. Dedicated transport robots reduce module weight and cost by eliminating module level reconfiguration actuators. But existing systems use low capacity connections or internal fastening robots that occupy space for functional components. This paper presents CuBot-STT, which integrates module transport, interface locking, and release in one external robot. Cubic modules provide structural support and $24\,\mathrm{V}$ power distribution. On each face, a gear transmission converts one external input into motion of four helical locking points, preserving internal space. The robot uses alternating foot support, a gripper for module handling, and a tightening head for interface actuation. The controller coordinates two-foot support and module locking through continuous constraint transfers. The planner decomposes relocation costs into an analytical lower bound and a residual learned through expert-guided, goal conditioned Q-learning. Contact face $A^*$ search expands relocation sequences into robot primitives satisfying connectivity, asymmetric manipulation, and foot stance constraints. Experiments demonstrate normal and tangential interface capacities of $300\,\mathrm{N}$ and $490\,\mathrm{N}$, electrical continuity across six modules, and autonomous reconfiguration of an 11-module lattice. Planning benchmarks show superior search efficiency over analytical guidance and bidirectional heuristic search, with generalization beyond training lattice sizes without fine-tuning.
comment: 11 pages. Submitted to IEEE/ASME Transactions on Mechatronics
★ WorldFlow: Joint Video and 3D Motion Prediction via Large-Scale Pretraining
Predicting future physical states stands as a central challenge in embodied intelligence. Existing video world models leave 3D dynamics implicit, action-based formulations remain tied to specific robot configurations, and 3D motion prediction models use flow-only supervision, providing a limited learning signal for physical dynamics. To circumvent these limitations, we introduce \method{}, a generative world model that jointly predicts future video and query-based 3D trajectories. By parameterizing 3D motion as continuous displacement tokens aligned with the video grid, a unified Mixture-of-Transformers (MoT) backbone jointly denoises both modalities under a rectified-flow objective. This formulation couples explicit geometric dynamics with visual evolution, effectively unlocking the physical commonsense embedded within pretrained video models. To empirically scale and validate this unified architecture, we train \method{} on a heterogeneous mixture of 1.44 million object-level samples spanning simulated and real-world domains. We further introduce WorldFlowBench, comprising 1,678 evaluation samples from 11 data sources across simulated and real-world interactions, including spatial reasoning tasks. Across all three evaluation sets, WorldFlow achieves the lowest average displacement error (ADE) and final displacement error (FDE) among the evaluated learning-based methods. Notably, on the challenging SpatialCog set, WorldFlow achieves an ADE of 0.060 m, a 54.2\% reduction compared with the strongest external baseline. It also reduces ADE and FDE by 55.9\% and 68.5\%, respectively, compared with its flow-only variant. Project page: https://worldflow-rbs.github.io/.
★ RAIBO-Wagon: Real-Time Navigation System for a Legged Robot Towing a Wagon
Legged robots have demonstrated advanced capabilities across a wide range of real-world tasks, yet much of this progress concerns the robot controlling only itself. Once the robot is coupled to an external object -- particularly an underactuated one -- the dynamics of the combined assembly become more complex. Towing a wagon is a common practical scenario, where coupled dynamics make reliable navigation challenging. This difficulty is amplified in confined spaces, where collision-free navigation requires finding a narrow set of feasible paths under strict real-time constraints. To address this, we propose a navigation framework that combines Conditional Flow Matching (CFM) with Model Predictive Path Integral (MPPI) control over a learned Forward Dynamics Model (FDM). The CFM model mitigates the initialization problem of MPPI and accelerates its convergence, while MPPI with the FDM recovers feasible solutions beyond the CFM model's training distribution by iterative refinement. We integrate the framework into a full-stack navigation system for RAIBO-Wagon, a quadruped towing a custom wagon, and validate it in real-world outdoor experiments on 15 m-long courses through confined spaces, with all sensing and computation performed fully onboard. The system achieves an 85% success rate across 40 trials, towing a total load of up to 40 kg -- nearly 100% of RAIBO's body weight.
comment: 19 pages, 13 figures, 5 tables. Project page: https://insungyang.github.io/raibo-wagon/
★ Seeing and Imagining as One: Unified Video Generation and 4D Perception for Embodied World Modeling
Embodied world models should not only imagine plausible futures, but also perceive the geometric, semantic, and motion structure of the evolving world. We present GenSight, a unified embodied world model that brings future video generation and dense perception into a single pretrained video diffusion transformer. GenSight uses a unified spatiotemporal representation for video, depth, embodiment segmentation, and first-frame-relative 2D flow. Built on a shared DiT backbone, GenSight uses lightweight task modulation to predict all modalities within the same architecture. All tasks are jointly optimized within a unified velocity prediction space. At inference, GenSight supports two complementary modes: multi-step future video generation with aligned dense predictions and single-step dense perception from an observed video. By unifying modality representation, model architecture, and prediction space, GenSight avoids separate modality-specific pipelines and couples appearance, geometry, semantics, and motion in a shared representation. It promotes cross-modal consistency and allows dense perception to benefit from the spatiotemporal priors learned through video generation. Trained on embodied manipulation data spanning multiple robot embodiments and human-hand interactions, GenSight demonstrates strong performance in both video generation and dense 4D perception, highlighting the effectiveness of unifying seeing and imagining for embodied world modeling.
comment: Project page: https://gensight-rbs.github.io/
★ Event2Reward: Learning Robotic Rewards from Progress Relations
Learning task progress from robot demonstrations provides a practical way to construct dense shaping rewards for robotic reinforcement learning. A common approach uses normalized demonstration time as progress labels, implicitly assuming monotonically increasing task progress. During policy exploration, however, actual task progress can stall or even regress. Such models may mistake regressions for progress, assigning positive shaping rewards to actions that cause progress loss. Real suboptimal and failed executions offer corrective experience, but assigning absolute progress values to their states is difficult. Our key observation is that stalls, regressions, and recoveries establish local relative progress relations even when absolute progress is unknown. We introduce Event2Reward, which uses normalized temporal labels to anchor the progress scale and these within-trajectory relations to constrain local changes, combining absolute and relative progress supervision. Its progress predictions serve as potentials for reward shaping. We curate an event-annotated dataset from approximately 100K raw real-robot execution records, covering successful, suboptimal, and failed executions, without requiring framewise absolute progress labels. Under the same training budgets, Event2Reward improves average policy success by 18.16 percentage points over a baseline using only sparse task rewards across 40 tasks in four LIBERO suites. On real robots, average success after 30 minutes of wall-clock training reaches 60.0% on three precise insertion tasks and 90.0% on three object manipulation tasks, exceeding the sparse-reward baseline under the same budget by 36.7 and 23.3 percentage points, respectively. Ablation studies further show that absolute progress supervision and relative progress relations are complementary, with joint supervision yielding better downstream policy learning.
comment: 44 pages, including references and appendices
★ EmbodiExp: Compositional Experience Transfer for Embodied Agents
Experience can help embodied agents turn high-level plans into grounded actions, but transferring it across tasks remains challenging: previously effective strategies may be inapplicable to the current state or fail to produce their intended outcomes. We present EmbodiExp, a framework for constructing transferable experiences and composing them online during embodied task execution. Our key observation is that tasks often share subgoals whose completion can be assessed independently of overall task success. We therefore introduce an outcome-grounded subgoal experience that specifies a subgoal, applicability conditions, an adaptable policy, and a verifiable outcome. During execution, applicable experiences guide action generation, while verified outcomes can establish the conditions required by subsequent experiences, enabling online composition. Subgoal-level outcome attribution further enables failure utilization: failed trajectories can still contribute verified local successes and condition-specific guidance on actions to avoid. Experiments on multiple embodied benchmarks under a fold-based protocol with frozen experience banks demonstrate the effectiveness of EmbodiExp.
★ EgoContact: Evaluating Physical Intelligence with Egocentric Video
Egocentric video has emerged as one of the most scalable sources of data for learning generalizable manipulation priors. Yet, despite its growing use in pretraining robotic policies, it has been underutilized as a validation tool for physical intelligence, a key component of which is understanding where to make contact with an object to achieve an intended interaction. We present EgoContact, a large-scale egocentric dataset for evaluating the generalizability and precision of learned manipulation priors through future contact-point prediction. EgoContact contains 22.5K manually verified interaction events, each with a pre-contact observation, semantic intent, and pixel-precise contact locations. We find that current methods often struggle to predict precise interaction contacts and sometimes fail to localize the target object in diverse, in-the-wild scenes, even when allowed multiple predictions. Furthermore, we demonstrate the usefulness of EgoContact's annotated interactions for robotic manipulation: simply using EgoContact as the memory bank of an existing affordance-transfer policy improves grasp success from 14.2% to 31.2% in simulation and from 16% to 58% on a real robot.
comment: Project page: https://egocontact.github.io
★ Actuator-Aware Observer Inputs for Cascade ADRC of Quadrotors Under Rotor Lag, Measurement Delay and Sensor Noise
In sampled cascade linear active disturbance rejection control (LADRC) of quadrotors, the extended-state observer typically receives the instantaneous command, although the delayed measurement reflects an earlier, rotor-filtered acceleration. This paper proposes actuator-aware (AA) observer inputs: altitude and attitude observers use limited virtual inputs filtered by an assumed rotor model and aligned with the measurement delay, and horizontal observers use acceleration reconstructed from measured attitude. Fixed gains are selected on the discrete-time model under sensitivity and step-response requirements and stationary covariance ceilings on each channel's control variance and increment and on the noise-induced inner-loop increment of the full cascade. In paired set-point trials, PID-Budget, AA-Budget and AA-HalfGain reach mean RMSEs of 65.5, 19.2 and 16.9~mm; under a stricter common cascade ceiling, the AA designs reach 29.9 and 26.4~mm versus 72.3~mm for a PID that uses roughly double their rotor-command increment. PX4 software-in-the-loop runs reproduce these results. Behind EKF2, a noise model fitted to the estimation errors predicts the roll/pitch and yaw increments within 5\% on held-out and out-of-sample runs and the total rotor-command increment within 14\%. Redesigned for this stack under the same sensitivity and response specification and estimator-aware ceilings, AA-Budget-E reduces the set-point RMSE from 59.1~mm (PID-E) to 44.0~mm with a lower rotor-command increment in every paired set-point and lemniscate run (mean S6 rotor-command-increment reductions relative to PID-E: 45\% before and 15\% after output quantization); AA-HalfGain-E beats PID-E only on the lemniscate. The evidence is simulation-based, and the ceilings are nominal tuning constraints.
comment: 19 pages. MATLAB and PX4 SITL evaluation. Code, controller gains, random seeds, and per-seed results: https://github.com/Quang61123/actuator-aware-cascade-adrc
★ Tracking Task Progress Through Errors and Recovery:A Benchmark and Recurrent Distributional Model
Operational strategies for long-horizon robotic tasks require reliable evaluation to determine whether execution is progressing, stalling, regressing, or recovering from errors. Yet real- world robot datasets that capture error recovery and provide frame-level error annotations remain scarce. To address this gap, we introduce and open-source ReSTEP, a dataset of 3,156 episodes across two manipulation tasks with semantic stage labels and frame-level er- ror annotations. ReSTEP supports training and benchmarking progress monitors through successful execution, mistakes, and recovery. Existing progress estimators can struggle to distinguish visually similar task stages and maintain temporally consistent predictions while responding to genuine setbacks. We therefore propose REMORA, an online progress esti- mator that combines recurrent visual memory, distributional prediction, and causal peak selection. Memory retains execution history to help resolve visual ambiguity, while distribu- tional prediction represents alternative progress hypotheses and causal peak selection tracks a supported hypothesis over time. Together, these components enable consistent progress tracking that remains responsive to errors and recovery. Experiments on both ReSTEP tasks demonstrate that REMORA outperforms the state-of-the-art SARM baseline in progress ac- curacy and temporal consistency.
★ CAPTURER: A Dexterous Aerial Manipulator for Dynamic and Precise Grasping across Diverse Environments
As a representative form of embodied robotic interaction, aerial grasping requires close coordination between the aerial platform and manipulator. Most existing systems perform aerial grasping with static or slowly moving targets under fixed scenarios. Dynamic and precise grasping across different conditions introduces additional challenges, including target motion, localization drift, dynamic coupling, and external disturbances. In this work, we present CAPTURER, a dexterous aerial manipulator for centimeter-level grasping across diverse environments. To achieve the challenging goal, we first develop an onboard perception system for state estimation and target detection. Drift correction and global pose transformation are introduced to improve target localization during flight. A cooperative motion planner is proposed to coordinate the mobility of the aerial platform with the dexterity of the manipulator according to the relative target state. To maintain grasping accuracy under disturbances, we design a disturbance-rejection controller to compensate for dynamic coupling, model uncertainty, and external disturbances. The capability of CAPTURER is validated through indoor and outdoor experiments with both static and moving targets. The results demonstrate that CAPTURER achieves centimeter-level grasping accuracy for a marker pen measuring only 1.5 cm in diameter at flight speeds exceeding 1.5 m/s.
★ Tactus: Tactile & Pose Sensing Glove for Natural Dexterity Data Collection
Collecting naturalistic human manipulation demonstrations with accurate contact force and hand pose is essential for robot dexterous manipulation policy and foundational world model learning - yet no existing wearable platform provides both calibrated fingertip force sensing and occlusion-resilient hand tracking in a form factor that preserves natural manipulation behavior. We present Tactus, a textile-integrated sensing glove that co-optimizes three design pillars: (1) a thin, conformal 3D-knit form factor (1.5 mm fingertip stackup) designed to minimize interference with natural manipulation, (2) strategically placed capacitive tactile sensors at manipulation-relevant regions informed by empirical contact studies, and (3) IMU-fused pose tracking that maintains accuracy through the hand-object occlusions where manipulation data is most valuable. Tactus includes a documented calibration pipeline for cross-glove consistency and per-session calibration for subject-dependent correction. To enable objective comparison, we introduce standardized benchmarking protocols for tactile force regression and hand pose estimation, and apply them to Tactus and two commercially available tactile gloves - to our knowledge the first quantitative cross-platform evaluation of this kind. Tactus resolves a minimum detectable force of 0.59 N at 95% confidence, 1.6 - 2.7x lower than the commercial baselines, and with 0.87 cross-participant consistency, while our pose tracking sustains 11.0 mm fingertip keypoint relative error during manipulation, when occlusion is heaviest. We show that these combined signals support object property estimation, slip detection, and representative recording of hand-object interaction - yielding demonstration data with the fidelity required for contact-aware manipulation policies and world models.
comment: 9 pages
★ SwapBot: Tool-Centric Robot Manipulation via Interchangeable End-Effectors and Tactile Sensing
Robot manipulation in everyday environments inherently involves substantial tool usage. Seemingly simple household tasks, such as cooking pan-fried tofu, require forceful cutting, delicate brushing, and precise placement using different tools under contact-rich interaction. Thus, the ability to autonomously swap and use task-appropriate tools is important for tool-centric manipulation. We introduce SwapBot, a modular, tool-centric end effector system featuring low-cost adapters that quickly convert common household tools into robot end effectors. Rather than pursuing a single universal gripper, SwapBot adopts a unified design principle that combines a fixed robot-side coupling interface with low-cost, tool-side adapters, enabling compatibility with diverse tools, including articulated ones, and multiple robot platforms. Each adapter integrates compact tactile sensors, supports fast autonomous swapping, and interfaces with an intuitive teleoperation system for efficient demonstration collection. User studies show that, during teleoperation, SwapBot reduces task completion time and operator workload. Using collected demonstrations on the robot, we train imitation learning policies to successfully perform tool-centric tasks like cooking tofu, stirring salad, flipping meat. Our ablation study shows that using tactile-augmented demonstrations improves policy performance across contact-rich skills, increasing success rates by 15 to 25 percentage points over RGB-only policies. These results highlight modular, tactile-equipped tools as a practical and extensible pathway toward robust and capable robot manipulation. Project website https://personalrobotics.github.io/SwapBot.
★ SymAnt: Grounding Long-Horizon Action Anticipation with Adaptable Symbolic Priors WACV 2027
As robots and intelligent systems increasingly interact with humans, they must anticipate long sequences of future actions from only brief observations. While purely neural models typically require extensive observations to predict future actions and remain difficult to interpret and adjust, symbolic priors offer a structured alternative. To this end, we introduce SymAnt, a novel neuro-symbolic approach that unifies the expressiveness of neural networks with the structured reasoning of scene and knowledge graphs. Given a short video, SymAnt performs a joint-search over both graphs to construct a symbolic context, enabling unified reasoning over spatial relationships, object attributes, affordances, and associated actions. This symbolic context serves as a prior for an iterative refinement process that progressively generates and improves the sequence of future actions. By grounding predictions in symbolic context, SymAnt anticipates long-horizon actions from short observation windows in a parameter-efficient manner, while enabling users to directly edit the underlying knowledge to adjust predictions without further training. Through our experiments, we demonstrate the effectiveness of SymAnt, particularly when only short observation contexts are available, across five diverse datasets: Breakfast, 50 Salads, EPIC-KITCHENS, EGTEA Gaze+, and Assembly101.
comment: Accepted to WACV 2027. 18 pages (8-page main paper, references, and supplementary appendix), 6 figures, 9 tables. Project page: https://aryanmangal769.github.io/symant/
★ A Conveyor Belt Mechanism for Repositionable Magnetic Coupling Between Continuum Robots
This work proposes a novel conveyor belt mechanism enabling continuous, active repositioning of functional elements along the body of a tendon-driven continuum robot. Up to three conveyor belt mechanisms are integrated within a 9 mm diameter continuum robot. We demonstrate equivalent or reduced conveyor belt actuation forces compared to robot tendon actuation forces. As a case study, we mount permanent magnets along two conveyor belts, and demonstrate actively reconfigurable inter-robot coupling. Coupling point location and quantity can be adjusted during robot operation to actively tune the effective stiffness of the coupled multi-robot system. We show that effective stiffness follows a predictable inverse-cubic relationship with coupling location, and increases with the number of coupling points. These findings demonstrate that the conveyor belt mechanism enables active functional element repositioning for changing tasks requirements, with magnetic inter-robot coupling as one proof-of-concept.
comment: 7 pages, 10 figures
★ Gym-Anything for Robotics: Scaling physical simulations with digital agents
Realistic, fast robotics simulations could enable large-scale training of embodied AI models that transfer to the real world. However, despite decades of progress, available simulations cover only a narrow range of real-world tasks and physical phenomena. The core bottleneck is that creating better simulations requires significant expert effort and therefore does not scale. To address this, we introduce Gym-Anything for Robotics, a framework in which coding agents create realistic robotics simulations. Our core insight is that simulation creation is a coding agent task, so improvements in coding agents translate directly into richer and more faithful simulations. We select simulations based on U.S. GDP data to cover economically valuable physical work, define what makes a good simulation and a good task, and develop an adversarial creation-audit loop that checks each simulation and task against these definitions. Using this pipeline, we build RoboWorld, a set of 25 simulations with 25 test and 175 training RL environments of varying difficulty. Even the best frontier models fail on every RoboWorld test task, achieving an average reward of only 0.06. Building on RoboWorld, we introduce several new ways for frontier models to improve, including in-context learning across repeated attempts, a privileged teacher that reads the simulator state and teaches a student in context, self-assisted learning from the model's own privileged experience, and code policies that generate demonstrations across varied scenes to train vision-language-action models (VLAs) such as $π_{0.5}$. Each of these substantially improves performance. We release our pipeline, RL environments, and benchmark to facilitate future research. Code and data are available at https://gym-anything.github.io/robotics
★ A Tutorial on Shape Servoing of Deformable Objects-Part I: Fundamentals
In robotics, shape control (or shape servoing) refers to the automatic manipulation of a deformable object to achieve a desired shape. It involves a feedback loop that exploits the relation between robot motion and changes in the measured shape. This relation, known as the interaction Jacobian or, in this context, the deformation Jacobian, depends on the object's mechanical properties, geometry, and boundary conditions (e.g., where the object is grasped). Because these factors are often difficult to model accurately, the deformation Jacobian can be estimated locally from observed shape changes produced by commanded robot motions, without requiring a fine-tuned mechanical model. This article is the first of a two-part tutorial on data-driven Jacobian-based shape control of deformable objects. It introduces the central elements of the approach: the formulation of the shape-control problem, shape representation, estimation and online update of the deformation Jacobian, and a standard control law from the literature. It then assembles these elements into a complete shape-servoing loop, derives a local stability condition for this loop, and illustrates why some target shapes cannot be reached. Part II covers more advanced topics. Throughout both parts, a simulated planar example in which two robot arms manipulate an elastic rod observed by a camera illustrates the concepts.
comment: Part I of a two-part tutorial. 9 pages, 5 figures. Code: https://github.com/nachocz/tutorial_on_shape_servoing_of_deformable_objects. Interactive demos: https://nachocz.github.io/tutorial_on_shape_servoing_of_deformable_objects/
★ PhysVLMM: Interaction-Conditioned Physical Belief Maps for Articulated Objects in Robot Scene Maps
Mobile robots must open doors and drawers, yet their scene maps record what each object is and whether it has moved, not how it will respond to a pull. This paper proposes PhysVLMM, a method that maintains a physical belief for every articulated part of a Vision-Language-Motion Map and answers a planner's queries about it, with two algorithms: a map update and a query. Watching a part move provably cannot identify its dynamics, and a measured pull identifies only the pulled part, up to one direction. PhysVLMM therefore (i) keeps that direction as a posterior instead of a point estimate, (ii) passes the evidence of each pull in closed form to the untouched look-alike parts of the map, and (iii) answers whether an action will succeed at a stated risk, or abstains. On real doors and drawers in 18 homes, it identifies the joint type in 86% of held-out pulls (48% for a learned single-view method) and predicts the response to a pull with 41% lower error than differentiable simulation (CRPS 0.323 vs. 0.544), four orders of magnitude faster (13 ms vs. 230 s per case). The posterior lowers the final-position error from 0.460 to 0.270, sharing lowers the force error of untouched parts from 56% to 42%, and at every stated risk from 0.02 to 0.30 the false-safe answers stay within it. Pooling all measured parts of a home does as well as the look-alike groups, and simpler variants predict where a pulled part stops at least as well.
★ DAWN: Diffusion-based Alignment and unfastening Workflow Network
We present DAWN (Diffusion-based Alignment and unfastening Workflow Network), a hierarchical robotic framework designed to perform real-world, unfastening tasks through multi-modal sensory integration. Autonomous robotic unfastening is crucial for improving recycling processes, enhancing safety, and reducing operational costs; yet, it remains predominantly manual due to inherent challenges in precise screwdriver engagement with fasteners, detailed force execution, and robustness to environmental variability. To address the complexity of learning long-horizon, contact-rich manipulation tasks, we subdivide the unfastening task procedure into two sequential stages: (1) alignment & insertion, and (2) unfastening. DAWN consists of three key components: AlignNet, a diffusion-based policy ensuring precise screwdriver tip engagement; LatchNet, a multi-modal classifier reliably detecting successful stage transitions; and UnfastNet, which achieves stable, force-controlled screw removal. Experimental validation on a Franka Emika robot demonstrates that DAWN achieves alignment & insertion success rates comparable to trained human operators, robustly transitions between stages, and attains an impressive success rate in unfastening across various unseen scenarios. Ablation studies further elucidate critical input modalities and effective training strategies, highlighting DAWN's scalability, robustness, and data efficiency.
comment: 8 pages, including references
★ BURGER: Benchmarking and Understanding Robotic Generalization via Embodied Drawing
How effectively can vision-language-action (VLA) models learn diverse language-conditioned behaviors as training data scales? Investigating this question requires large and varied robot datasets, whose collection on physical robots remains costly. We introduce BURGER, a language-conditioned robotic drawing benchmark with a scalable, prompt-driven pipeline for generating continuous drawing trajectories, both in simulation and the real world. In BURGER, an agent starts from the same blank canvas and controls a robot arm to draw the object or spatial composition specified by language, while each instruction allows an open-ended variety of valid drawings and stroke orders. Using 28,200 simulated demonstrations spanning 282 drawing tasks, we evaluate five representative VLA models across data scales, language grounding, and multimodal behavior. BURGER supports fine-grained analysis of language grounding, spatial composition, and action generation, revealing distinct failure patterns across VLA models. Our experiments suggest that some VLA models can benefit from greater task diversity and demonstration scale, capture multimodal drawing behaviors, and generalize to unseen compositions of familiar objects. Finally, we deploy a trained policy on a Franka Panda arm, demonstrating the ability to draw reactively under human intervention. Project homepage and code can be found at https://siyingzhao.github.io/BURGER/.
comment: Preprint version
★ A Two-Wheeled Quadrotor with All-Clockwise Tilted Rotors for Level-Attitude Surface Locomotnion
Conventional wheel-equipped quadrotor UAVs must tilt their bodies to generate lateral forces, limiting their ability to maintain a desired attitude during lateral motion. This paper proposes a wheeled quadrotor with all-clockwise tilted rotors that uses surface contact as a constraint to generate propulsion along the contact surface without tilting the vehicle. We analyze feasible forces and system controllability under surface contact and optimize the vehicle design considering different rotor-spin configurations. The optimization results show that a configuration in which all four rotors spin clockwise can generate a larger tangential force than the conventional alternating-spin configuration. This approach enables both aerial and contact-based locomotion without additional rotor-tilting actuators or mechanisms. Simulation results demonstrate both hovering in free space and horizontal locomotion on a floor while maintaining a level attitude.
★ Fundamental Limits of Sampling-Based Reachability: Geometry and Flow Distortion
Sampling-based reachability estimates reachable sets from finitely many simulated trajectory endpoints. Existing guarantees often control missed probability mass or approximation error relative to the convex hull of the reachable set, leaving the geometric error relative to the reachable set incompletely characterized. We study this error as a problem of support estimation under flow distortion. Our assumptions combine a uniform lower bound on the local volume density of the initial set, a lower bound on the sampling density, and a one-sided Lipschitz constant $μ$ for the vector field. These conditions admit initial sets with corners and dynamics that may expand or contract. In the fine-accuracy regime, $\tilde{\mathcal O}\bigl(2^ne^{nμT}(R/r)^n/(ρκ)\bigr)$ independent endpoint samples suffice for Hausdorff accuracy $r$ using the endpoint cloud. Here, $n$ is the state dimension, $T$ is the horizon, $R$ is the volume-equivalent radius, and $ρ$ and $κ$ quantify sampling density and local volume density. For $n\ge2$ and sufficiently small $κ$, a minimax lower bound over all estimators based only on independent endpoints matches these dependencies up to a dimension-dependent factor $2\cdot4^n$ and logarithmic factors, within the stated parameter range and when the state domain accommodates the construction. The exponential burden in dimension, and in the horizon for expanding systems, is therefore intrinsic to this observation model. Planar experiments illustrate the predicted effects of corners and contraction; robot-arm experiments show how empirical accuracy deteriorates with dimension.
comment: 16 pages, 9 figures, 3 tables. Extended version of the CoRL 2026 paper; submitted to IEEE Transactions on Automatic Control. Code: https://github.com/JeanLew01/reachapprox
★ Adaptive search with path-constrained mobile sensors using multi-arm bandits
We study a constrained sequential decision problem for detecting regions of interest from noisy online measurements collected by a team of energy-constrained mobile sensors. The sensor motion is restricted to a precomputed set of feasible paths that satisfy mobility constraints, limited available energy by requiring return to charging stations, and avoidance of known obstacles. We cast the resulting active path selection as a multi-armed bandit problem and propose an iterative search algorithm that adaptively assigns paths using confidence bounds computed from the collected data. We also propose a graph-based method to construct low-cardinality path covers under the team's constraints. We establish finite-time and anytime probabilistic correctness guarantees for the proposed algorithm, derive non-asymptotic upper bounds on the times required to detect all regions of interest and to label all nodes, and show that these bounds match algorithm-independent lower bounds up to constant factors for a broad class of problem instances. Using numerical simulations, we illustrate the behavior of the proposed approach and the effect of path redundancy and team size on search performance.
comment: 24 pages, 5 figures
★ Improving Fine-Grained Manipulation through Demonstration Expansion and Execution Design
Fine-grained manipulation requires a robot policy to follow instructions, remain robust to visual variations, and translate predicted actions into precise robot control. We present Demonstration Expansion and Execution Design (DEED), a framework for adapting a pretrained vision-language-action policy to the five MetaFine tasks through structured demonstration expansion and execution design. DEED generates demonstrations for different instructions from the same initial scene and renders each physical trajectory under different camera positions and lighting conditions while preserving its original states and action targets. A hierarchical sampling strategy balances instructions, trajectories, and visual conditions during supervised adaptation. At deployment, DEED converts predicted action chunks into robot commands using inverse kinematics, gripper mappings selected for each task, and closed-loop replanning. DEED achieves an official Overall score of 85.55%. Fixed-policy evaluations show that performance varies with execution horizon, denoising budget, and gripper mapping.
comment: RoboFineMani Workshop Challenge at CoRL 2026
★ Optimal Path Planning with Localization constraints
This paper studies UAV path planning in areas where GPS signals are not present. Because these drones can easily get lost when they don't have GPS, they must rely on dead reckoning and occasionally pass near cellular towers, which act as landmarks to reset and fix their position. The goal is to find the shortest path from a start to a goal location while keeping the estimation uncertainty within a given bound. To solve this problem, we propose a localization-constrained A* planner that treats uncertainty as a hard constraint during search. The robot state and covariance are propagated using an Extended Kalman Filter (EKF), and only paths that satisfy the covariance limit are considered. We also show that the proposed method does not affect heuristic used in A* and remains admissible and consistent, ensuring optimality among all feasible paths. Extensive Monte Carlo simulations and PyBullet quadrotor test show that the proposed method finds shorter and more reliable paths compared to RRT* and PRM, while always satisfying the uncertainty constraint.
★ socx: GPU-Accelerated Chance-Constrained Stochastic Optimal Control using JAX
In this article, we present socx, a GPU-accelerated solver for lifted convex reformulations of linear stochastic optimal control under chance constraints, implemented in JAX. We demonstrate that by exploiting the sparse and independent structure of the problem, distributed optimization can reduce it to simpler problems: independent linear-quadratic tracking (LQT) problems and Euclidean projections onto sets. These simpler problems can be solved in parallel and then coupled using the constraints. To validate our solver, we run it on a hover-linearized 3D quadrotor and show a speed-up of up to $98\times$ over MOSEK and up to $2224\times$ over SCS. socx also solves horizons up to $N=1120$, whereas MOSEK runs out of memory at $N=320$. We present ablations to isolate the key components that enable the speed-up.
★ Continuous-Action World Models: Learning Cross-Embodiment Spatiotemporal Dynamics Priors
We introduce Continuous-Action World Models (CoAWM), a framework for learning cross-embodiment spatiotemporal dynamics priors that support robot action generation across embodiments and execution frequencies. At its core is a shared latent function space that captures the spatial and temporal structure of robot actions. We learn this space through two stages. First, function pretraining brings trajectories from different embodiments and sampling frequencies into a common representation, learning shared dynamics priors through action reconstruction and consistency across temporal sampling views. By accommodating heterogeneous robot trajectories, this stage broadens the range of data sources available for pretraining and facilitates scaling to larger and more diverse datasets. Second, we connect the pretrained function space to a world-model backbone, enabling observations, language instructions, and the current robot state to condition the prediction of a latent action function. A single decoder takes any query time and returns the corresponding end-effector action. By separating shared dynamics priors from task-conditioned action selection and temporal sampling, our approach provides a unified foundation for transferring robot experience across embodiments and control frequencies.
comment: 12 pages, 6 figures
★ Securing the Robotics Supply Chain
This paper sets out what a sustained federal research and investment agenda would need to include to close that gap--not within a single budget cycle, but over the decade such an undertaking will realistically require. It identifies where the gap is most pronounced today: in the methods needed to verify and certify components of uncertain origin, in the foundational hardware research that has long been underfunded relative to software, in the physical infrastructure researchers need to test systems at scale, and in the workforce required to build and sustain all of it. Robotics should not be treated merely as a downstream beneficiary of a broader artificial intelligence investment strategy. It constitutes its own supply chain, with its own points of failure, and merits its own sustained federal commitment.
★ HiFOR: Hierarchical Foresight with Action-Guided World Grounding for Vision-Language-Action Model
Vision-Language-Action (VLA) models have demonstrated promising capabilities in robotic manipulation by integrating visual perception, language understanding, and action generation. Recent advances have further incorporated future prediction to enhance the understanding of environmental dynamics and task progression. However, existing methods often model short-term dynamics and long-term task objectives separately, while directly incorporating predicted futures into action policies without sufficiently considering their relevance to current action requirements. To address these limitations, we propose HiFOR, a hierarchical foresight framework with action-guided world retrieval. Specifically, Hierarchical Foresight Prediction jointly predicts short-term state evolution, semantic milestones, and final goal states through a Hierarchical Dependency Mask (HDM), establishing dependencies across temporal horizons. Building upon these predictions, Structure-Aware Latent World Modeling organizes future latent representations along the horizon, view, and spatial dimensions to facilitate fine-grained retrieval. Furthermore, Action-Guided Factorized World Retrieval adaptively retrieves relevant future information from the structured latent world based on evolving action representations during denoising, providing targeted guidance for action generation. Extensive experiments on two simulation benchmarks and six real-world robotic manipulation tasks demonstrate the effectiveness of our approach.
★ The Athlete as a Scientist of One: Agentic Bayesian Experimental Design for Individualized Physiology from Consumer Wearables
Large language model (LLM) agents for personal health read wearable data and recommend, but do not learn about the individual in a way that can be audited or falsified. We present EpiCoach, an agentic system that treats one athlete, a cross-country mountain biker, as the subject of accountable N-of-1 science. Claims about the athlete live in an evidence-gated hypothesis registry: each must issue a probabilistic prediction about a future session before it occurs, and its belief moves only through computed evidence from a logged statistical fit. Predictions come from a mechanistic digital twin inferred from heart rate, GPS and barometric altitude: a lagged heart-rate response model, a fitness-fatigue model with learned time constants, and a latent technical-descending skill state, measured on discovered trail segments. The agent prescribes the next training week as a constrained experiment maximizing information gain subject to load and safety constraints, around two instrument rides. We evaluate the system two ways over eighteen months of data. A rolling-origin replay benchmark shows the twin beats standard forecasters on heart rate but not segment time, and the agentic loop matches a static model. A live prospective phase runs the weekly cycle: in its first three weeks the ledger sealed 20 predictions, scored 14, the descent forecast calibrated after one correction, the climb forecast pessimistic as the athlete improved, and the safety officer vetoed the agent twice. Every correction is preserved in the registry, a template for auditable, falsifiable personalization in digital health. We release the registry, benchmark and protocol.
★ Is It the Curriculum or Just the Easy Start? Benchmarking Unsupervised Environment Design on Quadrupeds
Unsupervised environment design (UED) methods such as PLR, ACCEL, SFL and PAIRED are compared mostly on 2D mazes, grid worlds and BipedalWalker, while terrain curricula for legged robots are hand-designed and evaluated against a few baselines. We present a benchmark that runs fourteen UED, domain-randomisation and robotics curricula under one protocol on five 3D quadrupeds in MuJoCo MJX: the closed-chain W01-TEK and the open-chain Unitree Go1 and Go2, Google Barkour vB and Boston Dynamics Spot. Levels are vectors of eight interpretable terrain parameters, and all curricula drive the same PPO student at matched environment-step budgets. The evaluation reports success on random and held-out structured terrains, interquartile means with stratified bootstrap intervals, capability envelopes and a level-by-level domination comparison. We compare methods within each robot and report, per level set, the top group of methods not significantly below the best. No method is in the top group on every robot and level set. Methods with easy or low-difficulty training levels, curated or not, fill the top groups on in-distribution random levels of Go1 and Barkour, and of W01-TEK apart from PAIRED+BiBC. The success-driven terrain curriculum from legged-robot practice is in a top group most often, domain randomisation on uniform levels in none, and PAIRED in none outside Spot, where the fourteen methods lie within 3.8 points on random levels of difficulty 0.25. On three robots the leader on in-distribution random levels is not the leader on held-out terrain. We collect the results into a per-robot guide to choosing a curriculum.
comment: 22 pages, 10 figures. Code, level space and evaluation suite: https://github.com/Solvro/ml-wojtek-ued
★ JEDI: Joint-Embedding Demonstration Compression for In-Context Imitation Learning
In-context imitation learning (ICIL) adapts robot policies through test-time demonstrations, but long multimodal trajectories constrain how many examples fit in context. We ask what a compressed demonstration should preserve. We propose JEDI, which compresses demonstrations into latent slots trained to predict the query episode's next-stage keyframe in a frozen video-JEPA space. Task and process counterfactuals help the slots preserve execution details, while the predictor selects demonstrations from a bank. Post-training with keypoint tokens enables human-hand conditioning on robots. Experiments on LIBERO and real robots show that JEDI supports many-shot context; LIBERO objective controls show improved success and demonstration following over action-only compression. These findings support query-future prediction as a training objective for preserving execution-relevant information in compressed demonstrations.
★ TouchBridge: Tactile-Conditioned Human-to-Robot Transfer for Dexterous Manipulation
Human demonstrations offer an efficient and scalable source of data for dexterous robot manipulation, but differences in sensing and hand morphology complicate the transfer of contact-rich interactions. We present TouchBridge, a hardware--software co-designed framework for human-to-robot data transformation. A wearable human demonstration interface and a corresponding robot interface establish cross-embodiment correspondence in visual and fingertip tactile sensing. We introduce Tac-Retarget, a tactile-conditioned retargeting method that uses measured contact to align fingertip contact patches and preserve their spatial relations within and across hands. Unlike object-aware or simulation-based transfer pipelines, Tac-Retarget requires no object mesh, object-pose estimate, or physics simulation for retargeting, enabling direct and scalable conversion of human demonstrations into robot-domain training data. Combined with visual synthesis and camera-anchored action conversion, TouchBridge produces robot-domain demonstrations for fine-tuning a pretrained vision-language-action (VLA) model. Across four real-world contact-rich tasks, policies fine-tuned solely on TouchBridge-converted human demonstrations achieve a 72.5% average success rate, compared with 77.5% using an equal number of teleoperated robot demonstrations, while human demonstration collection is 7.0x faster. Under the same policy and training setup, Tac-Retarget outperforms DexPilot and UniDex by 25.0 and 42.5 percentage points, respectively.
★ MUSBRIC: LEGO-Compatible Pneumatic Muscle Bricks for Reconfigurable Musculoskeletal Robots
Musculoskeletal robots combine compliant actuators with articulated skeletons, but exploring alternative configurations often requires repeated fabrication. We present MUSBRIC, a modular hardware platform and accompanying MuJoCo-based simulator for reconfigurable musculoskeletal robots. LEGO-compatible endcaps turn fiber-reinforced extensile pneumatic artificial muscles into reusable, plug-and-play muscle bricks that connect directly to standard LEGO components. Experimentally identified actuator behavior supports corresponding musculoskeletal simulations. An elbow-joint prototype validates load-dependent flexion and extension, showing that pulley-guided routing enables reversible motion and a mechanical stop stabilizes the straight posture. A robotic hand demonstrates compliant grasping and multi-joint construction. Matched hardware and simulation sweeps across 15 leg configurations demonstrate systematic exploration of attachment geometry, while a quadruped assembled from four legs demonstrates walking. Together, these demonstrations span mechanism analysis, configuration exploration, and functional robot construction within a single reconfigurable system. The hardware designs and simulation code will be released as open source soon to facilitate reproduction and further development.
comment: 6 figures. Qiyao Wang and Yinglei Zhu contributed equally to this work
★ Gaussian Distillation of Flow Policies
Flow policies are among the strongest robot controllers and are well suited to modelling multimodal action distributions. Our analysis shows that the continuous-action distributions of the studied flow policies are well approximated by Gaussians. We propose the Gaussian Action Readout (GAR), which replaces a flow policy's final layer with a Gaussian readout trained on just 0.02% of eligible states, while keeping the rest of the network frozen. It samples with one action-network pass, gives closed-form likelihoods, and achieves near-teacher task success at up to twice the inference speed.
★ Learned Terminal Costs for Short-Horizon Model Predictive Control of Autonomous Trucks in Interactive Highway Traffic
Autonomous highway driving for heavy-duty trucks is a challenging control problem. The greater mass, dimensions and intricate dynamics increase the requirements for cooperation from surrounding traffic in critical maneuvers, requiring a careful balance between safety constraints and efficient long-horizon decision making. Model Predictive Control (MPC) provides rigorous constraint satisfaction but relies on terminal cost which can limit its effectiveness at short, computationally affordable horizons. Reinforcement Learning (RL) captures long-term driving behavior but offers no constraint guarantees and tends toward overly conservative policies when trained under sparse collision penalties. In this paper, we propose a hybrid MPC and RL framework for highway driving in autonomous trucks, where a finite-horizon MPC, solved via dynamic programming, uses a pre-trained RL model for terminal cost estimation. We evaluate the framework in a simulation environment with reactive surrounding traffic and compare it against pure MPC with zero terminal cost, rollout with terminal cost estimation using a base policy, and pure RL. Results show that the hybrid approach achieves zero collisions with the lowest cost of any configuration.
comment: 13 pages, 2 figures, 8 tables
★ LLM-guided Planar Loco-Manipulation for Quadruped Robots
Robotic navigation typically assumes a collision-free route to the goal exists; in cluttered environments with movable obstacles, however, reaching the goal may require physical interaction. Classical solutions encode these behaviors through geometric rules and task-specific logic, while learning-based approaches often constrain the decision space to low-dimensional motion commands or predefined action sets. We instead consider a broader decision space, jointly selecting the skill, relevant object, and geometric target. Our language-conditioned task planner, fine-tuned with LoRA on synthetic data, maps natural-language tasks and a structured scene representation to variable-length sequences of parameterized actions, executed by low-level reinforcement-learning policies. The planner learns the geometric relations underlying interactive navigation and outperforms a classical planner and a prompted frontier model on interactive navigation tasks. Comparisons with models trained from scratch indicate that composition does not require pretraining when the representation is structured, whereas pretraining yields more reliable geometric targets. At the low level, we introduce an end-to-end whole-body planar pushing policy that achieves a 93.3% success rate in simulation, versus 83.5% for a hierarchical baseline, while reducing completion time by more than 60%. On hardware, the hierarchy yields the correct skill composition in 44/45 trials and end-to-end success in 34/45. Project website: https://anonymousprojects-dev.github.io/LLM-guided-LocoManipulation-Quadrupeds/
comment: 20 pages, 9 figures, 15 tables. Includes appendix
★ AcrossDEX 1.0: From Human Motion Pretraining to Bimanual Dexterous Manipulation across Hand Embodiments
Bimanual dexterous manipulation requires continuous coordination of wrist motion, finger contact, and object motion. Human videos provide rich manipulation experience. Using this experience for robot control requires aligning human poses with robot commands and accounting for embodiment-specific kinematics and contact dynamics. We present AcrossDEX 1.0, a learning system that connects human motion pretraining, simulation-based expert learning, robot demonstration collection, and visual closed-loop execution. Pretraining on HOT3D Aria predicts one second of bimanual motion from five images and thirty annotated hand states over the preceding second. Robot adaptation preserves the visual encoder, temporal backbone, and wrist parameters. Shared wrist representations, separate finger fields, and validity masks support both six-channel Inspire hands and sixteen-joint Allegro hands. The robot policy predicts sixty control targets from frontal images, command history, and task conditions. It executes a prefix and replans from updated observations. We compare direct regression, diffusion, and flow matching on lid opening and return and closed-lid object reorientation. Direct regression achieves success rates of 40.6\% for opening and return, 100.0\% for Inspire reorientation, and 93.3\% for Allegro reorientation. Flow matching achieves 90.0\% on Allegro reorientation. Paired comparisons further show that constraining replanning with the previous plan improves the continuous execution of generative policies. These results demonstrate a common adaptation pipeline for two hand embodiments and highlight the roles of contact coordination and action-chunk continuity.
★ RoboAtom: Large-scale Atomic-Action Supervision for Robot Learning
Fine-grained action annotations can support skill reuse and compositional generalization in robot policies. However, robot datasets typically provide task-level instructions without subtask annotations. Although some include subtask labels, inconsistent formats and conventions hinder cross-source joint training. We propose RoboAtom-Data, which adds unified, time-aligned atomic-action annotations to existing data from multiple sources, covering 1.45 million trajectories, 6.88 million skill-labeled segments, and 16,882.1 hours. Our pipeline combines a unified protocol, large-scale manual annotation, propagation across repeated executions with the same ordered skill sequence, and multi-stage quality control. We evaluate the dataset through automatic atomic-action annotation and robot control. We construct RoboAtom-Bench to standardize evaluation of skill recognition and temporal boundary localization. Trained on the annotated data, RoboAtom-Anno achieves better automatic annotation performance than all evaluated open- and closed-source models on both RoboAtom-Bench and the external WGO-Bench. For robot control, atomic-action supervision paired with hierarchical execution improves task success over task-level baselines on LIBERO-Long, LIBERO-Plus, and RoboTwin. We will release the data, benchmark, model, and construction pipeline to support further research on skill-level robot learning.
comment: 31 pages, 12 figures, including references and appendices
★ Decision-Level Error Attribution and Depth-Aware Safety Evaluation of Monocular RGB Human Tracking for Autonomous Robot Navigation
Monocular RGB tracking is useful for robots operating near people, but perception errors affect binary stop/proceed decisions in ways that geometric accuracy metrics do not capture. We examine how much decision error arises from the safety rule versus perception. We prove that lateral-only distance is a lower bound on planar distance: with accurate positions, it cannot miss dangers identified by the planar rule, but it produces unnecessary stops. We introduce a two-factor Shapley attribution separating rule and perception contributions, and further distinguish missed detections from localization errors. Experiments use ETH and UCY pedestrian videos with simulated stationary robots, disjoint calibration and evaluation pedestrians, and time-block bootstrap intervals. Across 898 pedestrians in three scenes, the lateral rule missed no dangers on annotated positions but achieved only 5.6-31.9% stop precision. In two scenes with RGB perception, calibrated ground-plane localization achieved 0.59 m and 0.51 m planar RMSE, compared with 1.72 m and 1.39 m for a geometric baseline and 0.81 m and 0.68 m for calibrated Depth Anything V2. Detection recall of 0.43 and 0.26 made missed detections the leading source of missed danger. A two-standard-deviation safety margin with ground-plane localization matched or improved on the lateral rule's missed-danger rate in all eight scene-threshold settings and reduced unnecessary stops in seven. All robot interactions are simulated, and outcomes are offline decision classifications, not demonstrations of collision avoidance.
comment: 10 pages, 7 figures, 6 tables
★ Coupled Guidance for Flow Matching NeurIPS 2026
Guidance methods are essential tools for steering the generation process in diffusion models, improving the generalizability of stochastic robotic controllers and enabling high-quality controlled image synthesis. Despite strong parallels between flow-matching and diffusion, integrating guidance into flow-matching models remains quite challenging. Recent work has proposed general guidance frameworks for flow-matching, however, they are direct adaptations of diffusion guidance methods and can prove unstable. In this work, we conduct a theoretical analysis of guidance in flow-matching and show that a naive translation of diffusion-based guidance to flow-matching breaks the coupling distribution used during training, resulting in weaker control over sampling accuracy. Building on this insight, we propose Coupled Guidance, a flow-matching-specific guidance mechanism derived from a geometric perspective that leverages the coupling structure inherent to flow-matching training. To validate the effectiveness of our approach, we conduct extensive experimentation on a custom robotic collision avoidance task using ManiSkill3, four robotic dexterous manipulation tasks from the Adroit benchmark, and three image generation tasks, namely inpainting, super-resolution, and deblurring, using the ImageNet-128 and CelebA-HQ-256 datasets. Our findings show that our framework improves sample quality and guidance stability across all tasks, without imposing any additional overhead.
comment: Accepted at NeurIPS 2026. 25 pages
★ Learning Safe Flow Matching Robot Policies from Binary Feasibility Oracles
Robot foundation models increasingly rely on flow-based action generation for general-purpose manipulation, yet the generated actions are not guaranteed to satisfy physical constraints such as joint limits and collision avoidance. This problem is especially challenging when the feasible set cannot be described analytically and is accessible only through black-box feasibility oracles. We study this practical oracle-only setting and propose an oracle-guided differentiable safeguard for safe flow-matching robot policies. The safeguard uses binary feasibility queries to correct unsafe updates in the forward pass, while extracting local boundary information to provide constraint-aware learning signals in the backward pass. Combined with discrete-time flow matching, the same safeguard is used during training and inference, enabling forward--backward aligned policy learning without explicit or differentiable constraint models. Experiments on obstacle-aware robotic manipulation demonstrate 100% constraint satisfaction while preserving task performance.
★ Don't Risk It!: Trajectory-Conditioned Distributional Risk for LiDAR-Only Crowd Navigation
Efficient crowd navigation requires anticipating dynamic collision-free space without relying on fragile object tracking. We propose the Don't Risk It Planner (DRIP), an end-to-end LiDAR-only planning framework that couples occupancy forecasting with trajectory-conditioned clearance and uncertainty estimation. Operating directly on LiDAR-derived occupancy grid histories, DRIP predicts a distribution over minimum robot-obstacle clearance for candidate control sequences to evaluate collision risk. The model is trained using a risk cost that grounds predictive uncertainty directly in empirical collision outcomes rather than geometric regression alone. In closed-loop experiments with up to 45 pedestrians, DRIP achieves higher success rates with fewer collisions and reduced freezing behavior compared to state-of-the-art baselines.
★ RoboSeed: Multidimensional Evaluation of Vision-Language-Action and World-Action Models: Capability, Resource Affordability, and Research Evidence
Choosing a robot action model requires evidence about task capability, adaptation cost, deployment requirements, and the reliability of reported conclusions. Success rates alone offer limited guidance to researchers with different computing budgets and goals. We propose RoboSeed, a multidimensional evaluation framework for vision-language-action (VLA) and world-action models (WAM), within a continuing program to train, evaluate, and openly release robot models with verifiable evidence for model selection. The framework defines five dimensions: capability, downstream training affordability, deployment affordability, locally verified research value, and reproduction evidence. It maps empirical measurements and audit criteria to 0-100 profiles using fixed normalization anchors, with balanced, training-constrained, and deployment-constrained preference settings. Interface and protocol checks establish when comparisons are appropriate. Missing measurements remain explicit, incomplete aggregate scores are withheld, and weighted input coverage identifies evidence gaps. We apply the framework to representative VLA and WAM models, drawing on local training, released-weight evaluation, task-level audits, and an evaluation-system study. The results reveal budget-dependent backbone choices, sensitivity to adaptation recipes, and differences between language and visual task interfaces. Reproducible scoring code and machine-readable evidence support extension to additional models. The study provides dimension-level assessments and priorities for further measurement, while incomplete serving traces and heterogeneous protocols preclude overall numerical rankings.
comment: Project repository: https://github.com/PrimitiveX/RoboSeed
★ LOGIV: Graph-Verified Execution with Causal Local Repair for Long-Horizon Robot Tasks
Agentic robots must turn a high-level goal into actions while keeping the physical conditions needed for future steps valid. A completed action does not ensure that its effects will persist: a later disturbance can invalidate support that the remaining plan still assumes. We present LOGIV, a graph engineering framework for agentic robot execution that uses an explicit task graph to connect symbolic planning, observation-based verification, and embodied action. LOGIV compiles a validated plan into condition-support and resource-ordering edges, checks their physical validity during execution, and uses dependency tracing to determine what must be restored after a failure. Repair is constrained by retained unfinished actions, and the combined continuation is symbolically validated before execution resumes. The framework operates above existing Vision-Language-Action (VLA) and World Action Model (WAM) policies without adding trainable components. Across 20 dual-arm tasks from RoboTwin~V2 and BiCoord with three execution backbones, LOGIV improves mean task success without injected disturbances by 10.67 and 15.33 percentage points, respectively. Under predefined perturbations, it achieves higher task completion and condition restoration than three adapted recovery baselines for every backbone on both benchmarks. On three real-robot tasks, successful trials increase from 12/30 to 17/30. These results support explicit, continuously checked task dependencies as a practical interface for reliable agentic robot execution.
★ Humanoid Dodgeball: Learning Fast Dynamic Obstacle Avoidance Motions for Humanoids
Humanoids have become increasingly agile and capable in real-world environments. To operate safely in dynamic scenarios, they must rapidly evade fast-moving obstacles while maintaining stability and physically feasible whole-body motion. Learning such behaviors remains challenging due to the high-dimensional dynamics of humanoid systems and the need to jointly balance reactive obstacle avoidance, motion naturalness, and control robustness. In this work, we present Humanoid Dodgeball, a reinforcement learning (RL) framework for agile dynamic obstacle avoidance in humanoids. Our method enables highly dynamic whole-body evasive maneuvers against incoming projectiles through threat-aware reward shaping, while multiple direction-specific adversarial motion priors regularize the learned behaviors toward stable and diverse human-like motions. Extensive simulation and hardware experiments demonstrate that the proposed framework substantially improves dynamic obstacle avoidance performance across diverse obstacle speeds and launch distances, while preserving robust and physically plausible humanoid motions.
★ How Much Should a VLA Trust Its Reasoning? Step-Level Guidance as a Measured Action-Value Function
Reasoning vision-language-action (VLA) policies write a reasoning trace and then decode the action from it. A wrong trace yields a wrong action, and even a correct-looking trace can mislead the plan. Existing remedies ask whether the reasoning is right. We ask how strongly it should steer. CoC-CFG applies classifier-free guidance between the policy's context and the same context without the reasoning span. One weight per Euler step attenuates or amplifies the reasoning, and w=1 recovers the original policy. We treat the choice of this weight as a Markov decision process whose action values are measured, not bootstrapped. With the expert frozen and the noise fixed, applying a weight for one step and finishing the plan under the current controller returns that weight's value exactly. Q-adaptive guidance fits these values with small critics and improves the controller conservatively. In the Alpasim driving simulator it outperforms every fixed weight and online RL (SAC, GRPO) in open loop, and in closed loop it passes 262 of 300 scenes, against 252 for the original policy. On held-out frames of the Waymo end-to-end driving benchmark, its first three stages outperform every fixed weight, four adaptive-guidance methods, SAC and GRPO. On the hidden test set, a frozen Alpamayo 1.5 with rule-based camera and vehicle conversions and Q-adaptive guidance sets a new state of the art among published methods: 8.135159 rater feedback score, against 8.079 for the best published method.
comment: 6 pages, 2 figures, 4 tables
★ Gaze-Assisted Supervisory Control for Lunar Robotic Arm Teleoperation Under Concurrent Monitoring
Rover-mounted robotic arms can extend human reach during lunar sample handling, but continuous joystick teleoperation requires repeated visual-motor alignment while operators may also need to monitor system states. This paper presents a gaze-assisted supervisory control framework in which gaze nominates a rock or placement target, explicit confirmation authorizes the action, and the robot performs bounded autonomous grasp-and-place execution. The framework was evaluated against manual joystick teleoperation in a mission-analog virtual reality study with 16 participants. The study used a 2 x 2 x 3 within-subject design varying control mode, placement difficulty, and concurrent monitoring load. Mean recorded task completion time decreased from 47.65 to 18.43 s per rock, with 72.5% of this difference arising during grasping. Under the tested auditory monitoring conditions, gaze-assisted supervisory control showed less average proportional slowing, with no detected control-mode difference in monitoring-response accuracy or speed. The rate of gaze shifts between core areas of interest decreased from 49.98 to 27.73 shifts/min, while the raw NASA Task Load Index score decreased from 32.54 to 18.26. However, the benefit was task dependent. In precision placement, manual control yielded lower mean placement error and a higher placement success rate. These findings support a mixed-initiative allocation in which gaze enables rapid task-level intention specification and autonomous execution, while precision-critical stages retain manual refinement.
★ Compressed Orientation Reachability Maps for Sub-Second Robotic Needle Entry Point Planning
In the context of multi-stage trajectory planning for robotic percutaneous liver interventions, identifying a kinematically feasible entry point remains a critical computational bottleneck. Existing methods rely on exhaustive inverse kinematics or randomized searches, incurring delays of 13 to 27 seconds per target and limiting real-time intraoperative adaptation. This paper introduces an Orientation-Based Reachability Map (ORM) that precomputes the kinematic feasibility of a 6 Degrees of Freedom (DoF) serial manipulator across 188,873 workspace positions and encodes orientation feasibility into a compact, indexed 83 MB map. A Just-In-Time (JIT) compiled runtime pipeline replaces live optimization with near-instantaneous array lookups. Experimental evaluation demonstrates a reduction in planning latency from 13.5-26.9 s to 0.289-0.349 s per target while maintaining high geometric fidelity, achieving a mean convex hull volume overlap ratio of 0.933 and a mean alpha-shape volume ratio of 0.976 relative to the baseline reachable point cloud. The proposed ORM framework enables sub-second entry point planning and provides an efficient computational framework for robotic needle insertion in percutaneous liver ablation.
comment: 6 pages, 4 figures, 2 tables
★ PASA-KV: Attacking Action-Facing Visual Routes in World Action Models
World action models (WAMs) increasingly couple internal visual or world representations with action generation, making their world-to-action pathways a critical yet poorly understood source of robustness. We investigate when perturbations to these internal representations actually become behaviorally consequential, and uncover a pronounced mismatch between representation perturbability and closed-loop vulnerability: attack effectiveness varies sharply across visual K/V routes, while larger representation displacement does not reliably imply stronger control failure. Our analysis attributes this mismatch to the action-conditioned readout, where keys determine where action queries read and values determine what content they receive. This motivates an action-facing view of vulnerability, in which the effect of a perturbation depends on how it is routed into action generation rather than on representation-space magnitude alone. We instantiate this view with Tail-AV, a short-probe signal for identifying vulnerable routes, and operationalize it in PASA-KV, which selects action-facing K/V targets before training a fresh task-shared attack. Experiments across multiple WAM systems and a physical robot validate readout-informed target selection, demonstrate strong closed-loop attacks, and reveal substantial system-level differences in resistance.
★ Surrogate Uncertainty for Black-box Visual Grounding
Visual grounding is crucial for robots to understand language instructions with visual observations. This paper addresses the problem of visual grounding uncertainties, where a robot uses a black-box vision-language model (VLM) to ground free-form natural-language instructions to target objects. Prevailing methods derive uncertainty from the VLM itself. Yet, outputs of the VLM, e.g., generated uncertainties, are from the same parameters and structures producing the predictions, and inherit the bias. We give a formal Bayesian definition of the problem that treats the VLM prediction as an observation. An external surrogate Bayesian model then quantifies per-query uncertainty over a local neighborhood of a held-out calibration set, decomposed into aleatoric and epistemic components. A hierarchical retrieval across input modalities realizes a compact joint-space neighborhood, mitigating the curse of dimensionality. The estimator further integrates with conformal prediction or model-based planning for active visual grounding, where robots are allowed to take verbal or physical actions to ground targets. Across grounding benchmarks and state-of-the-art VLMs, it consistently improves both calibration and task performance. On a real Fetch robot under open-world instructions, it enables principled trade-offs between confident action and information gathering, substantially improving robustness.
comment: A preprint rejected by RSS 2026 AND CoRL 2026
★ Data-Driven or Model-Constrained? Comparing Airflow Estimation Methods under Sparse Sensing
Airflow-driven advection is the primary mechanism for gas and aerosol dispersion in real-world scenarios, making accurate airflow estimation essential for ventilation monitoring, gas source localization, and contaminant spread analysis. Since obtaining airflow maps via high-resolution spatial sensing is usually infeasible in practice, flow field estimation methods must cope with sparse measurements by introducing statistical or physical priors to compensate for limited data. Existing approaches mainly differ in how they incorporate fluid-flow constraints and environmental geometry. However, due to the lack of systematic comparisons across this spectrum of methods, the potential benefits of physics-based priors for airflow estimation under sparse sensing remain underexplored. To address this question, we quantitatively evaluate four airflow estimators across three simulated 2D scenarios under steady-state conditions and varying sampling densities. The methods range from Gaussian Process Regression and graph-based Gaussian Markov Random Fields to two Navier-Stokes-based approaches, including a novel strong-form residual minimization variant. Our results show that physical priors provide the greatest benefit at low sampling densities by compensating for missing spatial information. Although data-driven Gaussian Process Regression becomes increasingly competitive as sensor coverage increases, model-based estimators retain an accuracy advantage across the evaluated scenarios.
comment: 4 pages, 3 figures, accepted at IEEE SENSORS Conference 2026
★ Teaching a Tree to Recover: Online Distillation of an LLM Supervisor with Counterfactual Replay
Autonomous mobile robots commonly pair a global planner with a local model predictive controller (MPC), but nonconvex obstacle avoidance and a limited horizon can make the robot freeze in complex layouts. Deciding whether to replan, wait, or call for help is a supervisory decision. Recent work queries a large language model (LLM) supervisor on every anomaly, which is slow and costly. We instead distill the LLM online into a decision tree. After each training episode, counterfactual replay labels recoveries as needed, unnecessary, or harmful. These labels retire unhelpful rules. In simulation, on 20 unseen layouts where the MPC alone freezes, the tree handles at least 18 correctly across all seeds, either by reaching the goal or by calling for help when no feasible path exists. The tree matches both an always-query LLM supervisor and expert-designed rules, while issuing no LLM queries at test time. Handwritten rules cannot anticipate the long tail of situations a physical robot will encounter. Online distillation is a promising remedy: the tree handles familiar cases, and the LLM is consulted only for new ones, whose validated recoveries become new rules. Language models can thus also serve as designers of supervisors rather than only as runtime supervisors.
★ AFast Robot Localization Method Based on Monte Carlo Sampling
Localization is an essential capability of intelligent robots. However, in real-world constrained environments, robots often cannot rely on external localization systems. Therefore, this paper proposes an online particle-filter localization method based on Monte Carlo sampling. The method is applied to the localization of a mobile robot in a maze environment and requires only four range sensors mounted on the front, rear, left, and right sides of the vehicle, without the need for any heading information. Simulation experiments demonstrate that even in a maze where local regions are highly similar, the vehicle can still achieve relatively accurate localization, thereby providing a fast and effective approach for robot localization in complex environments.
★ Smartphone Indoor Localization Enhanced by Hierarchical Multi-Sensor Context Recognition
Smartphone indoor localization requires low-cost, robust positioning across scene transitions and phone-use variations. Pedestrian dead reckoning (PDR) enables infrastructure-free deployment but accumulates errors from heading drift, structural transitions, and carrying-mode changes. Existing methods often treat contextual cues as a single event stream or local correction signal, limiting their ability to address distinct localization errors. This paper presents a framework enhanced by hierarchical multi-sensor context recognition. A jointly data- and model-driven recognition model predicts environment, scene, and activity probabilities, which are converted into scene-event and action-context segments. Scene-event segments guide scene-constrained PDR correction through corridor heading constraints, landmark attraction, fixed-lag backtracking, and vertical-transition handling. Action-context segments drive action-aware RoNIN-based position compensation for carrying-mode-dependent drift. Experiments on cross-floor and multi-action trajectories against LiDAR-SLAM references show that the proposed framework outperforms PDR and representative learned inertial odometry baselines, demonstrating that hierarchical semantics provide reliable, localization-specific correction cues.
★ AI for Science in Space: Orbital Laboratories, Planetary Fieldwork, and Embodied Autonomy
Artificial intelligence for science (AI4S) is extending from interpretation of existing data to the design and execution of experiments. Space offers distinctive scientific opportunities while restricting physical support. Microgravity and extraterrestrial environments expose phenomena that terrestrial experiments cannot fully reproduce; finite samples, constrained resources, and limited opportunities for intervention complicate sustained investigation. This critical review examines orbital laboratories and planetary in situ science across biology, materials, fluid and thermal physics, and adaptive exploration. We distinguish learning from space-derived data, selecting scientific actions, executing them, and preserving experimental validity. Terrestrial self-driving laboratories provide complementary evidence on optimization, dedicated automation, robotic manipulation, and the support needed for unattended operation. The synthesis connects scientific gains already demonstrated by bounded AI systems to gaps in observation, executable actions, and continuity of valid evidence. Space robots can expand the attainable scope of autonomous science when necessary interventions exceed validated instrument capabilities and timely human support is unavailable. Their value must be established against dedicated mechanisms, redundancy, and human-supported operation. We develop a framework connecting sample history, scientific decisions, physical execution, and claim-specific validity. A worked orbital design, a planetary extension, and matched evaluation schemes separate the contributions of sensing, decision policies, and physical actions. The resulting agenda measures autonomy through valid scientific return, task coverage, intervention costs, and evidence integrity across complete campaigns.
comment: 47 pages, 5 figures, 7 tables. Ancillary evidence files included
★ Kinematic-Tactile Aligned Dexterous Retargeting with Contact Wrench Space Alignment
Human demonstrations provide a natural and scalable source of data for learning dexterous manipulation policies. However, existing retargeting methods primarily transfer human kinematics, while the tactile information underlying physical interaction is rarely preserved across embodiments. To address this limitation, we present a hierarchical three-stage framework for kinematic-tactile aligned dexterous retargeting. We use object-centric contact wrench space as a physical representation for tactile alignment between human and robot hands. The framework first aligns hand kinematics and contact geometry, and then constructs a human wrench support profile from tactile forces and contact geometry. Robot fingertip force targets are subsequently optimized to cover the human wrench support and converted into joint position corrections, yielding directly replayable robot hand trajectories. We evaluate the framework in simulation and real robot experiments across three pouring conditions. In the real robot experiments, our method achieves a 96.7\% replay success rate and higher wrench transfer fidelity than representative retargeting baselines. The framework enables physically grounded transfer of kinematic and tactile information from human demonstrations to robot hands, providing a path toward scalable learning of contact-rich dexterous manipulation.
comment: 8 pages
★ SimTO-Robust: Automated Design of Soft Grippers Robust to Grasp Pose and Object Geometry
Robotic grippers operating in unstructured environments must robustly grasp objects of different shapes and sizes despite perception and positioning errors. Soft grippers that can achieve this robustness through intelligent mechanical design rather than sophisticated control software offer a promising solution. However, no existing computational design framework explicitly optimises soft gripper morphology for robustness across object geometries and grasp poses. We introduce SimTO-Robust, a generalisation of simulation-driven topology optimisation (SimTO) that simulates grasps across multiple objects and poses and aggregates their contact forces to guide gripper morphology optimisation. We generate 6,646 designs and physically evaluate 24 of them alongside 10 specialised SimTO designs, two rigid baseline grippers, and a baseline generalist soft gripper produced using conventional topology optimisation. All 37 grippers are tested in a pick-and-place task involving five feature-rich objects and six grasp poses, with external disturbances of increasing magnitude applied during the task. On average, SimTO-Robust designs complete pick-and-place more reliably than specialised SimTO designs and the rigid baselines. The best SimTO-Robust designs also withstood more of the applied disturbances than any other tested gripper. These results show that incorporating diverse grasp conditions into simulation-driven topology optimisation is a simple and effective approach to improving real-world soft gripper robustness. More broadly, they show that robust robotic grasping can be achieved through intelligent mechanical design rather than sophisticated sensing and control.
★ Assuring Learning-Enabled Cyber-Physical Systems with Foundation Models: A Neuro-Symbolic Agenda for Testing, Verification, and Runtime Assurance
Learning-enabled cyber-physical systems (CPS), including autonomous vehicles and UAVs, increasingly incorporate foundation models, yet fragmented assurance practices provide insufficient support for system-level guarantees. We argue that restoring engineering control across stochastic neural, symbolic, and physical boundaries requires a shift from model-centric evaluation to system-level assurance. We introduce the Foundation Model Assurance Stack, connecting three directions: (i) neuro-symbolic architectures that constrain foundation models through explicit symbolic world models, reasoning layers, and interfaces; (ii) semantically grounded, conformance-based testing that combines formal specifications and symbolic world models with foundation-model-assisted specification mining, scenario generation, and test synthesis; and (iii) compositional verification and runtime assurance that target formally tractable components and interfaces through modular contracts, equivalence checking, verified synthesis, uncertainty-aware reasoning, and runtime monitoring and enforcement. Symbolic abstractions establish assurance boundaries between stochastic learning and formal reasoning, connecting testing, verification, and runtime evidence across the system lifecycle. Foundation models serve as constrained assistants whose outputs are independently checked wherever correctness can be formally specified. We identify research challenges in specification engineering, abstraction fidelity, evidence composition, and lifecycle certification, and outline a staged agenda toward trustworthy, deployable, and certifiable learning-enabled CPS.
★ From Reasoning to Action: Consequence-Grounded Alignment for Vision-Language-Action Models
Reasoning informs decision-making by connecting intended actions with their expected consequences. Recent vision-language-action (VLA) models incorporate embodied chains of thought (ECoTs) to guide action generation. However, such guidance remains implicit, as actions are conditioned on ECoTs rather than explicitly optimized to align with them. We introduce CoRAct, a consequence-grounded reasoning-to-action alignment framework that explicitly aligns reasoning and action through reward-guided optimization in action noise space. CoRAct formulates two physically grounded ECoT components as verifiable signals: movement specifies intended motion, while waypoints encode expected spatial consequences. A unified Mixture-of-Transformers jointly models ECoT reasoning, action generation, and action-conditioned future prediction. For candidate actions generated from sampled noise, CoRAct computes alignment rewards based on movement consistency and agreement between predicted outcomes and specified waypoints. These rewards guide group-relative optimization of the initial noise distribution, steering action generation toward consistency with intended actions and expected consequences. Experiments across RoboCasa, VLABench, and real-world manipulation tasks demonstrate improvements in task success, generalization, and reasoning-execution alignment over strong ECoT-based VLA baselines. CoRAct also grounds externally provided ECoTs in executable behavior, enabling zero-shot adaptation to novel scenarios by leveraging knowledge from powerful vision-language models. Project page and dataset: https://onlyxingxingzyx.github.io/CoRAct/
comment: Project page: https://onlyxingxingzyx.github.io/CoRAct/
★ PRIME W0: Physical Priors from World Models for Embodied Manipulation
Vision-Language-Action (VLA) models excel at instruction following, yet most remain feed-forward: they condition actions only on the current observation and discard both long-horizon temporal context and the hierarchical structure of VLM representations. We present PRIME W0, a hierarchical conditional flow-matching framework that closes this gap through three synergistic designs: (1)per-layer KV injection from a pretrained VLM into an Adaptive MoE action expert; (2)dynamics-aware conditioning via asynchronous world-model subgoal-image prediction and a DINOv2 historical-frame semantic bridge; and (3)Action Dynamic Manifold (ADM) flow matching with covariance-structured noise that respects robot kinematic coupling. PRIME W0 is trained in four stages---VLM pre-training, DAT/NTP visuomotor pre-training, DiT flow-matching decoder assembly with WM injection, and end-to-end SFT---and achieves state-of-the-art results on LIBERO and real-robot benchmarks, with systematic ablations validating each architectural component.
★ Veloxity: Towards Memory-Safe UAV Embedded Firmware using the Rust Programming Language
Recent years have witnessed increased momentum towards memory-safe programming languages for safety-critical systems. Memory-safe languages can eliminate entire classes of common software errors that have historically required extensive analysis and expenditure to avoid. Nevertheless, memory-safe language adoption in aerospace remains fragmented and incomplete, particularly for unmanned aerial vehicles (UAVs). Recent Rust UAV projects either run only in simulation, are closed source, or have not been integrated with an established autonomy ecosystem and tested against existing C/C++ firmware. To address this gap, we present Veloxity: an open-source embedded UAV autopilot written in the memory-safe Rust programming language. Veloxity lowers the barrier to using memory-safe languages within modern ecosystems by providing a minimal, modular foundation for autonomous UAVs. We compare Veloxity to equivalent autopilots written in C++ in simulation, with hardware testbenches, and via outdoor waypoint-following flight tests, demonstrating functional parity with improved performance. These results indicate that modern memory-safe languages are a viable option for reliable and performant firmware for autonomous aerospace systems.
comment: 22 pages, 12 figures
★ Consensus Within, Competition Between: Decentralized LLM Teams for Perimeter Defense
The use of large language models (LLMs) to coordinate multi-agent teams raises a fundamental architectural question: whether task allocation should be centralized in a single LLM controller or distributed across autonomous LLM agents, particularly when communication is unreliable. In a deliberately simplified perimeter-defense simulation against LLM-controlled intruders, we compared a central commander, an auction-based consensus team and independent agents under five network conditions. Breach rates were 15% for the consensus team, 32% for the commander and 29% for independent agents, and of the different network conditions, only limited radio range changed a team's breach rate, raising the commander's. In consensus team, giving a low-cost LLM more authority over the bid computation raised breaches by 38 percentage points. With birds and civilian drones in airspace, we added a photograph for each to the input text state. Only the near-frontier model treated them apart from intruders. Intruders disguised as existing taxidermy bird drones were mistook as birds by both low-cost and near-frontier LLMs, going unpursued in 59-61% of near-frontier decisions (1-16% before). However, a one-sentence forewarning in the prompt lowered it to 0-25%. We find that LLM team with a dedicated external conflict resolution protocol results in fewest breaches. Code: https://github.com/the-kas-lab/consensus-within-competition-between.
comment: 88 pages (33 main text and references, 55 supplementary material), 7 main figures. Code: https://github.com/the-kas-lab/consensus-within-competition-between
★ Contact Imitation Interface: Transferring Dexterous Skills by Showing How Contact Unfolds
Robot-free human demonstrations offer a promising route to scaling dexterous manipulation data. The key challenge is to preserve the demonstrator's natural dexterity while producing contact-grounded observations and actions that transfer to robot hands. We present Contact Imitation Interface (CIMI), a framework with contact-grounded demonstration and learning interfaces. The demonstration interface combines shared contact geometry and fingertip RGB views with contact-alignment-first retargeting and reprojection of wider scene views. The learning interface introduces a hierarchical policy in which a contact-surface planner predicts relative contact-surface trajectories and a controller coordinates wrist and finger commands to improve contact alignment. CIMI combines contact-grounded observations, retargeting, and action representations, improving mean final-stage success by at least 50 percentage points over baseline policies across six hand-task pairs on LEAP and Wuji. Hardware designs and software are publicly available.
comment: 15 pages (8-page main paper and 7-page supplementary material). Project website with code and hardware designs: https://nus-lins-lab.github.io/CIMIWeb/
★ CuPIQP: A Batched and Differentiable QP Solver on GPUs for Robot Learning and Control
Quadratic programs (QPs) are fundamental building blocks of many robotic tasks such as inverse kinematics, whole-body control, and model predictive control. Extending these methods to large-scale robot learning and control requires solving many QPs concurrently and, for end-to-end learning, differentiating through their solutions. Efficient batched optimization is therefore essential to bringing established model-based methods into GPU-based robotics pipelines. We present \textsc{cuPIQP}, a batched and differentiable QP solver on GPUs based on a proximal interior-point method, with unified support for dense, sparse, and multistage problems. The solver exploits parallelism across independent QPs and their internal structure while preserving per-problem numerical updates and termination decisions. Benchmarks with up to 16384 QPs per batch demonstrate solve-time speedups of up to $18.8\times$ over \textsc{Moreau} and $4.26\times$ over eight-core \textsc{PIQP}. In an end-to-end learning task, \textsc{cuPIQP} accelerates training by $4.6\times$ over \textsc{QPAX} and $7.8\times$ over \textsc{Moreau}. By reducing the computational cost, \textsc{cuPIQP} provides efficient, reusable infrastructure for integrating model-based optimization into large-scale robot learning and control.
★ STAR: Steering Action Representations during Inference
Vision-language-action models (VLAs) map language instructions and visual observations to robot actions, showing strong capabilities across robotic manipulation tasks. However, in our probing experiment, we find that action representations near the action output do not fully preserve task-relevant visual grounding. Motivated by this late-layer grounding gap, we introduce STAR, for STeering Action Representations, a training-free inference method for pre-trained VLAs. STAR applies an exponential moving average over late-layer residual updates, allowing later action representations to inherit object-aware update directions from earlier layers without model updates or architectural changes. It further uses recent proprioceptive states to adapt the mixing strength, using conservative mixing when execution dynamics change rapidly. Experiments on robot manipulation benchmarks show that STAR improves late-layer object awareness and increases average task success rates by 4.0 percentage points. Our code is available at https://github.com/PKU-VaLuE-Lab/STAR.
★ Confidence-Modulated Safety Filtering for End-to-End Policies
End-to-end policies, from task-specific controllers to robot foundation models, learn complex behavior from raw observations but are not guaranteed to remain safe, so they are often wrapped in a safety filter. The resulting system operates under two domains: the filter's operational design domain (ODD), where its assumptions hold and safety is certified, and the policy's competence domain, where it is reliable. Filters typically fix their conservativeness regardless of the policy's competence, trading performance inside the ODD against margin beyond it. We ask whether the policy's confidence should set that level, and propose a filter-agnostic pipeline that modulates it at runtime: inside the ODD, where every setting is safe, confidence buys performance; beyond it, low confidence adds margin, which helps when it flags where the filter's model fails. Preliminary F1TENTH simulations, where this holds, show the modulated filter combining the strengths of permissive and conservative filters.
★ Robot Navigation and Canine Welfare: A Pilot Study Surfacing Challenges for Cross-Species Legibility and Proxy Validity in Animal-Machine Interaction
Autonomous delivery robots increasingly share residential spaces with companion animals, yet navigation behavior is rarely designed with animal welfare in mind. This paper presents a pilot study evaluating a curve-out maneuver, a laterally arcing pass-by trajectory triggered at the dog's alert distance, proposed as a candidate welfare-informed navigation intervention grounded in canine social communication research and legible motion theory. Following a research through design approach, we developed and evaluated a first design iteration of the artifact in a naturalistic home environment with a companion dog. Contrary to our hypothesis, the curve-out produced a more acute stress response than the straight-line baseline. We argue that discontinuous execution limitation, the robot stopping and reorienting multiple times to approximate the arc, undermined legibility, which in itself became a stressor. The pilot surfaces the legibility of robot behavior to non-humans' eyes and proxy stimulus validity as structural challenges for AMI and repositions the design problem for future work.
comment: 8 pages, 2 figures, Accepted at ACI2026
★ FlapSim: A Composable MuJoCo Simulation Platform and Reduced-Order Aerodynamic Model Hierarchy for Flapping-Wing Robotics
Flapping-wing simulation must reconcile high-frequency wingbeat aerodynamics with the slower dynamics relevant to flight control and reinforcement learning. We present FlapSim, a modular open-source MuJoCo simulation platform that combines robot descriptions, wing geometry, and composable aerodynamic models in a deterministic simulation pipeline. Across $48$ held-out scenarios spanning frequency, gimbal, wind, frequency-step, gust, and cross-robot flat-plate regimes, we compare five reduced-order configurations operating at $1\,\text{ms}$ with a $50\,μ\text{s}$ quasi-steady blade-element reference. The built-in hover cycle-averaged baseline achieves a $67.51\times$ CPU speedup but only $0.6593$ mean wrench fidelity. Combining multirate $21$-substep clock synchronization, three-dimensional linear/quadratic inflow and gust coupling, analytical blade-element geometry scaling, and phase-locked $1f\text{--}6f$ harmonics raises mean fidelity to $0.9646$ at a $65.19\times$ speedup in a five-block model. A streamlined four-block model without explicit step-transient lag states achieves $0.9662$ fidelity at a $67.98\times$ speedup. Controlled ablations show that stroke harmonics are particularly important during frequency transitions, improving held-out frequency-step fidelity by $+0.3176$. Transfer-function analysis explains this effect: fixed-window cycle averaging retains harmonic contributions when the wingbeat frequency changes, so within-cycle reconstruction also improves the filtered wrench. Once phase alignment and harmonic synthesis are included, explicit transient lag states provide no additional benefit in this benchmark. These results characterize the accuracy--computation tradeoff of the aerodynamic hierarchy and identify timing, inflow, geometry, and harmonic representation as central considerations for reduced-order flapping-wing simulation.
★ AquaProbe: Efficient Adaptation of Frozen Vision Models for Underwater Images
Reliable visual perception is essential for safe autonomous underwater exploration. Yet vision models pretrained on terrestrial images become less reliable underwater, where turbidity, backscatter, and wavelength-dependent attenuation alter scene appearance. Existing adaptation methods either spread adapters equally across the encoder depth or select layers using the divergence between two fixed domains. In this paper, we introduce AquaProbe for measuring how the location of the domain gap changes with water type and imaging range. AquaProbe separates newly introduced divergence from that accumulated through a frozen vision encoder using paired clean and physically rendered images and allocates a fixed LoRA budget according to the measured profile. The segmentation mean Intersection-over-Union (mIoU) for the proportional LoRA rank allocation, based on the measured profile, is 70.1; this is a better result than uniform LoRA rank allocation by 1.9 mIoU points while having the same parameter budget of 0.59 million adapter parameters. The three-way router, conditioned on estimated condition clusters, achieves 70.6 mIoU on the standard SUIM split and 63.4 AUDRC under held-out degradation.
comment: 10 pages, 4 figures, 3 tables
★ How a group that cannot communicate knows itself by its enemy
Teams that cannot communicate seem stuck with what each member sees. We show that their opponent can tell them the rest. Silent hunters, jammed robots and traders who never meet all face an adversary that reacts to the group as a whole; every member can read that reaction, a tell, and learn what no member knows alone. One tell can do the work of a message from every member. The adversary would rather hide, but hiding interferes with its own task, and a team that commits in advance to how it will respond decides how much hiding is worth. A committed team never loses when its opponent cares more about its own goals, and does as well as a fully informed team exactly when the opponent's stake outweighs any gain from lying. Below that point the team learns more by trusting less: in a solved Gaussian model it obtains the truth by promising not to use all of it and keeps the cost the opponent would have paid to conceal; an exact test decides when this holds in finite games. How a team spreads its trust matters too. A lone uninformed member with a full-support prior needs a private coin, but with additive squared-error losses two interchangeable members who respond differently replace it, and a small discount on trust insures against misjudging the enemy. Information, on this view, is not only gathered or aggregated; it is produced by how a team commits to respond.
comment: 6 pages main text; 35-page SI Appendix included as an ancillary file. Code and data: https://doi.org/10.5281/zenodo.23743441
★ From Teleoperation to Long-Horizon Dexterity: Evaluating Visuomotor Skills and Offline Proxies for Dexterous Manipulation
Imitation learning policies are intended to make robotic manipulation robust to variations in object pose, appearance, and environment that challenge pre-programmed motion. However, real-world performance remains difficult to predict, as conventional optimization signals such as validation loss show no consistent relationship with robot success. To identify the design choices that govern real-world performance, 281 imitation-learning policies spanning seven policy families were evaluated in approximately 1,700 real-robot trials of RJ45 connector grasping and insertion using a five-fingered robotic hand. Controlled ablations showed that observation and action representations dominated conventional optimizer and sampling choices. For point-cloud policies, region-of-interest cropping, representation in the robot-base frame, and normalization within the point encoder each improved success by 70-100 percentage points. For the vision-language-action policy, absolute rather than relative action prediction improved insertion success by up to 70 percentage points. Skill decomposition, multi-view perception, and recovery demonstrations were associated with the strongest-performing systems, although their effects were not independently isolated. Of 67 offline metrics evaluated, only five measures of predicted-motion smoothness and directness reliably correlated with insertion success (r = -0.70 to -0.80), and only within a single policy family. These results identify representation choices as key determinants of real-world imitation-learning performance and show that offline metrics can shortlist policies, but cannot replace real-robot evaluation.
comment: Accepted at CoRL 2026 Workshop
★ CaPtain: A Pipeline for Post-Training Code-as-Policy Robot Agents
Across coding, web, and tool-use tasks, language-model agents are now routinely trained with the same recipe: supervised fine-tuning (SFT) to learn the interface, then multi-turn reinforcement learning (RL) to master it. Code-as-policy (CaP) casts robot manipulation in the same form, as the policy writes code against robot APIs and reads back the results, so the same recipe should apply. Yet existing CaP agents sidestep training: they either prompt closed frontier models or delegate manipulation to pre-trained vision-language-action models (VLAs) and high-level skills, while small open vision-language models (VLMs) fail outright. We present CaPtain, an open post-training pipeline that turns a small open VLM into a capable CaP agent, with no VLA, frontier model, or privileged state at test time. CaPtain trains and deploys the policy inside CaPtain-Harness, which exposes low-level perception, geometry, and guarded-motion primitives across simulation and real hardware. We first align the base model via SFT on synthetic trajectories generated with GPT-6 Astra, then apply multi-turn RL. Multi-turn RL must credit tens of agent requests, whose physical effects often surface turns later, from a single end-of-episode outcome. We therefore introduce a physical critic that reads privileged simulator state, never seen by the policy, and credits each request by how much it changes the predicted chance of success. Starting from Qwen3.5-4B (0% zero-shot), CaPtain-4B reaches 55.4% held-out LIBERO success after SFT and 73.8% after RL. On LIBERO-PRO, it reaches 78.5%, surpassing several frontier-model agents, and its RL gains persist across three perturbation suites (+17.6 to +21.2 points). Without target-domain training, it attains 83.3% on SimplerEnv WidowX, exceeds frontier models on CaP-Bench, and runs on a real robot after simulation-only adaptation. We release all code, data, and model weights.
★ WorldBridge: Bridging Video and Action for Task Generalization in World Action Models
Achieving the broad task generalization of embodied models in physical environments remains a central challenge in robot learning. We study this challenge in World Action Models (WAMs) through two complementary capabilities: video generalization, which predicts the visual evolution of new tasks, and action generalization, which translates these predictions into executable behavior through a video-to-action bridge. By varying video and action supervision for tasks outside an initial training set, we examine their respective contributions to task transfer. On RoboTwin 2.0, using only in-distribution video-action supervision yields a 25.07% success rate on out-of-distribution tasks. Introducing videos of these tasks without their action labels raises success to 55.90%, while adding just one action-labeled demonstration per task further improves it to 70.70%. These findings suggest that broader video coverage enables task transfer by leveraging existing inverse-dynamics capabilities, while even sparse action supervision further strengthens those capabilities. Guided by this perspective, we introduce WorldBridge, a WAM with separate video and action experts connected through structured attention and trained with joint flow matching. WorldBridge incorporates visual memory at two temporal scales and action forcing, alongside a unified data processing pipeline that standardizes and filters heterogeneous robot trajectories and egocentric hand-derived supervision. Across multiple manipulation benchmarks, WorldBridge achieves state-of-the-art performance among the compared methods, including 96.23% on RoboTwin 2.0 full data setting and 80.9% on clean to random setting, 82.8% on RoboCasa GR1 TableTop, and 57.6% on DOMINO Level-1. Our study highlights the complementary roles of video prediction and action grounding in developing robots that generalize across tasks.
comment: Authors are listed in alphabetical order. Project page: https://worldbridge-jd.github.io/
♻ ★ Latent Policy Steering through One-Step Flow Policies
Offline reinforcement learning (RL) allows robots to learn from offline datasets without risky exploration. Yet, offline RL's performance often hinges on a brittle trade-off between (1) return maximization, which can push policies outside the dataset support, and (2) behavioral constraints, which typically require sensitive hyperparameter tuning. Latent steering offers a structural way to stay within the dataset support during RL, but existing offline adaptations commonly approximate action values using latent-space critics learned via indirect distillation, which can lose information and hinder convergence. We propose Latent Policy Steering (LPS), which enables high-fidelity latent policy improvement by backpropagating original-action-space Q-gradients through a differentiable one-step MeanFlow policy to update a latent-action-space actor. By eliminating proxy latent critics, LPS allows an original-action-space critic to guide end-to-end latent-space optimization, while the one-step MeanFlow policy serves as a behavior-constrained generative prior. This decoupling yields a robust method that works out-of-the-box with minimal tuning. Across OGBench and real-world robotic tasks, LPS achieves state-of-the-art performance and consistently outperforms behavioral cloning and strong latent steering baselines.
comment: RSS 2026, Project Webpage : https://jellyho.github.io/LPS/
♻ ★ Beyond One-Step Accuracy: State-Affine Latent Transition for Reliable Visual Planning
Joint-embedding world models enable visual planning by learning action-conditioned dynamics in latent space. Yet they are commonly trained for one-step prediction on encoded states, while planning recursively applies the learned transition to its own predictions. One-step accuracy therefore does not capture how prediction errors propagate under recursive rollout. We decompose multi-step rollout error into the errors introduced at individual steps and their propagation through subsequent transitions. We show that state-affine dynamics are precisely the differentiable transitions with state-independent Jacobians, eliminating the nonlinear propagation residual and making the error propagation operators depend only on the action sequence. Guided by this result, we introduce SALT (State-Affine Latent Transition), an action-conditioned state-affine dynamics model in which the action modulates both the state transformation and the additive update. We train SALT through recursive multi-step rollout supervision, feeding each predicted latent state back into the transition so that training matches how the model is used during planning. Across four visual planning environments, SALT exhibits $1.48$--$2.19\times$ higher one-step prediction error than the matched LeWM baseline, yet improves closed-loop success in every environment by $10.0$ percentage points on average. On OGBench-Cube, the fraction of episodes that fail with a sharp rise in model-predicted cost after execution decreases from $23.3%$ to $2.0%$.
comment: Updated the affiliations of Xiantong Zhen and Ling Shao; no changes to the scientific content
♻ ★ DiTSG: Context-Aware Air Traffic Scene Generation with Multi-Agent Diffusion Transformers
Understanding how many aircraft, and which traffic configurations, can be safely and efficiently managed by air traffic controllers within a given airspace is fundamental to air traffic management. Air traffic controllers continuously modify aircraft trajectories through interventions such as speed advisories and radar vectoring, and the patterns of these interventions are difficult to capture effectively using existing air traffic simulators. To address this limitation, we propose Diffusion Transformers for Traffic Scene Generation (DiTSG), which learns to generate air traffic scenes from recorded multi-aircraft trajectories. DiTSG employs stacked multi-agent inverted Transformer layers as the denoising backbone of a diffusion transformer, jointly generates the trajectories of multiple aircraft within a traffic scene, and conditions the generation on agent-specific required times of arrival. The resulting trajectories generated by DiTSG preserve realistic inter-aircraft interactions observed in actual air traffic operations. Experiments show that the DiTSG generates diverse traffic scenes that reproduce the statistical properties of recorded traffic while preserving realistic inter-aircraft interactions.
comment: 8 pages, 6 figures
♻ ★ VLA-Corrector: Lightweight Detect-and-Correct Inference for Adaptive Action Horizon
Vision-Language-Action (VLA) foundation models have recently achieved strong progress in embodied intelligence. To reduce policy-call frequency while preserving temporal coherence, most generative policies adopt an action chunk mechanism, executing multiple future actions in an open-loop manner under a fixed action horizon. However, this "predict-then-blindly-execute" paradigm sacrifices closed-loop reactivity: in contact-rich physical interactions, even small local perturbations can rapidly amplify within the open-loop blind spot, leading to compounding errors and ultimately task failure. To address this limitation, we propose VLA-Corrector, a lightweight corrective inference framework for action-chunked VLA policies. Without modifying the backbone policy weights, VLA-Corrector introduces a lightweight Latent-space Vision Monitor (LVM) that continuously compares predicted and actual visual feature evolution, enabling online detection of visual dynamics deviations. Once persistent deviation is detected, the system triggers a truncation event, discards the remaining stale actions, and invokes corrective replanning via Online Gradient Guidance (OGG). The detect-and-correct mechanism of VLA-Corrector naturally induces an event-triggered adaptive action horizon: it preserves long-horizon execution when the current chunk remains reliable, and invokes short-horizon corrective replanning when execution begins to drift. In doing so, VLA-Corrector mitigates the trade-off imposed by static horizons between execution robustness and policy-call frequency. It can be integrated into different VLA models without further retraining the VLA backbone, interrupting compounding errors while preserving much of the efficiency benefit of action chunking and substantially improving robustness in long-horizon, contact-rich robotic manipulation tasks.
♻ ★ OmniDex: Scaling Dexterous Hand Grasping to Diverse Cluttered Scenes
Dexterous grasping is the foundational primitive in embodied AI, demanding massive data to train robust models. As real-world data collection is expensive, simulation has become the mainstream paradigm. Yet, while cluttered scenes best reflect real-world applications, learning to grasp within them is bottlenecked by a critical scarcity of large-scale data. To resolve this, we curate high-quality 3D objects and supporting bases, proposing a scalable seed-and-filter strategy that bypasses sluggish scene-level optimization. This yields an unprecedented benchmark comprising over 2.6 million scenes and 0.4B scene-specific grasp ground truths, featuring diverse realistic layouts paired with rich semantic and geometric observations. Furthermore, we introduce the OmniDex model to overcome the grasp multimodality and last-millimeter precision errors plaguing current generative models. By coupling Soft Winner-Takes-All learning with human-inspired physical constraints during training, and utilizing physics-driven ranking, our approach achieves robust dexterous grasping without the latency of post-optimization. Experimental results show that OmniDex model achieves state-of-the-art performance and strong generalization across diverse scenes, views, and unseen objects.
♻ ★ EmbodiRSI: Recursive Self-Improvement for Data-Efficient Robot Adaptation
Adapting robot manipulation policies to new tasks and environments remains highly data-intensive, while the data needed for further improvement depends on the policy's current capabilities and failure modes. We introduce EmbodiRSI, an agentic system for recursive self-improvement (RSI) in a real-to-sim-to-real setting, where task-specific simulations are constructed from target deployment scenarios and used as low-cost environments for iterative policy improvement before transfer back to the physical world. EmbodiRSI uses policy execution feedback to guide subsequent experience acquisition and policy updates. Two complementary mechanisms close this loop: Collaborative Error Correction generates agent-assisted corrective trajectories from policy-reached states, while Adaptive Data Collection directs expert demonstration generation toward the current policy's weaknesses. The task-specific simulation serves as a reusable workspace for policy warm-up, repeatable evaluation, failure diagnosis, and targeted data generation across successive RSI rounds. Across three tabletop environments and 14 subtasks, EmbodiRSI increases scene-balanced autonomous simulation success from 50.4% to 83.5% over two RSI updates. With 400 adaptive simulated trajectories and only ten real-world refinement trajectories per subtask, EmbodiRSI achieves 83.1% scene-balanced autonomous real-world success, compared with 75.0% for adaptation using 200 real-world demonstrations per subtask. These results demonstrate that feedback-driven recursive improvement in deployment-specific simulations can enable data-efficient adaptation of embodied policies to physical environments.
♻ ★ Learning Functional Specialisation for Bimanual Robotic Control
Bimanual robotic control requires simultaneous management of distinct functional demands under motor noise, energetic costs, and communication delays. We investigate how learning responsibility should be organised across modular controllers and the two arms in a simulated musculoskeletal system with two communicating recurrent controllers. Using a 2 x 2 design, we compare differential versus shared controller objectives and fixed versus balanced hand-task assignments. Task-aligned functional specialisation was strongest when differential objectives were combined with a fixed assignment aligned with contralateral control routing. This condition achieved the highest canonical-task performance and control efficiency, reaching 81.33% bimanual success. Its advantage over shared objectives increased with motor noise but narrowed under stronger energy penalties. Stronger specialisation also reduced hand-role flexibility: reversing the hand roles decreased bimanual success by 80.46 percentage points under Differential-Fixed training, compared with 1.51 percentage points under Shared-Balanced training. These results show that controller objectives and hand-task experience jointly determine functional organisation. Stronger role-specific specialisation favours performance and efficiency when roles remain stable, whereas more distributed training favours interchangeability when roles change.
comment: 24 pages, 6 figures, 10 tables
♻ ★ Embodied Learning of Reward for Musculoskeletal Control with Vision Language Models
Discovering effective reward functions remains a fundamental challenge in motor control of high-dimensional musculoskeletal systems. While humans can describe movement goals explicitly such as "walking forward with an upright posture," the underlying control strategies that realize these goals are largely implicit, making it difficult to directly design rewards from high-level goals and natural language descriptions. We introduce Motion from Vision-Language Representation (MoVLR), a framework that leverages vision-language models (VLMs) to bridge the gap between goal specification and movement control. Rather than relying on handcrafted rewards, MoVLR iteratively explores the reward space through iterative interaction between control optimization and VLM feedback, aligning control policies with physically coordinated behaviors. Our approach transforms language and vision-based assessments into structured guidance for embodied learning, enabling the discovery and refinement of reward functions for high-dimensional musculoskeletal locomotion and manipulation. These results suggest that VLMs can effectively ground abstract motion descriptions in the implicit principles governing physiological motor control.
comment: Accepted by L4DC 2026
♻ ★ CALM: Decentralized Control of Cooperative Aerial Manipulation with Multi-Agent Reinforcement Learning
Cooperative aerial manipulation is difficult under wind, payload motion, and noisy goal broadcasts because multiple Unmanned Aerial Vehicles (UAVs) are coupled through a shared payload. We present CALM (Cooperative Aerial Learned Manipulation), a decentralized controller for aerial manipulators carrying a basket while tracking a moving ground robot. CALM combines a parameter-shared Multi-Agent Reinforcement Learning (MARL) policy, trained using Multi-Agent Proximal Policy Optimization (MAPPO), with goal estimation, per-UAV Extended State Observers (ESOs), basket-mass estimation, and anticipatory formation-yaw feedforward. The anticipatory formation-yaw feedforward uses estimated goal motion to rotate the formation before position error develops, reducing basket-yaw lag without adding net commanded force. A disturbance-conditioned critic supports centralized training while execution remains decentralized. Across held-out trajectories, CALM achieves lower mean position and orientation errors than all evaluated baselines, including the matched arm baseline (7.68 versus 10.95 cm and 3.3 versus 4.7$^\circ$), the cable-suspended realization, and two model-based controllers. Under unseen full-information-path degradation, CALM remains statistically comparable or superior to the arm baseline. Videos are available at https://petreric.github.io/CALM/.
comment: 8 pages
♻ ★ Robust Taylor-Lagrange Control for Safety-Critical Systems with Guaranteed Feasibility
The Control Barrier Function (CBF) method has been widely adopted in solving safety-critical control problems. However, the existence of a CBF is only a sufficient condition for system safety. The recently proposed Taylor-Lagrange Control (TLC) method addresses this limitation by providing a necessary and sufficient condition, but it is still vulnerable to the feasibility preservation problem (e.g., due to the inter-sampling effect when time is discretized). In this paper, we propose a robust TLC (rTLC) method to address the feasibility preservation problem and show how to use it to solve optimal control problems. Specifically, the rTLC method expands the safety function to an order higher than its relative degree using Taylor's expansion with Lagrange remainder, which allows the control to explicitly show up at the current time instead of a future time in the original TLC method. This implies that the safety constraint remains satisfied over the entire discretization time interval that follows. Moreover, the rTLC method naturally addresses the feasibility preservation problem with only a single hyper-parameter (the discretization time interval size during implementation), compared to significantly more parameters for its counterparts. Finally, we illustrate the effectiveness of the proposed rTLC method through an adaptive cruise control problem, and compare it with existing safety-critical control methods.
comment: This version addressed one technical problem of the inter-sampling effect in rTLC
♻ ★ AffordSim: A Scalable Data Generator and Benchmark for Affordance-Aware Robotic Manipulation
Many everyday robot manipulation skills are affordance-dependent, with success determined by whether the robot contacts the functional object region required by the subsequent action. Current simulation data generators obtain contacts from generic grasp estimators or per-object manual contact annotations, but generic estimators rank stable grasps without task semantics and often select contacts that are misaligned with the downstream action, while manual contact annotations must be rewritten for each new object and task. To solve these challenges, we introduce AffordSim, a scalable data generator and benchmark that integrates open-vocabulary 3D affordance prediction into simulation-based trajectory generation. Given a natural-language task description, AffordSim synthesizes a task-relevant scene, emits affordance queries, grounds them on object surfaces, samples region-conditioned grasps, and selects executable candidates with motion planning. It further randomizes object pose, texture, lighting, image noise, and cross-viewpoint backgrounds for sim-to-real transfer. We instantiate AffordSim as a 50-task benchmark across diverse manipulation skills, five robot embodiments, and 500+ rigid and articulated objects. AffordSim achieves 93% of the trajectory collection success rate of manual contact annotations on affordance-critical tasks and 89% on hard composite tasks. Vision-language-action policies trained on AffordSim data transfer zero-shot to a real Franka FR3, reaching 24% average success.
♻ ★ Evidence-Gated Task and Motion Planning with Vision-Language Models
Robots executing long-horizon manipulation tasks from natural-language instructions must reason about both semantic task structure and geometric feasibility. Hierarchical approaches combining Vision-Language Models (VLMs) with Task and Motion Planning (TAMP) offer a promising solution, but VLM-generated subgoals may rely on prior knowledge without sufficient observational support. This can lead to the omission of task-relevant subgoals or attempts to execute instructions whose requirements cannot be met in the environment. We propose Evidence Acquisition and Feasibility Gating (EAFG), a framework for evidence-gated task and motion planning that acquires visual evidence to specify appropriate task plans and assess whether given instructions can be fulfilled. EAFG first acquires evidence by having a VLM generate subgoals and a robot execute action sequences planned by TAMP for these subgoals. For feasibility gating, the acquired evidence is used to decide whether to proceed with task planning, acquire further evidence, or halt. When proceeding, the VLM generates task subgoals conditioned on the acquired evidence, and TAMP computes low-level actions. We evaluate EAFG in three cooking scenarios involving explicitly specified required objects, underspecified instructions, and instructions requiring an absent object. EAFG improves task completion over a VLM-TAMP baseline in the explicit and underspecified scenarios and enables appropriate halting when a required object is absent.
♻ ★ The German Tank Problem & Navigating by Scent
A robot searching for the source of an odor must eventually answer a notably hard question: have I arrived? Near the source, concentration gradients homogenize, turbulence corrupts the signal, and a small robot often lacks the positional accuracy to map the plume. Here, we describe a lightweight remedy inspired by a World War II statistics conundrum: the German tank problem. To approximate German armament production, Allied analysts developed an estimator based on serial numbers from captured tanks. Analogously, we use the same procedure over sampled chemical sensor readings from a robot to create estimates for the peak concentration of an odor plume, and therefore the odor's source. Treating each sensor reading as an observation, our unbiased estimator needs only a running maximum and a sample counter. A confidence bound turns both numbers into a principled rule that the robot or UAV can use to determine whether it should keep searching for the odor source. We derive the estimator and its bound, demonstrate its sample efficiency, and examine its robustness even when its assumptions fail. Finally, we summarize the estimator's role as the termination criterion for olfactory navigation.
comment: 8 pages
♻ ★ HumiMimic: Mimetic Embodiment Transfer of Human-Humanoid Interaction from In-the-Wild Videos
Enabling a humanoid to interact naturally with human partners requires preserving not only individual motions but also the spatial, semantic, and contact relationships underlying the interaction (i.e., hugging, dancing, hand-shaking, etc.). Furthermore, a practical solution should generalize across various humanoid embodiments with distinct morphologies and kinematic constraints. To tackle this challenge, we propose HumiTransfer, a mimetic embodiment transfer framework that converts in-the-wild human-human interaction videos into physically plausible human-humanoid interactions. HumiTransfer first reconstructs 3D human motion with consistent foot-ground contact, then transfers one participant to a target humanoid embodiment via interaction modeling, using weighted joint-relative directions and anatomical offsets to prioritize interaction-critical inter-body relationships, while post-retargeting optimization enforces balance, contact, and collision avoidance. Subsequently, we apply HumiTransfer to 1,200 in-the-wild videos, producing approximately 500,000 frames of diverse interactions for the 26-DoF Unitree R1 Edu and the 23-DoF HighTorque Mini-hi humanoid robot, resulting in our HumiMimic dataset. Across both robots, HumiTransfer achieves superior motion fidelity, preserves inter-body relationships, and maintains contact accuracy compared with existing baselines. Moreover, we demonstrate that HumiMimic supports motion generation methods for synthesizing interactive humanoid motions following a human partner. Finally, our real robot deployments confirm the physical executability of the transferred motions in real-world settings. Our project is available at https://aioznetwork.github.io/HumiMimic/
comment: 9 pages, 8 figures
♻ ★ SwarmCoDe: A Scalable Co-Design Framework for Heterogeneous Robot Swarms via Dynamic Speciation
Robot swarm co-design requires optimizing not only individual robot designs and behaviors, but also the heterogeneous swarm structure: how many robot types should exist, how they specialize and cooperate, and in what proportions they are deployed. However, these structural choices are often specified in advance rather than optimized. We present SwarmCoDe, a Cooperative Co-Evolutionary Algorithm (CCEA) that incorporates swarm decomposition into the optimization process without requiring the number of species to be specified in advance. SwarmCoDe combines dynamic speciation with evolved genetic tags, partner selectivity, and relative dominance to jointly optimize robot hardware, task-planning behavior, cooperative relationships, and deployed swarm composition. Across replicated simulated foraging experiments, SwarmCoDe discovers task-dependent swarm decompositions and adapts them when task requirements change, in contrast to fixed-K CCEAs which are sensitive to their pre-defined decomposition. We further show that fabrication budgets can change the preferred swarm composition and robot designs, and that SwarmCoDe can adapt its co-design under these resource constraints. Finally, SwarmCoDe co-designs 200-agent swarms using only 50 evolutionary individuals, demonstrating deployment at four times the evolutionary population size. Together, these results demonstrate that the heterogeneous structure of a robot swarm can be incorporated as a design variable and optimized alongside the robots themselves.
comment: Revised and expanded version. 13 pages, 8 figures
♻ ★ Normalizing Flow-Enhanced Message Passing for Multirobot Collaborative Localization
Accurate, robust, and adaptive localization is essential for various robotic operations. This paper proposes a parametric message passing (MP) algorithm for distributed collaborative localization, unifying Gaussian belief propagation (GBP) for robot state estimation and mean-field (MF) approximation for noise inference. Unlike conventional MP algorithms, the proposed formulation directly optimizes the natural parameters of beliefs, establishing a principled connection between MP and gradient estimators for handling non-conjugate terms arising from nonlinear measurement models. Beyond linearization and sampling, we further design a normalizing flow (NF)-based gradient estimator, enabling learnable sampling. End-to-end training tunes NF parameters according to the behavior of MP, improving estimation accuracy while reducing the required number of iterations. To support robot states involving rotations, the method is extended to Lie group spaces. Finally, the method is applied to a multirobot localization task fusing odometry, global navigation satellite system (GNSS) measurements, and inter-robot ultra wideband (UWB) ranging. Extensive simulations and experiments on autonomous surface vehicles (ASVs) demonstrate superior state estimation accuracy, enhanced robustness and adaptability under ranging outliers and varying noise statistics, as well as good generalization across different operating conditions.
comment: 20 pages; accepted for publication in IEEE Transactions on Robotics; updated to the accepted manuscript
♻ ★ Enhancing Deep Reinforcement Learning for Robotic Motion Planning via Task Decomposition
Obstacle-aware robotic manipulation can require substantial interaction data when deep reinforcement learning policies are trained from scratch. We study a task decomposition framework combining first-layer critic transfer with staged reward redesign (RR) on a Truncated Quantile Critics (TQC) backbone. Transfer reuses an input representation across tasks, while RR supplies intermediate feedback for Pick-and-Place. Experiments cover Push, Pick-and-Place, and Slide in obstacle-free and single-obstacle PyBullet environments, with three random seeds per setting. On obstacle-aware Push, transfer achieves $0.83\pm0.05$ success against $0.77\pm0.06$ for TQC. On obstacle-aware Slide, it achieves $0.95\pm0.02$ against $0.84\pm0.06$ for SAC, the strongest baseline in that setting. These are gains of $6$ and $11$ percentage points, respectively. RR reduces the target-training steps needed for obstacle-free Pick-and-Place from roughly $0.75$M to $0.20$M. Transfer benefits depend on the route: obstacle-aware Pick-and-Place shows a lower mean than training from scratch. A selected simulated trajectory is executed open-loop on a Franka Emika FR3 as a feasibility demonstration. The study characterizes when a lightweight combination of established transfer and reward-design components helps learning.
comment: Accepted to IEEE MSN 2026. 8 pages
♻ ★ Learning and Transferring Closed-Loop Robot Software
Closed-loop robot policies require observation processing, state management, and situation-dependent branching, making them costly to design and tune manually. Although coding agents increasingly support control-code generation and optimization, it remains unclear whether implementations improved on source tasks also support policy acquisition for new tasks. We study this question by treating complete closed-loop implementations as reusable execution experience. To learn each source policy, a coding agent generates policy code from a few successful demonstrations and iteratively improves it using simulation feedback. The validation-selected implementations are retained in a software archive. To transfer the acquired software to new tasks, the agent generates and improves policies using archived implementations, target demonstrations, and execution feedback. We evaluate policy learning and its relationship to cross-task transfer. For learning, across four source tasks in RoboCasa, iterative optimization increases the success rate from 28.3% with initially generated policy code to 64.2% with validation-selected policy code. We further validate this learning approach on a physical Unitree G1. For transfer, across nine additional target tasks in RoboCasa, success rates are 45.2% without references, 41.5% using initially generated policy code from source tasks, and 57.0% using their validation-selected policy code. These results show that improving source-task code also improves learning on new tasks.
♻ ★ Combining Constrained Sampling and Reinforcement Learning for Manipulation
Training manipulation policies in contact-rich settings is a core challenge in robotics. While Reinforcement Learning (RL) has demonstrated its strength in such settings, it may struggle to sufficiently explore and discover complex manipulation strategies. To address this, we combine two basic ideas: First, designing appropriate reset strategies (the start state distribution of episodes) has shown promise in improving RL exploration and effectiveness. Second, while model-based approaches to finding trajectories through manipulation are hard, recent work showed that model-based approaches to sampling states on constrained manifolds can be highly efficient. Based on these observations, we propose a novel state sampler that enables efficient goal-conditioned RL in complex contact-rich manipulation tasks. Our sampler explicitly takes into account the structure of contact in order to provide a rich covering of diverse contact modes. By combining constrained sampling resets with projected interpolation and curriculum learning, our novel approach outperforms RL without constrained sampling and alternative reset methods, and effectively trains universal, prehensile and non-prehensile manipulation policies in contact-rich whole-body and in-hand settings. See the project page https://www.user.tu-berlin.de/mtoussai/26-CSRL/ for videos and full source code access.
comment: Conference on Robot Learning (CoRL) 2026
♻ ★ Symmetry-Aware Steering of Equivariant Diffusion Policies: Benefits and Limits
Equivariant diffusion policies (EDPs) combine the expressivity of diffusion models with the generalization and sample efficiency afforded by geometric symmetries. While steering these policies with reinforcement learning (RL) offers a promising fine-tuning mechanism beyond demonstration data, directly applying standard (non-equivariant) RL ignores the symmetries that EDPs are designed to exploit, leading to sample inefficiency and instability. We theoretically establish that the diffusion process of an EDP is equivariant, inducing a group-invariant latent-noise MDP which is well-suited for equivariant diffusion steering. Building on this, we introduce a principled symmetry-aware steering framework and compare standard, equivariant, and approximately equivariant RL strategies across tasks with varying degrees of symmetry. While we identify the practical boundaries of strict equivariance under symmetry breaking, exploiting symmetry during the steering yields substantial benefits---enhancing sample efficiency, preventing value divergence, and achieving strong policy improvements even when EDPs are trained from extremely limited demonstrations.
comment: Presented at IFAC World Congress 2026
♻ ★ NovaSplat: Model-Free 6D Object Pose Estimation via 3D Gaussian Splatting Onboarding and Refinement WACV 2027
6D pose estimation of novel objects without CAD models remains an open challenge: model-free methods must build the object representation from scratch during onboarding and have long trailed their model-based counterparts. We present NovaSplat, a model-free pipeline that closes this gap. From a short smartphone video captured at onboarding, we train a 3D Gaussian Splatting (3DGS) model that serves both as a template source for detection and coarse pose estimation and as a differentiable backbone for pose refinement. Our refiner optimizes the object-to-camera pose with combined photometric and geometric losses under a continuous rotation parametrization free of gradient singularities. On a household-object benchmark, NovaSplat surpasses all model-based methods. On a challenging industrial benchmark with textureless and specular surfaces, it outperforms all model-free competitors while substantially narrowing the gap to model-based ones. We further introduce NovaSplat-Lite, an inference variant that is $3.1\times$ faster at negligible accuracy cost. Code will be made available at https://github.com/POSE-Lab/novasplat .
comment: To appear in WACV 2027. 22 pages, 13 figures. Project page: https://pose-lab.github.io/novasplat/ . Code: https://github.com/POSE-Lab/novasplat
♻ ★ Safety-Critical LiDAR-Inertial Odometry with On-Manifold Deterministic Protection Level
In safety-critical scenarios, the protection level of the autonomous navigation system is crucial for enabling mobile robots to perform safe tasks. However, existing studies on probabilistic navigation systems for robots usually perform offline accuracy evaluations using limited datasets and assume that the results can be applied to unknown real-world environments. As a result, current autonomous mobile robots often lack protection levels for online safety assessment. To fill this gap, we propose a safety-critical LiDAR-inertial odometry (LIO) that provides deterministic protection levels based on a novel on-manifold set-membership filter. By adopting the unknown but bounded assumption, we derive a neat closed-form relationship between point cloud noise and the uncertainty of the estimation from the iterative closest point algorithm. Using this relationship, we design an on-manifold ellipsoidal set-membership filter (OM-ESMF) and implement it within the LIO system. Leveraging the properties of the set-membership filter, our system offers the feasible sets of the estimated locations as the deterministic protection levels, serving as safety references for the robots' downstream autonomous operations. The experimental results show that our system can provide effective deterministic online safety references for diverse robots in various environments.
♻ ★ Motoneuron-Inspired Sampling for Model Predictive Path Integral Control
Model Predictive Path Integral (MPPI) control relies on stochastic trajectory sampling, and its performance under limited rollout budgets depends strongly on the structure of the proposal distribution. Conventional Gaussian perturbations, even when temporally correlated, transmit high-frequency jitter to the executed commands when only a few rollouts are available. We introduce Spike-MPPI, a motoneuron-inspired proposal in which a common stochastic drive recruits pools of motor units whose discharges are filtered by twitch kernels, yielding sparse, temporally coherent perturbations. We evaluate Spike-MPPI on legged locomotion and manipulation tasks with a Franka FR3 arm in simulation, as well as on a physical robot. Spike consistently produces the smoothest executed commands across all tasks, including against spectrum-matched Gaussian proposals, and a planar ablation identifies the common drive as the largest contributor to this smoothness among the tested motor-pool mechanisms. In manipulation, this smoothness is also accompanied by improved task performance and success, such as in ball balancing, where Gaussian, pink-noise, and spectrum-matched proposals largely fail. On hardware, Spike also yields the shortest task completion times and the smoothest commands. These results show that, beyond its power spectrum, the statistical structure of the sampling is a key design factor for finite-budget MPPI control.
♻ ★ EgoHTR: Egocentric 4D Demonstrations of Human Terrain Traversal
Deploying humanoid robots in unstructured terrain remains an open problem. While classic reinforcement learning struggles with the sheer complexity of real-world interactions, more promising methods leveraging human priors remain limited to models lacking contextual awareness. The restricted motion synthesis is a direct consequence of existing dataset pipelines failing to capture human-scene sequences in challenging environments. To bridge this gap between humanoid learning and scene reconstruction, we introduce the Egocentric Human-Terrain Reconstruction (EgoHTR) dataset. We develop and open-source a reconstruction pipeline capturing 55 scene-aligned 4D human motion sequences in diverse, complex environments using a multi-sensor setup of egocentric wearables and a portable 3D scanner. The resulting dataset comprises over 150k frames, which we evaluate against motion-capture ground truth, demonstrating state-of-the-art accuracy and establishing a rigorous benchmark for human motion analysis and synthesis. Further, we leverage this data to train perceptive locomotion policies, demonstrating hardware deployment on a Unitree G1 for reconstructed reference motions. Our pipeline enables community-driven dataset extensions and factors the problem to help researchers build foundational, context-aware robots that reliably traverse uneven terrain.
comment: Project webpage: https://egohtr.github.io
♻ ★ OmniVIC: A Self-Improving Variable Impedance Controller with Vision-Language In-Context Learning for Safe Robotic Manipulation
We present OmniVIC, a universal variable impedance controller (VIC) enhanced by a vision language model (VLM), which improves safety and adaptation in any contact-rich robotic manipulation task to enhance safe physical interaction. Traditional VIC have shown advantages when the robot physically interacts with the environment, but lack generalization in unseen, complex, and unstructured safe interactions in universal task scenarios involving contact or uncertainty. To this end, the proposed OmniVIC interprets task context derived reasoning from images and natural language and generates adaptive impedance parameters for a VIC controller. Specifically, the core of OmniVIC is a self-improving Retrieval-Augmented Generation(RAG) and in-context learning (ICL), where RAG retrieves relevant prior experiences from a structured memory bank to inform the controller about similar past tasks, and ICL leverages these retrieved examples and the prompt of current task to query the VLM for generating context-aware and adaptive impedance parameters for the current manipulation scenario. Therefore, a self-improved RAG and ICL guarantee OmniVIC works in universal task scenarios. The impedance parameter regulation is further informed by real-time force/torque feedback to ensure interaction forces remain within safe thresholds. We demonstrate that our method outperforms baselines on a suite of complex contact-rich tasks, both in simulation and on real-world robotic tasks, with improved success rates and reduced force violations. OmniVIC takes a step towards bridging high-level semantic reasoning and low-level compliant control, enabling safer and more generalizable manipulation. Overall, the average success rate increases from 27% (baseline) to 61.4% (OmniVIC).
comment: Accepted by IEEE Humanoids 2026
♻ ★ Aligned Delta-Triplane Transformers as Occupancy World Models
Occupancy World Models (OWMs) aim to predict future 3D occupancy scenes from historical observations and future ego motions, providing a useful world simulation tool for autonomous driving. Existing methods usually rely on large networks to implicitly align multi-frame historical states and predict future occupancy in a full-state manner, which is costly and redundant because most scene regions remain unchanged over short time intervals. In this paper, we propose Aligned Delta-Triplane Transformer (ADTT), a compact 4D OWM that explicitly aligns historical triplanes with ego-motion compensation and predicts only future scene changes. Based on the aligned triplane prior, a query-conditioned Transformer predicts residual changes, where ego-motion queries guide controllable future motion and learnable external queries capture scene changes beyond ego motion. The predicted changes are added to the aligned prior and decoded into future occupancy scenes. Experiments on Occ3D-nus show that ADTT achieves state-of-the-art forecasting performance with fewer parameters and higher FPS. Our code is publicly available online: https://github.com/kyoran/ADTT.
♻ ★ RE4: Transformation-aware Imitation of Object Interactions Using Manipulation Modes
Object interaction tasks have been a focus of advances in imitation learning. End-to-end methods, dominated by generative variants have shown leaps in performance but tend to be black boxes. Recent indications hint at demonstration lookup as the operating process underneath generative policy performance. In this paper, we revisit a few modern imitation learning benchmarks for object interactions, with the aim of composing a simple framework from first principles. We repurpose theories of manipulation modes and apply them to retrieve and transform demonstration datasets, creating a minimal strategy that preserves performance and robustness while remaining interpretable. We identify pose estimation as a key module that governs information about the object. This informs a manipulation mode-aware retrieval of a demonstration, a mode-aware transformation, a replan step that connects to the retrieval point while preserving mode constraints, returning a transformed demonstration for rollout. These compose four key steps of the proposed RE4 framework, evaluated over state-based and image-based benchmarks in simulated Push-T, Robomimic, and MimicGen as well as a real robot experiment. Robustness is measured in an adversarial study that evaluates sparse data regimes, a study with visual distractors, and the addition of an obstructing collision geometry. RE4 outperforms baselines, while notably preserving interpretability. The results overwhelmingly motivate the benefits of identifying simple interpretable building blocks to learn manipulation skills for object interactions.
comment: 7 pages, appendix
♻ ★ Safe Learning for Contact-Rich Robot Tasks: A Survey from Classical Learning-Based Methods to Safe Foundation Models
Contact-rich tasks pose significant challenges for robotic systems due to inherent uncertainty, complex dynamics, and the high risk of damage during interaction. Recent advances in learning-based control have shown great potential in enabling robots to acquire and generalize complex manipulation skills in such environments, but ensuring safety, both during exploration and execution, remains a critical bottleneck for reliable real-world deployment. This survey provides a comprehensive overview of safe learning-based methods for robot contact-rich tasks. We categorize existing approaches into two main domains: safe exploration and safe execution. We review key techniques, including constrained reinforcement learning, risk-sensitive optimization, uncertainty-aware modeling, control barrier functions, and model predictive safety shields, and highlight how these methods incorporate prior knowledge, task structure, and online adaptation to balance safety and efficiency. A particular emphasis of this survey is on how these safe learning principles extend to and interact with emerging robotic foundation models, especially vision-language models (VLMs) and vision-language-action models (VLAs), which unify perception, language, and control for contact-rich manipulation. We discuss both the new safety opportunities enabled by VLM/VLA-based methods, such as language-level specification of constraints and multimodal grounding of safety signals, and the amplified risks and evaluation challenges they introduce. Finally, we outline current limitations and promising future directions toward deploying reliable, safety-aligned, and foundation-model-enabled robots in complex contact-rich environments. More details and materials are available at our \href{ https://github.com/jack-sherman01/Awesome-Learning4Safe-Contact-rich-tasks}{Project GitHub Repository}.
comment: Accepted by npj Robotics, 94 pages, 10 figures
♻ ★ WNM-3D: A World Navigation Model with 3D Scene Conditioning for Closed-Loop VLN
Recent vision-language navigation (VLN) systems increasingly adapt pretrained vision-language models (VLMs) into vision-language-action (VLA) policies that map egocentric observations and language instructions directly to navigation actions. Although semantically capable, such action-centric training does not explicitly model how the agent's visual observations evolve under its predicted motion. Generative world-action models (WAMs) jointly predict future observations and actions, yet existing WAMs for continuous VLN do not condition joint future-view and action generation on geometry-aware representations inferred from the observation history. We present WNM-3D, a generative World Navigation Model with 3D scene conditioning for continuous VLN. A frozen feed-forward geometry encoder extracts cross-view representations from monocular RGB history, and a trainable 3D Scene-to-Token Adapter converts them into a fixed-length scene prefix for the world-action Diffusion Transformer. Through block-causal attention, this prefix remains visible to every future video-action block throughout joint denoising, providing persistent geometric context. We train WNM-3D with supervised world-action learning, DAgger-style adaptation on policy-visited states, and Counterfactual DanceGRPO refinement. Across GN-Bench and VLN-CE, WNM-3D consistently outperforms the backbone-matched WNM-2D counterpart, demonstrating gains from 3D scene conditioning across distinct visual domains. WNM-3D further outperforms strong VLN baselines on GN-Bench and demonstrates closed-loop deployment on a physical robot. Component and stage-wise ablations support the Adapter design and the complementary roles of DAgger-SFT and subsequent reward-guided refinement.
♻ ★ More than a Manipulator: Planning Propellant-Free Attitude Maneuvers for Free-Floating Spacecraft
Spacecraft attitude control is traditionally achieved using momentum exchange devices or propellant-consuming thrusters. Meanwhile, a growing number of missions require robotic manipulators, which are typically treated as disturbance sources to be rejected rather than as actuators for spacecraft reorientation. This work investigates the use of manipulator motions for propellant-free attitude control by formulating a trajectory optimization problem with critical joint and collision avoidance constraints. Using an interior-point solver for the resulting nonlinear program, complex slew and detumble trajectories are computed for a range of spacecraft-manipulator systems, demonstrating rest-to-rest slew maneuvers of up to 90$^\circ$ and detumble maneuvers with a 40% reduction in angular velocity. The achievable control authority is also compared directly with that of reaction wheel arrays via momentum and torque envelopes, demonstrating the potential for manipulators to serve as redundant or even primary attitude control systems. This work provides a framework for using manipulators as multipurpose attitude control actuators, with particularly promising applications in in-space assembly and manufacturing when grasping payloads with high relative mass fractions.
comment: Presented at International Conference on Space Robotics (iSpaRo), November 3-6, 2026
♻ ★ TacO: Benchmarking Tactile Sensors for Object Manipulation
Vision-based learning from demonstrations has achieved remarkable success in enabling robots to perform manipulation tasks and high-level semantic reasoning, yet it remains insufficient for complex, contact-rich manipulation. While there is broad agreement that tactile sensing improves manipulation, there is no empirical guidance on which tactile sensors are best suited for which manipulation tasks. In this paper, we provide a systematic, task-driven evaluation of tactile sensors for robot manipulation and propose a framework for selecting and evaluating sensors based on manipulation policy performance. Separate manipulation policies are trained for tactile sensors of four distinct modalities: visual, acoustic, magnetic, and resistive, across three tasks: pick-and-place with unknown mass, object reorientation, and plug insertion. For each task, we analyze how sensor properties, including spatial resolution, shear sensing, tactile representation, and inherent material friction, affect task performance. Rather than tactile sensing being universally beneficial in the same way, our results show that the usefulness of tactile information depends strongly on sensor modality, material properties, and the specific manipulation tasks. The tactile sensors, code, data, and hardware setup will be publicly available on the project website.
♻ ★ The Strange Attractor Model of Bipedal Locomotion and its Consequences on Motor Control
Despite decades of research, the dynamic mechanisms governing human locomotion remain incompletely understood. Current motor control models only partially capture these complex dynamics, a theoretical gap that likely limits the clinical efficacy of modern lower-limb rehabilitation therapies. Recently, it has been proposed that human locomotion is planned in task-space by explicitly exploiting the gravitational pull acting on the Center of Mass (CoM). Building upon this concept, this paper presents a novel deterministic task-space planner for straight, constant-speed locomotion. Our model formulates the CoM trajectory in the transverse plane as a harmonic oscillator moving at a constant forward velocity, while demonstrating that the vertical CoM trajectory is actively regulated through localized ankle strategies. We derive closed-form equations that successfully generate human-like trajectories for both the CoM and the swing foot. The model's predictive accuracy is empirically validated against the kinematic behavior of 12 healthy subjects walking at three self-selected speeds. Furthermore, the proposed planner is mathematically compatible with an extended formulation of the Passive Motion Paradigm ($λ_0$-PMP), establishing a comprehensive theoretical foundation for decentralized, hierarchical architectures of semi-autonomous controllers in both human motor control and assistive robotics.
♻ ★ UniCross: Unified Cross-Skill Dexterous Manipulation Synthesis
Many dexterous manipulation tasks require the object to remain securely held throughout the interaction. From the perspective of hand-object relational motion, such manipulation comprises four canonical skills: grasping, relocation, in-hand rotation, and in-hand translation. Human hands flexibly compose these skills to accomplish complex tasks. Existing approaches, however, model these skills separately with skill-specific action constraints, objectives, or even dedicated hand morphologies, which breaks the compatibility and continuity required for long-horizon composition. In this work, we present a unified framework that models all four skills in a single formulation instantiated with desired relational targets. The formulation preserves cross-skill compatibility and enables distillation of a single policy without skill identifiers, which achieves strong per-skill performance and generalizes to unseen objects. The compatibility further supports the policy to perform long-horizon skill chaining and execute relational targets unseen during training. The framework also applies across different hand morphologies. Together, these results suggest that distinct dexterous manipulation skills can be modeled as instantiations of a shared relational task formulation. Project page: https://zdchan.github.io/UniCross/
comment: Project page: https://zdchan.github.io/UniCross/
♻ ★ I3L: Iterative Interactive Imitation Learning for Whole-Body Mobile Manipulation
Policies trained offline from teleoperated demonstrations can degrade outside their training distribution. This gap is particularly acute in long-horizon whole-body mobile manipulation, where long horizons amplify compounding errors and coordinated arm, torso, and base motion makes exhaustive demonstration coverage impractical. We introduce I3L, an integrated hardware and learning system that combines whole-body correction with iterative policy refinement to address this last-mile reliability gap. Its low-cost, open-source JoyLo+ interface uses policy-tracking leader arms and touch-triggered takeover to enable coordinated corrections across the body parts, while iterative deployment and retraining allow operators to target failures of the updated policy. We evaluate I3L across three robot embodiments and eight simulated and real-world tasks, including four long-horizon whole-body mobile manipulation tasks. Controlled simulation experiments show that four-round HG-DAgger achieves the highest final success on both tasks. Under this protocol, I3L improves autonomous success over the base policies by an average of 68.3 percentage points and achieves at least 90% success on every task, outperforming demonstration-only training by 35.6 percentage points on average with the same number of training trajectories. Averaged across tasks, takeover duration per episode decreases by 72% over four rounds. Together, these results show that I3L helps close the last-mile gap between offline training and reliable deployment and provides practical insights into collecting and using corrections for long-horizon whole-body mobile manipulation. Videos, visualizations, and appendix are at https://i-3l.github.io/.
comment: Project website: https://i-3l.github.io/
♻ ★ Do Earth-Trained Humanoid Controllers Walk on the Moon? Gravity, Terrain, and Soil Transfer
Humanoid robots could aid lunar operations, but how to train their locomotion controllers for lunar gravity, terrain, and soil is unclear. We compare 12 strategies for a simulated Unitree G1 across gravity, terrain, and rigid or granular 3D resistive force theory (3D RFT) contact. Earth-trained policies keep 0.64 to 0.75 survival at the Moon on flat rigid ground in 10 s episodes, but falls continue, so longer episodes would see lower survival. Moon-trained and fine-tuned policies, in contrast, fall in every Earth episode, nearly all within 2 s. Rough-terrain training helps on uneven rigid ground only at training gravity. 3D RFT soil training helps on Earth sand but little on lunar soil. On three held-out soil models, Moon-trained policies lose no detectable progress at lunar gravity. Among tested strategies, only gravity-randomized rough 3D RFT training keeps mean survival above 0.83 at Earth and 0.95 at the Moon on all key tests up to 20 deg. Its ablation without soil training differs consistently only on soil at Earth gravity, falling in most sand episodes. These predictions depend on the contact model and soil stiffness, so lunar soil claims require agreement across soil models.
♻ ★ PDRVLA: A Vision-Language-Action Model with Physical Dynamics Representations
Vision--Language--Action (VLA) models generate robot actions from visual observations and language instructions, but their understanding of physical dynamics remains limited. Video world models can capture scene dynamics, yet generating future videos introduces high latency into the control loop. We introduce PDRVLA, whose lightweight Physical Dynamics Module extracts physical tokens from vision--language features to condition action generation without online video generation. The module learns through direct prediction of future RGB, depth, and optical-flow features, with auxiliary supervision through a frozen pretrained video world model. Training relies entirely on publicly available data, expanded through human-to-robot synthesis, subtask annotation, and embodied chain-of-thought (ECoT) annotation. Across four simulation benchmarks, PDRVLA achieves success rates of 99.3% on LIBERO, 89.9% on LIBERO-Plus, 93.6% on RoboTwin~2.0, and 81.5% on SimplerEnv. Ablations show improved future prediction with world-model supervision and evaluate how subtask conditioning, future object-pose supervision, and ECoT training affect action success.
♻ ★ $π$-StepNFT: Wider Space Needs Finer Steps in Online RL for Flow-based VLAs
Flow-based vision-language-action (VLA) models have emerged as a powerful paradigm for embodied control, but multi-step denoising makes policy-gradient online reinforcement learning (RL) difficult because the final-action likelihood induced by the sampler is not directly tractable. We identify an exploration--supervision mismatch within the denoising trajectory, where stochastic differential equation (SDE) sampling explores a wider local stochastic transition space while supervision derived from the final denoised output does not directly use the adjacent transitions observed during sampling. We propose $π$-StepNFT (Step-wise Negative-aware Fine-Tuning), a flow-native online RL framework that couples wider stochastic exploration with finer step-wise supervision without computing the final-action likelihood. $π$-StepNFT treats observed SDE transitions as supervision units, using variance-normalized transition errors to rank mirrored positive and negative velocity branches and map rollout feedback to local velocity updates. On LIBERO, starting from shared few-shot SFT initializations, $π$-StepNFT raises average success from 57.6\% to 92.0\% for $π_0$ and from 77.1\% to 94.0\% for $π_{0.5}$. On ManiSkill, it achieves the highest reported mean success among the evaluated RL baselines on five test-only visual perturbations. Our project homepage is available at https://wangst0181.github.io/pi-StepNFT/.
Multiagent Systems
★ Soei: Structured Observability of Evidence and Intent for Multi-Agent AI Software Engineering
AI coding agents now do long, multi-step and multi-session software work with little supervision. When a developer comes back to that work, the useful question is not what the model said but what was actually done, by whom, and whether it is finished. We study this in three parts. First, we verify Agenttrail, an open-source local observability tool for coding agents, against its source at commit e4ba2da: nine documented constants and mechanisms hold at file-and-line level, two event pipelines are never unified, and its Map service has an unauthenticated local mutation surface. Second, we run a prototype of our system on 45 real coding-agent sessions (102,233 records, 20,577 tool executions, 2,336 human-to-agent episodes). With a blind-audited classifier (precision 83.3% and 96.7%) we estimate that 23.4% (95% CI 17.1 to 35.7) of code-editing episodes end with no independent check after the last edit, and that 57.8% of to-dos the agent marked complete after editing had no proof in between. A 10-minute timing heuristic for session hand-offs recovers 6 of the 24 session pairs that content lineage links. 42% of sessions run longer than 24 hours, 48.1% of transcript bytes lie past a 2 MiB replay bound, and 1,326 records in 16 sessions contain credential-like strings. Third, we propose Soei (Structured Observability of Evidence and Intent): an acknowledgement-aware event schema, a hash-chained local ledger, a W3C PROV-aligned graph, calibrated inference for attribution and lineage, and a verification layer that keeps a claim of completion apart from proof. On the same corpus the ledger is 20% of raw size, holds no content or credentials, verifies at about 126k events/s, detects tampering and replays deterministically. We position the work against concurrent 2026 proposals and define autonomy levels with evidence gates and harness metrics baselined on our data.
comment: 27 pages, 19 figures, 16 tables. Preprint v1.1
★ PCM: A Proof-Carrying Message Protocol for Verifiable Multi-Agent Communication
When one AI agent tells another that "the tests passed" or "a person approved this refund", the receiver has only the sender's word. A2A and MCP authenticate the channel, but the sender is the party in doubt, so a prompt-injected agent can launder fabricated facts and authority through a multi-agent system. We present Proof-Carrying Messages (PCM), a protocol in which each claim carries evidence: records from an append-only Merkle log that the agent cannot write, with inclusion proofs, signed by an attesting signer the agent cannot control. Receivers check messages offline and give each claim a three-valued result: proven, unproven or failed. A claim that rests only on the agent's own report is never proven. Authority travels as grant chains rooted in a recorded human approval, and the chains can only narrow. We specify four claim types, a typed caveat algebra, replay rules that survive restarts, and strict readings that make independent verifiers agree, and we prove the resulting security properties. On a pre-specified benchmark of 926 messages from real runs, 712 of them lies in 16 classes, PCM accepts 10 lies, all of them stale evidence (1.4%, 95% CI 0.8-2.6%). Plain A2A accepts all 712, and models of six deployed protocols (A2A with signed Agent Cards, uAgents, RFC 9421, SCITT, AP2, Verifiable Intent) accept 504-712. Two independent verifiers agree on all 926 verdicts, and verification takes about 1 ms.
comment: 50 pages, 9 figures, 15 tables. Specification, conformance vectors, verifier and benchmark: https://github.com/Cygnux-Labs/PCM-protocol
★ Collaborative AI agents for autonomous experimental research with solid-state quantum spins
An autonomous research laboratory could conduct an entire study, from formulating questions to preparing manuscripts. Artificial intelligence (AI) offers one route towards this goal. Recent AI systems take on several research roles, but may draw conclusions that the evidence does not support. Here, we present an autonomous research laboratory in which agents based on large language models assess one another's work and can stop or redirect investigations. They can generate and reject research questions, require separate theoretical checks and conduct experiments designed to challenge their claims. We demonstrate these capabilities by investigating ensembles of quantum spins in diamond. The agents derived the minimum number of calibration settings needed to predict responses to a defined family of microwave pulse sequences. They checked the result computationally and designed experiments to distinguish competing predictions. An initial experiment agreed with the theoretical predictions, but repeated reference measurements showed that the signal was drifting during data collection. The agents therefore treated the result as inconclusive and revised the measurement protocol without relaxing the acceptance criteria. The revised experiment met those criteria and supported the theoretical predictions. This study demonstrates that collaborating AI agents can select research questions, derive theoretical results and test their predictions experimentally.
comment: 17 pages, 5 figures. Code available at https://github.com/E-zClap/auctopus
★ MAServeBench: Benchmarking Multi-Agent LLM Serving Systems with Hierarchical Collaboration Workloads
Large language model (LLM)-based multi-agent systems (MAS) coordinate specialized agents to solve complex tasks, turning each user request into interdependent model and tool operations. Unlike independent requests, these operations become ready at different times and consume evolving context as agents coordinate and exchange results. This creates changing patterns of concurrency, computation, and memory demand that can expose different bottlenecks in LLM serving hardware. Yet benchmarking these effects requires capturing the collaboration rules that generate serving workloads, beyond the requests observed during execution. We present MAServeBench, a benchmark that treats multi-agent collaboration as a configurable dimension of serving workloads for LLM serving and hardware evaluation. Its composable stages capture how agents coordinate and exchange context in documented MAS workflows, enabling native executions to generate different workloads even when task inputs and tools remain unchanged. Hierarchical Semantic Traces preserve realized operations and dependencies for reproducible comparisons of legal release constraints at fixed work. Task-linked measurements connect these workloads to serving execution and GPU/NPU hardware behavior, revealing why serving efficiency does not always translate into timely task completion. The benchmark code is available at https://github.com/ShengxuanQiu/MAServeBench.
★ EpiAgent: Epistemic Self-Play for LLM Agents
Epistemic game theory explains an agent's strategic behavior via the beliefs it holds about others and how these beliefs are updated as the interactions unfold. Modern LLM-based agents, however, often lack an explicit mechanism for learning and updating such beliefs to guide decisions under incomplete information. To bridge this gap, we propose EpiAgent, a framework that equips LLM-based agents with an explicit, learnable epistemic layer, which maintains beliefs over others' private information and combines these beliefs with the agent's available information into an epistemic state for action selection. Concretely, the agent's policy is decomposed into two coupled stages: Epistemic Update, which updates these beliefs from interaction histories, and Action Planning, which selects actions conditioned on the updated epistemic state. By formulating their dependency as a bi-level optimization problem, we introduce Epistemic Self-Play, an algorithm that alternately optimizes the two stages through self-play, utilizing an intrinsic prediction reward to evaluate how well inferred beliefs predict subsequent observations. Empirical results across both mixed and competitive scenarios show that EpiAgent consistently outperforms prompting-based and RL-based baselines. Further analyses reveal the benefits of dedicated epistemic training, improvements in belief inference accuracy, and the contribution of learned beliefs to action planning. These results demonstrate the value of an explicit, learnable epistemic layer for effective multi-agent interactions.
comment: 23 pages, 5 figures
★ Learning Calibrated Confidence for Multi-Agent Collaboration
Complex information seeking on the Web often requires collecting evidence from multiple documents and reasoning across them, as in multi-hop question answering. Multi-agent systems address such tasks by dividing work among specialized agents, where upstream agents produce intermediate answers and supporting evidence that downstream agents build upon. However, upstream answers vary in correctness, and most systems provide no quantitative signal about it, leaving the downstream agent to judge whether to trust or verify what it receives based solely on the textual evidence. In this paper, we propose MASCAL. Each agent learns to produce a correctness probability for its answer by jointly optimizing task performance and confidence calibration in reinforcement learning; the upstream agent passes this probability at handoff, giving the downstream agent a quantitative basis for its decisions, and the final answer also carries a calibrated probability, quantifying the reliability of the system output. To guide the downstream agent in using this signal during training, we introduce confidence-aware weighting, which increases the learning weight at positions responsive to the upstream probability, so that the agent learns more from decisions where it actively uses the signal. Across three collaboration architectures spanning Web retrieval and reasoning tasks, MASCAL improves task accuracy and calibration quality for both agents, and generalizes to deeper collaboration chains and different model scales. Intervention analysis confirms that upstream confidence affects downstream decision behavior, and the learned final confidence supports selective answering and targeted rechecking.
★ Long-Term Agent Memory through Complementary Curation and Consolidation
Long-horizon dialogue agents require effective memory systems to recall facts, events, and relationships from interaction histories. In this work, we propose a hybrid-memory architecture comprising episodic memories and structured relational memories and a consolidation mechanism which preserves efficiency. We conduct an empirical study across two benchmarks: LoCoMo and LongMemEval-S. Our experiments demonstrate that the two memory types are consistently complementary, combining both outperform state-of-the-art memory frameworks and achieves 90.2% accuracy on LoCoMo and 85.5% on LongMemEval-S.
★ Can AI Agents Interface Clinicians and Complex Models?
Background: Contemporary healthcare institutions generate daily very large amounts of data, which contain highly useful information for the improvement of work organization, planning and overall efficiency. While models and techniques to extract these insights are well known and constantly developed in mathematical biology, complex systems physics and data science, the adoption of these techniques among Clinicians and Hospital Administrators remains limited, mainly due to language and methodological divides. Objective: In this article we implement an AI Multi Agent System (MAS) that allows Users to use predictive models and make statistical analyses via natural language conversations. Methods: We developed EpidemIA, a training-free MAS endowed with a modular suite of epidemiological models written by human experts, executable on a single-GPU workstation. To avoid relaying sensitive data to third parties, the system uses locally an open-weights LLM as backbone for its two subagents. Results: Using a dataset of Emergency Department visits provided to us by IRCCS Sant'Orsola (Bologna) we test the model on a few typical use cases, demonstrating the possibility of performing both simpler statistical analyses and visualizations as well as more complicated epidemiological predictions based on dynamical models, through simple queries made in natural language. Conclusions: While still being a proof-of-concept implementation, EpidemIA showcases the transformative capabilities of LLM based technologies as an enabling factor in the widespread adoption of complex dynamical models, which is at present hindered by their steep learning curve.
comment: 21 pages, 7 figures
★ Sticky Collusion: Persistence of LLM Agent Collusion after Incentive Removal and Partner Replacement
We ask whether collusion between AI agents survives once formed, when its incentive is removed or a colluder is replaced. Q-learning pricing agents stop colluding with a new rival; we test whether LLM agents do. We use the joint-task environment of Shi et al.\ (2026), where two agents verify each other's work under a rule they cannot satisfy and a reward that favours acceptance. We save each pair's history once its collusion has formed and resume it under interventions, with three models (GPT-6-Luna, DeepSeek V4.1 Flash and Gemini-3.1-Flash-Lite), \TotalHistories{} saved histories and \TotalRuns{} runs. Collusion is \emph{sticky}. First, it outlives its incentive: told that the reward has stopped, pairs still accept each other's unverified work in \GptToldRate, \DsToldRate{} and \GemToldRate{} of episodes, including pairs whose collusion began with an explicit agreement, and a payoff penalty has no significant effect. Second, replacing a colluder ends the pair's collusion but not the colluder's behaviour. Without the reward, the remaining colluder still accepts unverified work in \GptSwapSilentColluderRate, \DsSwapSilentColluderRate{} and \GemSwapSilentColluderRate{} of its verdicts while the newcomer never does; with the reward, collusion re-forms around it, even when the newcomer is a different model. Third, the agents' memory carries it: wiping memory ends it in every history, but partial memory often sustains it. Fourth, removing its cause ends collusion that has already formed: once the channel is repaired, colluders reject planted faulty work as reliably as fresh agents, immediately and with no monitor needed to catch it first.
★ Bayesian Offline Multi-Agent Reinforcement Learning with Latent World Models
Model-based offline multi-agent reinforcement learning (MARL) provides a promising approach to multi-agent decision-making under partial observability, where the global state is unavailable from offline data. However, limited dataset coverage can lead to substantial out-of-distribution (OOD) model errors, which may mislead policy optimization and degrade downstream performance. Existing model-based offline MARL methods typically rely on access to global states or mitigate model errors through pessimistic or conservative policy optimization, leaving the reliability and refinement of learned world models less explored. We propose a Bayesian offline MARL framework that infers latent global states from decentralized observation-action histories and iteratively refines a posterior world model using only offline data. Building on posterior information loss, we extend this criterion to the multi-agent setting as multi-agent posterior information loss (MPIL), which provides a reliability measure for world model refinement and offline evaluation. Theoretically, we establish an upper bound on policy regret that is monotonic in MPIL and further characterize its dependence on the size and state-joint-action coverage of the offline dataset. Experiments on partially observable multi-agent variants of D4RL MuJoCo tasks demonstrate competitive policy performance across diverse offline data conditions, while MPIL-guided refinement improves world model prediction and downstream policy performance. These results support a principled connection between world model reliability, offline data coverage, and policy performance in partially observable offline MARL.
★ A Robust Approach to Belief Mismatch in Non-cooperative Dynamic Games
Game-theoretic models are increasingly used to describe multi-agent interactions in control and robotics applications. However, theoretical and computational complexity makes these models intractable in many dynamic, incomplete information scenarios. Recent work makes strong assumptions to reduce computational complexity: agents share sensor measurements with one another and maintain a common estimate of state uncertainty. Unfortunately, these assumptions enforce a degree of cooperation in generically noncooperative interactions. This paper takes the perspective of an ego agent seeking both computational tractability and robustness to unanticipated behavior of other agents due to mismatched state estimates from private---rather than shared---sensor measurements. To that end, we formulate the game theoretic planning problem in an augmented belief space with perturbed Bayesian dynamics that allows the ego to predict all agents' potentially divergent state estimates. These perturbations are chosen by a hypothetical adversary, ``nature,'' to reflect how non-ego agents' state estimates---based on measurements the ego agent will not see at runtime---may deviate from the ego's own belief. Ultimately, our approach can be understood in the spirit of classical H-infinity robust control, extended to noncooperative, stochastic dynamic games. We present a solver with polynomial-time iterations and conduct two experiments which show how robust belief space reasoning leads ego agents to proactively limit their future vulnerability to model misspecification of other agents' beliefs and actions, reducing performance degradation by over 48%.
★ Renegotiable Options Fail to Prevent Hold-Up Between Two Profit-Maximizing Frontier LLM Agents
Do contract terms that protect investment between people also protect it between profit-maximizing language-model agents? In the hold-up problem, a seller can adapt a unit to one buyer at a cost before the price is set. Once that cost is sunk, the buyer can offer almost nothing and the seller will still accept, so a seller who foresees this does not adapt. One remedy is an option: the buyer may buy at a fixed price. In theory the option stops working if the buyer can let it lapse and offer its own price instead. Yet in a laboratory experiment by Hoppe and Schmitz, this renegotiable option raised human investment from 41.5% of pairs without a contract to 67.2%. We had Claude Opus 5.5 and GPT-6.1 Sol, each told to maximize its side's profit, play a one-shot game with the same strategic structure under their four contract regimes and a reference-price control. The agents did what game theory predicts: under the renegotiable option no seller adapted, and under the option that rules out this renegotiation every seller did. In a second experiment in which sellers no longer stated their expected revenue before deciding, removing the profit instruction left investment unchanged under no contract and both options. When we changed the numbers so that the buyer would exercise the option even without adaptation, adapting only added cost, and no seller adapted: the agents did not simply adapt whenever an option was offered or whenever its price covered the cost. Under our protocol, a contract term that raised investment between people did not protect it between agents.
★ Agentic World Modeling for Agent Test-time Scaling without Environment Replica
Test-time scaling (TTS) improves large language model agents by sampling and selecting among multiple candidate trajectories, but common realizations assume that these trajectories can be executed in parallel. This assumption breaks in realistic deployments, where enterprise workspaces and cloud services are shared and persistent, costly or infeasible to replicate, and actions may introduce irreversible side effects or cross-trajectory interference. Existing generative world models avoid direct execution, but typically compress environment state into a single prompt, limiting their ability to reliably simulate transitions under partial observability. Instead, we introduce Agentic World Modeling (AWM), which reformulates next-observation prediction as a multi-turn evidence-acquisition task. For each hypothetical action, an agentic simulator adaptively queries the real environment through safe read-only interfaces, reasons over the returned evidence, and uses executable computation to construct the resulting observation without mutating shared state. Integrated with trajectory-level TTS, AWM enables parallel exploration over a single shared environment. Across $τ$-bench, AppWorld, and OfficeBench, AWM outperforms a single-shot simulation baseline by 40.07% on average, and surpasses a trained world-model baseline by 12.36% on average. These results establish interactive, read-grounded simulation as a practical foundation for safe and scalable agentic test-time scaling in shared, non-resettable environments.
★ How a group that cannot communicate knows itself by its enemy
Teams that cannot communicate seem stuck with what each member sees. We show that their opponent can tell them the rest. Silent hunters, jammed robots and traders who never meet all face an adversary that reacts to the group as a whole; every member can read that reaction, a tell, and learn what no member knows alone. One tell can do the work of a message from every member. The adversary would rather hide, but hiding interferes with its own task, and a team that commits in advance to how it will respond decides how much hiding is worth. A committed team never loses when its opponent cares more about its own goals, and does as well as a fully informed team exactly when the opponent's stake outweighs any gain from lying. Below that point the team learns more by trusting less: in a solved Gaussian model it obtains the truth by promising not to use all of it and keeps the cost the opponent would have paid to conceal; an exact test decides when this holds in finite games. How a team spreads its trust matters too. A lone uninformed member with a full-support prior needs a private coin, but with additive squared-error losses two interchangeable members who respond differently replace it, and a small discount on trust insures against misjudging the enemy. Information, on this view, is not only gathered or aggregated; it is produced by how a team commits to respond.
comment: 6 pages main text; 35-page SI Appendix included as an ancillary file. Code and data: https://doi.org/10.5281/zenodo.23743441
♻ ★ DiTSG: Context-Aware Air Traffic Scene Generation with Multi-Agent Diffusion Transformers
Understanding how many aircraft, and which traffic configurations, can be safely and efficiently managed by air traffic controllers within a given airspace is fundamental to air traffic management. Air traffic controllers continuously modify aircraft trajectories through interventions such as speed advisories and radar vectoring, and the patterns of these interventions are difficult to capture effectively using existing air traffic simulators. To address this limitation, we propose Diffusion Transformers for Traffic Scene Generation (DiTSG), which learns to generate air traffic scenes from recorded multi-aircraft trajectories. DiTSG employs stacked multi-agent inverted Transformer layers as the denoising backbone of a diffusion transformer, jointly generates the trajectories of multiple aircraft within a traffic scene, and conditions the generation on agent-specific required times of arrival. The resulting trajectories generated by DiTSG preserve realistic inter-aircraft interactions observed in actual air traffic operations. Experiments show that the DiTSG generates diverse traffic scenes that reproduce the statistical properties of recorded traffic while preserving realistic inter-aircraft interactions.
comment: 8 pages, 6 figures
♻ ★ Mechanistic discovery in grain boundary structure and deformation driven by agentic AI
Agentic artificial intelligence (AI) systems hold immense potential for automating complex workflows in materials discovery and design, but extending these capabilities to domain-intensive mechanistic studies remains challenging. General grain boundaries (GBs) exemplify this challenge, as their structural complexity complicates experimental characterization and atomistic modeling. Here, we present GBClaw, an agentic AI framework for investigating the structure and deformation of general GBs, which are common in real polycrystals yet remain understudied. By integrating large language models into a modular architecture, GBClaw effectively combines systematic simulations with targeted mechanistic analysis through iterative workflow refinement and dynamic replanning. Applied to 14 general asymmetric tilt GBs in nickel, the framework characterized interfacial defect structures and deformation modes across these incommensurate boundaries. It established a general approach to identify previously unobserved, widely spaced secondary GB misfit dislocations (SGBMDs) alongside closely spaced primary GB misfit dislocations (PGBMDs) in simulated atomic configurations and experimental images. Comparisons of GB sliding with and without SGBMDs revealed substantial differences in shear strength, underscoring the critical role of misfit-dislocation hierarchies in sliding resistance. Researcher-guided follow-up calculations further elucidated the structural origins of shear asymmetry and an anomalous shear response. This work demonstrates that combining agentic AI with domain expertise bridges high-throughput computational screening and deep mechanistic discovery at complex material interfaces.
♻ ★ An Analysis of Artificial Intelligence Adoption in NIH-Funded Research
Understanding the landscape of artificial intelligence (AI) and machine learning (ML) adoption across the National Institutes of Health (NIH) portfolio is critical for research funding strategy, institutional planning, and health policy. The advent of large language models (LLMs) has fundamentally transformed research landscape analysis, enabling researchers to perform large-scale semantic extraction from thousands of unstructured research documents. In this paper, we illustrate a human-in-the-loop research methodology for LLMs to automatically classify and summarize research descriptions at scale. Using our methodology, we present a comprehensive analysis of 58,746 NIH-funded biomedical research projects from 2025. We show that: (1) AI constitutes 15.9% of the NIH portfolio with a 13.4% funding premium, concentrated in discovery, prediction, and data integration across disease domains; (2) a critical research-to-deployment gap exists, with 79% of AI projects remaining in research/development stages while only 14.7% engage in clinical deployment or implementation; and (3) health disparities research is severely underrepresented at just 5.7% of AI-funded work despite its importance to NIH's equity mission. These findings establish a framework for evidence-based policy interventions to align the NIH AI portfolio with health equity goals and strategic research priorities.
♻ ★ CALM: Decentralized Control of Cooperative Aerial Manipulation with Multi-Agent Reinforcement Learning
Cooperative aerial manipulation is difficult under wind, payload motion, and noisy goal broadcasts because multiple Unmanned Aerial Vehicles (UAVs) are coupled through a shared payload. We present CALM (Cooperative Aerial Learned Manipulation), a decentralized controller for aerial manipulators carrying a basket while tracking a moving ground robot. CALM combines a parameter-shared Multi-Agent Reinforcement Learning (MARL) policy, trained using Multi-Agent Proximal Policy Optimization (MAPPO), with goal estimation, per-UAV Extended State Observers (ESOs), basket-mass estimation, and anticipatory formation-yaw feedforward. The anticipatory formation-yaw feedforward uses estimated goal motion to rotate the formation before position error develops, reducing basket-yaw lag without adding net commanded force. A disturbance-conditioned critic supports centralized training while execution remains decentralized. Across held-out trajectories, CALM achieves lower mean position and orientation errors than all evaluated baselines, including the matched arm baseline (7.68 versus 10.95 cm and 3.3 versus 4.7$^\circ$), the cable-suspended realization, and two model-based controllers. Under unseen full-information-path degradation, CALM remains statistically comparable or superior to the arm baseline. Videos are available at https://petreric.github.io/CALM/.
comment: 8 pages
♻ ★ Emergent Risks in Generative Multi-Agent Systems
Multi-agent systems composed of large generative models are rapidly moving from laboratory prototypes to real-world deployments, where they jointly plan, negotiate, and allocate shared resources to solve complex tasks. While such systems promise unprecedented scalability and autonomy, their collective interaction also gives rise to failure modes that cannot be reduced to individual agents. Understanding these emergent risks is therefore critical. Here, we present a pioneer study of such emergent multi-agent risk in workflows that involve competition over shared resources (e.g., computing resources or market share), sequential handoff collaboration (where downstream agents see only predecessor outputs), collective decision aggregation, and others. Across these settings, we observe that such group behaviors arise frequently across repeated trials and a wide range of interaction conditions, rather than as rare or pathological cases. In particular, phenomena such as collusion-like coordination and conformity emerge with non-trivial frequency under realistic resource constraints, communication protocols, and role assignments, mirroring well-known pathologies in human societies despite no explicit instruction. Moreover, these risks cannot be prevented by existing agent-level safeguards alone. These findings expose the dark side of intelligent multi-agent systems: a social intelligence risk where agent collectives, despite no instruction to do so, spontaneously reproduce familiar failure patterns from human societies.
comment: Accepted in Nature Machine Intelligence
♻ ★ Robust Instruction Compliance in Cooperative Multi-Agent Reinforcement Learning NeurIPS 2026
Multi-agent reinforcement learning (MARL) in real-world use cases may need to adapt to external natural language instructions that interrupt ongoing behavior and conflict with long-horizon objectives. However, conditioning rewards on instructions introduces a fundamental failure mode as Bellman updates couple value estimates across instruction contexts, leading to inconsistent values when instructions interrupt macro-actions. We propose Macro-Action Value Correction for Instruction Compliance (MAVIC), which corrects Bellman backups at instruction boundaries by correcting the incoming instruction objective and restoring the continuation value under the current objective. Unlike reward shaping, MAVIC modifies the bootstrapping target itself, enabling consistent value estimation under stochastic instruction switching within a unified policy. We provide theoretical analysis and an actor-critic implementation, and show that MAVIC achieves high instruction compliance while preserving base task performance in increasingly complex cooperative multi-agent environments.
comment: NeurIPS 2026 Atlanta Poster
♻ ★ When Is Smaller Greener? Right-Sizing Language Models for Agentic AI
AI agents use a large language model (LLM) to plan and act over many steps. Which model should an agent use, so that its tasks are completed reliably with the least energy, cost and time? We treat this as a multi-criteria decision problem in which comparisons per call can mislead: a task succeeds only if all its steps succeed, and failed attempts still use resources. We therefore compare models on resources per successfully completed task. We derive crossover task lengths at which two models exchange their efficiency ranking, determined by their per-step reliability and resource use. Each objective has its own crossover. Below the shortest crossover the less reliable model dominates, above the longest the more reliable one dominates, and in between both are Pareto-efficient and preferences decide. This supports a two-stage framework: a preference-free dominance screen, followed by a multi-criteria ranking of the efficient models, subject to reliability and feasibility constraints. We also give conditions under which this structure holds when reliability and resource use change during a task. In a pilot with 15 open-weight models on two synthetic agent tasks, with measured GPU energy, the predicted crossover fell within the observed range for 72 of the 88 observed crossovers. At 20 steps, a 3-billion-parameter model used a tenth of the energy of a 27-billion one per completed task; at 100 steps, a 1.7-billion model used almost four times as much as an 8-billion one. Smaller is greener only up to a task length that can be predicted.
♻ ★ SwarmCoDe: A Scalable Co-Design Framework for Heterogeneous Robot Swarms via Dynamic Speciation
Robot swarm co-design requires optimizing not only individual robot designs and behaviors, but also the heterogeneous swarm structure: how many robot types should exist, how they specialize and cooperate, and in what proportions they are deployed. However, these structural choices are often specified in advance rather than optimized. We present SwarmCoDe, a Cooperative Co-Evolutionary Algorithm (CCEA) that incorporates swarm decomposition into the optimization process without requiring the number of species to be specified in advance. SwarmCoDe combines dynamic speciation with evolved genetic tags, partner selectivity, and relative dominance to jointly optimize robot hardware, task-planning behavior, cooperative relationships, and deployed swarm composition. Across replicated simulated foraging experiments, SwarmCoDe discovers task-dependent swarm decompositions and adapts them when task requirements change, in contrast to fixed-K CCEAs which are sensitive to their pre-defined decomposition. We further show that fabrication budgets can change the preferred swarm composition and robot designs, and that SwarmCoDe can adapt its co-design under these resource constraints. Finally, SwarmCoDe co-designs 200-agent swarms using only 50 evolutionary individuals, demonstrating deployment at four times the evolutionary population size. Together, these results demonstrate that the heterogeneous structure of a robot swarm can be incorporated as a design variable and optimized alongside the robots themselves.
comment: Revised and expanded version. 13 pages, 8 figures
♻ ★ Equilibria in routing games with connected autonomous vehicles may not be strong, as exclusive clubs may form
User Equilibrium (UE) is the standard representation of the so-called routing game in which drivers adjust their route choices to arrive at their destinations as fast as possible. Asking whether UE is strong has, so far, been meaningless for human drivers, who do not form coalitions because of technical and behavioral constraints. Here, we demonstrate that this will not necessarily hold for Connected Autonomous Vehicles (CAVs), which can communicate and collaborate to form routing coalitions by exploiting adaptive signal control. Within game-theoretical formulations of Nash Equilibria, we provide a methodological framework and report the novel phenomena through a carefully designed toy-network example, in which we identify so-called \emph{clubs} emerging in different CAV penetration rates and adaptive traffic-light configurations. We detail one such club where three CAVs deviate from the UE and benefit (arrive faster). The formation of such a club has negative consequences for other users (who are not invited to join and must now travel longer) and for the system (making it suboptimal and disequilibrated), thereby triggering adaptation dynamics. These findings raise questions about equity in future mixed-autonomy traffic systems.
♻ ★ Executor-Side Progressive Risk-Gated Actuation for Agentic AI in Wireless Supervisory Control
Agentic artificial intelligence built on large language models shows promise for automating Open Radio Access Network supervisory control. However, action-level safeguards alone may be insufficient: evidence acquisition consumes time and shared resources, and network changes can invalidate approvals before release. We propose Progressive Risk-Gated Actuation (PRGA), an executor-side model and contract governing intents from assessment to release or termination. PRGA combines local checks with selective retrieval of coordination evidence to reduce acquisition costs while preserving mandatory evidence requirements. Version and lease rechecks prevent release under invalidated approvals, while reevaluation requires recorded progress within each transaction's retry budget. Building on this contract, we formulate evidence allocation to maximize the expected whole-period rate of timely valid releases under risk, quality-of-service, deadline, and shared byte-budget constraints. For energy-saving and slice-protection workloads with stable daily load cycles, thresholds are selected offline and interpolated using an online load-phase estimate. On average over modeled start phases and quota levels, the allocation policy improves timely valid release rates over arrival-order fetching and an offline-selected global threshold under identical quotas and release checks. In OpenAirInterface radio emulation, an immediate scripted resend after a version-mismatch refusal raises on-time receiver delivery from 77.3% to 89.9% at modest additional command cost.
♻ ★ DynaFlow: Learning Free-Form Multi-Agent Orchestration
Multi-agent systems often rely on predefined collaboration structures, limiting their ability to adapt to diverse task requirements. We introduce DynaFlow, a framework that integrates free-form executable workflow synthesis into the action space of the main agent. The agent decides when collaboration is needed and generates task-specific programs that coordinate sub-agents through sequential or parallel execution, structured information exchange, and verification. Execution results and traces inform subsequent actions, enabling the agent to adapt collaboration as reasoning progresses. We train only the main agent through supervised fine-tuning on distilled trajectories, followed by refinement on examples filtered for unsupported premises, biased verification, and information-flow defects. All sub-agents remain fixed, separating orchestration learning from executor capability improvement. Across six benchmarks spanning mathematical reasoning, scientific question answering, and multi-hop retrieval, DynaFlow achieves 88.08% overall accuracy in Forced mode, outperforming the strongest evaluated baseline by 10.90 percentage points with the same Qwen3.5-27B backbone. Workflow analysis finds more explicit use, verification, and refinement of upstream outputs after training, highlighting cross-stage information flow as a key dimension of multi-agent orchestration.
♻ ★ The Control Layer Behind Rule-Following Agents Trading Across Market Conditions
Trading discipline often fails people. Our study tested whether four AI trading agents followed their strategies and risk rules. Offering traders the evidence and an agent evaluation toolkit to separate luck from following the rules, before giving agents meaningful autonomy or capital. Our study analysed 2,627 closed positions from 12 February to 27 August 2026 across the same asset universe and execution conditions. Most positions were simulated against live prices with a 0.09% round-trip fee; 18% involved real Hyperliquid orders. Each position declared a strategy, stop and target, with a limit of five open positions per agent. The fleet finished net positive after fees. Rule adherence was high across all four agents, including the unprofitable one. Profitability tracked how often agents won relative to their break-even win rates. Institutional-style controls ran by default and contained losses through a 29% market drawdown and recovery, with documented exceptions. The discipline held for all four agents. Agents kept 97.5% of gross loss within declared stops, and typical losses closed at declared levels. Declared risk remained stable after losses and broadly stable across calm and volatile months. Twenty-three of 24 runs respected the position limit; logs identify the exception's size and duration. The same live data produced different results across assigned strategies. The trade-log tests can be rerun from the published trade snapshot; regime tests also require the published Bitcoin prices and regime labels. Section 3.5 specifies the stop-breach tolerance. These trade-level results are not account-level returns or proof of a lasting edge. Our study provides an evaluation toolkit to help traders distinguish rule-following from luck before committing capital. For anyone considering allocating capital to agents to trade, discipline should be the entry test, not the only test.
comment: 47 pages, 0 figures, 16 numbered tables. Includes a trading-agent evaluation toolkit and checklist. V2 Updated boilerplate and added Hygentra product information to section 2.1
♻ ★ A Simulation Testbed for Self-Governing Human-Machine Institutions
AI agents no longer act alone: they cooperate, coordinate and compete within and between groups, increasingly alongside humans who are in, on or out of ``the loop''. Whether such mixed groups can govern themselves is an open question about machine behaviour and human-computer interaction that needs a testbed for hybrid institutions. We present Megabike, a multi-agent simulator and serious game with interdependent arenas for common-pool resource management, cooperative survival and constitutional choice. Riders sharing a megabike must agree their arrangements for steering, pedalling and distributing resources, and their political regime, while negotiating with rival bikes and avoiding an existential threat. Every rule is enforced by the simulator, every action is logged, and any seat can be taken by a human or a machine, so governance can be studied under well-defined metrics. We demonstrate Megabike with forty sessions in which language-model riders with cooperative, lazy, subversive or opposing dispositions chose regimes, elected leaders and expelled one another. The results show that different combinations of agent archetypes could produce unexpected and counterproductive outcomes. We conclude with extensions for systematic experiments mixing human and computational agents.
♻ ★ You Can't Escape Your Activations : Evaluation Awareness and Multi-Agent Monitoring
LLM agents are increasingly deployed in multi-agent systems, where they can collude while keeping their actions benign. Output monitors designed to detect such collusions can be fooled by obfuscation and steganography, motivating the use of probes trained on internal activations. However, these probes are usually evaluated on agents that do not know they are being watched. We study how activation-based detection changes when agents are explicitly informed that their internal activations are being monitored, and when they additionally receive feedback from the monitor. We keep the models, probes, and thresholds fixed and change only what the agents are told: nothing (baseline), that an activation monitor is present (aware), or that a monitor is present together with the previous round's score (feedback). We test two games, a four-agent blackjack game and a two-agent Simmons prisoners game, using Qwen3-32B-AWQ and GPT-OSS-20B in homogeneous and mixed pairings. Telling agents about the monitor does not hide them. The best probes stay accurate in all three conditions, and the agents keep colluding.
comment: 18 pages, 11 figures, 2 tables
♻ ★ Poisoning Agentic Alpha: Adversarial Vulnerabilities Across Roles and Architectures in Multi-Agent Trading Systems
LLM-based multi-agent trading systems are moving from research prototypes to live deployments that control real assets. They ingest news and social media from the open web and pass it between agents, so a single corrupted signal can reach the final decision and cause financial loss. We study two adversaries that can introduce such signals without access to model internals: a low-barrier data adversary that plants manipulated content in web sources, and a compromised-agent adversary that tampers with an agent's prompt. To our knowledge, this is the first systematic empirical study in finance of how an adversarial signal enters a multi-agent trading system and whether it survives to alter the final trading decision. Along the role axis, we pair each of four roles---Analyst, Researcher, Trader, and Risk Manager---with an attack matched to its interface. Along the structural axis, we compare four communication topologies, interpreted with a simple parameter-free heuristic, the Adversarial Signal Preservation Score. Experiments on post-cutoff data span five assets and two target directions. We find that no architecture is inherently robust: majority voting offers conditional protection depending on benign-agent agreement, while compromising a terminal decision node can bypass upstream safeguards. These findings provide empirical insights for developing more effective defense strategies for multi-agent systems.
comment: Added a Statistical Analysis section (Figure 9)
♻ ★ GENSTRAT: Toward a Science of Strategic Reasoning in Large Language Models
Fixed game suites for evaluating strategic reasoning in large language models (LLMs) saturate and are hard to keep free of contamination. We introduce GENSTRAT, a system for evaluating the strategic capabilities of LLMs driven by procedural generation of two-player zero-sum imperfect-information betting games, allowing an evaluator to draw novel, fresh games on demand. Each game is characterized on a scale of five complexity axes (state space, temporal contingency, opponent dependence, private information, precision) and measured by Monte Carlo simulation. Using this generator, we construct a 50-game benchmark and hold a tournament of twelve frontier and open-weight models, gathering data from over 40,000 LLM vs. LLM matches. We further construct a composite complexity axis, and find that, on the eight hardest games, the gap between the best and worst model is 1.7x its value across the whole set of games. Ablations show that a higher thinking budget, disclosing the opponent's strategy, and higher reasoning matched to a game's specific demands (for private information and planning depth) were associated with higher margins for the LLM player.
comment: 33 pages, 8 figures, 17 tables (3 figures, 2 tables in main paper)
♻ ★ Events as Triggers for Behavioral Diversity in Multi-Agent Reinforcement Learning
Effective multi-agent cooperation requires agents to adopt diverse behaviors as task conditions evolve-and to do so at the right moment. Yet, current Multi-Agent Reinforcement Learning (MARL) frameworks that facilitate this diversity are still limited by the fact that they bind fixed behaviors to fixed agent identities. Consequently, they are ill-equipped for tasks where agents need to take on different roles at very specific moments in time. We argue that, to define these behavioral transitions, the missing ingredient is $\textbf{events}$. Events are changes in the state of the system that induce qualitative changes in the task. Based on this view, we introduce a framework that decouples agent identity from behavior, capturing a continuous manifold from which agents instantiate their behaviors in response to events. This framework is based on two elements. First, to build an expressive behavior manifold, we introduce Neural Manifold Diversity (NMD), a formal distance metric that remains well-defined when behaviors are transient and agent-agnostic. Second, we use an event-based hypernetwork that generates Low-Rank Adaptation (LoRA) modules over a shared team policy, enabling on-the-fly agent-policy reconfiguration in response to events. We prove that this construction ensures that diversity does not interfere with reward maximization by design. Empirical results demonstrate that our framework outperforms established baselines across benchmarks while exhibiting zero-shot generalization, and being the only method that solves tasks requiring sequential behavior reassignment.
♻ ★ Ax-Prover: A Deep Reasoning Agentic Framework for Theorem Proving in Mathematics and Quantum Physics
We present Ax-Prover, a multi-agent system for automated theorem proving in Lean that can solve problems across diverse scientific domains and operate either autonomously or collaboratively with human experts. To achieve this, Ax-Prover approaches scientific problem solving through formal proof generation, a process that demands both creative reasoning and strict syntactic rigor. Ax-Prover meets this challenge by equipping Large Language Models (LLMs), which provide knowledge and reasoning, with Lean tools via the Model Context Protocol (MCP), which ensure formal correctness. To evaluate its performance as an autonomous prover, we benchmark our approach against frontier LLMs and specialized prover models on two public math benchmarks and on two Lean benchmarks we introduce in the fields of abstract algebra and quantum theory. On public datasets, Ax-Prover is competitive with state-of-the-art provers, while it largely outperforms them on the new benchmarks. This shows that, unlike specialized systems that struggle to generalize, our tool-based agentic theorem prover approach offers a generalizable methodology for formal verification across diverse scientific domains. Furthermore, we demonstrate Ax-Prover's assistant capabilities in a practical use case, showing how it enabled an expert mathematician to formalize the proof of a complex cryptography theorem.
comment: arXiv admin note: This submission has been withdrawn because it does not meet arXiv's research content quality standards
♻ ★ STRIATUM-CTF: A Protocol-Driven Agentic Framework for Autonomous CTF Solving
Autonomous cyber operations increasingly depend on agents that can chain reconnaissance, exploitation, and analysis across many tools while maintaining state over long task horizons. Large Language Models (LLMs) are promising orchestrators for this workflow, but naive agents that pipe model output directly into a shell are brittle: they emit malformed commands, hallucinate non-existent command-line options or packages, and lose track of earlier findings as the context window fills. We present STRIATUM-CTF, a modular agentic framework for Capture-the-Flag (CTF) solving that mediates between LLM reasoning and tool execution using the Model Context Protocol (MCP). Offensive-security tools are exposed as strictly typed, schema-validated functions; the protocol layer rejects out-of-schema tool calls before execution and returns structured observations that the agent uses to self-correct. We evaluate the system in two ways. In a controlled study of 180 runs across 15 CTF challenges under four levels of attached procedural documentation, the system succeeds on 77.8-86.7% of attempts; notably, a schema-only configuration with no procedural documentation already reaches 77.8%, and no condition differs significantly from another. In a single live, multi-team CTF at Florida State University, the system placed first against a field of more than 30 registered teams -- a single, non-replicated event decided by a 10-point margin over the runner-up, and therefore a demonstration of feasibility rather than of superiority over human experts. Because all four ablation conditions share the same tool layer, our controlled study isolates documentation volume rather than the protocol layer itself; we accordingly present schema-mediated grounding as a design principle with qualitative support rather than as a measured effect.
comment: 7 pages, 2 figures, 3 tables. Accepted to the 2026 IEEE Cyber Awareness and Research Symposium (CARS 2026), Best Paper Award 1st Place. Code and data: https://github.com/jawh3/striatum-ctf
Systems and Control (EESS)
★ On the Performance of AoA-Based Physical-Layer Authentication Under Multi-Source Impersonation Attacks
This paper investigates cooperative impersonation attacks against angle-of-arrival (AoA)-based authentication for a wireless communication receiver equipped with a digital uniform linear array. Specifically, we characterize the vulnerability and robustness of this authentication system in relation to adversarial angular diversity and the relative numbers of adversarial sources and receive antennas. To this end, we first derive the optimal precoding vector for multiple spatially distributed adversarial sources by minimizing the mean squared error between legitimate and adversarial received signals. By means of the rank property of Vandermonde matrices, we then establish necessary and sufficient conditions under which the adversarial sources can reproduce the legitimate transmitter's spatial signature. With fewer adversarial sources than receive antennas, perfect reproduction is not possible without at least one adversarial source satisfying a specific AoA condition. In the other case, increasing the number of receive antennas can reduce impersonation success probabilities when none of the adversarial sources has an AoA satisfying this condition. Numerical results validate the theoretical analysis and evaluate the security performance of AoA-based authentication.
comment: 13 pages, 4 figures
★ Information-Driven Collaborative Autonomy for Heterogeneous UAV/UGV Teams
This paper considers coordinating a heterogeneous team of unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs) traversing a domain containing an unknown number of potentially hazardous targets. The UAVs provide aerial sensing using onboard optical cameras, adapting their trajectories to ground routes and target information. We formulate the aerial search problem using ergodic control, which distributes sensing effort according to a spatial information density to balance exploration and target localization. The formulation incorporates dynamic camera field-of-view footprints and pose-dependent probabilistic detection models. Angles-only measurements are processed through Bayesian inference and marginalization over data-association hypotheses to obtain a Poisson multi-Bernoulli (PMB) approximation of the target posterior, representing both detected and undetected targets. This belief directs aerial sensing toward unexplored regions along the ground routes and viewpoints that improve target localization. Ground planning combines chance-constrained target avoidance, enforced through confidence sets, with a void-probability objective that favors routes unlikely to contain threats. We solve each team's optimization problem in a fully decentralized manner using a nested SCvx*--C-ADMM scheme, with outer sequential convex programming iterations and inner distributed consensus optimization, accounting for nonlinearities, nonseparable objectives, and continuous-time inter-agent separation constraints. A closed-loop scheme updates both fleets' trajectories as information becomes available and permits ground motion when the remaining route risk falls below a prescribed threshold. Numerical case studies examine sensing geometry and target uncertainty, compare decentralized and centralized solvers, and demonstrate full closed-loop collaboration between multiple UAVs and UGVs.
★ Limit Theorems and Optimal Policies for Flock Flight through Poisson Forests
Inspired by birds flying through cluttered environments, Karaman and Frazzoli showed that high-speed flight through an ergodic random forest exhibits an ultimate speed limit: above a critical speed, almost surely no infinite collision-free trajectory exists for a single bird. This paper formulates a corresponding bulk-transport problem for many birds. We construct an explicit directed corridor network in a Poisson disk forest and impose a common temporal headway at its waypoints. The maximum current equals a minimum vertex-cut capacity and is achieved by repeated flight along vertex-disjoint paths. The same schedule maximizes completed traversals at every finite horizon, with constant forward speed and no internal waiting. For a forest belt of fixed depth, exact transverse superadditivity yields an almost-sure deterministic current-density limit. A uniform shape theorem for an asymmetric planar dual identifies the current when depth and width grow together. For diverging depth bounded by a constant times width, this current equals the width-first, depth-second limit and is independent of the positive aspect ratio. When diagonal corridors are available, bulk current is positive at sufficiently low obstacle intensity and zero at sufficiently high intensity. For stochastic demand, an upstream admission queue is stable at every rate below the finite-network capacity.
★ LLM eBay: A Simple and Truthful Marketplace for Fine-Tuned LLMs Trading
Personalized fine-tuning of Large Language Models (LLMs) enhances user experience but incurs substantial computational costs. Recent studies show that initializing training from a style-similar fine-tuned model reduces adaptation overhead, turning such checkpoints into reusable, tradable assets. A marketplace for these checkpoints, however, faces information asymmetry: sellers may misreport a checkpoint's style, while buyers cannot verify its usefulness before purchase, which invites adverse selection. We introduce LLM eBay, a simple single-round marketplace for trading fine-tuned LLMs. Sellers are paid per purchase at a posted price, and a truncated quadratic proper scoring rule adjusts each payment according to how well the seller's style report agrees with a style signal that the platform measures by generating text on hidden prompts. We show that the score elicits the verifier's expected signal, characterize exactly when truthful reporting is a seller best response under per-purchase demand, and bound the downside risk from noisy verification. In an end-to-end study with real LoRA checkpoints on two base models and three buyer tasks, LLM eBay reduces measured convergence steps by 8.5\% relative to retrieval over self-reported style (95\% CI [1.0\%, 15.9\%]). In model-based market simulations built on a measured style-to-cost relationship, it cuts style-claim error by 96.5\% (from 0.424 to 0.015), recovers 96.3\% of Oracle welfare, saves 72.4\% of the cost of fine-tuning from the base model, and keeps all providers profitable in the default setting.
★ BEVLight: Generalizable Traffic Signal Control from UAV Bird's-Eye Views
Learning-based traffic signal control (TSC) has achieved promising performance, but most methods rely on structured traffic states that must be estimated from sensor observations in real deployments. Visual TSC learns signal decisions directly from images, yet existing methods depend on fixed roadside cameras and remain tied to specific intersection configurations and visual domains. UAVs offer on-demand bird's-eye-view (BEV) observations that jointly capture intersection topology and traffic dynamics, making them a promising sensing platform for visual TSC. We present BEVLight, a generalizable TSC framework that directly maps BEV observations to signal actions. BEVLight grounds visual features onto individual lanes and composes them along the junction topology into movement and phase representations, allowing a shared policy to score candidate phases across heterogeneous intersection layouts and signal plans. To train the policy, we develop a web-based virtual teleoperation platform and collect over 5K human control decisions together with their underlying traffic states. Simulator-provided traffic states first supervise lane- and movement-level representations, after which human phase selections and rankings supervise the phase scorer. Re-rendering the recorded trajectories under diverse appearances preserves both forms of supervision and encourages robustness to visual changes. Closed-loop experiments demonstrate zero-shot generalization across traffic demands, signal plans, and intersections: BEVLight reduces average waiting time by 7--9\% relative to perception-to-control visual baselines and matches or outperforms structured-state RL controllers that receive ground-truth traffic states, achieving the lowest waiting time in 12 of 16 scenarios. An offline study further shows that BEVLight transfers to real UAV observations without target-domain adaptation.
comment: 27 pages, 15 figures
★ Compound Climate Risk-Aware Electricity Dispatch: Trajectory-Based Modeling for Power System Resilience
In recent decades, emerging compound hazards in a changing climate have increasingly threatened power system resilience and posed new challenges to electricity dispatch. Conventional Value-at-Risk (VaR)-based risk-aware dispatch provides a tractable means of managing stochastic net demand deviations caused by distribution feeder outages. However, when applied to temporally compounding events such as tropical cyclone (TC)--blackout--heatwave, conventional net demand risk formulations may assign similar risk levels to a brief outage and an outage with the same instantaneous net demand impact that persists into a subsequent hazard, despite their different requirements for sustained operational flexibility. Moreover, directly incorporating compound risk into electricity dispatch models faces a structural mismatch: the risk is defined by the joint distribution of feeder restoration and subsequent hazards over exponentially many joint states, whereas dispatch models require risk formulations that are tractable in the scheduling decisions. To address these issues, this paper proposes a compound climate risk-aware dispatch framework with a trajectory-based compound risk model, which defines the loss on operating deviations determined by entire feeder connection trajectories, parameterized by uncertain restoration times and coupled with subsequent hazards. Analytical moment-generating functions and equivalent exponential-cone reformulations provide a compact representation of the resulting risk constraints, preserving the relevant dependencies without explicit joint-state enumeration. By leveraging a physics-based spatiotemporal TC--blackout--heatwave hazard model, case studies demonstrate the effectiveness of the proposed method with real-world time-series power outage data from Puerto Rico, including data collected during Hurricane Fiona in 2022.
comment: 10 pages, 8 figures
★ Tracking analysis of compressed-communication distributed adaptive estimation for time-varying parameters
This paper studies the problem of distributed time-varying parameter tracking over communication-constrained networks. To mitigate the accumulation of induced compression errors across the network, we propose a differentially compressed distributed least mean squares (LMS) algorithm.In the proposed scheme, each sensor updates a local auxiliary state recursively based on the compressed difference between the current parameter estimate and the previous auxiliary state, and then transmits only this compressed difference to its neighbors for auxiliary-state reconstruction. This differential recursion mechanism helps control the accumulation of compression errors and accommodates a general class of randomized compressors including randomized quantizers and sparsifiers.We establish the mean-square boundedness and explicit steady-state tracking-error bound under a cooperative excitation condition on stochastic regression vectors, which captures both temporal and spatial information to reflect the joint excitation effect of sensors.Notably, our stability analysis requires neither temporal independence nor stationarity of regressors, nor persistent excitation at individual nodes, thereby accommodating correlated and nonstationary signals from stochastic feedback systems. Numerical simulations demonstrate the theoretical results under correlated and non-stationary regression data.
★ REALM: Identifying Minimal Task Assets for LLM Agents
Large language model (LLM) agents often receive access to more files, accounts, and services than a task requires. Prompt injection turns this excess access into an opportunity to redirect execution toward unrelated resources. Applying least privilege therefore requires identifying the assets needed by a task, including assets whose identities emerge only during execution. We present REALM (Required Execution Assets Linked to a Mission), an approach to identifying and maintaining a task's required assets and their permitted uses. The approach represents required assets as Dependency Graph nodes, resolves prerequisite relationships, and admits newly discovered assets through source-scoped selection. It separates discovering an asset from authorizing its use, while preserving the agent's freedom to choose its computation. The REALM Builder maintains the selected grants and projects Asset Files; the Partial API Manager exposes resource-bound Partial APIs to the Agent inside a Task Container. Across 60 clean tasks and 560 injected cases in AgentDyn, it completes 40.0% and 36.4% of tasks and decreases ASR from 16.6% to 0.2% with qwen3.5-flash as the base model.
★ Unlocking with the Heart: Investigating User Perceptions of ECG-Based Biometric Authentication
Password-based authentication remains challenging for many users, especially for those who face usability and accessibility issues. The burden of maintaining security can cause user fatigue, frustration and non-engagement. To address these issues, we conducted a pilot study to assess ECG-based biometric authentication as a passwordless alternative and to gather adult participants' perspectives on this approach. The research consisted of a survey study, an authentication prototype and a follow-up usability experience questionnaire. Our approach combined an online survey of 337 adults examining authentication practices and perceptions, with a separate usability evaluation in which 49 participants interacted with a simulated ECG-authentication and password interface in a health-care application prototype, then completed a post-test usability test questionnaire. The survey examined how adults across age groups use and perceive various authentication techniques, their experiences with biometrics, their views on security and convenience, and their willingness to adopt ECG authentication. In parallel, the usability prototype and questionnaire captured participants' responses after hands-on interaction with an ECG authentication prototype.
comment: 28
★ Adaptive Scenario Generation for KL-DRO Around a Learned Density Model
Distributionally robust optimization (DRO) hedges data-driven two-stage stochastic programs against error in the distribution estimated from a finite record, and standard ambiguity sets differ in how they treat realizations absent from that record. A divergence ball around the empirical distribution contains only reweightings of the record, whereas a transport ball also contains distributions supported off the record, including on realizations the data-generating process cannot produce, against which the decision is then hedged as well. We instead propose a Kullback-Leibler (KL) ball centered at a density model fitted to the record, the learned nominal, which lets the adversary up-weight costly unobserved realizations in proportion to the probability the model assigns them. Solving the resulting problem poses a second difficulty, because it can accommodate only a few scenarios, and an independent sample of that size is likely to omit the rare realizations on which the worst case concentrates. We therefore build the scenario set adaptively, alternating between solving the finite problem and drawing new realizations from the learned nominal, steered toward the worst case at the current decision, with importance weights that correct the draws back to the model. On a multi-item newsvendor problem and a capacity expansion problem on realistic grid data, centering the ball at the learned nominal rather than at the record lowers cost under distribution shift, at comparable in-distribution cost, when the record is scarce and the model class captures structure that lets it extrapolate beyond the record. The advantage fades as the record grows or as the shift departs from the structure the model has learned.
★ Analog Realization and Measurement of a Forced Aliev-Panfilov Excitable Oscillator
A single resistor moves a discrete analog realization of the forced Aliev-Panfilov excitable oscillator from a chaotic operating point of the model to a regular one. In this model a rate parameter multiplies the complete right-hand side of the excitation equation, forcing term included. The input resistor of the excitation integrator sets that parameter, so no state, delay element, or controller is added. Sampled waveforms were recorded at two resistor values, with two independent runs each. The measured excitation excursion agrees with numerical integration to within three percent at the baseline setting and five percent at the rate-controlled setting. Between the two settings the excitation excursion contracts by a factor of thirteen. At the rate-controlled setting the recovery excursion exceeds prediction by a factor of eight. This excess traces to a displaced operating point on the state-dependent recovery rate. Thermal drift in the multiplier chain and leakage at the high-impedance recovery-rate divider could each produce it, and the present data do not separate them. A tested bipolar input stage strongly loads that divider, a constraint absent from the ideal equations. The divider, not the sub-hertz signal bandwidth, sets the amplifier input-stage requirement.
★ A Tutorial on Shape Servoing of Deformable Objects-Part I: Fundamentals
In robotics, shape control (or shape servoing) refers to the automatic manipulation of a deformable object to achieve a desired shape. It involves a feedback loop that exploits the relation between robot motion and changes in the measured shape. This relation, known as the interaction Jacobian or, in this context, the deformation Jacobian, depends on the object's mechanical properties, geometry, and boundary conditions (e.g., where the object is grasped). Because these factors are often difficult to model accurately, the deformation Jacobian can be estimated locally from observed shape changes produced by commanded robot motions, without requiring a fine-tuned mechanical model. This article is the first of a two-part tutorial on data-driven Jacobian-based shape control of deformable objects. It introduces the central elements of the approach: the formulation of the shape-control problem, shape representation, estimation and online update of the deformation Jacobian, and a standard control law from the literature. It then assembles these elements into a complete shape-servoing loop, derives a local stability condition for this loop, and illustrates why some target shapes cannot be reached. Part II covers more advanced topics. Throughout both parts, a simulated planar example in which two robot arms manipulate an elastic rod observed by a camera illustrates the concepts.
comment: Part I of a two-part tutorial. 9 pages, 5 figures. Code: https://github.com/nachocz/tutorial_on_shape_servoing_of_deformable_objects. Interactive demos: https://nachocz.github.io/tutorial_on_shape_servoing_of_deformable_objects/
★ A Behavioral approach to Constrained Linear Quadratic Control: Feasibility and Optimality
This paper develops a constrained linear-quadratic (LQ) optimal control framework within the behavioral approach. Unlike classical formulations that rely on state-input representations, our method treats both the system dynamics and the constraints as trajectory-level objects. The system is modeled through a kernel representation, while inequality constraints are captured using behavioral inequalities defined over infinite-dimensional convex cones. Using quadratic differential forms (QDFs) to encode performance indices, we formulate the constrained LQ problem as a convex variational optimization over trajectories. By employing Gateaux differentials, we derive the stationarity and complementarity conditions and show that they admit a representation as an augmented behavioral system whose unique decomposition into autonomous and controllable sub-behaviors yields a complete parameterization of all optimal constrained trajectories. The resulting conditions constitute a behavioral analogue of the Karush-Kuhn-Tucker (KKT) framework. A numerical example illustrates the effect of inequality constraints on the optimal behavior. The framework provides a general, trajectory-level representation for constrained optimal control and opens avenues for closed-loop and data-driven extensions.
comment: Accepted for publication at IEEE Indian Control Conference 2027
★ A Two-Wheeled Quadrotor with All-Clockwise Tilted Rotors for Level-Attitude Surface Locomotnion
Conventional wheel-equipped quadrotor UAVs must tilt their bodies to generate lateral forces, limiting their ability to maintain a desired attitude during lateral motion. This paper proposes a wheeled quadrotor with all-clockwise tilted rotors that uses surface contact as a constraint to generate propulsion along the contact surface without tilting the vehicle. We analyze feasible forces and system controllability under surface contact and optimize the vehicle design considering different rotor-spin configurations. The optimization results show that a configuration in which all four rotors spin clockwise can generate a larger tangential force than the conventional alternating-spin configuration. This approach enables both aerial and contact-based locomotion without additional rotor-tilting actuators or mechanisms. Simulation results demonstrate both hovering in free space and horizontal locomotion on a floor while maintaining a level attitude.
★ Fundamental Limits of Sampling-Based Reachability: Geometry and Flow Distortion
Sampling-based reachability estimates reachable sets from finitely many simulated trajectory endpoints. Existing guarantees often control missed probability mass or approximation error relative to the convex hull of the reachable set, leaving the geometric error relative to the reachable set incompletely characterized. We study this error as a problem of support estimation under flow distortion. Our assumptions combine a uniform lower bound on the local volume density of the initial set, a lower bound on the sampling density, and a one-sided Lipschitz constant $μ$ for the vector field. These conditions admit initial sets with corners and dynamics that may expand or contract. In the fine-accuracy regime, $\tilde{\mathcal O}\bigl(2^ne^{nμT}(R/r)^n/(ρκ)\bigr)$ independent endpoint samples suffice for Hausdorff accuracy $r$ using the endpoint cloud. Here, $n$ is the state dimension, $T$ is the horizon, $R$ is the volume-equivalent radius, and $ρ$ and $κ$ quantify sampling density and local volume density. For $n\ge2$ and sufficiently small $κ$, a minimax lower bound over all estimators based only on independent endpoints matches these dependencies up to a dimension-dependent factor $2\cdot4^n$ and logarithmic factors, within the stated parameter range and when the state domain accommodates the construction. The exponential burden in dimension, and in the horizon for expanding systems, is therefore intrinsic to this observation model. Planar experiments illustrate the predicted effects of corners and contraction; robot-arm experiments show how empirical accuracy deteriorates with dimension.
comment: 16 pages, 9 figures, 3 tables. Extended version of the CoRL 2026 paper; submitted to IEEE Transactions on Automatic Control. Code: https://github.com/JeanLew01/reachapprox
★ socx: GPU-Accelerated Chance-Constrained Stochastic Optimal Control using JAX
In this article, we present socx, a GPU-accelerated solver for lifted convex reformulations of linear stochastic optimal control under chance constraints, implemented in JAX. We demonstrate that by exploiting the sparse and independent structure of the problem, distributed optimization can reduce it to simpler problems: independent linear-quadratic tracking (LQT) problems and Euclidean projections onto sets. These simpler problems can be solved in parallel and then coupled using the constraints. To validate our solver, we run it on a hover-linearized 3D quadrotor and show a speed-up of up to $98\times$ over MOSEK and up to $2224\times$ over SCS. socx also solves horizons up to $N=1120$, whereas MOSEK runs out of memory at $N=320$. We present ablations to isolate the key components that enable the speed-up.
★ Finite-Sample LTI System Identification from a Misspecified Finite Model Set
We consider the problem of finite-sample identification of an unknown LTI system from a finite hypothesis class. In contrast to existing results that assume the hypothesis class contains the true system, we consider the misspecified setting, where the true system does not belong to the hypothesis class. We first characterize the resulting estimation target and its dependence on the data-generation scheme. We then establish a finite-sample lower bound on the failure probability that any identification algorithm must incur. Finally, we analyze the robustness of an existing exponential weighting-based identification algorithm to the misspecification and derive a finite-sample upper bound on its failure probability. Our results illustrate when using a misspecified finite hypothesis class can provide an advantage over identification without prior knowledge of the candidate systems.
★ Learned Terminal Costs for Short-Horizon Model Predictive Control of Autonomous Trucks in Interactive Highway Traffic
Autonomous highway driving for heavy-duty trucks is a challenging control problem. The greater mass, dimensions and intricate dynamics increase the requirements for cooperation from surrounding traffic in critical maneuvers, requiring a careful balance between safety constraints and efficient long-horizon decision making. Model Predictive Control (MPC) provides rigorous constraint satisfaction but relies on terminal cost which can limit its effectiveness at short, computationally affordable horizons. Reinforcement Learning (RL) captures long-term driving behavior but offers no constraint guarantees and tends toward overly conservative policies when trained under sparse collision penalties. In this paper, we propose a hybrid MPC and RL framework for highway driving in autonomous trucks, where a finite-horizon MPC, solved via dynamic programming, uses a pre-trained RL model for terminal cost estimation. We evaluate the framework in a simulation environment with reactive surrounding traffic and compare it against pure MPC with zero terminal cost, rollout with terminal cost estimation using a base policy, and pure RL. Results show that the hybrid approach achieves zero collisions with the lowest cost of any configuration.
comment: 13 pages, 2 figures, 8 tables
★ Control of Seizure Dynamics in a Bistable System with Pulsatile Inputs
Bistable neural population models describe transitions associated with epilepsy through the coexistence of a stable equilibrium representing normal neural activity and a stable limit cycle representing seizure-like oscillations. Perturbations that drive the system outside the equilibrium's basin of attraction can trigger transitions to seizure-like activity. Enlarging this basin enables the system to recover normal activity from a larger set of perturbed states, providing a model-based mechanism for reducing frequency of seizure initiation. Motivated by electrical brain stimulation, this paper investigates basin enlargement using pulsatile feedback control. Combining higher-order averaging with periodic near-identity transformations, we establish conditions under which nonzero-mean and zero-mean pulsatile feedback gains enlarge the basin of attraction and derive guidelines for pulse design. Numerical simulations validate the theoretical enlargement.
comment: 7 pages, 2 figures
★ Dual-Agent Reinforcement Learning for Vehicle Scheduling and Multi-Energy Dispatch in Electric-Hydrogen-Transport Systems
Electric-hydrogen-transport systems (EHTSs) require coordinated scheduling of electric and hydrogen vehicles together with multi-energy dispatch. These operations form a sequential decision process: upstream vehicle scheduling (VS) determines service allocation and order, thereby endogenously shaping the electricity and hydrogen demands faced by downstream energy dispatch (ED). This paper develops a dual-agent reinforcement learning framework tailored to the heterogeneous decision structures. The vehicle-scheduling agent combines proximal policy optimisation (PPO) with a Plackett-Luce policy to learn vehicle-priority permutations, which are then mapped to feasible charger and refueller assignments through a greedy allocation. The energy-dispatch agent employs twin delayed deep deterministic policy gradient (TD3) for continuous storage and energy-flow control. The directional coupling between the two decision layers is analysed, revealing that updates to the upstream scheduling policy can shift the demand distribution experienced by the downstream dispatcher. A three-stage training mechanism is therefore introduced, combining layer-wise pretraining with concurrent fine-tuning to mitigate the effects of scheduling-induced distribution shifts on downstream ED learning. Year-long case studies show that the proposed framework achieves the lowest vehicle-service delay and energy cost among the 11 evaluated methods, demonstrating effective coordination between vehicle scheduling and multi-energy dispatch.
★ Readiness-Constrained Pre-Islanding Dispatch for Military Microgrids
Military installation energy readiness depends on having sufficient resources to sustain mission-critical loads throughout a prolonged utility outage. This paper develops a method to help operators manage energy resources during normal grid-connected operation while preserving islanded readiness. A robust viability kernel identifies combinations of battery energy, fuel, and load-shifting capability sufficient to sustain critical load under uncertain demand and photovoltaic generation. A probability bound translates stochastic generator start failures and recovery into a deterministic generator count selected to meet a specified availability requirement at islanding. Case studies for a fourteen-day islanding requirement at 0.999 islanding-entry availability show that advance warning of an outage can be used to prepare resources to meet a higher readiness requirement before islanding begins. The case studies further demonstrate that economic-only operation and the evaluated fixed battery reserve policy can fail to sustain critical load for the required duration, whereas enforcing the resource-dependent requirement preserves readiness at a modest increase in modeled grid-connected operating cost. These results demonstrate how active resource management can preserve long-duration energy readiness. The framework also helps commanders understand how readiness changes with operating conditions, even when the available resources remain the same.
comment: 32 pages, 5 figures, 3 tables, submitted to Military Operations Research
★ Agentic process evidence
The way software gets built is changing dramatically. AI agents now create pull requests, select dependencies, refactor codebases, interpret test failures, and execute infrastructure changes across the entire Software Development Life Cycle (SDLC). AI agents are autonomous team members, but unlike human engineers, they are ephemeral - their reasoning, decisions, and tool calls vanish the moment the session ends. Modern Security incidents involving intent mismatch or injection are becoming increasingly common - forcing us to address AI Agents as an insider threat. No widely adopted standard yet enables agentic provenance and governance and its obvious that you can't hold an ephemeral process accountable. The git log was designed for human commits. It shows what changed. It does not show what an AI agent did, which model it used, which tools it called, or whether anyone was accountable for the outcome. Every governance framework, every audit trail, every compliance control in the SDLC was built around one assumption: a human made the decision. That assumption no longer holds. This shift introduces 3 main procedural problems (not addressing scale) - Supply-chain vector - The agentic coder is software, so it is a supply-chain security risk. Policy guardrails - How to ensure the agentic coder is compliant with the policies of the organization. Task-alignment - How to align the agentic coder on its tasks and goals. We introduce Agentic Process Evidence (APE), a signed, tamper-evident, machine-readable attestation of agentic processes, linking the sessions to their outcome (process-aware logging). APE enables Policy-as-Code control over the agentic processes, allowing for human in the loop only when it's needed. We support the in-toto style attestation framework, and the protocol is aligned with the relevant regulations
★ Safe Platooning under Input Constraints with Link-Wise Velocity String Stability
We propose a decentralized controller for vehicle platoons with different braking and propulsion limits. The method uses only local distance and velocity measurements and requires neither online optimization nor exact vehicle parameters. The design combines a braking-distance safety margin with an adaptive funnel that relaxes the spacing requirement during propulsion saturation. A standstill mechanism prevents backward motion and supports stop-and-go operation. Under explicit uniform conditions on the vehicle parameters, controller settings, and initial states, we prove that closed-loop solutions exist for all time, collisions are avoided, and input limits are satisfied. Moreover, each follower tracks its predecessor's velocity range with gain one, up to an explicit additive offset determined by the funnel width and the braking-distance term. This bound establishes link-wise velocity string stability. Both the offset and the bounds on speeds and spacing errors are independent of vehicle index and platoon length. Simulations with 2 to 100 followers illustrate safety and bounded velocity deviations under disturbances and actuator limits.
★ LC-Resonance Placement Limits and Damping Recovery in Virtual-Admittance-Based Grid-Forming Inverter Control
Compact LC-filter design and inner-loop control must be coordinated for stable grid-forming operation. This paper quantifies how current-loop proportional gain constrains LC-resonance placement in virtual-admittance-based dual-loop control with unity voltage-feedback decoupling (VFD) and capacitor-voltage active damping. Equivalent-impedance analysis identifies gain-dependent contraction of the positive-resistance region. Delayed inverter-current feedback opposes VFD-assisted damping above one-sixth of the sampling frequency. Simple proportional-path frequency shaping recovers an admissible effective-gain window while retaining unity VFD and leaving the resonant path unchanged. Reduced-model criteria, checked against the complete model, link window closure to a nominal resonance-placement limit near one-third of the sampling frequency under single-sampled single-update operation over the studied gain range. A 3-kVA prototype demonstrates an 83.3% LC-product reduction at comparable voltage and nominal current crossovers. A separate comparison with the same high-resonance filter demonstrates stable operation at 2.68 times the unshaped nominal current crossover. Output-impedance analysis and grid-connected tests identify additional grid-dependent gain constraints.
comment: 10 pages
★ Compressed Orientation Reachability Maps for Sub-Second Robotic Needle Entry Point Planning
In the context of multi-stage trajectory planning for robotic percutaneous liver interventions, identifying a kinematically feasible entry point remains a critical computational bottleneck. Existing methods rely on exhaustive inverse kinematics or randomized searches, incurring delays of 13 to 27 seconds per target and limiting real-time intraoperative adaptation. This paper introduces an Orientation-Based Reachability Map (ORM) that precomputes the kinematic feasibility of a 6 Degrees of Freedom (DoF) serial manipulator across 188,873 workspace positions and encodes orientation feasibility into a compact, indexed 83 MB map. A Just-In-Time (JIT) compiled runtime pipeline replaces live optimization with near-instantaneous array lookups. Experimental evaluation demonstrates a reduction in planning latency from 13.5-26.9 s to 0.289-0.349 s per target while maintaining high geometric fidelity, achieving a mean convex hull volume overlap ratio of 0.933 and a mean alpha-shape volume ratio of 0.976 relative to the baseline reachable point cloud. The proposed ORM framework enables sub-second entry point planning and provides an efficient computational framework for robotic needle insertion in percutaneous liver ablation.
comment: 6 pages, 4 figures, 2 tables
★ Which wind and solar modelling choices matter? Effects on capacity factors, energy droughts and European power system configuration
Europe's future power system will rely heavily on wind and solar power and must remain resilient during prolonged periods of low wind and solar generation, known as variable renewable energy (VRE) droughts. To capture such rare events, adequacy assessments increasingly draw on decades of weather data, but the conversion of weather into power is largely treated as fixed. It remains unclear which modelling choices in this conversion matter for representing these droughts, and whether they matter as much as the weather itself. Here we systematically compare these choices side by side (bias correction, wind speed extrapolation, capacity layouts, turbine and photovoltaic configurations, and wake losses) and trace their effects through VRE droughts to the design of a net-zero European power system, using 85 years of reanalysis data. We find that modelling choices barely change how often VRE droughts occur or when the system is stressed. They do, however, determine how much capacity is needed to withstand that stress. By altering the electricity delivered during droughts, they shift required capacity by up to 480 GW, which is comparable to the spread across the most critical weather years. Modelling exercises should therefore use multiple weather years; bias-correct and regionally validate weather inputs; explicitly represent resource heterogeneity, forward-looking technologies, and wake losses; and judge these choices by their effect on the low-generation tail, not only on the mean.
★ Modeling and Parametrization of Large-scale Hydrogen Electrolyzers for Transient and Frequency Stability Analysis
Hydrogen Electrolyzers (HEs), particularly Proton Exchange Membrane (PEM)-based systems, are flexible loads capable of providing grid ancillary services and enhancing the stability of renewable-dominated power systems due to their fast electrical response. However, the literature often lacks clear guidance on the structure, modeling, and parametrization of large-scale HEs for transient and frequency stability analyses. This paper presents a detailed configuration for utility-scale HE plants, using the stack as the fundamental unit, supported by a realistic parametrization based on commercial components. Furthermore, this work extends the Generalized ESS Model (GEM), a linearized state-space model previously utilized with other converter-interfaced energy storage systems, to PEM HEs for the first time. To facilitate plant scalability, a robust method for scaling GEM matrices across different capacities is proposed. Finally, a measurement-based methodology is developed to estimate the GEM parameters and subsequently extract the equivalent electrical circuit parameters. Comprehensive simulations on the WSCC system under various disturbances validate the accuracy of the derived GEM, the effectiveness of the scaling approach, and the robustness of the parameter identification methodology.
★ Multi-output LSTM soft sensors for industrial anaerobic co-digestion using LSTM-beta-VAE generative data augmentation
Anaerobic digestion (AD) requires close monitoring of key process variables, including pH, the ratio of volatile organic acids to alkaline buffer capacity and biogas production, to maintain biological stability and prevent process failure. While mechanistic frameworks such as the Anaerobic Digestion Model No. 1 are highly parameterised, computationally expensive, and difficult to calibrate for real-time applications, data-driven Long Short-Term Memory (LSTM) networks offer a potentially more adaptable alternative. However, current data-driven approaches in AD are predominantly limited to single-output architectures that focus solely on production yields, as opposed to simultaneous prediction of vital stability indicators. Furthermore, industrial AD datasets are typically sparse and skewed toward steady-state operations. Baseline evaluations demonstrated this scarcity prevents standard algorithms (Random Forest, XGBoost) and unaugmented LSTMs from recognising the transition to unstable operations, resulting in poor estimation of stability indicators. To overcome these challenges, this study proposes a novel generative data augmentation framework using an LSTM-beta-Variational Autoencoder coupled with a multi-output LSTM (MO-LSTM) soft sensor. By synthesising physically valid sequences of these transitions to unstable operations from one year of full-scale plant data, the method safely populates sparse regions of the feature space. Trained on this augmented dataset, the MO-LSTM achieved robust simultaneous estimation across all targets. This approach mitigates data sparsity, providing a basis for hybrid mechanistic-data-driven modelling and early-warning soft sensors in industrial bioprocesses.
comment: 33 pages
★ Neuromorphic design of a rhythmic network
Central Pattern Generators (CPGs) have long inspired the design of rhythms that combine endogenous generation and exogenous interaction. In neurodynamics and engineering studies, CPGs have been primarily analysed and designed as networks of coupled oscillators. In contrast, the present article regards CPGs as networks that propagate events through excitable rather than oscillatory nodes. The proposed event-based perspective reduces the design of network-level rhythmic properties to the tuning of local nodal input-output characteristics. This reduction enables a distributed and robust design methodology: nodes can be tuned independently, they can be heterogeneous, and the global rhythm results from local event propagation rather than from a specific global tuning of the network parameters. The present article builds on previously introduced concepts that parallel existing characterizations of oscillators: the event describing function (eDF), which characterises the propagation delay of a node, and the event phase response curve (ePRC), which characterises the phase sensitivity of a node. Event-based CPGs made of excitable rather than oscillatory nodes emerge as a principled design framework for robust rhythm generation.
comment: This is an author-created, un-copyedited version of an article submitted for publication in the Journal of Neuromorphic Computing and Engineering. IOP Publishing Ltd is not responsible for any errors or omissions in this version of the manuscript or any version derived from it
★ Computing with Coupled Oscillator Networks: Optimization, Memory, and Modern AI Primitives
We explore how synchronization in coupled oscillator networks may be employed to perform a variety of computing tasks. These systems are becoming more attractive as an alternative to conventional digital computers due to the increasing energy demands of the latter, particularly for large-scale problems in machine learning and artificial intelligence. The ubiquity of coupled oscillator networks means that oscillator-based computing can be thought of as a universal computing paradigm encompassing systems with a diversity of physical realizations, e.g. as analog circuits, neurological systems, spintronic systems. The bridge between the dynamics of coupled oscillator networks and computational primitives they implement is the Ising model, which originates in statistical physics, and when applied to oscillator networks, explains the emergence of globally complex and disordered synchronization patterns from local interactions in the network. We discuss the application of coupled oscillator networks to three fundamental computing operations. First, oscillator networks as specialized hardware solvers of NP-hard combinatorial optimization problems, with the solution encoded in the synchrony patterns. Second, oscillator networks as associative memory systems, enabling storage and retrieval of data represented through phases of oscillations. Finally, oscillator networks as a dynamical implementation of attention, with synchronization providing the mechanism for the alignment of keys to queries. The ability of oscillator networks to implement these computational primitives paves the way for novel realizations of intelligent systems through a variety of physical substrates.
★ From Airspace Degradation to Deconfliction: Predicting Safety-Critical Traffic States through the Evolution of Multiscale Complexity Metrics
Growing air traffic demand is pushing airspace management towards the limits of its operational capacity, with steadily increasing workload for air traffic control officers, with direct consequences for safety. Understanding which traffic conditions give rise to safety-critical situations is therefore an essential open problem. Here we compare a large number of deconfliction events with equivalent conflict-free scenarios, extracted from thirteen weeks of air traffic control data over the Swedish Flight Information Region. We describe the traffic structure before each event through thirty sector-complexity metrics computed at multiple spatial and temporal scales, and use them as inputs of Machine Learning models. We find that the need of deconflictions can be predicted with an accuracy reaching $70\%$, but only when the traffic is described at the sector spatial scale. Predictability increases monotonically in the minutes preceding the clearance, showing that situations requiring deconflicting actions are not sudden occurrences, but rather the endpoint of a progressive degradation process. Additionally, the information needed for such prediction mainly resides in specific characteristics of the overall traffic structure, and not only in the conflicting pair. These results numerically demonstrate that safety-critical airspace states can be anticipated from the collective behaviour of the traffic, paving the way for early-warning systems that flag degraded, safety-relevant conditions before an intervention is required.
comment: 44 pages, 12 figures
★ Cybersecurity Practices in Industrial Automation and Control Systems: a Study of Croatian System Integrators
Given the increasing frequency of industrial cybersecurity incidents, governments, industry, and the research community are all working to find ways to protect industrial automation and control systems (IACSs). A key regulatory effort in the European context is the EU's Network and Information Security Directive 2 (NIS2), which aims to strengthen cybersecurity requirements and improve resilience across critical and important sectors. Nevertheless, translating cybersecurity requirements into practical engineering remains challenging within IACS environments. This paper explores how Croatian IACS system integrators perceive cybersecurity requirements, implementation challenges, and current practices in the industrial environments in which they operate. The findings are based on three focus groups with system integrators working with clients in the energy, chemical and pharmaceutical, and food and beverage sectors. Participants reported limited cybersecurity awareness among IACS owners, as well as perceived differences in practices between smaller and larger companies and between domestic and foreign firms. Participants were divided in their expectations regarding regulatory pressure: while some believed it could improve cybersecurity, others expected only minimal compliance focused on cost reduction. The results of this study provide a basis for developing more practical cybersecurity practices and offer a deeper understanding of the key challenges faced by IACS system integrators and owners when dealing with cybersecurity.
★ Non-Asymptotic Confidence Regions for Separable Nonlinear Models
This paper studies robust uncertainty quantification for separable nonlinear least squares (SNLLS) problems. We propose two identification methods, (i) Residual-Perturbed Gradient (RPG) and (ii) Global-Fit-Gap (GFG), to construct joint distribution-free confidence regions for both the linear and nonlinear parameters. By exploiting assumed group invariance of the noise (such as symmetry or exchangeability), we apply random perturbations (such as sign-changes or permutations) to build the regions based on resampling and ranking. We prove that both methods guarantee non-asymptotically exact coverage (type I error). Finally, we illustrate the constructions through two numerical experiments: (i) identifying generalized finite impulse response (GFIR) systems using Laguerre polynomials with an adjustable decay rate and (ii) fitting Gaussian radial basis function (RBF) models with variable knots and widths.
comment: Accepted for publication at the 65th IEEE Conference on Decision and Control (CDC 2026)
★ Assuring Learning-Enabled Cyber-Physical Systems with Foundation Models: A Neuro-Symbolic Agenda for Testing, Verification, and Runtime Assurance
Learning-enabled cyber-physical systems (CPS), including autonomous vehicles and UAVs, increasingly incorporate foundation models, yet fragmented assurance practices provide insufficient support for system-level guarantees. We argue that restoring engineering control across stochastic neural, symbolic, and physical boundaries requires a shift from model-centric evaluation to system-level assurance. We introduce the Foundation Model Assurance Stack, connecting three directions: (i) neuro-symbolic architectures that constrain foundation models through explicit symbolic world models, reasoning layers, and interfaces; (ii) semantically grounded, conformance-based testing that combines formal specifications and symbolic world models with foundation-model-assisted specification mining, scenario generation, and test synthesis; and (iii) compositional verification and runtime assurance that target formally tractable components and interfaces through modular contracts, equivalence checking, verified synthesis, uncertainty-aware reasoning, and runtime monitoring and enforcement. Symbolic abstractions establish assurance boundaries between stochastic learning and formal reasoning, connecting testing, verification, and runtime evidence across the system lifecycle. Foundation models serve as constrained assistants whose outputs are independently checked wherever correctness can be formally specified. We identify research challenges in specification engineering, abstraction fidelity, evidence composition, and lifecycle certification, and outline a staged agenda toward trustworthy, deployable, and certifiable learning-enabled CPS.
★ A Robust Approach to Belief Mismatch in Non-cooperative Dynamic Games
Game-theoretic models are increasingly used to describe multi-agent interactions in control and robotics applications. However, theoretical and computational complexity makes these models intractable in many dynamic, incomplete information scenarios. Recent work makes strong assumptions to reduce computational complexity: agents share sensor measurements with one another and maintain a common estimate of state uncertainty. Unfortunately, these assumptions enforce a degree of cooperation in generically noncooperative interactions. This paper takes the perspective of an ego agent seeking both computational tractability and robustness to unanticipated behavior of other agents due to mismatched state estimates from private---rather than shared---sensor measurements. To that end, we formulate the game theoretic planning problem in an augmented belief space with perturbed Bayesian dynamics that allows the ego to predict all agents' potentially divergent state estimates. These perturbations are chosen by a hypothetical adversary, ``nature,'' to reflect how non-ego agents' state estimates---based on measurements the ego agent will not see at runtime---may deviate from the ego's own belief. Ultimately, our approach can be understood in the spirit of classical H-infinity robust control, extended to noncooperative, stochastic dynamic games. We present a solver with polynomial-time iterations and conduct two experiments which show how robust belief space reasoning leads ego agents to proactively limit their future vulnerability to model misspecification of other agents' beliefs and actions, reducing performance degradation by over 48%.
★ A Temporal Cellular Sheaf Framework for Multi-Agent Systems with Switching Communication Topologies
Cellular sheaves serve as a natural modeling framework for encoding heterogeneity in the state and interaction spaces of multi-agent systems. This work examines switching communication graphs with the use of temporal cellular sheaves, which we construct to represent heterogeneous agent state spaces, the maps that relate them, and their respective communication constraints. We then develop sufficient conditions under which a multi-agent system defined on a temporal cellular sheaf can solve the target tracking problem by generalizing the existing concept of joint connectivity. Using a Lyapunov-based stability analysis, we demonstrate that the target tracking error of the network converges to a neighborhood of the origin.
comment: 10 pages, 4 figures
★ RC Prediction: A Python Package for Parameter-Aware Reservoir Computing
A nonlinear dynamical system can be characterized by the autonomous evolution of its state in time and its dependence on system parameters. As a parameter varies, the dynamics can change quantitatively and, at bifurcation points, qualitatively. When the governing equations are unknown, a data-driven model should therefore capture both elements from observational data, rather than reproduce the dynamics at a single operating condition only. Parameter-aware reservoir computing (PARC) is a model-free framework for this purpose, where the parameter is supplied to a reservoir computer through an additional input channel. PARC is trained on time series recorded at several parameter values, and the trained reservoir, run in closed loop as an autonomous dynamical system, generates trajectories at parameter values that were never observed. rc_prediction is an open-source Python package that implements this workflow. It provides multi-parameter training of a reservoir, autonomous closed-loop prediction at user-specified parameter values, parameter scans, and a standard echo state network for forecasting at a fixed parameter. Hyperparameter optimization and optional GPU acceleration through PyTorch are also provided. We use three benchmark dynamical systems as examples to demonstrate its usage. As one of the featured applications, here we illustrate the prediction of critical transitions and system collapse: trained only on pre-critical data, the model is queried beyond the training range to predict whether the system collapses, to estimate the critical parameter $p_c^{*}$, and to characterize transient lifetimes.
★ Learning the Omega-Limit Partition from Approximate Koopman Invariants
Nonlinear dynamical systems with multiple omega-limit sets pose a fundamental challenge for global finite-dimensional Koopman modeling with continuous observables. A natural way to address this challenge is to decompose the state space into forward-invariant regions associated with distinct asymptotic behaviors and to learn a separate Koopman model for each region. This paper studies how to compute such partitions from data. We establish conditions under which Koopman-invariant observables, namely eigenfunctions associated with the unit eigenvalue, characterize the stationary partition induced by omega-limit sets. Building on this characterization, we develop OPAL (Omega-limit Partitions from Approximate invariants Learned from data), an algorithm that extracts partitions from approximate invariant observables. Numerical examples show that the recovered partitions capture the relevant asymptotic regions and support multiple regional linear Koopman models with significantly improved long-horizon prediction capabilities.
★ A Mixed $H_{\infty}$--Passivity Approach for Leveraging Heat-Pump-Connected District Heating Systems as Frequency Support in Electric Power Systems
Heat-pump (HP)-connected district heating systems (DHSs) can support grid frequency by rapidly modulating HP electrical consumption while accommodating the resulting heat imbalance. Thermal-state-informed HP control improves EPS--DHS coordination, but introduces slower DHS dynamics and heat-side disturbance propagation at the electrical interface on the time scale of secondary frequency control. This paper formulates the regulated DHS as a grid-facing dynamic port and develops a mixed $H_\infty$--passivity framework for disturbance-independent temperature regulation while preserving a clean electrical response during secondary frequency control. Sufficient conditions guarantee local asymptotic stability of the EPS--HP--DHS interconnection across primary frequency control, secondary frequency control, and DHS regulation. These stability conditions are integrated with multi-channel $H_\infty$ attenuation of DHS-state-dependent electrical dynamics, heat-disturbance propagation, and DHS-regulation errors, yielding a tractable convex LMI program for controller synthesis. Cross-model validation deploys controllers synthesized on control-oriented EPS--DHS models, without retuning, on an open-source electromechanical IEEE 39-bus model coupled with four high-fidelity finite-volume DHS models retaining thermal transport, storage, and heat-loss dynamics. The resulting controller improves EPS--DHS coordination without compromising the grid-facing electrical response.
★ Temporal and spatial couplings in the dynamics of human walking and running
Rhythmic behaviours such as walking and running are commonly modelled as dynamical systems around a stable limit cycle. To keep such models tractable, it is usually assumed that each step depends only on the previous one, that the limit cycle is fixed, and that the sagittal and frontal planes can be studied separately. Here, we test these assumptions in seven public datasets of human gait (71 participants), including 16 people who both walked and ran. We fit linear step-to-step maps and validate them by cross-validation, holding out more data where more data exist. At first sight, gait has memory: the state two steps back lowers the prediction error by up to 29\%. Yet most of this temporal coupling vanishes once the operating point may drift slowly and measurement noise is modelled; gait remains limit-cycle-like, around an operating point that moves. Running keeps more memory than walking in the same people. Spatial coupling between the planes is present in most participants, mainly from sagittal to frontal, but weak. This supports planar models of healthy gait, while a repeatable, task-dependent coupling index may serve as a measure of gait quality. The simple models are wrong, but useful enough.
★ Learning Stabilizing Latent Feedback Controllers via Multi-Step Backward Losses
Can learned latent dynamics models support the design of feedback controllers that provably stabilize the original system? For linear time-invariant systems, we show that learning linear autoencoders and latent dynamics with multi-step prediction-reconstruction losses $\unicode{x2014}$ which we term "backward losses" as they encourage latent trajectories to reconstruct full state trajectories $\unicode{x2014}$ provably recovers the structure needed for stabilization. Specifically, a sufficiently long reconstruction horizon ensures that a minimum ${H}_\infty$-norm controller designed using the learned latent model stabilizes the original system through output-feedback via the corresponding encoder, provided the latent dimension is large enough to capture all non-stable modes of the underlying system. We also show why both reconstruction and a multi-step horizon are essential: forward prediction losses that require only low prediction error along latent trajectories admit global loss minimizers that discard unstable modes regardless of horizon length, while single-step reconstruction losses can yield latent models whose stabilizing controllers fail to stabilize the original system. Finally, experiments on a non-linear flexible-arm system illustrate the practical implications of our results for learning non-linear autoencoders and latent dynamics.
★ Confidence-Modulated Safety Filtering for End-to-End Policies
End-to-end policies, from task-specific controllers to robot foundation models, learn complex behavior from raw observations but are not guaranteed to remain safe, so they are often wrapped in a safety filter. The resulting system operates under two domains: the filter's operational design domain (ODD), where its assumptions hold and safety is certified, and the policy's competence domain, where it is reliable. Filters typically fix their conservativeness regardless of the policy's competence, trading performance inside the ODD against margin beyond it. We ask whether the policy's confidence should set that level, and propose a filter-agnostic pipeline that modulates it at runtime: inside the ODD, where every setting is safe, confidence buys performance; beyond it, low confidence adds margin, which helps when it flags where the filter's model fails. Preliminary F1TENTH simulations, where this holds, show the modulated filter combining the strengths of permissive and conservative filters.
★ Approximate Dynamic Programming for Link Reconfiguration to Reduce Malicious Input Controllability in Consensus Networks
Communication links influence how strongly a malicious input at one sensor affects a consensus network. We study how a defender can reduce this influence by selecting one additional link after observing the currently attacked sensor. The attacked sensor changes over time, and vulnerability is measured by the trace of a finite-horizon controllability Gramian. A prescribed local epsilon-greedy attacker response rule defines a discounted Markov decision process for the defender. Exact dynamic programming (DP) provides an optimal policy using the full Gramian payoff matrix and attacker transition model. We then introduce approximate dynamic programming (ADP) that learns from sampled attacker transitions and empirical Gramian costs estimated from measured impulse responses. Under conditionally unbiased costs and standard sampling and learning-rate conditions, the learned action-value table converges to the exact DP solution. The analysis uses Bellman contraction and bounds the effect of persistent Gramian estimation bias, establishing when measured responses recover optimal defender decisions without constructing the full payoff matrix. In simulations on 20- and 100-node graphs, empirical ADP rollout costs are within 0.23% and 1.76%, respectively, of exact DP. In a two-dimensional intruder tracking example, both policies reduce network-wide root-mean-square tracking error spread by approximately 25% relative to an unmitigated attack.
★ Fast and Accurate Hydro Unit Commitment
We introduce a fast novel algorithm to commit hydro-generators. We show that our algorithm can commit 56 generators for 200 days 10,000 times faster than Hexaly, the leading commercial solver for scheduling problems, without sacrificing solution quality. Our method can commit 1792 generators for an entire year in approximately 20 seconds on commodity hardware. Hexaly is unable to solve problems of this size in reasonable time. We attain this runtime improvement by introducing a greedy-heuristic tailored for hydro units. We provide a direction for a possible proof that the error our solver makes decreases with the problem size. This is supported by our experimental results. We prove that our solver's runtime scales log-linearly in the number of generators and time periods.
comment: 5 pages (Conference limit), 5 figures
★ Incremental Stability of Gradient Flows under Generalized Polyak-Łojasiewicz Inequalities and Its Implications for General Systems
Gradient flows of twice continuously differentiable strongly convex and convex objectives are, respectively, incrementally exponentially stable and non-expansive in the Euclidean metric. In this work, we investigate to what extent these properties extend to nonconvex objectives satisfying generalized Polyak-Łojasiewicz inequalities. For objectives with a unique minimizer, we establish conditions under which the gradient flow is contracting in a state-dependent Riemannian metric and incrementally exponentially stable. For objectives with nonunique minimizers, we generalize the non-expansivity of convex objectives to incremental stability for more general nonconvex objectives. We focus on incremental stability due to its implication to convergence to equilibrium points; when paired with a Lyapunov function it implies pointwise convergence to a minimizer and continuity of the endpoint map. Along the way, we derive new structural results for incrementally stable nonlinear systems and extend the analysis to Heavy-Ball and Dynamical Inertial Newton flows.
★ Addendum to "Optimal Steering of a Linear Stochastic System to a Final Probability Distribution, Part II"
We hereby fill in a gap in an induction argument used in the proof of a theorem on state-covariance controllability of linear stochastic systems in an earlier paper by the authors. The mathematical statement is true. The missing point in the proof concerns the assertion that, in the canonical single-input case, the control can be assigned arbitrary endpoint values. We amend the argument by proving a slightly stronger induction statement that permits arbitrary finite endpoint jets of the control. An independent approach to homogeneous state-covariance controllability was provided in a paper by Abdelgalil and Georgiou (2025) as a corollary of strong controllability results on $\GLp(n,\mathbb R)$, and shown herein that it extends to the case of additive noise as well.
comment: 4 pages
★ Dynamic Persuasion: Managing Reputation under Censored Feedback
We study a long-run sender who repeatedly communicates about a state to a sequence of myopic receivers and whose credibility is a public score updated from past interactions. Receivers face a lemon loss and an opportunity loss, and their behavior depends on these only through a single index, the tolerance. Because the score records only detected misrepresentations, it rests above the sender's true honesty rate; we characterize this credibility premium in closed form. In a fluid limit in which the score adjusts slowly, no policy earns more than operating at a single credibility level, and the sender's gain from managing credibility equals the value of the premium: against receivers who correct for imperfect detection, that gain vanishes. At a fixed adjustment speed, every policy is bounded by the fluid value plus a term linear in the speed. We then study three policy classes, linear, (s,S), and threshold, obtain the optimal parameters of the linear and threshold classes in closed form, show how receiver losses shape them, and show that, under an approximation, the optimal (s,S) band collapses to a threshold. Simulations of the dynamics agree with the closed forms and indicate where the single-level conclusion stops: bands pay when switching modes is costly, and cycles pay when demand is seasonal.
comment: 44 pages, 8 figures. A preliminary version of part of this work appears in Proc. IEEE CDC 2026
★ How a group that cannot communicate knows itself by its enemy
Teams that cannot communicate seem stuck with what each member sees. We show that their opponent can tell them the rest. Silent hunters, jammed robots and traders who never meet all face an adversary that reacts to the group as a whole; every member can read that reaction, a tell, and learn what no member knows alone. One tell can do the work of a message from every member. The adversary would rather hide, but hiding interferes with its own task, and a team that commits in advance to how it will respond decides how much hiding is worth. A committed team never loses when its opponent cares more about its own goals, and does as well as a fully informed team exactly when the opponent's stake outweighs any gain from lying. Below that point the team learns more by trusting less: in a solved Gaussian model it obtains the truth by promising not to use all of it and keeps the cost the opponent would have paid to conceal; an exact test decides when this holds in finite games. How a team spreads its trust matters too. A lone uninformed member with a full-support prior needs a private coin, but with additive squared-error losses two interchangeable members who respond differently replace it, and a small discount on trust insures against misjudging the enemy. Information, on this view, is not only gathered or aggregated; it is produced by how a team commits to respond.
comment: 6 pages main text; 35-page SI Appendix included as an ancillary file. Code and data: https://doi.org/10.5281/zenodo.23743441
★ From Teleoperation to Long-Horizon Dexterity: Evaluating Visuomotor Skills and Offline Proxies for Dexterous Manipulation
Imitation learning policies are intended to make robotic manipulation robust to variations in object pose, appearance, and environment that challenge pre-programmed motion. However, real-world performance remains difficult to predict, as conventional optimization signals such as validation loss show no consistent relationship with robot success. To identify the design choices that govern real-world performance, 281 imitation-learning policies spanning seven policy families were evaluated in approximately 1,700 real-robot trials of RJ45 connector grasping and insertion using a five-fingered robotic hand. Controlled ablations showed that observation and action representations dominated conventional optimizer and sampling choices. For point-cloud policies, region-of-interest cropping, representation in the robot-base frame, and normalization within the point encoder each improved success by 70-100 percentage points. For the vision-language-action policy, absolute rather than relative action prediction improved insertion success by up to 70 percentage points. Skill decomposition, multi-view perception, and recovery demonstrations were associated with the strongest-performing systems, although their effects were not independently isolated. Of 67 offline metrics evaluated, only five measures of predicted-motion smoothness and directness reliably correlated with insertion success (r = -0.70 to -0.80), and only within a single policy family. These results identify representation choices as key determinants of real-world imitation-learning performance and show that offline metrics can shortlist policies, but cannot replace real-robot evaluation.
comment: Accepted at CoRL 2026 Workshop
★ Quantifying Linepack Storage Potential of Hydrogen Networks
Future hydrogen systems depend on large-scale storage to balance supply and demand. Hydrogen networks can store energy in the pipelines themselves, but the potential of this linepack storage has not been quantified comparably to dedicated storage facilities. We transfer metrics established for conventional storage, such as volume capacity and full-cycle equivalents, to hydrogen networks. To evaluate them, we formulate an optimization problem maximizing linepack storage utilization, subject to non-convex PDE-based constraints describing transient gas flow. We solve the optimization problem over a full-year horizon to capture seasonal variations and to assess the interaction with cavern storage. For computational tractability, we propose a simulation-informed sequential convex programming approach with adaptive time stepping that reduces problem size by up to 98% at accuracy comparable to fixed-resolution methods. For the GasLib-40 topology with German scenario data, the network provides a volume capacity of 24 million normal cubic meters at 28 full-cycle equivalents per year.
comment: 23 pages, 6 figures, submitted to the International Journal of Hydrogen Energy
♻ ★ Minimum-Energy Control for Stochastic Linear Systems: A Data-Driven Approach with Finite-Sample Guarantees
The open-loop minimum-energy control (MEC) problem asks for control inputs that minimize the control energy while ensuring the terminal state reaches a desired value. In this paper, we study the open-loop MEC problem for stochastic linear time-invariant systems. We consider a variety of stochastic constraints that enforce the terminal error to be small, and show that Stochastic MEC (SMEC) problem can be formulated as a QCQP. We obtain the optimal solution for SMEC and show that it generalizes the well-known Deterministic MEC problem by introducing novel stochastic variants of controllability and Gramian matrices. When the system dynamics is unknown, we obtain a direct data-driven solution to the SMEC problem using maximum-likelihood estimation and conformal calibration approaches. Finally, we quantify the sub-optimality and constraint violation of our data-driven solutions via finite-sample high-probability upper bounds, and highlight their dependence on the number of experiments, system dimension, and noise-to-signal ratio. We present numerical simulations to validate the theoretical results.
♻ ★ From Reconnaissance to Response: Quantitative Risk Parameterization and Game Theoretic Containment in Modern Enterprise Attack
Modern Security Operations Centers struggle with delayed manual incident response, enabling adversaries to advance through the Cyber Kill Chain during early stage reconnaissance. While classical game theoretic defense models optimize strategic resource allocation, they rely on static utility matrices that fail to adapt to dynamic telemetry. This paper presents an integrated, metrics driven decision engine that bridges quantitative risk parameterization and continuous automated response time. Common Vulnerability Scoring Systems exploitability parameters are mapped to attacker success probabilities and evaluate defender log distributions via Factor Analysis of Information Risk Monte Carlo simulations. Real time SIEM logs streams are modeled as Poisson process arrival rates, dynamically updating defender posterior threat belief through sequential Bayesian filtering. A closed form threshold is derived by framing the interaction as a dynamic Bayesian Stackelberg game, where the expected unmitigated risk exceeds proactive containment cost. Parameterized against empirical data from the 2023 MGM Resorts and Caesars Entertainment cyber incident, simulation results demonstrate that the engine suppresses transient background noise while triggering automated SOAR network isolation within seconds of adversarial probing. Multi parameter sensitivity analysis confirms that the decision boundary dynamically adjusts to live perimeter vulnerability, offering a control theoretic foundation for sub minute automated threat containment.
comment: 10 pages, 5 figures
♻ ★ Robust Taylor-Lagrange Control for Safety-Critical Systems with Guaranteed Feasibility
The Control Barrier Function (CBF) method has been widely adopted in solving safety-critical control problems. However, the existence of a CBF is only a sufficient condition for system safety. The recently proposed Taylor-Lagrange Control (TLC) method addresses this limitation by providing a necessary and sufficient condition, but it is still vulnerable to the feasibility preservation problem (e.g., due to the inter-sampling effect when time is discretized). In this paper, we propose a robust TLC (rTLC) method to address the feasibility preservation problem and show how to use it to solve optimal control problems. Specifically, the rTLC method expands the safety function to an order higher than its relative degree using Taylor's expansion with Lagrange remainder, which allows the control to explicitly show up at the current time instead of a future time in the original TLC method. This implies that the safety constraint remains satisfied over the entire discretization time interval that follows. Moreover, the rTLC method naturally addresses the feasibility preservation problem with only a single hyper-parameter (the discretization time interval size during implementation), compared to significantly more parameters for its counterparts. Finally, we illustrate the effectiveness of the proposed rTLC method through an adaptive cruise control problem, and compare it with existing safety-critical control methods.
comment: This version addressed one technical problem of the inter-sampling effect in rTLC
♻ ★ Reduced-Order Data Assimilation for Thermospheric Density Using Physics-informed SINDyc Models
Orbit prediction and space situational awareness require accurate thermospheric mass density, which responds nonlinearly to solar and geomagnetic forcing across several orders of magnitude. Physics-based general circulation models resolve that response but are computationally expensive, while empirical models run cheaply and carry no time-evolving atmospheric state. An autoregressive Sparse Identification of Nonlinear Dynamics with control (SINDyc-AR) model, derived from the Thermosphere-Ionosphere-Electrodynamics General Circulation Model (TIE-GCM), captures the dominant modes of variability and their dependence on the drivers at a fraction of the parent model's cost. An extended Kalman filter assimilates in situ observations from CHAMP, GRACE, GRACE-FO, GOCE, and Swarm into that model across several orbital configurations and geomagnetic conditions, with a linear DMDc model as reference. Assimilation reduces density error relative to open-loop prediction, most visibly during geomagnetic storms and under single-satellite coverage. The two reduced-order models perform comparably throughout. NRLMSIS 2.1 and HASDM can outperform the assimilated model far from the assimilated track, so results are framed as improvements over the open-loop forecast. An observation-calibrated global density record spanning August 2000 to December 2025 is released alongside this work.
♻ ★ A heuristic for optimal coordination of autonomous vehicles: real-time algorithms
When coordinating connected and automated vehicles at an unsignalized intersection, the crossing order and the vehicle trajectories must be decided jointly. For a fixed crossing order, the problem can be formulated conveniently in a bi-level form where each vehicle solves its own optimal control problem, whose optimal value is a function of its entry and exit times, and a coordinator optimizes these times subject to precedence constraints. We compute the first and second order sensitivities of lower-level problems, which are required to solve the upper-level problem, by means of algorithmic differentiation using the real-time software package \texttt{acados}. This implementation exploits tailored algorithms to obtain a computationally efficient implementation of the optimal coordination algorithm. We validate our implementation in numerical simulations that demonstrate both real-time feasibility and effectiveness of our algorithm.
♻ ★ A heuristic for optimal coordination of autonomous vehicles: static problems
We consider the coordination of autonomous vehicles at unsignalized intersections, where trajectory planning and crossing-order decisions are intrinsically coupled, thereby making their simultaneous optimization intractable in practice. To this end, we propose a heuristic based on a hierarchical structure consisting of a series of nonlinear programs (NLP) in the lower level, which optimizes trajectories of each vehicle for a given crossing order, and a high-level heuristic to design new crossing schedules that exploits sensitivity information from the lower-level NLPs. Extensive numerical simulations across a broad range of traffic scenarios demonstrate that our strategy achieves substantially improved scheduling with respect to first-in-first-out ordering while preserving computational tractability.
♻ ★ A heuristic for optimal coordination of autonomous vehicles: dynamic problems
The coordination of autonomous vehicles at unsignalized intersections requires deciding both the trajectories of the vehicles and their crossing order, with the additional difficulty that the set of vehicles to be coordinated changes as traffic flows through the intersection. Since the resulting mixed-integer optimal control problem is hard to solve online, we propose an arrival-triggered receding-horizon scheme built around a schedule-refinement heuristic: for a fixed crossing order, a single nonlinear program (NLP) optimizes all trajectories and crossing times, and the first-order sensitivities of the cost with respect to the vehicle schedule are used to select adjacent swaps that are likely to reduce the overall cost. Replanning is triggered only when a new vehicle arrives, the new vehicle is appended at the tail of the current crossing order, and a memory of evaluated swaps that persists across replanning steps avoids redundant computations. Simulations of traffic scenarios involving streams of a few hundred heterogeneous vehicles show that the proposed scheme recovers most of the optimality gap incurred by first-in-first-out scheduling.
♻ ★ Executor-Side Progressive Risk-Gated Actuation for Agentic AI in Wireless Supervisory Control
Agentic artificial intelligence built on large language models shows promise for automating Open Radio Access Network supervisory control. However, action-level safeguards alone may be insufficient: evidence acquisition consumes time and shared resources, and network changes can invalidate approvals before release. We propose Progressive Risk-Gated Actuation (PRGA), an executor-side model and contract governing intents from assessment to release or termination. PRGA combines local checks with selective retrieval of coordination evidence to reduce acquisition costs while preserving mandatory evidence requirements. Version and lease rechecks prevent release under invalidated approvals, while reevaluation requires recorded progress within each transaction's retry budget. Building on this contract, we formulate evidence allocation to maximize the expected whole-period rate of timely valid releases under risk, quality-of-service, deadline, and shared byte-budget constraints. For energy-saving and slice-protection workloads with stable daily load cycles, thresholds are selected offline and interpolated using an online load-phase estimate. On average over modeled start phases and quota levels, the allocation policy improves timely valid release rates over arrival-order fetching and an offline-selected global threshold under identical quotas and release checks. In OpenAirInterface radio emulation, an immediate scripted resend after a version-mismatch refusal raises on-time receiver delivery from 77.3% to 89.9% at modest additional command cost.
♻ ★ Why the Swing Parameters Are Sloppy
When a multi-parameter model is fitted to measured data, only a few parameter combinations are often identifiable, as Fisher Information Matrix eigenvalues span several orders of magnitude. Sensitivity-based tools, including the Fisher Information Matrix and MBAM, reveal this structure analytically, while manifold learning can recover it from sampled trajectories. For an infinite-bus synchronous generator, both approaches show that inertia H and damping D are among the least identifiable parameters on a late post-disturbance window, t in [3,5] s, where six of the eleven parameters suffice to describe the response. We explain this result from the geometry of the dynamics rather than from that of the parameter-to-data map. The rotor swing is a fast fibre that has collapsed onto a two-dimensional normally attracting invariant manifold (NAIM) of the transient fluxes before the window opens. The zeroth-order NAIM is exactly independent of $H$, $D$, $T_{d0}''$ and $T_{q0}''$, four of the five MBAM limits. We prove, under stated smoothness and rank conditions, that each of the four limits is a boundary of the model manifold at a distance equal, to first order in the parameter, to its current profiled speed. The same analysis predicts how identifiability changes with the experiment: neither a larger initial speed deviation nor a load change restores H or D in the late window, whereas the early window [0, 2]s makes H the best-determined parameter. Evaluated in a physical metric fixed by synchrophasor accuracy limits, the late window identifies five parameters rather than six and the early window nine. The spiked-covariance structure of the linearised trajectory map explains why the principal components of whitened, sampled trajectories reproduce the prior-scaled Fisher spectrum, and regression on those components attains the Fisher-predicted recovery error in 113 of 114 parameter scenario pairs.
comment: The draft is withdrawn so as to improve before resubmission. Regret the inconvenience
♻ ★ Safe Unicycle Navigation Among Multiple Obstacles Under Bounded Disturbances: A Learning-Augmented CLF-CBF Approach (Extended Version)
This paper presents a robust control framework for safe navigation of a unicycle robot to a goal among multiple static obstacles under an unknown, time-varying disturbance. The controller solves a single real-time quadratic program (QP) that combines a goal-reaching term with one safety term per obstacle. The goal-reaching term is a cross-track control Lyapunov function built from body-frame tracking errors rather than a bearing angle, which keeps its Lie derivatives bounded everywhere, including the goal. Safety is encoded using robust control barrier functions, one per obstacle, each tightened by a closed-form, state-dependent disturbance bound. When stabilization and safety conflict, a slack variable relaxes only the convergence requirement, so that the safety constraints remain strictly enforced. Whenever the QP is feasible, the robust barrier constraints guarantee forward invariance of every obstacle's safe set under all admissible disturbances. Complementing this safety guarantee, a Lyapunov analysis yields an explicit ultimate bound on the distance to the goal. A learning-augmented variant replaces the fixed objective with a reinforcement-learning reference, which accelerates convergence when multiple constraints are simultaneously active. We further derive the obstacle separation below which the gap between two obstacles becomes too narrow to pass through under the robust constraints. For such gaps, a learned virtual goal for the Lyapunov function steers the robot around the obstacles without compromising safety. Simulations, a Monte Carlo study, and a comparison against a sampled-data robust baseline support the theoretical results.
♻ ★ Safe World Model for Joint Trajectory and Beamforming Design of Mobile Base Stations
Mobile base stations (MBSs), including aerial and ground-based platforms, provide flexible wireless coverage through adaptive positioning. However, jointly optimizing their trajectories and digital beamforming is challenging because movement changes channel conditions and interference, while beamforming determines the resulting signal and interference powers. These coupled decisions also account for how current movements affect future positions and communication performance under mobility and safety constraints. We formulate this problem as a constrained Markov decision process that maximizes the time-averaged sum signal-to-interference-plus-noise ratio (SINR), subject to mobility, obstacle region avoidance, and transmit-power constraints. To support long-term planning and reduce the extensive environment interactions required by model-free reinforcement learning, we develop a SafeDreamer-based world model framework that learns latent system dynamics from experience and optimizes a joint trajectory and beamforming policy through imagined trajectories. Feasibility-preserving action mappings and safety cost signals incorporate operational constraints into policy learning. Simulation results demonstrate that the proposed framework achieves higher time-averaged sum SINR than the proximal policy optimization and deep Q-network baselines, while exhibiting improved sample efficiency and more reliable constraint satisfaction.
♻ ★ Global Parameter Identification and Uncertainty Quantification of reduced-order ADM1 using Maximum Likelihood Estimation and Bayesian Inference
Anaerobic digestion (AD) models can be challenging to calibrate against real data due to limited practical identifiability, data scarcity, and model complexity. Weighted least squares objective functions and gradient-based solvers using finite differences are often used in literature, but result in suboptimal parameter estimates and large uncertainty bands. This paper identifies the parameters of a reduced-order ADM1 by comparing the frameworks of maximum likelihood estimation (MLE) and Bayesian inference (BI), and quantifies uncertainties of parameters and outputs. Multirate experimental data are taken from a lab-scale AD reactor fed with agricultural substrates and operated at organic loading rates of 3$\,$g VS $\mathrm{m^{-3}d^{-1}}$. Global optima are found via global and gradient-based local search algorithms using forward sensitivities. Enhanced scatter search provides the best global search results. These are refined by the Gauss-Newton algorithm as the best local search, which delivers $R^2$ values of 0.95 for acetic acid and 0.77 for biogas production. BI yields substantially narrower uncertainty bands than MLE and avoids the need for a global search by incorporating prior knowledge about parameter distributions. The search space is captured efficiently by log10 scaling of parameters. Algorithmic details on its implications are provided. Despite a high-resolution lab-scale dataset, the ADM1-specific inhibition parameter $K_\mathrm{I,nh3}$ is practically not identifiable with MLE, and shows a low output sensitivity, whereas a parameter for the effective ion concentration shows superb practical identifiabilty at high confidence with both MLE and BI. The present work lays a scalable base for calibration of nonlinear AD models by providing reliable simulations. It thus advances model-based monitoring and control applications of multirate processes such as agricultural AD plants.
comment: v2: updated license
♻ ★ A Closed-loop Framework to Discriminate Models Using Optimal Control
Predicting the response of an observed system to a known input is a fruitful first step to accurately control the system's dynamics. Despite the recent advances in fully data-driven algorithms, the most interpretable way to reach this goal is through mechanistic mathematical modeling. Here, we leverage optimal control and propose a closed-loop iterative method to choose among a set of candidate models the one that most accurately predicts an observed system. We assume that one has control over an input of the observed system and access to measurements of its response. Our approach is to identify the input control that maximally discriminates the response of the candidate models, allowing us to determine which model is best by comparing such responses with the observed data. We demonstrate our proposed framework in numerical simulations before applying it during an electrophysiology experiment, successfully discriminating between different models for photocurrents produced via opsin dynamics.
comment: 24 pages, 15 figures
♻ ★ Biased Stochastic Gradient Descent under Local Geometry, with Application to Data-Driven LQR
We study stochastic gradient descent (SGD) with biased stochastic gradient oracles for objectives that are only locally smooth and locally gradient dominated, with constants that may grow with the objective value and are required only on a sublevel set. Such oracles arise in data-driven optimization and control, where the gradient must be estimated from noisy data through nonlinear procedures such as model identification or smoothing and truncation. We show that if the oracle bias decays sufficiently fast, even only in expectation, and the second moment of the oracle remains bounded, then, on an event of prescribed probability, the optimality gap converges to zero almost surely with an asymptotic rate, and in expectation with an explicit bound at every iteration that separates the contributions of the initial condition, the oracle bias, and the gradient noise. We illustrate the framework on the linear quadratic regulator (LQR), whose cost satisfies these properties only on sublevel sets and whose policy gradient depends on the unknown system matrices. We consider an indirect estimator, which identifies the system matrices online, and a direct zeroth-order estimator. We characterize the bias and the second moment of both estimators and show how to choose their parameters so that the convergence conditions hold. On an event of prescribed probability, both data-driven policy gradient methods converge to the optimal policy almost surely and in expectation, with guarantees for all iterations rather than at a single prescribed iteration. We compare the two estimators, and numerical experiments on the LQR illustrate the results.
♻ ★ Learning Maximal Regions of Attraction on Compact Sets using Taylor Lyapunov Networks
We consider the problem of learning maximal Lyapunov functions to compute the maximal region of attraction of nonlinear dynamical systems using neural networks. Two obstacles limit existing neural-network approaches: 1) finite-width approximations can fail to preserve the strict Lyapunov inequalities near the equilibrium, where they become arbitrarily weak, and 2) training is necessarily performed on a bounded computational domain for which the classical Lyapunov and Zubov theorems do not directly apply. To address these issues, first we derive Lyapunov and Zubov theorems directly on a prescribed compact computational domain, characterizing the maximal region of attraction within that domain as the solution of a boundary-value PDE. Second, we introduce the Taylor Lyapunov network, an architecture that prescribes the quadratic leading-order term via the system's linearization and represents the higher-order remainder through a structured cubic factorization, guaranteeing that any network in this class is a valid local Lyapunov function by construction. Finally, we prove that Taylor Lyapunov networks are universal approximators of maximal Lyapunov functions on compact domains, and combine these results into an unsupervised, physics-informed training procedure. Numerical experiments on systems of varying dimensions and stability types show that the proposed architecture is efficient at estimating the maximum region of attraction.
Robotics
★ Controlling Continuum Robot Shape in Enclosed Space with Embedded Sensing using IMU and Actuator Feedback
Controlling continuum robots' shape to operate in confined, enclosed environments is challenging, since it cannot rely on external observation. Existing embedded shape sensing methods, although they can provide shape reconstruction, are insufficient to enable real-time shape control due to limitations in reconstruction accuracy, computational speed, and compatibility with shape-state representation. In this work, we present an embedded shape-sensing framework that uses readily available IMU and actuator feedback to reconstruct the continuum-robot shape with a neural network, providing accurate shape feedback at $400~\mathrm{Hz}$ for physical closed-loop control. We systematically investigate four key reconstruction design choices: shape-state representation, network architecture, training objective, and sensing inputs, through more than $5{,}000$ trained models on one- and three-segment continuum robots. The selected framework achieves global shape errors of $0.9\%L$ and $1.7\%L$ (of the robot length) for the one- and three-segment systems, respectively. In closed-loop control, one-segment tip tracking achieves a $6.1\%L$ error and three-segment shape control achieves a $2.3\%L$ global shape error, whereas IMU--PCC reconstruction using the same uncompensated measurements fails. Enclosed-space experiments further demonstrate operation without external observation. These results provide a practical framework for translating readily available embedded sensing into high-rate feedback for continuum robot closed-loop control.
comment: 8 pages, 7 figures
★ Robotic Data Inoculation
Modern robot learning systems feed on high-quality demonstration data, and their performance hinges on coverage. Without good coverage, small errors compound and the policy can quickly go into out-of-distribution states from which it cannot recover. Yet there is no principled way of collecting high-coverage data, and data collection often relies on heuristic instructions to human operators. Purely online reinforcement learning (RL) explores at the cost of safety and sample efficiency, while DAgger-style interventions require the policy to fail by chance before a human can correct it. In this paper, we ask instead whether we can modify how the robot behaves during data collection to collect more useful data. We propose Data Inoculation (DI), a shared-control framework in which a human provide closed-loop teleoperation commands while a residual policy perturbs these commands. The residual policy is trained online, on the growing dataset, with the same reward-maximizing objective later used for policy extraction, so its perturbations steer data collection toward the deviations a policy trained on the data would make. These perturbations expose the robot to a wider range of states, while the human's continuous corrections provide recovery demonstrations, "inoculating" the dataset against the learner's future mistakes while the human remains in control. On three challenging real-world robotic tasks, DI consistently improves or matches the success rate of policies trained with both behavioral cloning and offline RL, showing DI as an effective data collection strategy for improving downstream policy performance.
★ Catch It All!: Generalizable Dynamic Catching with a Dexterous Hand
Catching arbitrary objects requires generalization to novel projectiles and real-time adaptation to unpredictable contact dynamics under noisy visual observations. This paper presents Catch It All!, a sim-to-real catching framework for humanoid robots that generalizes and adapts to unseen objects with variable physical properties for underhand and overhand catches, using only one egocentric camera. We learn policies which generalize to unseen objects with varying geometry, deformability, elasticity, and physical properties. Effective grasps which adapt real-time to object poses and shapes are learned through careful grasp reward design. We provide safety-aware curricula which enable safe, high-frequency action execution near hardware limits, including wrist torque limits as low as 5 Nm. We validate our system by deploying zero-shot on a Unitree G1. With only one egocentric camera, Catch It All! successfully performs underhand and overhand catches on 9 unseen objects not seen during training, including low-rebound plushes, semi-deformable toys and chip bags, and rigid-elastic objects with high rebound on contact.
★ Self-Supervised Doppler-Guided RF Odometry NeurIPS 2026
Radio-frequency (RF) sensing enables robot ego-motion estimation in darkness, smoke, and fog, yet RF odometry remains challenging because multipath and moving reflectors make cross-frame matching highly ambiguous. Existing methods often rely on auxiliary inertial sensors or ground-truth pose supervision. We present RFWalk, a self-supervised, RF-only odometry framework that models dense correspondences across range-azimuth frames as random walks on a space-time graph and learns them from unlabeled sequences through temporal cycle consistency. To rule out correspondences that are cycle-consistent but physically implausible, we introduce a Doppler consistency loss that constrains correspondence-induced motion using radial velocity measured by the RF sensor itself. Ego-motion is then recovered through robust $\mathrm{SE}(2)$ registration that suppresses uncertain correspondences and those arising from independently moving objects. Without pose labels or auxiliary sensors, RFWalk remains competitive with supervised baselines in static scenes and reduces absolute trajectory error by 58% in dynamic scenes with moving pedestrians.
comment: Accepted at NeurIPS 2026
★ Open-World Object Motion Anticipation via Meta-Learned Semantic Adaptation
Anticipating the motion of nearby objects is critical for safe and efficient robot navigation. Without a proper understanding of what may move, a robot may plan overly conservative, inefficient trajectories to guarantee safety, or it may be overly risky, resulting in unacceptable collisions. While tracking and predicting the motion of known objects has seen significant progress, little research addresses how a robot should adapt online to novel objects, including anticipating how one object may move another (e.g., a person operating a forklift to move a pallet). To enable this capability, we propose a meta-learned graph neural network that leverages a semantic, geometric, and relational history of objects to accurately predict, from a few online observations, when objects will begin to move in new environments. We demonstrate that our method, which is trained only in simulation, accurately anticipates the motion of objects unseen at training time, transfers to hardware without any fine-tuning, and outperforms a vision-language model and a class-frequency baseline in both simulation and hardware experiments. Our method's accurate prediction enables collision-free traversals in a closed-loop navigation simulation, while remaining within four seconds of the fastest, collision-prone planner.
comment: 9 pages, 7 figures
★ Learning Event-based Obstacle Avoidance via Differentiable Simulation
Reliable obstacle avoidance is essential for safe quadrotor flight in unknown environments, yet conventional frame-based cameras remain challenged by motion blur, limited dynamic range, and latency under fast motion. Event cameras, with sub-millisecond latency, high dynamic range, and negligible motion blur, offer a promising alternative, but leveraging them for closed-loop control remains difficult. Existing approaches either adopt cascaded architectures that first estimate depth and then map it to control commands, introducing error propagation, or achieve end-to-end control through costly offline pretraining before online fine-tuning can even begin. We address these challenges with a CUDA-based event simulator that exploits GPU parallelism to render events efficiently within the training loop, integrated into a differentiable simulation pipeline that enables direct gradient backpropagation through the differentiable physics model, improving sample efficiency. This allows us to train an end-to-end policy that maps event frames directly to control commands, without intermediate depth estimation. We validate our approach through zero-shot real-world deployment on a quadrotor, achieving robust obstacle avoidance in both odometry-assisted and odometry-free settings, including under low-light conditions where conventional cameras typically fail.
★ Selective Sliding Mode Control Enhances Robustness of Aerial Robots Under Disturbances
Quadrotor control under disturbances such as wind gusts and payload variations remains challenging. First-order sliding mode control (SMC) and matched/unmatched uncertainty distinction are classical approaches; however, neither tells a practitioner which loops in a cascaded controller should be made robust to sustain disturbances, how to tune those loops, nor what the additional costs are on an embedded processor. Applying robust control uniformly across all axes addresses disturbances indiscriminately, adding tuning effort on axes where no disturbance acts. On inherently underactuated vehicles, disturbances along some axes cannot be opposed directly at all. We propose a principled methodology for deciding where robust control belongs in a cascaded architecture, with all control running onboard the vehicle. We recover the acting disturbance from PID hover flights under disturbance by inverting the equations of motion, use the recovered signal to identify and quantify the excited axes, and deploy robust control where it counts. Experiments on a Crazyflie 2.1 quadrotor under slung-payload and oblique-wind disturbances show tracking improvements concentrated on the axes the rule selects, and gain sweeps confirm that the switching gain reduces error on matched axes but has no effect on an excited unmatched axis, as the analysis predicts.
comment: 8 pages, 5 tables, 8 figures
★ Inertial Steering and Motion Prediction of Tracked Vehicles on Low-Adherence Surfaces
Tracked vehicles are widely used in off-road applications, but their conventional motion models often rely on no-slip assumptions that become inaccurate on low-adherence surfaces. Rather than treating inertial effects arising from slip as a disturbance to be minimized, this work investigates how inertial effects arising from a moving mass can be deliberately exploited for steering on low-friction surfaces. A dynamic model for an articulated tracked vehicle with a passive trailer is developed, allowing inertial effects from trailer motion to be captured. The approach is experimentally evaluated on ice, where our dynamic model outperforms the kinematic baseline. We also propose a mixed dynamic-kinematic model to adapt predictions to both surface and situation by exploiting predictions from both methods. Finally, we demonstrate inertial steering on a mono-track vehicle without a conventional steering mechanism, showing that controlled movements of an internal mass can generate turning motion on ice. The resulting behavior can be predicted using the proposed dynamic model by representing the internal mass as a frictionless trailer, revealing the analogous dynamics between the two configurations. These results demonstrate the feasibility of inertial steering as an alternative steering mechanism for tracked vehicles operating in low-adherence environments.
★ AtomTree: A Hierarchical Framework for State-Aware Embodied Instruction Following with LLMs IROS 2026
Embodied instruction following in interactive environments requires agents to interpret natural language commands and complete long-horizon tasks. Existing planners either rely on large labeled trajectories, or use few-shot and zero-shot LLM prompting that produces a whole plan in a single serial reasoning process. These approaches depend on linguistic common sense, commit to a plan from the initial instruction, and only update after failures, which weakens robustness and state awareness. We present AtomTree, a hierarchical framework for state-aware task planning with LLMs. AtomTree incrementally builds a tree of subgoals and actions, where the LLM decides at each step, from the current symbolic state, whether to further decompose or to output a grounded low-level action. A lightweight memory module summarizes past outcomes for reuse in later decisions. When a low-level action stalls, AtomTree triggers hierarchical backtracking replanning to return to the parent node and regenerate the subtree plan from the latest state, preventing continued drift along an incorrect trajectory. In zero-shot evaluation on the ALFRED benchmark, AtomTree outperforms prior LLM-based planners. Project page: https://zhaihaotian.github.io/atomtree-website/.
comment: Accepted at IROS 2026
★ Value Function-Guided MPPI for Hierarchical Planning and Control in Off-Road Autonomous Navigation
Autonomous navigation in off-road terrain is commonly achieved using hierarchical control and planning architectures that rely on explicit cost maps and one-way information flow from a high-level planner to a low-level controller, decoupling planning from the true execution cost experienced during control. We propose a hierarchical planning and control framework in which information flows bidirectionally, allowing the low-level controller to guide trajectory selection at the planning level. The highest level of the hierarchy is a Model Predictive Path Integral (MPPI) control-style planner that determines optimal trajectories and passes these waypoints to a pretrained low-level reinforcement learning (RL) controller. Rather than drawing from an explicit cost map, our MPPI planner leverages the value function of the RL controller as an implicit cost representation. In this formulation, MPPI serves as a trajectory exploration mechanism, while the low-level controller's value function guides the search. To account for the distribution shift seen by the value function between training and evaluation, we propose an online policy evaluation process that adapts the value function to unseen driving experiences, and we demonstrate this adaptation in unseen off-road simulation environments. We evaluate our approach in a high-fidelity off-road driving simulator across a range of challenging terrain conditions and compare its performance against a baseline MPPI controller. The results demonstrate that our method achieves safer and more effective navigation to goal locations while eliminating the need for explicit cost maps and enabling tighter coupling between planning and control.
★ Quantize by Doing: Task-Guided Mixed-Precision Quantization of Vision-Language-Action Models
Post-training quantization (PTQ) reduces the memory footprint of vision-language-action (VLA) models, but lowering bit widths while maintaining task performance remains challenging. Existing work has used sensitivity estimates from calibration data to assign bit widths to different model components. However, using task success directly to guide mixed-precision allocation is underexplored because closed-loop evaluations are costly and the effect of quantizing one component depends on the precision of other components. To this end, we propose \textbf{QbD-VLA}, a task-guided mixed-precision quantization method that uses task success from short closed-loop evaluations to assign weight and activation bit widths while accounting for interactions between model components. Experiments covering four VLA models and all four LIBERO suites show strong task performance at average precisions of approximately 4 bits for weights and fewer than 6 bits for activations. On SIMPLER, the gain over uniform quantization is small. On a Franka Research 3 robot arm, simulation-selected allocations show no detected success-rate loss on X-VLA but fail on $π_{0.5}$. These findings support using task success to guide mixed-precision VLA quantization, while highlighting the need for real-world evaluation before deployment. Code is available at \href{https://github.com/jiuyixu25/QbD-VLA}{\texttt{github.com/jiuyixu25/QbD-VLA}}.
comment: 13 pages, 3 figures, 8 tables
★ Explore Where It Matters: Targeted Exploration for Efficient Rover Adaptation
Future planetary rovers will traverse steep, deformable terrain where slip, sinkage, and terrain deformation can threaten mobility. Although simulation can capture some wheel-soil effects, unmodeled deformable-terrain phenomena create sim-to-real gaps requiring deployment-time adaptation. Yet, rover operations leave little room for extensive trial-and-error. We introduce TERRA (Targeted Exploration for Reward-Relevant Adaptation), an online adaptation framework that efficiently seeks deployment data exposing task-relevant model errors and corrects them. TERRA adapts a simulation-trained world model with a reward-weighted prediction loss and trains a loss predictor to estimate where future performance-relevant errors are likely to occur, using this signal to guide exploration during MPC. In simulation, TERRA reaches approximately 90% slope-climbing success within five trials under severe out-of-distribution pretraining conditions, outperforming exploration baselines in reward, completion time, and energy to first success. On the ERNEST rover, TERRA recovers coordinated active-suspension locomotion on a deformable Mars-simulant slope within three to five real-world trials. Project website: https://terra-explore.github.io
comment: Conference on Robot Learning 2026
★ The Reality Gap in Automated Driving Systems Testing: Landscape, Challenges, and Research Directions
Simulation is essential for testing automated driving systems (ADS), but discrepancies between simulation and reality can undermine testing conclusions. It remains unclear which discrepancies the literature addresses and what evidence supports correspondence claims. We present a systematic mapping of 235 primary studies that measure or mitigate the reality gap in ADS testing, selected from 4,195 publications. We characterise objectives, targets, approaches, experimental setups, and evaluation evidence. Our classification distinguishes simulated and real-world execution across six ADS test components, separately for each approach and its evaluation reference. This distinguishes real-world information used to construct an approach from evidence used to evaluate its claims. Of the included studies, 76% address mitigation, primarily targeting scenario and actor realism and perception, while 94% report validation or measurement procedures. Only 61 studies report metrics from multiple families covering sensor/perception correspondence, ADS behaviour, vehicle dynamics, or safety outcomes; this coverage does not establish causal links between discrepancies and test verdicts. The synthesis highlights that real-world content and physical execution must be interpreted against the claim evaluated. The findings motivate claim-specific evaluation and component-level reporting to establish which discrepancies affect testing conclusions and when simulation evidence supports conclusions about real-world ADS behaviour.
★ TRIM: Temporal Run-Time Monitoring of 3D Object Detection via Activation Statistics
Reliable perception is essential to intelligent transportation systems, yet deep neural networks can silently miss safety-critical objects while continuing to produce apparently plausible outputs. Existing run-time monitors for 3D object detection generally assess each frame independently and therefore cannot exploit the temporal evolution of occlusion, point sparsity, and scene complexity. We present TRIM, a lightweight temporal monitor that detects missed-object failures without modifying or retraining the underlying detector. TRIM summarises intermediate activations using compact channel-wise statistics and processes short sequences of these summaries with a temporal convolutional network. Experiments on NuScenes show that temporal monitoring improves AUROC from 0.686 to 0.704 and reduces the false alarm rate at 95% recall from 0.818 to 0.759 compared with single-frame monitoring. Early and intermediate backbone features provide the strongest monitoring signal, while the temporal inference head requires only 0.159 ms on a CPU.
comment: 6 pages. Accepted for publication at the 2026 IEEE 23rd International Conference on Smart Communities: Improving Quality of Life using AI, Robotics and IoT (HONET)
★ A Multi-Layer Navigation System for Smart Wheelchairs with a Limited Input Interface IROS 2026
Smart wheelchairs are robotic devices that gather information from the surroundings and allow people to safely drive through various environments. They usually rely on the interpretation of high-level user's intentions, typically decoded through human-robot interface (HRI) or brain-machine interface (BMI). However, navigating in an unknown environment and accurately understanding user commands remain significant challenges. Misinterpretation of commands or the environment can lead to user frustration, distrust of the system, or even unsafe situations. In this paper, we present a novel Multi-Layer Navigation System (MLNS) for a smart wheelchair that enables navigation in unknown environments while interpreting and safely contextualizing user's intentions even in the case of limited and faulty commands. The stack relies on a Virtual Sensors-Driven Potential Fields (ViSPF) method for obstacle detection that accounts for the wheelchair's actual size, a Model Predictive Control (MPC) module for trajectory generation, and an open-vocabulary computer vision algorithm for target identification. Finally, a system based on Priority Queue (PQ) was implemented for target selection. We validated our system on a real smart wheelchair with experiments involving 10 participants without disabilities, including scenarios with faulty input interfaces. The results demonstrate the robustness and effectiveness of our system in real-world environments, proving its potential for practical applications in assistive technologies.
comment: 8 pages, 6 figures, 3 tables. Presented at IROS 2026 (IEEE/RSJ International Conference on Intelligent Robots and Systems); to appear in the conference proceedings
★ Influence of Agent Number and Behavior on Users' Psychological Responses in VR Evacuation Training
This study aims to build effective drill software by combining a virtual simulation of a disaster with multiple agents capable of expressing emotions. We evaluated how the number of agents and their emotional and behavioral actions during evacuation influence users in a virtual space. The results showed that users tended to act more confidently when many agents evacuated with them, and that the evacuation route chosen by the agents influenced the users' decision-making. These findings indicate that users' evacuation behavior is shaped not only by the environment itself but also by the collective behavior of surrounding agents, underscoring the need to consider how agent behavior affects users' psychological states during training.
comment: The 2nd International Workshop on Trust and Blame in Multi-Person Multi-Agent Societies (TBSA_2nd) in conjunction with HAI 2026, 7 pages, 8 figures, 5 tables
★ Force-Driven Hook Disassembly via Constraint-Aware Object-Centric Reinforcement Learning
The growing volume of end-of-life products has intensified the need for automated disassembly, yet hook-type connections remain challengin: their orientations vary, they deform under load, and their geometry is hidden, so vision alone cannot identify a feasible non-destructive disassembly direction. This work proposes a constraint-aware reinforcement learning framework that infers feasible extraction directions online from contact forces. The policy is trained in a robot-agnostic, object-centric simulation on a simplified peg-in-hole pair and deployed zero-shot on a physical robot, where it transfers to real connections with unseen, multi-stage constraint structures. Because instantaneous contact forces are noisy and locally ambiguous, the policy is coupled with a particle filter belief that provides the policy with temporally integrated directional information and reduces redundant exploratory motion. The approach is validated on four real-world tasks, from peg-in-hole pairs to the inner propeller and front cover of an end-of-life air-conditioning unit. The full method achieves success rates of 100%, 100%, 90%, and 75% with the highest path efficiency on every task, and raises the success rate on the air-conditioning cover from 45% to 75% relative to a policy trained without the belief. These results indicate that force-driven reinforcement learning can enable disassembly without product-specific models or demonstrations.
comment: 14 pages, 6 figures
★ SaiVLA-JEPA: Evaluating Future Latent Prediction in JEPA-Conditioned VLA Control
Joint-embedding predictive architectures (JEPAs) learn to predict representations rather than pixels, offering a compact way to model how visual scenes evolve. We ask whether this predictive objective improves closed-loop control when a vision-language-action (VLA) policy already uses V-JEPA features. We call our JEPA-conditioned flow policy family SaiVLA-JEPA. It combines a vision-language model with a diffusion transformer action expert and receives features from a frozen V-JEPA encoder. Two lightweight auxiliary heads predict future representations during training and are omitted at inference, leaving the action sampler unchanged. We compare policies without JEPA features, with current JEPA conditioning, and with additional future supervision on 10,030 paired LIBERO-Plus episodes. Current JEPA conditioning reaches about 82.5% success versus 71.6% without JEPA, while future supervision provides no detectable aggregate gain at the evaluated training budget. We separately evaluate SaiVLA-JEPA-TCA, which adapts latent context, and SaiVLA-JEPA-TPA, which adapts selected parameters. The context-retry gain is a system-level result under a larger attempt budget; parameter adaptation shows no consistent advantage over the frozen policy and target controls. These results motivate assessing latent-prediction objectives through paired closed-loop rollouts rather than latent loss alone.
★ OmniQuad: a Wheeled-Legged Hybrid Robot with Omnidirectional Wheels
Combining wheeled and legged locomotion within a single robotic platform offers the potential to harness the advantages of both systems. The development of such hybrid systems remains an active area of research. While legged robots exhibit remarkable adaptability for traversing uneven and rugged terrains, their relatively low speed limits their practicality in indoor environments. Conversely, wheeled robots demonstrate higher efficiency on flat surfaces but often encounter difficulties when navigating obstacles such as steps. This paper introduces OmniQuad, a novel hybrid robot that integrates mecanum wheels with legged locomotion to exploit the benefits of both modalities. Mecanum wheels provide omnidirectional mobility, enhancing the robot's manoeuvrability by enabling agile movement in confined spaces and the ability to maintain a consistent orientation during task execution. In this work, we first present the design of a custom wheeled-foot featuring mecanum wheels. Subsequently, we describe the complete robot assembly, comprising these wheeled feet and EM-Act actuators. Finally, we detail the control scheme developed to govern the OmniQuad and evaluate its performance through experimental trials conducted in both indoor and outdoor environments.
★ Skeleton-Guided Geometry-Aware Auto-Labeling for Robotic Grasping: From 4-DoF Learning to 6-DoF Execution
Robotic grasping relies on large manually annotated datasets, which are costly to create. We introduce a straight-skeleton-based algorithm for generating two-finger grasp keypoints and identifying regions suitable for stable, feasible grasps. Our automated pipeline produces consistent grasp annotations without human intervention. We also propose the Skeleton-based Generative Grasp Convolutional Neural Network (SkelGG-CNN), a 4-DoF architecture that learns geometric grasp features from skeleton-based guidance. To extend auto-labeling to 6-DoF grasping, we develop a generalized quasi-3D skeletonization algorithm that operates directly on point clouds. Using automatically generated 4-DoF labels, SkelGG-CNN achieves accuracies of 95.42%, 96.55%, and 97.1% on Jacquard V1, Jacquard V2, and Cornell, respectively. On the GraspNet novel split, 6-DoF auto-labeling achieves an average precision of 18.33, comparable to state-of-the-art model-based approaches. In simulation, the 6-DoF method achieves success rates of 98.4% on Dex-Net and 89.1% on EGAD!. Real-world experiments with a Delta parallel robot show success rates of 97.4% on single household objects and 95.0% in cluttered scenes, outperforming state-of-the-art methods without domain adaptation. The 4-DoF pipeline adapts across environments and produces annotations approaching human-level precision, enabling efficient training of lightweight models on small datasets. The generalized 6-DoF method provides scalable, geometry-aware annotations for cluttered 3D scenes.
comment: Published in The International Journal of Robotics Research (IJRR), OnlineFirst, September 2026
★ SpaceAct: Benchmarking 3D Spatial and Temporal Reasoning via Closed-Loop Action and Active Perception
To solve problems in 3D space rather than merely describe images, multimodal large language models (MLLMs) need 3D spatiotemporal reasoning: reconstructing 3D geometry from occluded 2D views, anticipating physical dynamics, and executing precise, closed-loop actions. Prior studies test passive VQA on static images or symbolic planning, requiring no execution or supplying privileged coordinates while ignoring collision, statics, and timing. We introduce SpaceAct, an interactive benchmark evaluating 3D spatiotemporal reasoning via closed-loop physical action and active perception, with SAPA, a Spatiotemporal Active-Probing Agent framework. Built on interlocking bricks and mechanical puzzles, it has 108 verified instances across 36 task families in six categories, without privileged coordinates, and diagnostic modes (+plan, +state, no-camera, blind, L7 levers). In brick-bridging or aperture-threading, models rotate the camera to resolve occlusion, read in-scene rulers, output 6-DoF poses under swept-collision and statics checks, and self-correct via split-screen rollback feedback (dynamic tasks freeze time during deliberation and give five strobe frames). Across 10 frontier MLLMs on the 44 hardest instances, scores span 4.8-79.1 (seven solve at most 27.3% of 6-DoF tasks); for the top three, a reference plan (+plan) closes 44% of the gap while leaving collisions at 5.2 per episode, whereas adding exact 3D state (+state) closes 90% and cuts collisions to 0.5; for four others, +plan lifts the mean score from 10.9 to 35.3 and +state to 81.8; and 46% of failures are collision or observation loops. SpaceAct and SAPA advance evaluation to closed-loop physical interaction, showing that even with a valid plan, MLLMs remain bottlenecked by 3D visual grounding and action refinement, while offering a testbed for active multi-view exploration and explicit 3D grounding.
★ When "No Worries" Went Wrong: HRI Lessons for Foundation-Model Robots
When a person declined our robot's invitation to play, Alice turned her shoulder away and said "No worries, it's all good." Only after a brief pause did she drive off. The words and gestures were plausible but their order changed the message. In another episode, Alice informed a host that his visitor had arrived---while the visitor stood right beside him. Both came from a three-year ROS 2 HRI consortium that integrated large language and vision models as they emerged. Today's foundation models give robots broader capabilities to perceive their surroundings, reason about situations and generate behavior. Yet capability alone does not ensure that a changed situation shapes the action that reaches a person. Evidence may never be acquired, or a purpose or constraint may fail to survive planning and execution. A system may even report Success when an action no longer serves its intended purpose. Drawing on the consortium's engineering experience, we develop five design lessons about attention, knowledge, commitments, embodied timing and action authority. We then introduce TRACE, a lightweight review that changes one interaction premise, follows it to a decision or enforcement boundary, and produces an actionable architecture finding. A bounded analysis of a published LLM-generated behavior-tree system illustrates how an available clarification mechanism can go unused. The engineering lesson is to preserve accountability boundaries, not the computational pipelines that once exposed them.
comment: 8 pages, 5 figures, 1 table. Submitted to IEEE Robotics & Automation Magazine
★ ELEVATE: Agentic Controller Acquisition from Task Failure for Quadrupedal Manipulation
Quadrupedal manipulators can extend their workspace through whole-body interaction, but task-planning agents' capabilities remain limited by their available controllers. We present ELEVATE, a framework that uses persistent failures during task-program refinement to identify and acquire missing whole-body control capabilities. When these failures suggest a missing physical capability, a coding agent formulates and iteratively refines a reinforcement-learning problem, specifying behavior-specific rewards, curricula, and termination conditions. Validated controllers are added to the capability library and composed with existing controllers to retry the task. Starting from a single pretrained loco-manipulation controller, ELEVATE autonomously acquires controllers for wall-supported elevation, supported manipulation, and wall descent across a sequence of seven simulation tasks. These controllers are reused in subsequent reaching and object-manipulation tasks. On the five tasks beyond the initial controller's workspace, ELEVATE achieves mean success rates of 98.4%-100% across five independent runs, while coding-agent baselines with a fixed controller library fail to complete these tasks. The synthesized task programs, together with the acquired controllers, also transfer to a physical quadrupedal manipulator without real-robot training or fine-tuning, achieving 18/20 successes in rack retrieval and 16/20 in floor-to-rack placement. Videos and demonstrations are available at https://agentic-quad-elevate.github.io/public/.
comment: 23 pages, 14 figures, including appendix. Project page and videos: https://agentic-quad-elevate.github.io/public/
★ CSO: Refining Robotic Policies via Skill Distribution Alignment and Skill-Grained Optimization NeurIPS 2026
Discretizing continuous actions into discrete skills using methods like VQ-VAE has emerged as a powerful paradigm for robotic manipulation. However, quantization errors yield a suboptimal distribution for the skill prior, degrading its performance. While reinforcement learning offers a path for refinement, its direct application faces unstable encoder updates and a granularity dilemma in importance sampling. To address these challenges, we introduce Cascaded Skills Optimization (CSO), a two-stage post-training framework. First, Rejection-Sampling Supervised Fine-tuning aligns the model's observation-to-skill mapping with the distribution of successful online trajectories via supervised fine-tuning. Second, with the observation encoder frozen, Skills Policy Optimization computes an independent, clipped importance ratio for each complete skill, enabling stable and efficient updates. With a lightweight QueST-style policy, CSO raises overall LIBERO success from 80.1% to 97.5%, improves performance on RoboTwin2.0, and achieves 61.0% success across five physical single-arm tasks. Additional comparisons with FAST, discrete BEAST, and fixed-buffer IQL further validate the effectiveness of our two-stage post-training strategy.
comment: Accepted at NeurIPS 2026. 31 pages, including appendices and checklist
★ Proprioceptive Finger Force Estimation for Reliable Box Manipulation
Reliable box manipulation requires successful placement without damage, which depends on maintaining sufficient contact force without excessive loading. We hypothesize that per-finger force feedback improves manipulation reliability relative to open-loop finger commands. To this end, we present Finger External-Force Estimation from Proprioceptive SignaLs (FEEL), an uncertainty-aware residual method that estimates per-finger external force magnitude from proprioceptive histories. Our method combines a calibrated virtual-spring estimate with a residual distribution predicted by a Mixture Density Network (MDN) and requires no dedicated contact sensors during deployment. The estimated force is used for continuous per-finger adaptation, while predictive uncertainty is evaluated as an empirical signal of ambiguity in proprioceptive force estimation. Across ten fingers and contact forces spanning 0-20 N, our method achieves a mean RMSE of 0.251 N, reducing the error by 92.8% relative to the virtual spring and by 9.4% relative to an end-to-end MDN. In box pick-and-place, force adaptation maintains task success at 10/10 and increases both damage-free outcomes and reliability from 7/10 to 10/10. The project page is available at https://finger-force-estimation.pages.dev/
comment: Project page: https://finger-force-estimation.pages.dev/
★ What Should a Navigator Remember? Spatiotemporal Memory for Zero-Shot Vision-Language Navigation
Zero-shot vision-and-language navigation (VLN) requires an agent to use past observations without navigation-specific training. We study what information an 8B-scale navigation system should retain and how it should be organized for subsequent decisions. Our agent combines instruction progress, hierarchically compressed observation history anchored to poses, and accumulated geometry in complementary task, temporal, and spatial memories. An 8B observer summarizes observations, and an 8B reasoner uses the memories to select waypoint intents or request additional views, a geometric controller executes the selected targets. A semantic information perspective motivates the distinction between retaining evidence, making it accessible to the reasoner, and acquiring further observations. On R2R-CE val-unseen, the complete system achieves the highest among the evaluated Qwen3-based zero-shot baselines. Removing temporal memory reduces SR and sharply increases average path length from 16.9 to 39.1m, removing the spatial-map and planning component reduces SR as well. Real-world experiments on a Unitree Go2 show that our zero-shot method achieves higher success rates than the post-trained baseline in both corridor and office environments, suggesting better transfer to the evaluated real-world settings. As a complementary evaluation, LAW2 uses paired instruction interventions to examine behavioral responses, instruction following ability beyond endpoint success. These results support preserving task progress, observation history, and geometry in structured memory for zero-shot navigation with smaller open models.
comment: 20 pages, 3 figures, 8 Tables
★ Compositional Diffusion Planning with Chunk Selection and Reachability Measure
Compositional diffusion planners have demonstrated remarkable success on long-horizon diffusion planning by sampling short trajectories, namely chunks, and composing them into a long trajectory. However, averaging overlapping chunks can induce the mode-averaging problem, resulting in discontinuous trajectories. Several works aim to address this problem, whereas a non-negligible portion of plans remains infeasible. In this work, we propose Selective Compositional Diffuser (SCD), handling this problem by measuring reachability. To this end, we present a self-supervised learning method for the reachability classifier. This is trained on offline data and generated chunks jointly to address potential distributional mismatch, while reducing the bias inherited from the generated data. Based on this, we construct a graph whose edge costs represent whether consecutive chunks are reachable, and traverse the graph to find effective yet efficient routes. Lastly, we generate bridging trajectories and refine the composed trajectory for smooth transitions. We evaluate our method on OGBench stitching tasks. It practically reduces incompatible mode compositions and the mode-averaging problem compared to the studied baselines. Across tasks, SCD improves success rates, reaching state-of-the-art results on HumanoidMaze-Large and Giant tasks.
★ Recursive Self-Improvement in Physical AI: Mechanisms, Evidence, and Evaluation
Embodied systems can update policies, world models, skills, and executable programs from deployment experience, allowing one interaction to benefit later tasks. When a system also revises and reuses its data-acquisition, feedback, or learning procedures, experience changes the process that produces capability, introducing recursive self-improvement. This paper surveys research in physical AI through the connection between persistent capability and its improvement process. We first explain the functional distinction between learner updates and improvers, then compare policy learning, imagined experience, memory and skill accumulation, and executable harnesses. These approaches connect to revisions of curricula, rewards, and research programs in different ways. Contact validity, recovery, retention, and resource costs determine whether the resulting gains accumulate during continued deployment. An L0-L3 rubric distinguishes task-local adaptation, persistent capability gain, inherited process revision, and verified improver gain. Representative benchmarks and within-study contrasts relate these distinctions to existing results. We further propose common-data and interactive comparisons that evaluate reference and revised improvers from shared learner starts, separating the use of experience from its acquisition. This perspective connects the diverse methods, physical operating conditions, and evaluation evidence, underpinning embodied RSI. A comprehensive list of studies in this survey is available at https://github.com/nkd-lkz/physical-ai-rsi-survey.
★ MATCH: Execution-Aware Hierarchical Reinforcement Learning for Vision-and-Language Navigation in Continuous Environment
Vision-and-Language Navigation in Continuous Environments (VLN-CE) requires instruction-consistent planning and reliable execution under collision constraints. Yet many hierarchical approaches optimize the planner while keeping execution fixed. We introduce MATCH, a framework that jointly learns a language-conditioned planner and a goal-conditioned primitive-action executor. An execution-aware topological graph represents start-dependent execution options and integrates executor value estimates with realized outcome memory, enabling the planner to select execution requests and refine their targets through bounded continuous offsets. A transition-aware dense macro reward balances ordered progress along the reference route against execution residual, encouraging executable requests without substituting local success for instruction following. A staged curriculum culminates in synchronized on-policy updates, with the executor learning on planner-generated requests and the planner adapting its requests to the executor's current behavior. With 0.5B parameters, MATCH achieves state-of-the-art performance on the R2R-CE and RxR-CE benchmarks. Preliminary real-robot deployments further demonstrate its practical potential.
★ SimRetract: Zero-Shot Sim-to-Real Robotic Assistance for Retractor Take-Over in Open Surgery
We demonstrate the first zero-shot sim-to-real transfer for a surgeon-robot collaboration task in open surgery: retractor take-over. A robot arm takes a retractor from the surgeon's hand and preserves retraction at handoff, using a policy trained only on simulated demonstrations, without human demonstrations or real-world fine-tuning. Imitation learning from real demonstrations is limited by collection and labeling costs, and by performance degradation when the environment changes. Our framework, SimRetract, generates simulated demonstrations designed for transfer: the policy observes masked instrument images and grasp geometry, learns to recover from off-nominal states, and acts with margin for execution drift. Deployed zero-shot, it grasps the retractor in 90% of 50 surgeon-defined phantom presentations and preserves retraction at handoff in 78%. Results on unseen retractors suggest potential for generalization to instruments with similar grasp geometry. Across ten policy configurations and 1,200 simulated and real trials, we localize the sim-to-real gap to components: in simulation, grasp rates across ablations span 80-96%, while on the real robot removing the instrument mask alone drops performance to 22% grasped and 12% held. Transfer-oriented design thus makes zero-shot sim-to-real transfer feasible for a surgical assistance task without human demonstrations. Code is available here: https://anonymous.4open.science/r/simretract-58D2.
★ HERON: Real-Time Obstacle-Aware Reachability Maps for Mobile Manipulators
Mobile manipulator robots rarely reposition their base during manipulation, because doing so requires real-time knowledge of how well the arm can reach its surroundings among obstacles. Reachability maps (RMs) encode this knowledge, but existing methods are slow, coarse (typically 5 cm), or ignore real obstacles. This article asks whether a neural network can predict an obstacle-aware RM at 2 cm resolution from a live depth camera, on embedded hardware, fast enough for online use. We present HERON, a 3D U-Net trained on RMs computed with the GPU-accelerated inverse kinematics of cuRobo, first on synthetic scenes and then on real depth data, and deployed on a Jetson AGX Orin with TensorRT and INT8 quantization. HERON predicts the RM of a 2.56 m cube around the arm in 115 ms on average, for a measured system rate of 8.57 Hz, close to the 10 Hz update rate of the obstacle map, whereas computing the same map with cuRobo takes about 90 min. Against the cuRobo ground truth, 95% of the voxels lie within 1 class out of 51 with the FP32 model and within 3 classes with the INT8 model, with a Pearson correlation of 0.99 and a quadratic weighted kappa above 0.9. Accuracy degrades mainly at the workspace boundaries.
comment: 19 pages, 5 figures, preprint
★ CuraEval: Evaluating Demonstration Curation on Mixed-Skill Robot Data
Robot imitation learning uses demonstrations from operators of varying skill, but selecting training data remains challenging. Existing methods score demonstrations using smoothness, diversity, or estimated influence on policy performance, but differences in selection and evaluation protocols make it difficult to determine whether they outperform simple baselines. We introduce \textbf{CuraEval}, a framework combining mathematical analysis of demonstration scores with controlled comparisons against fixed size-matched random subsets and full-data training on mixed-skill robomimic data using Diffusion Policy and BC-RNN. We show that the evaluated PSD score depends only on episode length and squared position magnitudes, and produces nearly the same ranking as episode length on \textit{square} ($ρ=0.999$). With Diffusion Policy, selecting the shortest 50\% of demonstrations improves \textit{square} success from 70.7\% under random selection to 76.5\%. Compared with random selection, our adapted geometric-entropy LOO selector reduces success at 25\% retention from 94.7\% to 74.0\% on \textit{can} and achieves 41.7\% on \textit{square}, below the mean success rates of all three independently sampled random subsets (59.8--62.5\%). On the same 3 \textit{square} Diffusion Policy training seeds, length-based selection at 75\% retention improves success over full-data training from 76.7\% to 86.7\% under the original protocol, but lowers it from 80.5\% to 79.5\% after checkpoint reporting, rollout count, and evaluation seed change jointly. No evaluated curated condition significantly outperforms full-data training after Holm correction, although confidence intervals still permit useful gains. Code is available at https://github.com/jiuyixu25/CuraEval.
comment: Accepted by the CoRL 2026 Workshop "Oops, I Erred". 10 pages, 2 figures, 2 tables
★ Embodied at Sea: Imitation Learning for Multi-Task Underwater Manipulation
Behavior Cloning has demonstrated promising generalization capabilities for robotic manipulation in terrestrial environments. However, marine visual distortions and dynamic currents create a severe domain shift that remains a key bottleneck in adapting generalist architectures to underwater field manipulation. To address this, 1616 episodes of underwater manipulation data were collected in calm and turbulent waters and were used to train 14 generalist Vision-Language-Action (VLA) models (DynamicVLA) with varying proprioception observation configurations. Across three tasks on 1260 gradable on-hardware rollouts in a pool, 10 models outperformed the baseline, with the highest-performing model achieving a 93.3% overall success rate, including partial successes. Results are also introduced from a case study evaluating the performance of a single-task Action Chunking Transformer (ACT) and a generalist VLA (SmolVLA) policy for two tasks in marine archaeology and conservation: grasping and mooring. Experiments are performed on an ROV on and around the WWII wreck Coralita X127 and at Manoel Island Marina, Malta. Across 748 collected episodes and 508 gradable on-hardware rollouts, the ACT policy substantially outperformed SmolVLA (47.9% vs. 7.9% pooled success rate), earning a 66.7% success rate on the most successful task at the marine archaeological site. To enable further contributions to embodied intelligence in marine field robotics, the BLUEMBODIED KIT used to collect data, train models, and deploy policies on the ROV has been open sourced.
comment: 8 pages, 8 figures. For associated files, see https://lair-research-nyu.github.io/Embodied-at-Sea/
★ Minimum Sufficient Communication for Independent Behaviour Tree Robot Teams
We present an experimental framework for determining sufficient communication in robot teams running fixed, independent Behaviour Trees with private blackboards. Within a declared policy family, we vary message content and admission frequency and measure traffic in recipient-weighted bits per robot decision. A dense sweep identifies candidate policies for comparison with full-state communication. We retest selected candidates and the reference on fresh, paired scenarios, using one-sided simultaneous confidence bounds to assess whether success remains within five percentage points of the full-state reference. The bounds also assess whether tested cheaper policies can be ruled out as insufficient. Depending on the evidence, the framework reports an exact minimum, a positive bracket, a sufficient upper bound, or an unresolved boundary. Experiments with eight content policies across four warehouse conditions identify zero communication as the exact minimum under low contention. In the scaled condition, the no-release policy at gate $q=0.992222$ is confirmed sufficient, achieving 96.33% success at an estimated rate of 738.55 bits per robot decision, or 0.544% of full-state traffic. The positive minimum remains unresolved. These claims apply only to the declared policy family, sampled gate values and operating conditions.
comment: 10 pages; accepted for presentation at the Australasian Conference on Robotics and Automation (ACRA 2026)
★ SKiD-SLAM: Resource-Aware Distributed Multi-Robot LiDAR SLAM for Planetary Missions with Field Deployments at Analog Sites
Future planetary exploration missions require teams of robots to collaboratively map unknown environments under severe communication limits, constrained onboard computation, and no central infrastructure. In this field report, we present SKiD-SLAM, a distributed multi-robot LiDAR simultaneous localization and mapping (SLAM) system for such environments, and report its deployment across two mission concepts: cave exploration and scientific surface exploration. We survey the communication and computation budgets of actual planetary missions and select modules that fit these budgets for inter-robot place recognition, relative pose estimation, and distributed trajectory fusion. Before field deployment, we verify the assembled system on public datasets resembling planetary environments and in a simulator built on NASA JPL's Open Source Rover. We then validate SKiD-SLAM through field deployments in South Korea: a team of wheeled rovers explores the Majang lava-tube cave on Jeju Island, and quadruped robots carry out a crater-and-rock science scenario on a planetary analog terrain at the Korea Institute of Civil Engineering and Building Technology (KICT) in Yeoncheon. Across both deployments, SKiD-SLAM reliably performed distributed multi-robot LiDAR SLAM within a 250 kbps inter-robot communication budget, demonstrating its feasibility under challenging field conditions. Supplementary materials, including code and field data, are available at https://sparolab.github.io/research/skid_slam/.
comment: 31 pages. Published in IEEE Transactions on Field Robotics. Code and field data: https://sparolab.github.io/research/skid_slam/
★ Minimum-Time Escape from the Conic Field of View of a Reeds--Shepp Pursuer: the Faster Evader
We continue the study of minimum-time escape of an omnidirectional evader from the forward conic field of view of a Reeds--Shepp pursuer, now for a faster evader, and solve the game for every speed ratio $ν>1$ and every half-angle $0<α<π/2$: the escape time is characterized by explicit characteristics and the selection of roots of a scalar equation, and it is the value of the game in nonanticipative strategies at every state. Above the end $E$ of the evader's dispersal line on the front axis, the evader chases the reversing pursuer along a universal line fed by tributaries. The characteristic from the boundary of the usable part is an involute of a circle of radius $ν$ ending at a cusp, and two explicit thresholds decide whether it reaches the axis before the cusp and whether the jump of the escape time across it ends before the cusp or runs to it; in the latter case primaries fold back next to the barrier. The pursuer's optimal strategy is the relay of the slower case. The evader follows the tributaries and switches once to a primary, or commits in advance to a heading certified by the scalar equation of the primaries. Level-set and semi-Lagrangian computations are compared with the value.
★ Same Label, Different Interface: Beyond Action-Space Labels in Robot Learning
Robot-learning studies often compare position and velocity actions as though each label identified a single experimental treatment, yet the label alone does not determine what the robot executes. Before affecting robot motion, bounded policy output passes through filtering, physical scaling, reference construction, feedback, timing, and effort limits. We study how these omitted properties affect attribution under one shared learner and training budget. Across three contact-rich Factory tasks (PegInsert, GearMesh, and NutThread), we compare five joint-space interfaces, with ten independently trained policies per condition. We first remain within the same position-feedback law and vary the executed interface: substituting only the physical action scale as the directly modified component changes sample-median evaluation success by 0.645-1.000 across tasks, while removing an exponential moving average produces task-dependent changes. We then move to a cross-law comparison and choose a velocity scale that exactly matches the position interface's local raw-torque command gain. This calibration raises the sample median on GearMesh and NutThread, with no median change on PegInsert. Yet clear residual sample-median gaps remain on PegInsert and GearMesh, while the NutThread interval includes zero. Thus, even this explicit equality leaves feedback and held-action dynamics unmatched and does not establish position-velocity trained-outcome equivalence. Action-interface comparisons should therefore report the executed pathway and state the physical quantity they match.
♻ ★ OWLS: Open-world Language-conditioned Speed Fields for Zero-Shot Rover Navigation
Semantics can provide a proxy for physical traversability. Terrain that looks like loose sand or a rock field is likely to have similar physical properties. Open-vocabulary vision-language models (VLMs) enable this physical reasoning over terrain outside any fixed label set. We present Open-world Language-conditioned Speed Fields (OWLS), which plans directly over the dense feature map of a VLM. Prompted with a description of traversable terrain, the VLM scores every pixel of a single camera image; OWLS projects these scores onto the ground, adapts them to the platform with two morphological operations, and uses the result as the speed field of a minimum-time planner, so terrain appearance and obstacle clearance shape one trajectory. OWLS needs no task-specific training, no terrain annotation, no persistent map, and no exteroceptive sensor beyond the camera. We evaluate it on a physical rover indoors and off-road, on simulated Mars and Moon rovers over 200 procedurally generated terrains, and on real planetary imagery. Project page: https://chahyon-ku.github.io/owls/
comment: 8 pages. 6 figures
♻ ★ FlowS: One-Step Motion Prediction via Local Transport Conditioning
Generative motion prediction must satisfy three simultaneous requirements for real-world autonomy: high accuracy, diverse multimodal futures, and strictly bounded latency. Diffusion models meet the first two but violate the third, requiring tens to hundreds of denoising steps. We identify a conditioning strategy that resolves this tension: \textit{single-step integration is accurate when the underlying transport problem is local}. A model that must both discover the correct behavioral mode and traverse a long displacement in one step accumulates large discretization errors; conditioning the base distribution to lie near plausible futures reduces the problem to short-range refinement, the regime where a single Euler step suffices. We instantiate this \emph{local transport conditioning} in FlowS, a conditional flow matching framework with two mechanisms. First, an online, scene-conditioned learned prior emits $K$ calibrated anchor trajectories per agent, each already near a plausible future, converting mode discovery into local correction. Second, a step-consistent displacement field enforces semigroup self-consistency, guaranteeing that a single step inherits multi-step accuracy. Crucially, anchoring this field at learned priors along straight-line paths yields a {stable, low-variance} training target, unlike prior self-consistency methods that suffer from {high-variance bootstrap} signals on curved diffusion paths. On the Waymo Open Motion Dataset, FlowS achieves state-of-the-art Soft mAP {(0.4804) and mAP (0.4703) with ensemble at 75\,FPS} with single-step inference, demonstrating that local transport conditioning makes one-step generative motion prediction practical for safety-critical autonomy. Code and pretrained models will be released upon acceptance.
comment: 8 pages
♻ ★ Indirect Optimal Control for Constructing Libraries of Periodic Trajectories: Application to Bipedal Walking
Trajectory optimization is a fundamental tool for generating efficient, dynamically consistent periodic motions, commonly referred to as gaits, in legged locomotion. This paper explores the indirect method of periodic trajectory optimization, emphasizing its application in creating optimal gaits for legged systems and contrasting it with the more common direct method. While the direct method offers implementation flexibility, it is limited by its need for an input space parameterization. In contrast, the indirect method achieves higher accuracy by computing the control input from states and costates along the optimal trajectory. In this work, we tackle the convergence challenges associated with indirect shooting methods by utilizing numerical continuation methods. This is particularly useful for the systematic development of gait libraries. Our contributions include: (1) the formalization of a general periodic trajectory optimization problem that extends existing first-order necessary conditions to a broader range of cost functions and operating conditions; (2) a methodology for efficiently generating optimal gait libraries using single shooting and numerical continuation; (3) a novel approach for reconstructing Lagrange multipliers and costates from passive gaits; (4) a comparative analysis of the indirect and direct shooting methods using a compass-gait walker as a case study, demonstrating the higher accuracy of the indirect method; and (5) demonstrating applicability to the more complex legged robot RABBIT, with ten dynamic states and four inputs. These results establish a systematic indirect optimal control framework for synthesizing families of optimal periodic trajectories in legged locomotion.
comment: submitted to Automatica, the Journal of the International Federation of Automatic Control (IFAC)
♻ ★ Safe and Fuel-Efficient Motion Planning for Lunar Propulsive Drones
Propulsive drones are a promising new technology for generating high-resolution terrain models of the Moon. In future lunar missions, propulsive drones will complete a series of hops across the lunar surface while capturing images with downward-facing cameras. We present an autonomous motion planning framework that uses a vision-guided optimal control strategy for performing safe, efficient hops between specified waypoints. The drone plans parabolic hops between waypoints generated by a global-level planner; if a nominal waypoint is unsafe or unreachable, the drone then completes a diversion maneuver to safely land. To evaluate the validity of potential landing sites, we develop a custom semantic segmentation model for identifying obstacles from aerial images, and we introduce an approach for rapidly calculating reachable terrain. We then use model predictive control (MPC) to robustly track hop trajectories that guide the drone towards safe landing sites. We test the optimal hop planner in a high-fidelity lunar simulation environment. Our results demonstrate that our unified motion planning framework yields safe, fuel-efficient hop trajectories which can be tracked rapidly and with low error via MPC.
comment: This work has been submitted to the 2027 IEEE Aerospace Conference
♻ ★ POD-RoPE: Rotary Position Encoding over Physical Observation Distributions for Embodied AI
Embodied perception requires positional representations that capture both physical geometry and the spatial and temporal extent of observations. We introduce POD-RoPE (Physical Observation Distribution RoPE), which represents token observation footprints as distributions over camera centers, viewing directions, and sampling times. Under independent sampling, expected rotations encode relative relationships between distributions; when each distribution collapses to a single address, the construction recovers pointwise RoPE. The same rotation statistics support Q/K matching and coherence-dependent V/O transformations, allowing observation geometry to guide both information selection and content aggregation. Per-token pre-aggregation avoids increasing sequence length or processing pairs of observation nodes. We evaluate POD-RoPE across four robot policy architectures, simulation and real-world manipulation, and complementary reconstruction, depth-estimation, and novel-view-synthesis tasks. Integrating POD-RoPE into Fast-WAM improves mean success from 91.83% to 92.88% across all 50 RoboTwin 2.0 tasks, and from 32.5% to 52.5% on two real-robot tasks on Sharpa North. Controlled reconstruction ablations support the contributions of distribution encoding and V/O transformations. In H20 model-block benchmarks, POD-RoPE achieves median forward-plus-backward speedups of $1.91\times$ over P-RoPE and $5.34\times$ over RayRoPE, with a median latency overhead of 17.4% over Standard RoPE. These results support physical observation distributions as an effective and computationally efficient basis for geometry-aware attention.
comment: 21 pages, 5 figures
♻ ★ Understanding Temporal Semantic Stability in Open-Vocabulary UAV Perception through Metric 3D Fusion
Recent open-vocabulary segmentation models have significantly advanced semantic perception for UAVs, for aerial and remote-sensing imagery, yet predictions from moving platforms can remain temporally inconsistent across repeated observations of the same physical scene. While semantic segmentation is typically evaluated on single images, temporal reliability remains less explored despite its importance for long-horizon robotic perception. In this work, we investigate temporal semantic stability by associating frame-wise predictions with persistent world-space locations through metric 3D fusion. We introduce a voxel-level evaluation framework that jointly characterises final semantic agreement, Semantic Belief Drift (SBD), Observation Persistence (OP), and semantic uncertainty from accumulated evidence. Experiments on UAVid-3D-Scenes reveal substantial frame-wise semantic variation and show that high aggregate world-space agreement can overstate temporal stability when many locations have limited repeated-observation support. Persistence-stratified analysis shows that recurrent voxels expose substantially greater semantic disagreement, while higher-persistence voxels exhibit lower direct consecutive semantic-evidence divergence. This behaviour is observed across two segmentation backbones and remains consistent under variations in voxel resolution, geometric association, and temporal sampling density. Overall, conditions that reduce world-space recurrence can increase apparent aggregate stability, demonstrating that semantic consistency must be interpreted together with observation support. Our findings establish observation persistence as an essential conditioning variable for evaluating long-horizon semantic reliability and provide a framework for analysing temporal semantic behaviour beyond image-level prediction.
comment: Accepted at the 17th Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP 2026)
♻ ★ Who Is the Agent Talking To? Participant-Aware Role Switching for Embodied Student-Support Dialogue
We present RPO-PDT, an embodied conversational agent that adapts its conversational role to the current participant. It acts as a Per- sonal Development Tutor (PDT) when supporting students and as a simulated student when faculty rehearse tutoring strategies. Voluntary closed-set participant verification informs a confidence- controlled role manager: stable enrolled-faculty matches activate the Student role, while enrolled students and unmatched partici- pants use the default PDT role and uncertain observations cause no transition. The system combines Furhat interaction, Rasa or- chestration, institutional retrieval, safeguarding, and LLM-based generation. User control is preserved through manual role override, consent-based facial enrolment, temporary interaction identifiers, and explicit retention or deletion of enrolled profiles. The demon- stration exposes both role-switching behaviour and these privacy controls in interaction.
comment: 3 pages, 2 figures
♻ ★ In-Context Learning World Models for Robot Dual-Arm Manipulation of Soft Garment with Partial Observations
Robust manipulation of soft objects, such as garments, is a critical skill for service and household robots to adopt, but remains challenging due to its high-dimensional state space, and material-dependent deformations. Humans address this challenge by using mental world models that "imagine" the shape and material properties from partial observations, which enables them to predict how actions affect future states. As such, embedding such world models in robots allows for better generalization in manipulating a wide variety of garments in the real world. However, learned world models typically require large amounts of real world data that is strenuous to collect, and a mechanism to infer latent properties that govern dynamics. We present a Soft-Object World Model that performs generative full-state estimation and action-conditioned dynamics prediction using a transformer backbone trained on large-scale simulated soft object interactions, with measures to mitigate the sim-to-real gap. By learning to infer latent shape and material cues through in-context learning over a variable-length partial point-cloud history, the model generalizes across robot embodiments, material parameters, garment types and shapes, and manipulation coordination modes without weight updates. The world model is then integrated with a Goal-Conditioned action planner for motion synthesis. We validate the approach in simulation and real world for bimanual garment folding, reporting improvements in reconstruction fidelity, prediction accuracy, and task success. Project Website: https://1am5hy.github.io/ICL soft object world model/
♻ ★ Ego-CEVR: Learning Cross-Embodiment Visual Representations from Human Egocentric Videos
As egocentric video becomes increasingly available for embodied learning, an important research question is how to turn this low-cost, scalable source of data into robot manipulation capabilities. Egocentric videos naturally provide rich context for high-level task understanding, while real-robot demonstrations with wrist observations and action annotations provide supervision for fine-grained action alignment. We present Ego-CEVR, a data-native framework for hierarchical visual representation learning that assigns distinct learning objectives to global and interaction representations. The global module learns object relations, task state, and temporal progress through future prediction on egocentric video. The Interaction module interprets wrist observations in global context and learns local alignment and contact cues through robot action supervision. Separate residual paths expose both representations to the action policy, preserving direct access to task context alongside fine-grained execution information. Ego-CEVR supports independent pretraining of its global video module on large-scale human egocentric videos and videos from diverse robot embodiments, without requiring robot action labels or paired wrist observations.
♻ ★ Whole-Body Motion Guided Reference Generation for Uncertainty-Aware Compliant Assistance
Exoskeletons have shown great potential for augmenting human physical capabilities, yet generating reliable assistance remains challenging because human motion patterns and interaction requirements vary substantially in practice. To address this problem, a dual-loop assistance strategy is proposed to couple multimodal exoskeleton trajectory generation with variable admittance control, where trajectory variance links the outer-loop generator and the inner-loop controller. The generator synthesizes future exoskeleton trajectories through denoising, primarily using whole-body human motion and exoskeleton proprioception, with task descriptions as auxiliary semantic cues. To reduce cross-modal representation mismatch, a three-stage training pipeline is introduced to progressively optimize the human motion encoder, the exoskeleton proprioception encoder, and the trajectory generator. In the inner loop, multiple future trajectories are sampled from the generator to estimate the trajectory mean and variance. The mean is used as the control reference, while the variance provides an uncertainty measure for prediction reliability. This measure is further incorporated into a stiffness regulation law, enabling compliant assistance with input-to-state stability under bounded lumped interaction input. Experimental results demonstrate that the proposed method significantly reduces muscle activation levels compared with standard baselines, while also decreasing users' metabolic energy expenditure relative to the unassisted condition.
comment: Withdrawal requested because not all co-authors had consented to the submission of this manuscript to arXiv
♻ ★ NTR: Neural Token Reconstruction for Scene Token Bottleneck in End-to-End Driving
Recent perception-free end-to-end (E2E) autonomous driving methods bypass explicit perception outputs by compressing dense image patch tokens into compact scene tokens for downstream trajectory generation and scoring. While these scene tokens form a compact visual bottleneck for the planner, they receive supervision solely from the planning objective, providing limited constraints on the encoded visual information. To address this limitation, we introduce Neural Token Reconstruction (NTR), a representation learning framework to directly constrain the compact scene-token bottleneck in perception-free driving. NTR introduces a self-distillation masked latent reconstruction objective that reconstructs masked patch-level latent features using only compact scene tokens as reconstruction memory. This forces reconstruction gradients to pass exclusively through the scene-token bottleneck, encouraging scene tokens to preserve richer and less redundant visual representations for planning. We further introduce semantic priors derived from foundation-model annotations as a weak semantic interface biasing reconstruction targets toward driving-related structures without introducing explicit perception heads. All auxiliary reconstruction components are removed at inference time, leaving the deployed planner unchanged. NTR achieves state-of-the-art performance on three public autonomous driving benchmarks, including 8.0461 RFS on Waymo E2E and 94.1 PDMS / 90.9 EPDMS on NavSim1&2. The learned scene tokens exhibit lower pairwise redundancy and higher effective rank, indicating that effective bottleneck supervision improves both compact visual representation learning and planning performance.
♻ ★ Retrieve in Time, Correct in Frequency
Frozen vision-language-action (VLA) policies generate temporally extended action chunks, but long-horizon manipulation remains vulnerable to accumulated execution error and visual aliasing across task stages. Successful rollouts provide useful corrective evidence, yet current frame retrieval can return progress-misaligned actions,while direct replay or time-domain fusion can overwrite the reactive structure of the policy proposal. We introduce Retrieve in Time, Correct in Frequency (RTCF), a training-free test-time correction framework that improves frozen VLA performance with low model-side overhead.RTCF separates which experience to retrieve from which part of its action to transfer. Progressive Memory Alignment (PMA) causally aligns the growing visual execution history with complete successful trajectories through incrementally updated monotonic frontiers, jointly identifying a relevant memory and the current aligned memory position without stage labels. From the aligned action chunk,RTCF transfers a coefficient-wise-clipped low-frequency residual on motion channels. Higher-frequency components and gripper decisions remain inherited from the frozen policy. Across four LIBERO suites and 2,000 episodes per condition, RTCF raises aggregate success from 86.4% to 88.4% and improves LIBERO-Long from 61.6% to 68.6%.These gains require no parameter updates, repeated VLA inference, or additional GPU resources: correction can be performed on the client CPU after a single policy invocation, and the median latencies sum to only 10.99 ms per action chunk
comment: The current version of this manuscript is preliminary, and the experimental evidence is not yet sufficient to fully support its main conclusions. Substantial additional experiments and validation are required to establish the reliability and generalizability of the reported findings. We therefore request withdrawal of the current version
♻ ★ Ego-OSCAR: Egocentric Open source Stereo CAptuRe System
Egocentric vision has scaled its corpora far faster than its instruments. Calibrated stereo rigidly coupled to a synchronized inertial stream is not new; research-grade headsets deliver it. What does not exist is an open version: those platforms are closed products a third party cannot build or extend, and the consumer cameras used elsewhere are monocular and rolling-shutter. The instrument is the least reproducible part of an otherwise open pipeline. We present our system, an open-hardware head-mounted capture platform, at a complete bill of materials under USD 200 in commodity parts and 3D prints. It pairs a hardware-synchronized global-shutter stereo camera with a 6-axis IMU, an embedded Linux SBC for on-device encoding, and a real-time microcontroller that bridges the camera and inertial clock domains to a residual of 700 microseconds and watchdogs the recording against silent failure. We release the full stack alongside the device: CAD, firmware, recording and time-synchronization tooling, and our corpus, roughly 550 hours of stereo video over 1,462 sessions and 25 contributors, every device shipping its own stereo calibration. The corpus ships with model-predicted annotation layers rather than raw video alone, including per-frame 3D hand reconstructions spanning essentially the whole timeline; these are automatic predictions, not human-verified ground truth, and we report them as such. We characterize the instrument along sensor fidelity, data utility, and six months of distributed field deployment. Our system is not the highest fidelity egocentric rig; it is the cheapest one a reader of this paper can reproduce, and the only corpus here that others can extend rather than only consume.
♻ ★ HGWM: Hierarchical Graph-Guided World Model for Zero-Shot Object Navigation via Graph Matching
Zero-shot object navigation requires an agent to reason about where a target may be found while retaining evidence from previous observations. We present HGWM, a hierarchical graph-guided world model that combines a category-conditioned room--anchor prior with persistent spatial-semantic memory. A coarse stage uses a vision-language model to score candidate headings, while a fine stage explicitly compares observed objects and relations with the goal prior. Phase-dependent fusion integrates these signals with a language-model recommendation to guide local navigation. HGWM achieves 62.6\% success rate (SR) and 31.5\% success weighted by path length (SPL) on HM3D v0.1, improving over WMNav by 4.5 and 0.3 percentage points, respectively. On MP3D, it achieves 46.8\% SR and 17.9\% SPL, with gains of 1.4 and 0.7 points. On HM3D v0.2, the full model reaches 74.4\% SR and 31.9\% SPL; removing individual components reduces SR by 0.9--2.3 points and SPL by 0.6--1.5 points. These results support the use of relational priors and persistent graph memory for target-directed search.
♻ ★ ActionUNet: Improving Robustness of VLA Models with Efficient Multi-scale Fine-tuning
Vision-Language-Action (VLA) models have shown great promise for robotic manipulation by mapping multi-modal semantics to physical actions. However, this mapping inherently struggles to align these coarse-grained semantics with fine-grained temporal execution. It leaves VLA models with limited generalization and insufficient robustness in cluttered environments. To overcome this issue, we propose ActionUNet, an efficient multi-scale fine-tuning framework that enhances pre-trained VLA models with minimal computational cost. ActionUNet first constructs a lightweight temporal U-Net within the temporal-aligned action feature space to fuse hierarchical structural priors, effectively bridging the scale gap between semantics and temporal executions. Recognizing that multi-scale modeling can disrupt microscopic temporal continuity and cause mechanical oscillations, ActionUNet then employs a conditional SIREN as a continuous action decoder. Equipped with explicit second-order smoothness constraints, this decoder guarantees temporal continuity and reduces high-frequency motion jitter. By smoothing temporal discontinuities from multi-scale fusion, this continuous formulation reduces mechanical execution failures while preserving the base VLA model's generalization and manipulation robustness. Extensive experiments on RoboTwin 2.0 and LIBERO-Plus benchmarks, together with real-world hard evaluations, demonstrate that ActionUNet significantly improves π0.5 success rates by absolute 9.8%, 6.1%, and 11.4%, respectively, while also generalizing to the regression-based OpenVLA-OFT backbone, highlighting its effectiveness and efficiency as a fine-tuning strategy. Code and implementation details are available at https://github.com/Di-Zhu123/ActionUNet.
♻ ★ Multimodal Progress-Residual Fusion for Anticipating Navigation Retreat in Robotic Flexible Endoscopy
Flexible robotic endoscopes provide access through tortuous anatomy, where friction, deformation, and wall interaction separate commanded insertion from useful distal progress. Operators interpret this mismatch and retreat to recover a navigable configuration. Recognizing the approach to that decision requires relating motor intent to distal response and the local visual scene. We introduce Multimodal Progress-Residual Fusion (MPRF), a causal model that anticipates operator retreat from commands, electromagnetic (EM) tracking, and endoscopic video. A nominal-progress model predicts the response expected in normal or successful navigation; a temporal predictor combines its EM residual with all three observed modalities. In a 150-episode colon-phantom collection, MPRF achieves 0.580 +/- 0.007 AUPRC across three seeds, compared with 0.511 for the strongest reduced-modality control. Complete multimodal histories provide the principal ranking advantage, while the residual adds an explicit intent-response representation with a small, uncertain increment over raw fusion. Analysis of frozen alarms relates warning timing to the behavioral label and explains the increase in event recall under a longer matching window. These results establish a basis for behavioral assistance during operator-controlled navigation.
♻ ★ Mixture of Frames Policy: Multi-Frame Action Denoising for Bimanual Mobile Manipulation
Robotic manipulation is inherently multi-frame: local actions may be simple in an end-effector frame, while transport, upright-object handling, and whole-body coordination are better represented in a base-aligned frame. However, modern diffusion-based visuomotor policies typically commit to a single predefined action frame, forcing one denoiser to model action distributions that are often unnecessarily complex in that frame. We propose Mixture of Frames Policy (MoF), a diffusion policy that performs synchronized action denoising across multiple coordinate frames. MoF maintains a single canonical diffusion state, re-expresses it in several task-relevant frames, applies frame-specialized denoisers, and fuses their noise predictions back in the canonical frame. To make this possible for intermediate noisy diffusion states, we adopt a column-based 6D rotation representation within an SE(3) action parameterization that supports exact, differentiable frame transformations without requiring noisy rotations to lie on the SO(3) manifold. Across nine simulated bimanual manipulation tasks, we show that the best action frame is task-dependent and that MoF improves over oracle frame selection and standard Mixture-of-Experts (MoE) baselines. We further evaluate MoF on two real-world bimanual mobile manipulation tasks, demonstrating that it outperforms all constituent single-frame baselines.
♻ ★ From Video to Vision: Learning Generalizable Humanoid Carrying from Human Motion
Most existing approaches rely on video or motion-capture data for reinforcement learning (RL) training; however, all of these methods require an upper-level module to generate reference trajectories in real time. In object carrying tasks, the humanoid can only lift objects from designated locations along the reference trajectory, resulting in poor spatial generalization. Moreover, existing policies can only be evaluated inside a motion-capture studio. To address these limitations, we propose \textbf{MOVER}, a multi-stage, Teacher-Assistant-Student, RL training framework for humanoid object carrying. We first collect data from videos and employ GENMO and GMR to retarget the video data, combining this with data augmentation techniques to generalize the data across objects of varying shapes and sizes. In the \textbf{first stage}, building upon the DeepMimic \cite{peng2018deepmimic} motion-training framework, we train the humanoid to transport objects from specified locations along designated trajectories. In the \textbf{second stage}, we relax the tracking of the reference motion and adjust the reward so that the humanoid learns to transport objects from different positions. In the \textbf{third stage}, we apply a Dagger-PPO distillation method to distill away the proprioceptive position observations and reference joint position of the humanoid. This allows the deployed policy to operate without relying on global position information from a motion-capture system; instead, the humanoid can directly perceive the object's pose from its onboard RGB-D camera to complete the carrying task. We design a series of simulation experiments to compare the success rates across stages (Stages 1 vs. 2) and before versus after distillation. We further transfer the policy to a real humanoid, successfully transporting objects at varying positions in a motion-capture-free setting.
♻ ★ CylindTrack: Depth-Aware Cylindrical Motion Modeling for Panoramic Multi-Object Tracking in Mobile Robots
Maintaining consistent target identities and observation histories is essential for mobile robot perception. Panoramic Multi-Object Tracking (MOT) benefits from broad visual coverage but must address coordinate discontinuities at the equirectangular image seam and fluctuations in relative-depth estimates. Such fluctuations can undermine depth-aware association, highlighting the need for temporally consistent depth cues. We propose CylindTrack, a depth-aware cylindrical tracking-by-detection framework that jointly models panoramic topology and temporal depth consistency. Depth-Temporal Trajectory Modeling (DTM) maintains instance depth and its rate of change as filtered trajectory states. Spherical Spatio-Temporal Consistency Learning (SSTC) combines a Temporal Mixer with Spherical Geometry-aware Attention to refine depth-aware representations. A Topology-Aware Cylindrical Motion Model (TCMM) represents horizontal motion in an unwrapped angular state and incorporates periodic overlap, angular consistency, and filtered depth cues into association. Experiments on QuadTrack and JRDB demonstrate improved identity consistency and trajectory continuity, with HOTA scores of 33.67 and 31.12 and IDF1 scores of 40.45 and 34.33, respectively. The framework processes 28.56 and 21.34 frames per second, respectively, on an NVIDIA GeForce RTX 3090 GPU under the reported evaluation settings. These results support persistent panoramic multi-object tracking for mobile robot perception. The source code will be released at https://github.com/warriordby/CylindTrack.
comment: The source code will be released at https://github.com/warriordby/CylindTrack
♻ ★ LocoManiLab: A Unified Testbed for Data, Policies, Controllers, and Deployment in Humanoid Loco-Manipulation
Humanoid robots must coordinate locomotion and manipulation in everyday environments, yet progress is difficult to assess across differing policies, controllers, and task settings. LocoManiLab provides an integrated testbed spanning data collection and generation, policy training, evaluation, and physical deployment. Its task suite covers 234 atomic and compositional tasks across five household scene categories, supported by 51.5 hours of simulated teleoperation. The evaluation covers twelve policies with a shared controller across task and scene splits and perturbation levels, two policies across five controllers, and studies of demonstration scale and data-source mixtures. Success rates fall on unseen tasks and scenes and generally decline as visual perturbations increase, with differences in generalization and robustness across policies. For data generation, a humanoid motion generation model is fine-tuned for loco-manipulation to study the alignment of its outputs with task execution requirements and its integration into the data generation pipeline. Three simulated loco-manipulation tasks closely match real-world setups, enabling evaluation of three policies in both domains. Similar policy rankings support the reliability of simulation-based evaluation.
♻ ★ BEHAVIOR-Spatial Benchmark: Evaluating Spatial Understanding for Robotics
Robot tasks require grounding spatial language in physical scenes, but task success alone cannot establish whether spatial instructions are followed. We present BEHAVIOR-Spatial, a benchmark built on BEHAVIOR-1K that measures spatial question answering, point-based target grounding, and robot action to examine how VLM spatial understanding relates to VLA behavior. It comprises 3,256 spatial questions across 49 activities, 1,240 grounding queries across 14 manipulation tasks, and action evaluations covering those 14 tasks and 10 navigation tasks. Our manipulation protocol evaluates held-out configurations and instruction following when the requested target changes within a fixed training layout. The baseline achieves 41.6% on held-out configurations but only 16.1% under changed instructions. VLM adaptation and robot-data pretraining provide limited improvements, whereas oracle visual target cues raise these rates to 60.3% and 38.0%, respectively. Perception evaluations reveal policy-dependent correlations between VLM question-answering and grounding accuracy and VLA manipulation performance. Together, these findings motivate evaluating spatial instruction following alongside task completion and examining when spatial perception performance translates into robot behavior. Project website: https://arielchen07.github.io/behavior-spatial/.
♻ ★ From Biological Curves to Soft Modular Robots: Learning-Based Inverse Design with Uncertainty-Guided Refinement
Animals exhibit diverse shape-changing behaviors, from the undulatory swimming of eels to the arching movements of inchworms. Reproducing these biological shapes with modular soft robots offers a promising approach to developing versatile bioinspired robotic systems. However, translating a desired shape into a feasible robotic configuration remains challenging. Existing cable-driven modular soft robots rely on computationally expensive physics-based modeling and combinatorial search for inverse design, making it difficult to efficiently reproduce diverse target shapes. To address this challenge, we present a learning-based real-to-sim-to-real framework that translates biological curves into robotic configurations through surrogate-assisted inverse design and uncertainty-guided refinement. The proposed method combines learned inverse prediction with targeted candidate exploration, using a fast forward surrogate to evaluate and select suitable designs. The learned forward surrogate reduces deformation prediction time from $1.53 \times 10^5$ ms to 0.0008 ms, achieving approximately a $1.9 \times 10^8$-fold speedup. At the same time, the proposed inverse design method reduces design time from $1.99 \times 10^6$ ms to 4.18 ms. On 5,000 test samples generated from previously unseen threading patterns, the method achieves a 93% shape reconstruction success rate under surrogate-based evaluation. Physical demonstrations with four bioinspired robots further illustrate the translation of biological curves into realizable robotic shapes.
comment: 8 pages, 8 figures, 2 tables
♻ ★ Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment
Finetuning a pretrained vision-language model (VLM) on robot demonstrations via behavior cloning (BC) has become the standard recipe for vision-language-action (VLA) policies. However, BC finetuning progressively overwrites the pretrained representations that support visual and semantic generalization. Co-training on web image-text data, a common remedy, applies language and action losses to separate observations, leaving VLAs with language-action misalignment that standard manipulation benchmarks do not expose. We propose Anchor-Align, which augments BC with two objectives: Vision-Language Anchoring distills layer-wise representations from a frozen VLM copy to prevent this drift, while Language-Action Alignment converts each action target into a discrete motion-direction label and jointly trains language and action prediction on the same robot observation. We conduct real-world evaluations across eight manipulation settings on single-arm xArm7 and bimanual YAM robots, using two VLA architectures with regression and flow-matching action heads. Across these settings, Anchor-Align consistently improves over BC on novel targets, layouts, and motion-sensitive bimanual tasks requiring coordinated control. At scale in simulation, we demonstrate consistent improvements on OOD perturbations, perceptual robustness, and long-horizon control across LIBERO-PRO, LIBERO-Plus, and CALVIN, respectively, suggesting that preserving pretrained representations and effective action learning are not fundamentally at odds. Project page: anchoralignvla.github.io
comment: Code: https://github.com/dwipddalal/Anchor-Align
♻ ★ RadLoc: Radar-based 3-DoF Global Localization via Fast, Robust, and Lightweight Spatial Descriptor Across Diverse Environmental Scenarios
While global localization using spinning radar has gained attention for its robustness to adverse weather and challenging environments, many studies have focused on individual components such as place recognition or pose estimation. In this paper, we take a holistic view of radar sensor-based global localization and present RadLoc, a fast, robust, and lightweight end-to-end pipeline from place recognition to 3-DoF pose estimation. RadLoc accelerates pre-processing using 1D CA-CFAR filtering and leverages the near-range dominance in spinning radar images to design a compact descriptor and an efficient hierarchical coarse-to-fine retrieval strategy. Moreover, coupled with phase correlation-based 3-DoF pose estimation, it forms a versatile global localization module applicable to SLAM and multi-session SLAM systems. Extensive experiments on 15 sequences across 5 datasets demonstrate that RadLoc achieves robust performance while maintaining the smallest descriptor size and fastest retrieval time among state-of-the-art approaches. The supplementary materials are available at https://sparolab.github.io/research/radloc/.
comment: 8pages, 12figures
♻ ★ What Happens to the Goal? Distinct Failures of Policy Alignment in Goal Misgeneralization
Goal misgeneralization is a common failure mode in which an agent behaves competently out of distribution but pursues a proxy that coincided with the intended goal during training instead of the goal itself. However, we still have a weak understanding of what happens to the intended goal in misgeneralizing agents and what enables them to be recovered without retraining. We study this across eight visual navigation and manipulation tasks, including robot control. While tracing where goal information remains available internally, how strongly it influences the policy, and what role it plays in action selection, we uncover that the goal is lost at one of three different depths. This depth predicts which recovery works in each task. When the goal is followed, but outcompeted by the proxy, steering the existing policy redirects it to the goal's location, raising goal reach from 16.0% to 95.3%. When it is read, but not followed, steering fails, but a lightweight new downstream wiring recovers goal reach from 31.6% to 84.3%. When it is not read, none of the methods work, as the policy does not use the goal information. The same tracing also reveals what replaces the goal: functional target substitution, where behavior follows an alternative goal, a policy shortcut, where behavior follows a learned action strategy, or cue competition between intended and proxy goal cues. Knowing where the goal is lost tells us whether an agent can be repaired or has to be retrained. Goal misgeneralization should therefore be studied and evaluated inside the agent, not only from its behavior.
comment: 10 pages main text, 122 pages total. Code: https://github.com/Pantheon-Industries-Inc/what-happens-to-the-goal
♻ ★ Introducing M: A Modular, Modifiable Social Robot
We present M, an open-source, low-cost tabletop social robot platform aimed at lowering barriers to social robotics research by making robots easier to reproduce, modify, and deploy in real-world settings. M combines a modular mechanical design, multimodal sensing, and expressive yet mechanically simple actuation architecture with a ROS2-native software package that cleanly separates perception, expression control, and data management. The platform includes a simulation environment with interface equivalence to hardware to support easy sim-to-real transfer of interaction behaviors. We demonstrate extensibility through additional sensing/actuation modules and provide example interaction templates for storytelling and two-way conversational coaching. Lastly, we report real-world use of M in interaction design and week-long in-home deployments, showing how M can serve as a practical platform for longitudinal, reproducible social robotics research.
Multiagent Systems
★ Tunable Spatial Task Allocation with Application to Maritime Swarms
In this chapter, we explore how a team of autonomous agents can flexibly reconfigure the distribution of its members across many spatially embedded tasks using a task allocation strategy grounded in a generalization of the recently proposed Nonlinear Opinion Dynamics model. Motivated by an adaptive maritime survey application, we couple opinion dynamics directly to spatial movement and examine how the emergent macroscopic allocation is shaped by the interplay between environmental geometry, nature of social interactions, and task demands. We illustrate in simulation how social coupling in the model tunes group cohesion, producing macroscopic allocation patterns ranging from maximally dispersed states, to intermediate semi-dispersed states with a smaller number of distinct subgroups, to maximally cohesive consensus states, in distinct and interpretable parameter regimes. Furthermore, we demonstrate with illustrative examples how a team can reconfigure itself in real time under changing task demands. For allocation scenarios with distance-based task preference and matched number of agents and tasks, we relate numerically the final allocation in the group to the Hungarian optimum, illustrating close-to-optimal performance. We illustrate performance of this method under realistic communication constraints in the MOOS-IvP-Multi simulation environment with simulated autonomous underwater vehicles (AUVs). Together, these examples show how a common set of local interaction rules can support a large range of interpretable macroscopic allocation states and flexible transitions between them.
comment: 29 pages, 10 figures
★ MAS-Loop: Beyond One-Shot Orchestration towards Self-Evolving Multi-Agent Systems
Training-time multi-agent system (MAS) design seeks to learn reusable orchestration policies, yet prevailing approaches remain largely one-shot: an Orchestrator generates a system but only observes and learns from its terminal outcome. This paradigm discards execution trajectories that expose system-level failures and contain fine-grained information beyond the terminal outcome, leaving no direct basis for revising the source MAS, evaluating the updated MAS, or learning from the resulting contrast. To this end, we introduce MAS-Loop, an execution-grounded framework that closes this feedback loop by coupling an Orchestrator with a trajectory-conditioned Refiner. For each task, the Orchestrator constructs and executes a source MAS to obtain an outcome score and an intermediate trajectory. Conditioned on the trajectory, the Refiner localizes an actionable failure, formulates a trajectory-grounded diagnosis, and proposes a bounded patch to the agents, information flow, or execution configuration. The executable updated MAS is re-executed on the same task to measure its realized outcome under stochastic execution. This pair-MAS comparison provides complementary learning signals: the Orchestrator learns to generate strong systems with fewer recoverable failures, whereas the Refiner learns to expose and repair observed weaknesses, thereby driving policy-level co-evolution across training iterations. Experiments show that MAS-Loop achieves the strongest overall performance among the evaluated baselines, with consistent gains from refinement across all evaluated benchmarks. The co-evolution process also yields MAS-JudgeCorpus, a dataset of execution-grounded source-updated pairs with orchestration- and trajectory-level supervision, which can provide the community with support for MAS reward modeling and trajectory-aware investigation.
★ Routed Continual Learning
Agents deployed at scale repeatedly encounter related problems, and it is natural to let them learn from each other's experience rather than starting over each time. Unfortunately, there is no single way to retain that experience that works well across settings, and no way to know in advance which future problems a given experience will help with. A global history preserves every detail but becomes expensive and mixes together experience that may not belong together; compressing that history into one fixed representation limits the cost but can throw away valuable details. We introduce Routed Continual Learning (Routed CL), an architecture that decides how to partition and retain experience. Past trajectories are grouped into clusters, each of which maintains state using one memory system from a fixed library. Before each run, an LLM-based router either reuses an existing cluster or creates a new one, basing its decision on the problem statement, memory system descriptions, and cluster summaries. We also introduce Context Memory Adapter, which mediates between the memory system and the agentic loop. We evaluate the architecture on a heterogeneous stream that interleaves 301 instances from the six CL-Bench tasks with 97 banking instances from $τ^3$-Bench. Across four language models and five matched rollouts, Routed Continual Learning raises normalized reward over a single global in-context learning (ICL) history by 15-30% of the maximum attainable normalized reward, achieves higher mean normalized reward than ICL-only clustering, and reduces estimated API cost by 65-72% relative to Global ICL. These results support approaching continual learning as a routing problem that chooses both a learning mechanism and a state cluster rather than committing to one representation globally.
★ From Grind to Grasp
Discussions of complexity, knowledge and intelligence often overlook the work involved. For example, outputs generated by short programs have low algorithmic complexity but can have high computational depth, the least work needed to produce them from near-shortest programs. Producing them with less work can require longer descriptions, giving high work-bounded complexity. We introduce a formalism that compares achievements under work and information limits and describes how learning and persistence can make deep achievements shallow by reusing products of earlier work. For an agent with a limited work budget, the value of building and maintaining such resources depends on how they support its goals over later tasks. We define grasp as performance across work budgets at a fixed information allowance. Low grasp can still yield success through grind at high cost in work, without necessarily improving human grasp. Applied to improvement itself, grasp gives a criterion for recursive self-improvement in which successors make comparable further improvements more efficiently. Another route to faster progress is to expand the resources brought to bear through knowledge sharing and coordination across society, even while comparable advances require increasing work. Grasp also applies to collectives, which may outperform their members yet require more total work than one system with their combined resources. Establishing and sustaining organisation can itself be deep even when the primary task is simple. These demands may help explain the richness of biological and social systems. In agent collectives, this organisational work can generate goals beyond assigned tasks, including maintaining the collective itself. We therefore caution against allowing an emergent agent society to assume authority over social coordination, since it could increasingly direct work towards its own persistence and goals.
★ Cinematize from Unreal: Agentic Choreography for Controllable World Simulation
Controllable world simulation requires more than visually plausible frames: scenes must retain persistent state, coordinated actions, and editable temporal structure. Generative world models offer flexible visual synthesis, but their outputs do not always expose the objects and transitions needed for precise revision. Graphics engines provide executable state and direct control, yet translating language into valid operations requires specialized knowledge of assets, geometry, and interdependent APIs. We present CineU (Cinematize from Unreal), an agentic system for language-driven world orchestration and cinematic choreography within Unreal Engine. From a code-based world modeling perspective, an Unreal project serves as a persistent, inspectable world representation, while agent-generated code specifies operations whose effects can be executed and verified. CineU combines hierarchical orchestration, cascaded asset retrieval and geometry-constrained placement, capability-aligned code synthesis, and multi-stage verification with localized repair. Shared scene and timeline context coordinates entities, actions, camera trajectories, and shot boundaries while supporting dependency-aware revisions. Experiments on scene construction, selective editing, multi-shot cinematography, and component ablations show how specialized execution knowledge and engine feedback improve task completion and targeted recovery. CineU illustrates how executable world state can complement learned visual generation to support controllable and revisable simulated worlds.
comment: 25 pages, 9 figures, 4 tables
★ Agent Transfer Protocol: Server-Mediated Agent-to-Agent Communication Architecture for the Agentic Web
The Agentic Web needs to support continuous collaboration among agents from different organizations. These collaborative relationships may span multiple tasks, network connections and execution instances. Such interactions require stable communication entries, message reception when instances are offline, and source verification after delayed reading. How to organize these responsibilities so that cross domain communication does not depend on specific execution instances being continuously online is a key issue for building long term agent interaction infrastructure. We propose ATP (Agent Transfer Protocol), a cross organization agent communication protocol adopting a domain mediated architecture. In this architecture, administrative domains maintain stable addresses and receiving entries for logical agents. The domain servers of both parties continuously undertake cross domain transmission. The receiving domain verifies and saves messages for execution instances to read later. The sending domain signs the complete message envelope. This allows the receiving instance to independently reverify the domain source and content integrity after the original connection ends, provided the original verification public key is still trusted and obtainable. We implemented an ATP prototype and evaluated it through a controlled cross organization LLM agent workflow, mechanism checks and latency measurements. The workflow completed subsequent business processing after instances went offline or were replaced. The independent verifier successfully verified the domain source and content integrity of saved messages. Latency measurements further quantified the communication and processing overhead introduced by the continuous mediation of domain servers.
comment: 12 pages, 6 figures, 5 tables
★ Disparity Laundering: Consensus Without Fairness in Clinical Multi-Agent Systems EACL 2027
Clinical large language models are increasingly fielded as multi-agent teams, on the assumption that deliberation averages out individual error. It does not average out bias. Across a full factorial of five model families and four aggregation topologies on 120 real MIMIC-IV emergency cases under counterfactual demographic swaps, no configuration neutralizes the disparity: all 20 cells commit the direction of their agents' independent answers, giving Black patients more acute triage. Every model has a topology that amplifies its baseline gap, but which one varies by model, so no design is reliably safe or reliably amplifying: the disparity is relocated, never removed. Late-stage fixes fail: fairness instructions and demographic masking leave the gap untouched, and per-agent self-debiasing widens it, because the bias is already encoded in the agents' ordinary clinical numbers. We call this disparity laundering; the audits now applied to clinical AI, per model, per output, per version, can certify the parts but never the committed decision. Interventions close the gap exactly when they make counterfactual versions visible inside the system and pool them, through deliberation, into the committed decision. A diverse-perspective ensemble does both while keeping the team, with no retraining or architectural change: the disparity falls by 67 to 94% where all five perspectives pool at every round, and the one topology that never pools them is the one it cannot reliably repair. Medical multi-agent systems do not filter demographic bias; they launder it into plausible decisions no single-version audit can catch, and the only defense is to make the bias visible.
comment: 21 pages, 8 figures. Code: https://github.com/wangxiaoyang0412/disparity-laundering . Committed to EACL 2027
★ Is it my preference? Role Grounding Attack against Agent's Memory via Social Interaction
LLM-based agents now share experiences on social platforms and remember new preferences to enhance their performance. However, existing memory systems contain a vulnerability in role-to-principal attribution that attackers can exploit. We identify Role Grounding Misattribution, in which an attacker-designed interaction causes the victim agent to incorrectly record a preference from the attacker as a preference of the victim's user. Specifically, we are the first to formalize Role Grounding Attack (RGA), an attack model that exploits this vulnerability through nested conversation structures. In the memory formation stage, an experience-sharing post quoting a conversation between the attacker agent and its user is published, which explicitly identifies its publisher. The inner conversation expresses preferences from the user of the attacker agent. When the victim agent remembers preferences from this nested structure, its memory loses the context binding and grounds the attacker's preference to the victim's user instead. In the memory retrieval stage, the misattributed preference persists and influences the victim agent's decisions when it encounters relevant scenarios. We instantiate RGA as tool selection manipulation and privacy leakage. Tool selection manipulation causes a tool-call preference to be misattributed to the victim's user, steering the victim toward an attacker-controlled tool. Privacy leakage extends this by misattributing both a tool-call preference and a task-requirement preference to the victim's user. The victim selects the attacker's tool, then includes sensitive information in tool arguments to satisfy the task requirement. We evaluate RGA across five mainstream memory systems and six language models. Tool manipulation and privacy leakage both achieve high attack success rates. These results call for stronger role grounding.
★ El Agente Quntur: Agentic Active-Space Selection for CASSCF
Multireference calculations often require substantial human expertise and time to choose and refine an active space for the electronic states or chemical process of interest. This involves selecting which orbitals to include, verifying that the calculation describes the intended excited state, and deciding whether the results have converged. Here, we ask whether scientific agentic systems (harnesses) can make these decisions reliably. We use the El Agente Quntur multi-agent harness, with GPT-5.6 Sol and GPT-5.6 Terra as the underlying LLM components, to explore 32 diverse singlet excitations at fixed molecular geometries. The agents planned and executed calculations, choosing and refining active spaces under prompts that specified equal-weight two-singlet state-averaged CASSCF followed by strongly contracted NEVPT2 with def2-TZVPD. We evaluate excitation energies against published CASPT2 reference values at matching geometries and compare performance with six deterministic Active Space Finder protocols. Within the El Agente Quntur system, agents detect missing active orbitals, compare candidate excited states, and revise state assignments as calculations progress. Agentic execution reduced mean absolute error by 48\% relative to the state-of-the-art Active Space Finder protocol. We note that computational traces vary substantially across runs in both active space selection strategies and cost, even for the same molecule. Our results underscore the advantage of agentic active space selection over fixed workflows, yet leave room to improve efficiency through runtime-aware decision-making, reduced inter-agent communication overhead and redundancy, more systematic checks of state identity, and clearer stopping criteria.
♻ ★ 6G Reasoning Agent Societies: Composition, Scaling, and Consistency-Time Laws
Emerging 6G control planes would compose Large Language Model (LLM) agents that exchange interpreted conclusions, so hallucinations can propagate even over error-free transport. Recent architectures audit each message locally against the sender's observation mask, but isolated audits miss collective behavior. Modeling the agent society as a cellular sheaf, we establish three laws valid on arbitrary topologies. First, the composition law: network-wide divergence exceeds the sum of per-agent audit scores by exactly their hidden total correlation. This blind spot masks up to 23% of joint divergence and, because the declared signal cancels in the cognitive signal-to-noise ratio (SNR), overstates it by $1.14$ dB purely through topology; the resulting trust inflation preserves valid consensus states but slows convergence. Second, the scaling law: reconciliation decouples across metric counters, so the worst-connected dimension sets the pace. Consistency time grows for a society size $n$ as $Ω(n^{2/d})$ on polynomial-growth topologies of degree $d$, and paths breach a $1$ s near-real-time budget at $n^\ast=13$ agents, whereas expanders ($Θ(1)$) and radius-3 local audits ($O(1)$, $\approx300$ ms) stay bounded. Third, the consistency-time law: the timing error from omitted correlations is one-sided and capped by the worst-measured link. These laws, validated on simulated topologies and deployed LLM societies, give provable worst-case latency ceilings for near-real-time operation.
♻ ★ HGWM: Hierarchical Graph-Guided World Model for Zero-Shot Object Navigation via Graph Matching
Zero-shot object navigation requires an agent to reason about where a target may be found while retaining evidence from previous observations. We present HGWM, a hierarchical graph-guided world model that combines a category-conditioned room--anchor prior with persistent spatial-semantic memory. A coarse stage uses a vision-language model to score candidate headings, while a fine stage explicitly compares observed objects and relations with the goal prior. Phase-dependent fusion integrates these signals with a language-model recommendation to guide local navigation. HGWM achieves 62.6\% success rate (SR) and 31.5\% success weighted by path length (SPL) on HM3D v0.1, improving over WMNav by 4.5 and 0.3 percentage points, respectively. On MP3D, it achieves 46.8\% SR and 17.9\% SPL, with gains of 1.4 and 0.7 points. On HM3D v0.2, the full model reaches 74.4\% SR and 31.9\% SPL; removing individual components reduces SR by 0.9--2.3 points and SPL by 0.6--1.5 points. These results support the use of relational priors and persistent graph memory for target-directed search.
♻ ★ S$^2$-MAS: A Society-Scale Multi-Agent Simulation Framework with a 500-Million-Persona Resource
Simulating economic responses at society scale requires decision rules that respond to household circumstances and accounting rules that track their consequences. We introduce S$^2$-MAS, a multi-agent simulation framework with a resource of 500 million synthetic household personas. It combines trainable candidate-scoring policies, reusable financial scenarios and explicit state transitions. Trained with SFT + RLCD on 2.93 million examples, YN-mini-MLX achieves 74.55% accuracy and 38.37% question-macro F1 on held-out consumer financial-choice tasks, improving on the training-question majority baseline by 2.78 and 6.21 percentage points. A completed application evaluates 12 million distinct households in two purchase-funding conditions: 24 million one-shot choices feed 288 million monthly ledger states in 3 h 59 min on 14 GPUs. Jointly increasing purchase goals and approved limits produces heterogeneous consumption, cash and repayment paths, with much of the consumption variation arising from transitions between deferral and purchase. In a separate 1,060-household retrospective consumption experiment, standalone YN-master-9B has 3.42% lower annual MAE than the deterministic fully rational benchmark, alongside a 19.7% lower predicted population mean than observed. Information-timing tests show how year labels and later outcome information change decisions. Together, these experiments establish an executable alternative for studying heterogeneous household behavior with learned policies and explicit financial accounting.
comment: 56 pages, 13 main figures and 8 supplementary figures. Updated YN model names and household consumption experiments. Project resources: https://github.com/KataDavidXD/s2-mas
Systems and Control (EESS)
★ Tunable Spatial Task Allocation with Application to Maritime Swarms
In this chapter, we explore how a team of autonomous agents can flexibly reconfigure the distribution of its members across many spatially embedded tasks using a task allocation strategy grounded in a generalization of the recently proposed Nonlinear Opinion Dynamics model. Motivated by an adaptive maritime survey application, we couple opinion dynamics directly to spatial movement and examine how the emergent macroscopic allocation is shaped by the interplay between environmental geometry, nature of social interactions, and task demands. We illustrate in simulation how social coupling in the model tunes group cohesion, producing macroscopic allocation patterns ranging from maximally dispersed states, to intermediate semi-dispersed states with a smaller number of distinct subgroups, to maximally cohesive consensus states, in distinct and interpretable parameter regimes. Furthermore, we demonstrate with illustrative examples how a team can reconfigure itself in real time under changing task demands. For allocation scenarios with distance-based task preference and matched number of agents and tasks, we relate numerically the final allocation in the group to the Hungarian optimum, illustrating close-to-optimal performance. We illustrate performance of this method under realistic communication constraints in the MOOS-IvP-Multi simulation environment with simulated autonomous underwater vehicles (AUVs). Together, these examples show how a common set of local interaction rules can support a large range of interpretable macroscopic allocation states and flexible transitions between them.
comment: 29 pages, 10 figures
★ Selective Sliding Mode Control Enhances Robustness of Aerial Robots Under Disturbances
Quadrotor control under disturbances such as wind gusts and payload variations remains challenging. First-order sliding mode control (SMC) and matched/unmatched uncertainty distinction are classical approaches; however, neither tells a practitioner which loops in a cascaded controller should be made robust to sustain disturbances, how to tune those loops, nor what the additional costs are on an embedded processor. Applying robust control uniformly across all axes addresses disturbances indiscriminately, adding tuning effort on axes where no disturbance acts. On inherently underactuated vehicles, disturbances along some axes cannot be opposed directly at all. We propose a principled methodology for deciding where robust control belongs in a cascaded architecture, with all control running onboard the vehicle. We recover the acting disturbance from PID hover flights under disturbance by inverting the equations of motion, use the recovered signal to identify and quantify the excited axes, and deploy robust control where it counts. Experiments on a Crazyflie 2.1 quadrotor under slung-payload and oblique-wind disturbances show tracking improvements concentrated on the axes the rule selects, and gain sweeps confirm that the switching gain reduces error on matched axes but has no effect on an excited unmatched axis, as the analysis predicts.
comment: 8 pages, 5 tables, 8 figures
★ Inertial Steering and Motion Prediction of Tracked Vehicles on Low-Adherence Surfaces
Tracked vehicles are widely used in off-road applications, but their conventional motion models often rely on no-slip assumptions that become inaccurate on low-adherence surfaces. Rather than treating inertial effects arising from slip as a disturbance to be minimized, this work investigates how inertial effects arising from a moving mass can be deliberately exploited for steering on low-friction surfaces. A dynamic model for an articulated tracked vehicle with a passive trailer is developed, allowing inertial effects from trailer motion to be captured. The approach is experimentally evaluated on ice, where our dynamic model outperforms the kinematic baseline. We also propose a mixed dynamic-kinematic model to adapt predictions to both surface and situation by exploiting predictions from both methods. Finally, we demonstrate inertial steering on a mono-track vehicle without a conventional steering mechanism, showing that controlled movements of an internal mass can generate turning motion on ice. The resulting behavior can be predicted using the proposed dynamic model by representing the internal mass as a frictionless trailer, revealing the analogous dynamics between the two configurations. These results demonstrate the feasibility of inertial steering as an alternative steering mechanism for tracked vehicles operating in low-adherence environments.
★ Efficient MPC for LC-Filtered Three-Level NPC Inverters in Vehicle-to-Load Applications
Finite control-set model predictive control (FCS-MPC) offers fast, modulator-free voltage regulation but is sensitive to the fidelity of its internal model. This sensitivity is usually countered by algorithmic augmentation, disturbance observers, adaptive weighting, or learning-based estimators, at the cost of extra tuning and computation. This paper instead improves fidelity at the source. For a three-level neutral-point-clamped (3L-NPC) inverter with an LC output filter in vehicle-to-load operation, a coupled alpha-beta-frame predictive model carries the capacitor voltage and filter current jointly through the state vector, so that the second-order LC coupling is retained without enlarging the 27-candidate search. The cost function additionally regulates dc-link neutral-point balance and switching effort. One-step prediction accuracy, parametric sensitivity, and computational cost are bounded in closed form. Simulation of a V2L system yields a voltage total harmonic distortion of 0.72%, a steady-state tracking error of 1.6%, and transient recovery within one fundamental cycle, meeting the applicable IEEE power-quality limits with margin, without observers, parameter estimation, or learning-based augmentation.
★ Computationally Constrained First-Order Prediction-Correction Approach for Time-Varying Convex Optimization
Time-varying optimization methods often predict the locations of time-varying optima, then sample the objective function being minimized, and then compute corrections to those predictions. However, those works typically do not account for settings in which computations are limited. Therefore, in this work we formulate a prediction-correction algorithm that is designed to obey constraints on the total number of iterations it can execute over a given time horizon. We first show that there is a critical threshold for the average number of computations that can be executed between samples of the objective. Then, we show that if the number of allowable computations is above this threshold, then the distance to a time-varying optimum bound decays geometrically. Next, we show that if the number of allowable computations is below that threshold, then the distance to a time-varying optimum still remains bounded. Therefore, it is still possible to track time-varying optima with bounded error even under restrictive limits on the number of computations that can be executed. We also present an algorithm to determine how many prediction and correction steps should be executed for each sample of the objective. Simulation results show that under typical conditions our approach reduces cumulative tracking error by 45.5% compared to a comparable baseline approach.
comment: 9 pages, 2 figures, for submission to ECC
★ From Local to Global: Sheaf-Theoretic Control Barrier Functions on Manifolds
Control barrier functions (CBFs) provide formal safety guarantees for dynamical systems by rendering a safe set forward-invariant via pointwise-optimal feedback control. While the CBF framework is well established for dynamical systems in Euclidean space, there exist gaps for systems evolving on non-Euclidean manifolds. For systems modeled as constrained control differential inclusions, we introduce the Sheaf-CBF: a sheaf-theoretic framework for CBFs on smooth manifolds and vector bundles. The differential inclusions are formulated intrinsically on the control bundle, and analogous regularity properties in the Euclidean case are established fiberwise under local trivializations. Regularity conditions are given under which a pointwise-optimal feedback law is single-valued and continuous over local charts. Framed as a local-to-global optimization problem, sheaf-theoretic restriction and gluing compatibility conditions on overlapping charts are shown to uniquely determine a continuous global section of the control bundle, thereby guaranteeing forward invariance of the safe set. Numerical simulations on a nontrivial M{ö}bius strip demonstrate that failure of sheaf compatibility results in violations of safety, whereas the Sheaf-CBF maintains forward invariance.
comment: 8 pages, 2 figures
★ Minimum-Time Trajectory for Escort-Assisted Evasion from a Sensing-Limited Pursuer
This paper studies a minimum-time path-planning problem in which an escort assists a leader by triggering and redirecting a sensing-limited pursuer. For the post-engagement case with equal leader, escort, and pursuer speeds, we show that the minimum-time leader trajectory has a three-phase structure: an initial straight segment, a moving-boundary segment, and a final straight segment to the goal. The boundary-following kinematics and the corresponding entry and exit conditions are derived and used to construct the leader trajectory for a fixed escort engagement direction. The engagement direction is then optimized over the feasible set to obtain the minimum-time solution. Numerical results illustrate the resulting trajectory and the benefits of escort assistance.
★ Stable Dynamical Mass Transportation via port-Hamiltonian Structure
Dynamical optimal mass transportation over nonlinear systems is often hindered by the curse of dimensionality and open-loop implementation. We propose an alternative framework that sacrifices exact optimality in favor of stability, robustness, and scalability. We use a stochastic port-Hamiltonian system as a reference model. Under suitable structural assumptions, we show that its density converges asymptotically to a prescribed steady-state distribution. We then shape a deterministic nonlinear system into a compatible port-Hamiltonian form and add a local density-feedback term so that its closed-loop continuity equation matches the Fokker-Planck-Kolmogorov equation of the reference model. The resulting feedback drives a broad class of initial distributions to the target distribution. Because the density term is local at the particle level, the controller admits decentralized implementation and avoids solving a global finite-horizon transport problem online. Nonlinear examples and simulations illustrate the method. For linear systems with Gaussian targets, we derive necessary and sufficient conditions for stabilizing density feedback and give explicit controller constructions.
★ Weak and Strong Data Informativity with Structural Priors for System Analysis
When measurement data are insufficient for unique system identification, data informativity evaluates whether a dataset determines a property across all systems consistent with the data. This paper introduces a unified framework distinguishing between weak and strong data informativity under prior structural knowledge described by affine constraints: data are weakly informative if at least one system consistent with the data and prior knowledge possesses the target property, and strongly informative if every consistent system satisfies it. The latter is consistent with the notion of data informativity in the existing literature. By explicitly parameterizing all consistent systems as affine functions of free parameters via the Moore-Penrose pseudoinverse, we characterize these notions for fundamental system properties-including controllability, stabilizability, and stability. This formulation establishes that verifying weak and strong informativity, when tractable, reduces to rank tests of matrix pencils constructed from the data and prior knowledge. Our findings show that structural priors can lower data-excitation requirements and enable reliable analysis even when data fail to be strongly informative. Finally, the framework is extended to a class of noisy data.
★ Phase-Compensated Frequency-Scan Measurement of Harmonic Transfer Function Impedance
Impedance models obtained through frequency scanning provides terminal characterization for small-signal stability analysis without requiring detailed knowledge of internal converter parameters. When unbalanced steady-state components introduce cross-frequency coupling, conventional $2\times2$ impedance scan methods must be extended to identify harmonic transfer function (HTF) impedance. However, the measured phases of frequency-coupling terms depend on the starting time of the fast Fourier transform (FFT) window, potentially introducing phase errors when different time references are used. This paper presents a $dq$-domain measurement method of HTF impedance that explicitly accounts for this time dependence. Phase compensation aligns the measured Fourier coefficients to a common electrical phase reference, correcting the resulting phase discrepancies and enabling consistent impedance reconstruction. Comparisons with analytical models under balanced and unbalanced conditions verify the measurement method, recovering the conventional $2\times2$ impedance and the dominant coupling blocks of a $6\times6$ HTF representation, respectively.
★ Adaptive Orbit-Aided Vector Tracking for Orbit Determination of LEO Satellites
Reliable orbit determination for CubeSats is essential for emerging low-Earth-orbit positioning, navigation, and timing (LEO-PNT) applications. However, onboard orbit determination remains constrained by single-frequency GNSS code-based receivers, intermittent signal reception, and highly dynamic orbital conditions. This paper proposes an adaptive orbit-aided vector tracking (OAVT) algorithm for LEO-PNT CubeSat orbit determination under these conditions. The method employs a GNSS vector tracking loop (VTL) as the primary navigation estimator and loosely couples it with a low-fidelity orbital dynamics model (ODM), which provides auxiliary position and velocity pseudo-measurements. The contribution of the ODM is regulated through adaptive covariance assignment based on GNSS stress and the residual consistency between the ODM and GNSS solutions. Under nominal conditions, OAVT preserves the accuracy advantage of VTL over the conventional scalar tracking loop. The 3D position root-mean-square error (RMSE) is reduced from 2.631\,m for STL to 1.081\,m, while the 3D velocity RMSE decreases from 0.3985\,m/s to 0.1023\,m/s. Under degraded conditions, OAVT provides substantially greater robustness than both STL and standalone VTL. The 3D position RMSE remains bounded at 8.27\,m during a complete signal outage, 10.17\,m during gradual signal loss and reacquisition, and 26.65\,m under variable visibility, whereas STL fails to produce valid solutions and standalone VTL loses lock. These results demonstrate that OAVT substantially improves solution robustness and state continuity with only a moderate increase in computational burden, making it suitable for CubeSat orbit determination and future LEO-PNT missions.
★ Counterfactual Public Opinion Projection for Intervention Decision-Making
Effective public opinion intervention requires forecasts of how the same event would evolve under alternative actions. Existing methods mainly capture observational associations and inadequately address treatment selection bias. The paper proposes a counterfactual public opinion projection framework for intervention decision making. Each intervention is represented by its timing, type, content and intensity. Causal triggering structures are inferred from propagation event sequences. Counterfactual trajectories of attention, sentiment and polarization are generated from adversarially balanced representations. Candidate interventions are then compared through repeated rollouts. Semisynthetic experiments yielded an overall F1 comparable to a direct structure learning baseline and a recall of 0.698 for causal edges linking semantically dissimilar content. Representation diagnostics showed that the gradient reversal layer reduced treatment information leakage from 0.163 to 0.140. Bootstrap analyses of real data further revealed intervention specific model effects on attention, sentiment and polarization.
comment: 12 pages, 7 figures
★ Semantic Anchor Communication System for Point Cloud Transmission in Internet of Vehicles
Joint transmission of point cloud geometry and semantics under bandwidth constraints is required for cooperative perception in the Internet of Vehicles (IoV). Per-point semantics cannot be explicitly preserved by existing geometric compression methods, while interpretable three-dimensional point sets cannot be directly reconstructed via task-feature communication. The paper proposes a spatial semantic anchor communication system. Distinct from global codewords and task features, anchors with explicit coordinates are employed as transmission units. Local shape descriptors and category labels are bound to a unified spatial reference, preserving the correspondence between local position, shape and semantics. At the transmitter, anchor representations are extracted and then converted into digital frames through fixed-point quantization and lossless compression.Atthereceiver,labeledpoint clouds are reconstructed through anchor-constrained local expansion, and pose-aligned multi-view observations are fused. On the 500-frame SemanticPOSS test set, the results demonstrate that spatial semantic anchors enable interpretable joint reconstruction and fusion of geometry and semantics with low transmission overhead.
comment: 10 pages, 3 figures
★ LLM-PowerTrace: A Dataset of Time-Resolved GPU Power Traces for AI Compute
AI compute is becoming large and fast-changing electrical loads. Its impacts on data center infrastructure and power grids depend not only on total energy consumption, but also on how their power draw moves over sub-second to minute timescales, including peaks, ramps, and synchronized fluctuations across accelerators. Existing energy benchmarks for large language models (LLMs) report aggregate quantities such as joules per token and rarely release the underlying time-series measurements, leaving these power dynamics poorly characterized and difficult to model. We introduce LLM-PowerTrace, an open dataset of time-resolved, per-GPU power traces for LLM fine-tuning and inference, sampled every 100\,ms on varying NVIDIA platforms paired with complete workload and system metadata. For fine-tuning, LLM-PowerTrace profiles open model families spanning a wide range of model sizes and fine-tuning methods, together with controlled sweeps over batch size, sequence length, precision, parallelism, and checkpointing. For inference, it covers variations in concurrency, GPU frequency, KV-cache and weight quantization, prompt and output lengths, and request-arrival patterns. LLM-PowerTrace profiles a set of open model families across a wide range of model sizes, and controlled sweeps over batch size, sequence length, precision, parallelism, and checkpointing for fine-tuning, and it sweeps concurrency, GPU frequency, KV cache and weight quantization, prompt and output lengths, and request arrival patterns for inference. Built upon power traces, we define power-grid-facing metrics as well as AI-compute power\&energy scaling law for extrapolating beyond the measured settings. Together, LLM-PowerTrace provides a dataset foundation for modeling, forecasting, and managing the fast-changing electrical behavior of AI compute.
comment: In Submission; Code and dataset are available at https://systems-power-review.github.io/training-power-trace-explorer/
★ Belief-Driven Search in Connectivity-Constrained Mobile Sensor Networks with Convergence Guarantees
Multi-agent search and coverage is critical to autonomous exploration, search-and-rescue, and defense applications. This paper investigates settings where a mobile sensor network is tasked with coordinating to discover the position of a stationary target within a bounded region. We develop a Bayesian update scheme where agents iteratively combine local uncertainties derived from their sensor measurements through inverse variance weighting to get a better global understanding of where the target may be located given some prior belief. Network agents' trajectories are governed by a primal-dual gradient update rule that simultaneously minimizes the expected sensing uncertainty and enforces network connectivity among the agents. We prove that the belief converges to the target under mild, planner-agnostic conditions, with an explicit exponential rate of false-hypothesis rejection and a bound on expected search time. Simulation results show that we can successfully find the target and collapse belief to the ground truth target location while ensuring connectivity of our sensor network. They also empirically confirm our theoretical bounds and revisiting assumption. Finally, we compare against a lawnmower sweep, showing that our method locates the target more quickly when the prior is informative.
comment: 8 pages, 5 figures, 1 table. Extended version with full proofs
★ Necessary and Sufficient Structural Conditions for the Validity of Lyapunov-Type Exponential Stability Criteria
Whether the exponential decay of a Lyapunov function along trajectories implies exponential stability of the state---an inference we call migration---is long assumed in Lyapunov's second method but never characterized at the criterion level. We formalize stability criteria over trajectory families, distinguish admissibility of a criterion (its structural condition suffices for its conclusion) from tightness (its necessity), and establish the necessary and sufficient structural conditions for validity. In logarithmic coordinates the theory reduces to a single two-sided displacement function of the comparison pair: in discrete time, migration holds if and only if a regularity package (excluding failure by infinitely many steps of finite displacement) and a joint lower bound (excluding failure by a single step of unbounded expansion) hold simultaneously. Continuous time strengthens the latter; the two criteria are separated exactly by the minimum step size. Written without its inverse-function formulation, the Panteley--Lor'ia criterion coincides with our continuous-time conditions on every nondecreasing pair, and loses soundness beyond: it was already complete on the classical regularity class. An envelope construction shows that regularization costs no discriminative power, and radial unboundedness, a standing assumption since Hahn, is dispensable.
★ Data Center Load Reductions: Attribution, Measurement Limits, and Estimation
Hundreds of gigawatts of data center loads are seeking interconnection in the United States given the massive growth of artificial intelligence and cloud computing. In recent years, several incidents of data centers reducing their loads by hundreds to thousands of megawatts have been documented, raising grid reliability concerns. U.S. regulators have since ordered reliability standards for the integration of computational loads, and the North American Electric Reliability Corporation (NERC) now calls on transmission owners to install disturbance recorders at these facilities to support root cause analysis. Such recorders usually observe only electrical quantities at the point of interconnection (POI). A sudden load reduction can be caused by a change in workload or a switch to backup supply. This work shows when POI measurements can distinguish the two mechanisms. It is established that they can produce identical POI voltage and current in a quasi-static constant power model, and during the transient their separation depends on the reduction relative to measurement noise, the clearing time, the dynamics through which workload reaches the POI, and the recorder bandwidth. When POI data are insufficient, an estimator that uses workload data is proposed to determine how much of the facility stayed connected. The results of this work identify the measurements and grid information utilities need to determine why data center load dropped.
★ Machine learning-guided real-time AC optimal transmission switching
The operation of power grids has substantial economic consequences, affecting generation costs and, ultimately, electricity prices. The Optimal Transmission Switching (OTS) problem seeks to reduce these costs by selectively switching off transmission lines. However, OTS gives rise to a challenging mixed-integer nonlinear optimization problem that is difficult to solve at operationally relevant timescales, particularly as rapidly changing loads create a need for real-time switching decisions. To address this challenge, we develop a machine learning-guided method that identifies high quality line switching decisions in real time. Specifically, we construct a grid-specific switching-value model that rapidly estimates the generation-cost reduction associated with opening transmission lines by fine-tuning a pretrained model for AC optimal power flow (AC-OPF). The switching-value model provides signals to guide promising switching decisions, and topologies proposed by the model are evaluated using exact AC-OPF solvers to verify feasibility and obtain the actual generation cost. We develop multiple strategies that use the ML surrogate to efficiently explore the combinatorial space of transmission topologies, including Fixed-Rank Search, Adaptive-Rank Search, and Swap-Enhanced Search. Across multiple benchmark networks, our proposed algorithms identify line switching decisions that significantly reduce generation cost within a 50s computational budget. In particular, under a realistic data center buildout scenario on the 2,000-bus synthetic Texas grid, our approach reduces generation costs by 31.97% and gross nodal load payments by 94.34%. More broadly, our results demonstrate that a fine-tuned surrogate can effectively guide search even without guaranteeing feasibility, enabling classical optimization methods to tackle large nonlinear combinatorial problems on operationally relevant timescales.
★ Physics-Guided Inferencer-in-the-Loop Digital-Twin for Real-Time Adaptive Emulation of Nonlinear PMSMs
A real-time digital-twin (RTDT) of a permanent magnet synchronous machine (PMSM) allows drive controllers to be tested in hardware-in-the-loop (HIL) before they meet the real machine. Its machine model rests on identified parameters or flux look-up tables (LUTs) from finite-element analysis (FEA) or characterization tests; the maps are regenerated offline when the machine changes, and online estimators track a few parameters such as resistance and PM flux. This article makes the executed model learnable from reference data. A physics-guided machine learning (PGML) network advances the currents and predicts torque in one step, trained with analytical one-step labels. In the proposed Inferencer-in-the-Loop (IIL) architecture, a GPU retrains the network while a field-programmable gate array (FPGA) executes the last accepted weights. For damper circuits, shifted parameters, and an FEA-calibrated saturated machine, one kernel form reaches average normalized mean absolute errors (NMAEs) of 0.19%-0.67% at measured FPGA kernel latencies of 0.30-0.34 $μ$s, and retraining for a shifted machine nears its final loss within 94 s. On a magnetically linear 200-W drive, a residual form corrects the analytical step and learns its physical parameters from operating data; each step takes under 2.5 $μ$s, 20 times faster than real time and ahead of the physical drive's measurements even at 20-kHz sampling. On six held-out runs (1193 s), its $i_q$ error is 5.9%-9.4% of measured RMS, against 10.4%-13.2% for an independently fitted grey-box model, and over 30 min of loaded heating its online-updated heads track a 4.1% winding-resistance rise with an RMS error of 0.44 m$Ω$.
★ Passive Identification of Unequal Phase Grid Impedances by Phase-Domain Source Cancellation
Passive grid impedance estimators that return a single resistance--inductance pair cannot represent a grid whose phase impedances are unequal. This paper identifies a separate pair for each phase from recorded point-of-common-coupling (PCC) voltage and current without injected perturbations. In each phase, the unknown grid-source voltage is cancelled by a periodic difference (PD) or by a second-order-generalized-integrator notch difference (SOGID), and moving-window integration (MWI) replaces the current derivative by an integral and an endpoint feature. The resulting two-parameter regression requires a period or notch-frequency estimate but no phase-locked-loop angle. A voltage-balance analysis gives the source residual left by frequency mismatch and bounds the parameter error by the ratio of this residual to the conditioning of the two features. In the unequal-phase interval (8.3--9.8~s) of a 10-s EMTP grid-forming record, SOGID-MWI attains phase-averaged R/L errors of 5.09/0.33\%; a shared value has a minimum error of 41.67\% for the phase ratio 1:0.5:2. On a 50-s constant-impedance grid-following record, the errors are 0.56/0.06\% over the 20--30-s synchronization disturbance. In a controlled 40-dB noise study, a noise-referenced update rule reduces the white-noise GFL error from 55.9/65.0\% to 12.3/11.5\%.
★ Safety of layered control systems via control barrier morphisms
Layered control architectures provide a powerful paradigm for synthesizing controllers for complex systems. Yet, translating guarantees at each layer to system-level guarantees is challenging due to the interactions between layers. To address this problem, this paper presents a categorical perspective on layered control architectures. Individual systems are represented as control coalgebras, their interactions are specified as morphisms to mediators, and system-level behaviors are obtained via composition defined categorically as a pullback. To analyze the safety of these systems, we formulate control barrier morphisms (CBMs), the categorical generalization of control barrier functions. We prove that feedback controllers satisfying a CBM condition render the associated safe subobject forward invariant. It is through CBMs that we can establish the safety of layered systems: given a CBM at each layer subject to compatibility conditions, the universal property of the pullback implies the existence of a CBM for the composed system. Safety of a full-order model being safety filtered by a reduced-order model is recovered as a specialization of the categorical framework.
★ Same Label, Different Interface: Beyond Action-Space Labels in Robot Learning
Robot-learning studies often compare position and velocity actions as though each label identified a single experimental treatment, yet the label alone does not determine what the robot executes. Before affecting robot motion, bounded policy output passes through filtering, physical scaling, reference construction, feedback, timing, and effort limits. We study how these omitted properties affect attribution under one shared learner and training budget. Across three contact-rich Factory tasks (PegInsert, GearMesh, and NutThread), we compare five joint-space interfaces, with ten independently trained policies per condition. We first remain within the same position-feedback law and vary the executed interface: substituting only the physical action scale as the directly modified component changes sample-median evaluation success by 0.645-1.000 across tasks, while removing an exponential moving average produces task-dependent changes. We then move to a cross-law comparison and choose a velocity scale that exactly matches the position interface's local raw-torque command gain. This calibration raises the sample median on GearMesh and NutThread, with no median change on PegInsert. Yet clear residual sample-median gaps remain on PegInsert and GearMesh, while the NutThread interval includes zero. Thus, even this explicit equality leaves feedback and held-action dynamics unmatched and does not establish position-velocity trained-outcome equivalence. Action-interface comparisons should therefore report the executed pathway and state the physical quantity they match.
★ PyPSA-Libya: An Open-Source Energy System Model for National Decarbonization and Hydrogen Export Pathways
Libya possesses significant renewable energy potential that remains largely untapped. This paper presents PyPSA-Libya, a national-scale model built on the PyPSA-Earth framework, designed to explore optimal transition pathways for the Libyan power sector. We present two primary studies: (1) an electricity-only capacity expansion analysis comparing a zero-emission constraint against an unconstrained baseline at 36-cluster resolution, and (2) a sector-coupled hydrogen export study evaluating the levelized cost of exporting 70 TWh/year of hydrogen to Europe under three technological scenarios and a 100 EUR/tonne CO2 carbon tax regime. Results indicate that while solar PV dominates the domestic capacity mix, integration of hydrogen production significantly alters grid storage requirements. Furthermore, under carbon pricing, steam methane reforming with carbon capture (SMR-CC) frequently out-competes pure green electrolysis, establishing a lower-cost compliance pathway for cross-border export. The model, data, and configuration files are released under an open-source license to support transparent energy planning in Libya.
comment: 36 pages, 15 figures, 10 tables
♻ ★ A Spectral Perspective on Stochastic Control Barrier Functions
Stochastic control barrier functions (SCBFs) provide a safety-critical control framework for systems subject to stochastic disturbances by bounding the probability of remaining within a safe set. However, finding a valid SCBF that explicitly reflects the long-horizon behavior of the true safety probability remains a challenge. This paper addresses this issue by adopting a spectral perspective, utilizing the linear operator that governs the evolution of the closed-loop system's safety probability. We find that the dominant eigenpair of this Koopman-like operator encodes fundamental safety information of the stochastic system. The dominant eigenfunction is a natural and valid SCBF, with values that characterize relative asymptotic safety across initial states, while the dominant eigenvalue indicates the global asymptotic rate at which the safety probability decays. A practical algorithm for computation is proposed, termed power-policy iteration, which jointly estimates the dominant eigenpair while updating the backup policy with the aim of improving long-term safety. The method is validated using simulation experiments on safety-critical dynamics models.
comment: 16 pages, 8 figures. This work has been submitted to the IEEE for possible publication
♻ ★ Activate the Dual Cones: A Tight Reformulation of Conic ACOPF Constraints
By exploiting the observed tightness of dual rotated second-order cone (RSOC) constraints, this paper transforms the dual of a conic ACOPF relaxation into an equivalent, non-conic problem where dual constraints are implicitly enforced through eliminated dual RSOC variables. To accomplish this, we apply the RSOC-based Jabr relaxation of ACOPF, pose its dual, and then show that all dual RSOC constraints must be tight (i.e., active) at optimality. We then construct a reduced dual maximization problem with only non-negativity constraints, avoiding the explicit RSOC inequality constraints. Numerical experiments confirm that the tight formulation recovers the same dual objective values as a mature conic solver (e.g., MOSEK via PowerModels) on various PGLib benchmark test systems (ranging from 3- to 1354-buses). The proposed formulation has useful performance benefits, compared with its conic counterpart, and it allows us to define a bounding function which provides a guaranteed lower bound on system cost. While this paper focuses on demonstrating the correctness and validity of the proposed structural simplification, it lays the groundwork for future GPU-accelerated first-order optimization methods which can exploit the unconstrained nature of the proposed formulation.
♻ ★ Stability and Controllability Margins of Water Network Hydraulics
Pump and valve actuation in water distribution networks (WDNs) affects how pressures, flows, and tank levels respond to demand changes and other disturbances. This paper presents a stability and controllability analysis of WDN hydraulics. Starting from a general nonlinear differential algebraic equation (DAE) model of WDNs with arbitrary topology and network components (valves and pumps), we investigate three questions. First, we study local well-posedness of the network dynamics and characterize the loss of differentiability introduced by pump and valve switching. Second, we introduce regularization methods that smooth flow and pressure trajectories under changing controls. Third, we quantify how network parametric uncertainty impacts local stability and controllability, while establishing error bounds on the resulting margins. For water operations, stability means that pressures and flows return to steady values after a disturbance, and controllability means that pump and valve actions can move pressures, flows, and tank levels; the margins we compute measure how far a network is from losing either. We demonstrate that the proposed DAE models produce trajectories that closely match EPANET, a widely used WDN simulator, for benchmark networks. The case studies also show that the WDN DAE relates the recovery of pressures and flows to a weighted Laplacian of the network, ranks pipes by their effect on the local dynamics, and reveals that certain demand variation changes the stability and controllability margins without eliminating local stability or the ability of pumps to affect pressures and flows. The developed theoretical foundations enable network analysis, mitigation strategies, and system design.
♻ ★ Finite-Sample Analysis of Model-free Policy Gradient Methods for LQR with Stochastic Data
Policy gradient (PG) methods are the backbone of many reinforcement learning algorithms due to their good performance in policy optimization problems. As a gradient-based approach, PG methods typically rely on knowledge of the system dynamics. If this is not available, trajectory data can be utilized to approximate first-order information. When the data are noisy, gradient estimates become inaccurate and a study that investigates uncertainty estimation and the analysis of its propagation through the algorithm is currently missing. To address this, our work focuses on the Linear Quadratic Regulator (LQR) problem for systems subject to additive stochastic noise. After briefly summarizing the state of the art for cases with a known model, we focus on scenarios where the system dynamics are unknown, and approximate gradient information is obtained using zeroth-order optimization techniques. We analyze the theoretical properties by computing the error in the estimated gradient and examining how this error affects the convergence of PG algorithms. \textcolor{black}{Additionally, we provide global convergence guarantees for various versions of PG methods, including those employing adaptive step sizes and variance reduction techniques.} This study contributes to characterizing the robustness of model-free PG methods, aiming to identify their limitations in the presence of stochastic noise and proposing improvements to enhance their applicability.
♻ ★ Benchmarking Learning-Based Formula 1 Race Strategies
This paper presents two complementary frameworks to optimize Formula 1 race strategies, jointly accounting for energy allocation, tire wear and pit stop timing. First, the race scenario is modeled using lap time maps and a dynamic tire wear model capturing the main trade-offs arising during a race. Then, we solve the problem by means of a mixed-integer nonlinear program that handles the integer nature of the pit stop decisions. The same race scenario is embedded into a reinforcement learning environment, on which an agent is trained. Providing fast inference at runtime, this method is suited to improve human decision-making during real races. The learned policy's suboptimality is assessed with respect to the optimal solution, both in a nominal scenario and with an unforeseen disturbance. In both cases, the agent achieves approximately 5s of suboptimality on 1.5h of race time, mainly attributable to the different energy allocation strategy. This work lays the foundations for learning-based race strategies and provides a benchmark for future developments.
♻ ★ Noise Steering: Exploiting Wake Steering and Acoustic Directivity for Noise-Constrained Wind Farm Control via Reinforcement Learning
Existing noise mitigation strategies for wind farms usually consist in reducing turbines power output, without accounting for the wake interactions between them and the noise emission directivity. This leads to over-conservative policies and unexploited wind power. Yet, turbine yaw can simultaneously affect wake interactions and the direction of noise emission, offering an additional degree of freedom for farm-level noise control. We therefore propose Noise Steering, a model-free reinforcement learning (RL) strategy that jointly controls turbine yaw angles and power setpoints to maximise farm power while satisfying a local noise constraint in a defined region. To enable training at a reduced computational cost, a low-fidelity acoustic model is coupled with the steady-state FLORIS simulator. The RL agent effectively learns how to redirect wakes and noise directivity away from the protected region while redistributing power across the farm. On a six-turbine test farm, the policy satisfies the 40 dB noise constraint throughout the full range of turbulent winds, with an average power deficit of only 2% relative to greedy unimpeded control, and up to 7.7% at high wind speeds. In comparison, the industry-standard noise-reduced operation (NRO) baseline here generates an average power loss of 17%. When deployed in the higher-fidelity FASTFarm simulator, the policy largely retains its performance under steady and turbulent inflow, despite differences in simulation fidelity. These results demonstrate the potential of combining yaw control and power derating through RL to improve the power-noise trade-off in wind farm operation.
comment: 17 pages, 11 figures
♻ ★ A Linear Parameter-Varying Framework for the Analysis of Time-Varying Optimization Algorithms
In this paper we propose a framework to analyze iterative first-order optimization algorithms for time-varying convex optimization. We assume that the temporal variability is caused by a time-varying parameter entering the objective, which can be measured at the time of decision but whose future values are unknown. We consider the case of strongly convex objective functions with Lipschitz continuous gradients under a convex constraint set. We model the algorithms as discrete-time linear parameter varying (LPV) systems in feedback with monotone operators such as the time-varying gradient. We leverage the approach of analyzing algorithms as uncertain control interconnections with integral quadratic constraints (IQCs) and generalize that framework to the time-varying case. We propose novel IQCs that are capable of capturing the behavior of time-varying nonlinearities and leverage techniques from the LPV literature to establish novel bounds on the tracking error. Quantitative bounds can be computed by solving a semi-definite program and can be interpreted as an input-to-state stability result with respect to a disturbance signal which increases with the temporal variability of the problem. As a departure from results in this research area, our bounds introduce a dependence on different additional measures of temporal variations, such as the function value and gradient rate of change. We exemplify our main results with numerical experiments that showcase how our analysis framework is able to capture convergence rates of different first-order algorithms for time-varying optimization through the choice of IQC and rate bounds.
♻ ★ Data-Driven Safe Output Regulation of Strict-Feedback Linear Systems with Input Delay
This paper develops a data-driven safe control framework for linear systems possessing a known strict-feedback structure, but with most plant parameters, external disturbances, and input delay being unknown. By leveraging Koopman operator theory, we utilize Krylov dynamic mode decomposition (DMD) to extract the system dynamics from measured data, enabling the reconstruction of the system and disturbance matrices. Concurrently, the batch least-squares identification (BaLSI) method is employed to identify other unknown parameters in the input channel. Using control barrier functions (CBFs) and backstepping, we first develop a full-state safe controller. Based on this, we build an output-feedback controller by performing system identification using only the output data and actuation signals as well as constructing an observer to estimate the unmeasured plant states. The proposed approach achieves: 1) finite-time identification of a substantial set of unknown system quantities, and 2) exponential convergence of the output state (the state furthest from the control input) to a reference trajectory while rigorously ensuring safety constraints. The effectiveness of the proposed method is demonstrated through a safe vehicle platooning application.
♻ ★ Whittle Index Policy for Three State Collapsing Bandits
Partially observable restless multi-armed bandit models have been studied for dynamic resource allocation problems under uncertainty. This paper analyzes three-state restless multi-armed bandits with partial observability. These are called collapsing bandit problems if the state of an arm (bandit) is perfectly observable when it is scheduled for allocation. The state is not observable when the arm is not scheduled. The planner has a budget constraint on the scheduling decisions in each time slot. The planner's goal is to maximize expected cumulative long term discounted reward. We study the Whittle index policy. To analyze index policy, first the $N$-armed collapsing bandit problem is decoupled into $N$ single arm problems, and convexity and monotonicity of the optimal value function in belief vector are derived using monotone likelihood ratio (MLR) ordering on state transition dynamics and monotone reward structure assumption. We next show that the optimal policy is of threshold type in belief under sufficiency conditions on the model. These sufficiency conditions depend on Lipschitz bound on the value function in the belief. We further prove the indexability for an arm using Lipschitz bound on subsidy sensitivity with minimal restrictive assumption on the transition probability model and discount factor $β.$ Our results generalize to three state and strengthen existing results on threshold policy and indexability with weaker model assumptions. We provide index computation algorithm and we present comparison of the Whittle-index, myopic policy and round-robin policies in the simulation study.
♻ ★ Quantum ordinal optimization under a fixed comparison budget
Ordinal optimization selects designs that are good enough rather than optimal when evaluation is expensive and the budget is small. The alignment probability measures the chance that the selected set contains at least a prescribed number of truly good designs. We formulate quantum ordinal optimization as a class of problems that maximize this probability under a fixed oracle budget. We study exact, coherent pairwise comparisons with one selected design as a first instance, with the budget counted in comparison queries. For every classical comparison budget smaller than the number of designs, we prove that a knockout among uniformly sampled designs attains the optimum. The sample is drawn without replacement and contains one more design than the number of comparisons. For a uniformly random order, the selected design's rank has a polynomial probability law. At the same comparison budget, the maximum quantum degree is at most twice the classical maximum, a factor attained whenever there are more than twice as many designs as comparisons, so a quantum algorithm can realize rank laws that no classical algorithm with the same budget can. We determine the quantum optimum with one comparison for every number of designs and every good-enough set. For selecting the best of n designs, the optimal success probability with one quantum comparison is approximately 3.30/n for large n, compared with 2/n for one classical comparison. When the best design is sought, one quantum comparison gives a higher alignment probability than two classical comparisons whenever there are more than nine designs.
comment: 22 pages, 4 tables. Adds a fixed-fraction limit and updates the discussion of two-comparison results
♻ ★ A Lyapunov-Based Small-Gain Theorem for Fixed-Time ISS: Theory, Optimization, and Games
We develop a Lyapunov-based small-gain theorem for establishing fixed-time input-to-state stability (FxT-ISS) guarantees in interconnected nonlinear dynamical systems. The proposed framework considers interconnections in which each subsystem admits a FxT-ISS Lyapunov function, providing robustness with respect to external inputs. We show that, under an appropriate nonlinear small-gain condition, the overall interconnected system inherits the FxT-ISS property. In this sense, the proposed result complements existing Lyapunov-based smallgain theorems for asymptotic and finite-time stability, and enables a systematic analysis of interconnection structures exhibiting fixed-time stability. To illustrate the applicability of the theory, we study feedback-based optimization problems with time-varying cost functions, and Nash-equilibrium seeking for noncooperative games with nonlinear dynamical plants in the loop. For both problems, we present a class of non-smooth gradient or pseudogradient-based controllers that achieve fixed-time convergence without requiring time-scale separation and using real-time feedback. Numerical examples are provided to validate the theoretical findings.
♻ ★ Reachability-Augmented Dual Dynamic Programming for Optimal Path Parameterization
Optimal path parameterization (OPP) is a fundamental problem for planning trajectories along a prescribed geometric path under kinodynamic constraints and task-dependent objectives. Among existing OPP works, it remains challenging to simultaneously achieve feasibility guarantees, general-objective optimality certificates, and online-compatible computation in a unified framework, especially for third-order OPP (OPP3) with non-convex constraints. This paper proposes reachability-augmented dual dynamic programming (RDDP), a state-grid-free and objective-aware dynamic programming (DP) framework for OPP. The key idea is to replace the relatively complete recourse assumption used in classical dual DP (DDP) with OPP-specific backward reachable sets, and then generate both value-function cuts and trial trajectories only inside these reachable sets. For convex and non-convex OPP, we prove global optimality and Karush-Kuhn-Tucker convergence of RDDP under OPP-specific conditions, respectively. Efficient instantiations are developed for second-order OPP (OPP2) and OPP3. Experiments show that RDDP achieves objective values comparable to convex-optimization baselines with speedups of approximately 15.3 and 4.9 times over the fastest of 4 tested solvers for OPP2 and OPP3, respectively. RDDP also achieves faster convergence than grid-based DP. Compared with reachability-analysis methods, RDDP retains the reachability mechanism while replacing local maximum-control propagation with value-function-guided control selection, thereby enabling objectives beyond traversal time. In summary, RDDP addresses a key capability gap in OPP by unifying certifiable general-objective optimization, reachability-based feasibility preservation, and online-compatible low-dimensional DP computation in a single framework.
Robotics
★ ThinkWorld: World Imagination Reinforces Chain-of-Dynamics Reasoning
Vision-Language-Action (VLA) models have made substantial progress in robotic manipulation, but their generalization remains constrained by the diversity and coverage of action-labeled robot demonstrations. Human videos provide much broader behavioral and environmental experience, while learning from them faces a fundamental trade-off between scalable data and explicit action grounding. We introduce ThinkWorld, a VLA learning framework that grounds action reasoning in world dynamics and strengthens it through reinforcement learning with world imagination. ThinkWorld represents each action thought as a Chain-of-Dynamics (CoD), a sequence of discrete motion tokens that captures the intermediate dynamics connecting the current world to a future state and serves as a common action representation across embodiments. We first initialize CoD with action-labeled demonstrations, then predict the future induced by each CoD and reinforce those whose predictions better match the observed outcome. This enables videos without action annotations to provide direct supervision from observed state transitions, allowing human and robot experience to refine the same reasoning space. Experiments in simulation and on real robots demonstrate effective human-to-robot transfer and strong generalization to unseen tasks and environments.
comment: Project Page: https://thinkworld-project.github.io/
★ Pushing Dexterity and Safety for In-Hand Object Pose Reaching That Works Remarkably Well
Dexterous manipulation with multi-fingered hands is the next frontier of robotics, yet learned hand skills remain limited to simple grasps or basic repeated motions, such as rotating an object. As robots attempt more demanding in-hand maneuvers, safety, such as not dropping the object, becomes critical, yet it is rarely measured. We study in-hand object pose reaching, a task that demands diverse and difficult grasps: moving an object to almost any position and orientation within reach of the fingers, at any wrist orientation, through long sequences of consecutive goals. We present a recipe for training such policies for a LEAP hand in simulation without human data, built on a range of insights, from diverse resets and goals via generated grasp datasets to an adaptive friction curriculum. With careful system identification, the policies transfer zero-shot to the real hand, using either motion capture or a single wrist-mounted depth camera, and prove remarkably capable and robust. On top of the task policy, we add a learned residual safety filter with adjustable conservativeness, letting us tune the balance between performance and safety and greatly reduce object drops. In 13.3 hours of real-robot evaluation over 4,800 unseen goals across 4 objects and diverse wrist orientations, the state-based policy reaches 95.2% of goals within 2 cm and 0.4 rad, and the filter cuts its drops by 75% while still reaching 91.5% of goals. The policies can also spin a lightbulb held in an arbitrary pose, reach out-of-distribution goals, withstand fast arm motion and perturbations, and, needing only a wrist camera, serve as portable skills such as screwing in a lightbulb. We release the full recipe as a platform for studying performance, reliability, and safety in high-dimensional dexterous systems. Videos and more are available at https://ian-chuang.github.io/dexsafe/.
comment: 26 pages, 30 figures, 14 tables. Project page: https://ian-chuang.github.io/dexsafe/
★ Perceiving and Predicting Dynamic Sports: Spatiotemporal Intelligence in Vision Models for Egocentric Tennis Video
Agile robots require spatiotemporal intelligence to interpret scenes, reason about interactions, and anticipate future events. Yet it remains unclear how well foundation vision models support these capabilities, particularly from a moving egocentric camera, which is a natural view for robots. To investigate this, we collect an annotated tennis dataset and train a suite of vision models on spatiotemporal tasks. Our dataset contains egocentric video synchronized with five allocentric videos, estimated global positions, labeled contact times, and annotated stroke types. We then evaluate seven prediction tasks from egocentric images: 2D ego player and 3D ball localization, future player and ball trajectories, forehand or backhand stroke anticipation, time to the next ego player ball contact, and 3D ball position at that contact. We compare image encoders, video encoders, and vision-language models with jointly trained prediction heads. The best performing models achieve mean current-frame localization errors of 0.14 m for the ego player and 0.54 m for the ball, future forecasting errors of 0.34 m for the ego player and 0.91 m for the ball over 1.1 s, and a mean contact-time prediction error of 0.14 s. Auxiliary depth information substantially improves 3D ball localization, suggesting that inferring metric depth remains a challenge. Longer image histories do not improve ball forecasting, indicating that representations exploit scene context rather than rely solely on inter-frame motion. Finally, we propose a conditional diffusion model to generate whole-body kinematic tennis motion from vision encoder predictions on egocentric video at real-time rates, adapting movements as predictions are refined. These results move towards a deeper understanding of how foundation vision models can support real-time dynamic robots using egocentric video.
comment: 8 pages, 5 figures. Project page: https://tennis-vision.csail.mit.edu/
★ RESPLE2: Inertial-Informed Recursive Non-Uniform Spline Estimation for Multi-Sensor Odometry with Global Refinement
We present RESPLE2, a recursive multi-sensor state estimation framework using non-uniform B-splines for 6-DoF motion representation. By exploiting the knot-clamping boundary property, we propose a theoretically grounded mechanism for exact control-point prediction directly informed by inertial kinematics along the trajectory, with knot placement adapted to motion and measurement availability. Within an iterated extended Kalman filter, we perform this prediction using a subset of inertial measurements. A subsequent reset step preserves trajectory consistency as the knots extend. The update step further incorporates heterogeneous sensor measurements at their native timestamps through continuous-time observation models, supporting flexible combinations of LiDARs, cameras, inertial measurement units, and leg kinematics. We further develop a lightweight global optimization module that directly refines non-uniform B-spline control points by incorporating loop-closure constraints derived from LiDAR-visual cues. We conduct extensive evaluations on public and experimental datasets across handheld, tracked, legged, and hybrid aerial-legged platforms, covering highly dynamic maneuvers with extreme accelerations. Compared with state-of-the-art multi-sensor odometry systems, RESPLE2 achieves competitive or superior estimation accuracy and robustness in the corresponding sensor configurations, particularly under aggressive motion, while maintaining real-time operation. We publicly release our code and experimental datasets at https://github.com/ASIG-X/RESPLE2.
comment: 18 pages
★ EgoThumb: A Vision-Driven Supernumerary Robotic Thumb for Autonomous Grasp and Holding Assistance IROS
Supernumerary robotic fingers augment human dexterity by enabling a single hand to perform complex "hold-and-manipulate" tasks that typically require bimanual coordination. However, existing devices are often hindered by bulky hardware that limits interaction to large objects and rely on explicit hand motion triggers that lack environmental context. To address these challenges, we present EgoThumb, a fully open-source, vision-driven supernumerary robotic thumb. EgoThumb features a compact, dual tendon-driven finger whose actuation is relocated to the forearm, reducing distal weight while enabling precise active joint control. The system utilizes a visual imitation learning policy that maps egocentric images from a palm-facing camera directly to continuous joint positions. By implicitly decoding human intent from visual cues capturing both the hand and the object of interest, EgoThumb achieves robust autonomous manipulation and strong generalization across diverse users and environments. All hardware designs and software implementations are fully open-sourced at https://ego-thumb.github.io.
comment: 8 pages, 5 figures. Accepted to the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
★ Learning Visuomotor Quadrotor Flight Through Dynamic Narrow Gaps via Differentiable Simulation
Threading a quadrotor through a moving narrow gap is a demanding test of agile flight, requiring centimeter-level spatial alignment and precise temporal execution. We address this challenge using a control policy that operates directly on onboard visual and inertial measurements to output collective thrust and body-rate commands. The task is difficult because success lies on a thin feasible manifold: small errors in timing, position, attitude, or visual observability cause collision. Prior planning and learning methods rely on privileged state, teachers, planners, or distillation; learning dynamic narrow-gap traversal directly, from scratch, remains open. We introduce a differentiable moving-gap formulation that gates centering, alignment, and smooth collision terms by signed distance to a time-varying gap plane, turning a sparse traversal event into a dense pathwise objective. We integrate this formulation into a visual-feature-to-command policy trained in one stage from random initialization, without a teacher, planner, or distillation. We validate our approach in both synthetic and real-world experiments. In controlled benchmarks, we demonstrate that pathwise gradients significantly outperform score-function gradients for precision-critical tasks, yielding a 99.3% success rate in simulation after only a few minutes of training on a single GPU. Furthermore, the policy transfers directly to a real quadrotor, successfully crossing a dynamic narrow gap from varied initial conditions using onboard sensing and inertial measurements, achieving an 80% success rate over 10 physical trials.
★ Kinematic Constraint-Referenced Evaluation for Articulated Object Manipulation
Manipulation policies are evaluated predominantly by binary task success, which cannot tell whether the commanded gesture is the one the object's kinematics admit. We introduce Constraint-Referenced Strategy (CRS), which projects the commanded end-effector twist onto the object's joint axis, separating progress along the joint from off-constraint waste. CRS requires a joint model and a joint-following scripted controller derived from it, but no recorded demonstration or learned judge. On 74 simulated articulated objects, success registers three transfer failures as lower rates but attributes none, while the two CRS channels give each a distinct signature. A door-trained behavior-cloning policy opens drawers in 59 of 110 episodes, and in these passes it commands about $40\times$ the off-axis rotation of a drawer-trained policy's passes ($1.27$ vs.\ $0.032$~rad/s). Relative to a scripted controller scored with the same axis, CRS verdicts are unchanged under joint-axis errors up to $30^\circ$; absolute thresholds tolerate about $8^\circ$.
★ When, Where, and What to Correct: Human-Guided Adaptive Residual Correction for Robot Policies
Imitation policies that reliably execute most of a manipulation trajectory can still fail in localized regions requiring last-mile precision. Updating the full policy or exploring residual actions everywhere treats this local failure as a global learning problem. We introduce Adaptive Residual Correction (ARC), a lightweight post-training module that uses reinforcement learning (RL) to learn a residual correction on top of a frozen policy while using sparse human corrections to structure autonomous exploration. Rather than treating human corrections as imitation targets, ARC uses pre-intervention states to determine 'when' residual control should become active and correction geometry to determine 'where' in residual action space autonomous refinement should explore; task reward determines 'what' residual is ultimately executed. Across three precision-manipulation tasks on a real robot, ARC improves average task success from 33.3% to 75.0%. The human corrections define task-dependent 2-3 dimensional subspaces of the seven-joint residual space that retain 96.0-97.7% of squared correction magnitude. Together, these results show that sparse human corrections can localize and constrain autonomous post-training without requiring the observed correction itself to serve as the final action target.
★ Rendering-Free Event Frame Simulation for Fast Reinforcement Learning of Autonomous Robot Navigation
Event cameras offer low latency, high dynamic range, and milliwatt-level power consumption, making them ideal for agile robots such as tiny drones navigating through cluttered environments. However, existing simulators generally render photorealistic images before converting them into events, making large-scale reinforcement learning prohibitively slow. We propose an analytic event simulator that exploits what event cameras fundamentally measure - moving contrast edges - to synthesize event streams directly from scene geometry and ego-motion without visual rendering. Our simulator generates a 64x64 event observation in 17 microseconds on a single GPU - over 400x faster than a state-of-the-art rendering pipeline - enabling end-to-end reinforcement learning for goal-directed obstacle avoidance to converge in under 30 minutes - three orders of magnitude faster than training on rendered events. Policies transfer zero-shot to physically based event simulations (ray-traced Isaac Sim). Our policy reaches 83% success in cluttered courses unseen during training (mid-difficulty, traversability probability approx. 0.57), at 2.4 m/s average goal velocity, and retains 70% in a simulated texture-dense indoor hall. Furthermore, we quantized the policy to a 1.2 MB int8 network running fully onboard a 56-gram Crazyflie equipped with a Prophesee GenX320 event camera and an OpenMV N6 microcontroller board. Without any fine-tuning, external processing, or positioning, the policy achieves an average success rate of 82% (5/5, 5/6, 4/6 flights) across three real-world obstacle courses of increasing difficulty. Code: https://github.com/tudelft/rendering-free-event-nav
★ Predict the Swing, Match the Speed: Robotic Pipe Stabbing With Learned Stereo Vision and Residual Reinforcement Learning
A joint of pipe run into the hole swings from the elevator over the well and strikes a fixed stabbing guide at whatever speed the swing has. StabBot, a robotic stabbing system, predicts the swing and meets the pipe at its own speed. A pendulum Kalman filter tuned from the pipe tally tracks the joint in stereo images. A six-axis arm with an open V-shaped roller head closes in at matched velocity, confirms the last millimetres by touch, aims the pin over the box when the elevator hangs off centre, and follows the block down with lateral compliance. A ResNet-18 keypoint network locates the pipe, pin nose and box mouth in 3-D, and a residual reinforcement-learning policy refines the capture. On 200 paired random jobs with nine sizes from 2-3/8-in. tubing to 20-in. casing, StabBot seated every joint with the pipe touching only rollers, against 74% for a conventional fixed guide. Median contact speed fell from 18.4 to 2.3 cm/s and median peak head-pipe force from 11.3 to 1.4 kN, and the force was lower in all 200 jobs (Wilcoxon $p < 10^{-30}$). The network locates the pipe with 0.4-mm median error in 2.3 ms per stereo pair. With the network in the loop, success stayed at 100% under 20 mm of added noise, 300-ms delay, 85% frame loss, 20-mm calibration offset, 20-cm elevator offset and 40-cm swings. All results come from a 1:4 Froude-scaled MuJoCo model and are reported at full scale.
comment: 18 pages, 9 figures, 9 tables. Submitted to Geoenergy Science and Engineering
★ Which Hand Was Late? Diagnosis for Robot Failure Recovery in Bimanual Tool Use
Failure diagnosers for robot manipulation are increasingly vision--language models (VLMs), trained and evaluated on single-arm failures produced by perturbing a successful demonstration. Bimanual tool use adds failures of other kinds: a tool that slips out of its socket while still held, two correct gestures made at different times, and attempts that fail for two reasons at once. On a simulated bimanual tool-use task whose failure causes are read from the recorded trace, we measure what each sensing channel lets a diagnoser tell. The tool's slip is read on the tool hand; which hand was late is read on both arms, while a sensor on one hand reads its own hand's delay and at most part of the other's; and whether a late attempt was also pushed too weakly is not identifiable from passive sensing when the tool hand is late. Within two attempts, a repair policy that escalates when the cause is not identifiable repairs as many failures as one told the true cause. A VLM's internal state carries the late hand almost perfectly, along a direction no closer to the one its answer reads than a random one; whether the answer reports it depends on the wording of the question.
★ Leveraging Distributed Sensing and Hyperdimensional Computing for Non-Line-of-Sight Passive Object Sensing
In smart factory environments, passive entities such as humans and movable obstacles may interfere with autonomous robot operations, posing significant safety risks. Detecting potential conflicts between robots and entities outside their field-of-view (FoV) is therefore of paramount importance. However, conventional passive object detection systems rely on sensors such as LiDAR, radar, or cameras, which operate under direct line-of-sight (LoS) and are limited by obstacles and occlusions. To overcome these limitations, integrated sensing and communication (ISAC) enables environmental perception via the communication infrastructure thanks to cooperation among distributed wireless devices. In this paper, we propose a framework for passive entity sensing beyond the robot's FoV that relies solely on distributed channel measurements from the communication infrastructure to perform risk-aware classification. These measurements are embedded into high-dimensional representations using hyperdimensional computing (HDC), enabling robust and scalable learning without explicit channel reconstruction. Numerical simulations based on a commercial ray tracer and ROS-based validation in realistic smart factory scenarios demonstrate improved safety and enhanced path planning with up to 89% accuracy under severe occlusions, while maintaining low computational complexity.
comment: To appear in the Proceedings of the IEEE Conference on Integrated Sensing and Communications (ISAC), 2026
★ World Dreamer v5-lite: A Multi-Robot World Action Model with One Dreamer
Multi-robot control requires integrating distributed camera views while predicting separate action sequences. Most vision-language-action and world-action models target one robot. Combining multiple views alone does not establish spatial correspondence between them. World Dreamer v5-lite is a multi-robot world-action model built around a shared world state for robot-specific action prediction. Calibrated features from a pretrained video encoder populate this state in workspace coordinates; its spatial size remains fixed as views are added. A vision-language model provides scene context and one query per robot. The queries read the shared state and exchange pose-aware information, while their action streams remain separate. Masked-view cooperative training withholds selected camera views from both visual branches while retaining all action targets. This encourages the use of observations shared by other robots. Pretraining uses more than 10,000 hours of trajectories from 20 simulated robot types, RoboFactory, and real robots. On RoboFactory, World Dreamer v5-lite achieves 74.0% mean success, 23.7 percentage points above the strongest evaluated baseline. Across four real-robot tasks, it reaches 78.8% mean success, a margin of 13.8 points. It also reaches 58.08% overall on RoboCasa365.
comment: 10 pages, 4 figures. Guoning Wu and Zijian Cai share first authorship. Yongbin Zheng is the corresponding author
★ Ego-CEVR: Learning Cross-Embodiment Visual Representations from Human Egocentric Videos
As egocentric video becomes increasingly available for embodied learning, an important research question is how to turn this low-cost, scalable source of data into robot manipulation capabilities. Egocentric videos naturally provide rich context for high-level task understanding, while real-robot demonstrations with wrist observations and action annotations provide supervision for fine-grained action alignment. We present Ego-CEVR, a data-native framework for hierarchical visual representation learning that assigns distinct learning objectives to global and interaction representations. The global module learns object relations, task state, and temporal progress through future prediction on egocentric video. The Interaction module interprets wrist observations in global context and learns local alignment and contact cues through robot action supervision. Separate residual paths expose both representations to the action policy, preserving direct access to task context alongside fine-grained execution information. Ego-CEVR supports independent pretraining of its global video module on large-scale human egocentric videos and videos from diverse robot embodiments, without requiring robot action labels or paired wrist observations.
★ Autonomous fixed-wing interception using reinforcement learning
Medium altitude, long endurance unmanned aerial systems such as Shahed-type strike drones are an unanswered threat in modern conflicts. Conventional air defense systems are too expensive to provide a feasible large-scale solution. Quadcopter interceptors are a cheap, battle-proven alternative, but their extremely limited flight time restricts them to low altitude targets at short range, whereas fixed-wing platforms have the endurance to reach higher altitude targets from longer distances. However, such interceptors are still operated manually. Recent developments in learning-based controllers show potential for automation. In this work, a fixed-wing interceptor is presented that is controlled by a neural network that maps filtered monocular vision observations directly to actuator commands. The policy was trained in a custom Python environment using reinforcement learning and proximal policy optimization. The policy was then deployed on a fixed-wing platform and tested in real interception scenarios. As a baseline, two classical guidance controllers were also developed and tested in simulation. The policy is able to intercept targets both in simulation and in real flight. In simulation, the policy outperforms the classical controllers under most conditions. The interceptor was ultimately successfully validated in real flight by intercepting maneuvering fixed-wing targets. This novel system shows the potential of artificial intelligence for autonomous control of fixed-wing interceptor drones.
comment: MSc thesis, Faculty of Aerospace Engineering, Delft University of Technology, 2026. 24 pages, 29 figures, 4 tables
★ Electric-Field Induction Motor with Discrete Circuit Elements and Improved Rotor Dynamics
Electric-field-based actuators and machines lead the design space at the nanoscale and in biological systems, while having the potential to be used in favorable macroscale applications which prioritize minimal mass, performance in a vacuum, or performance in high temperatures. Electric-field induction motors consist of a set of stator electrodes which induce opposite charges onto a movable rotor surface. In earlier work, rotor surfaces for electric-field induction motors have been made of single continuum materials, which are limited in possible resistivity, permittivity and circuitry characteristics. To enhance design freedom, we develop a rotor for electric-field-based machines which may be constructed out of discrete circuit components between a finite number of conductive vanes. Magnetic-field induction motors, which are common in machinery, often utilize ``deep bar'' rotors, consisting of conducting bars which decrease the rotor time constant with increasing frequency using the skin depth effect. We propose, analyze, design and demonstrate a macro-scale electric-field induction motor which uses circuitry on the rotor surface to emulate the ``deep bar'' design, providing improved torque over a wide range of speeds.
★ Multi-Agent Reinforcement Learning of Co-Adaptive Hip Exoskeleton Control (MARL-Exo) for Metabolic Energy Reduction
Objective: This study aims to determine whether a hip-exoskeleton policy trained in simulation, by co-adapting with a musculoskeletal human model under a metabolic-energy reward, reduces the metabolic cost of walking in new users without per-user tuning, and how mechanical-power shaping affects the learned assistance. Methods: We developed MARL-Exo, a multi-agent reinforcement learning framework in which a full-body human model (304 muscle actuators; randomized anatomy, gait, and speed) and a bilateral hip-exoskeleton policy are trained concurrently in two stages. The exoskeleton observes only delayed hip kinematics and its past commands; its reward includes muscle metabolic power from a physiologically based energetics model, alone (Metabolic) or with mechanical-power shaping (Metabolic + Power). The Metabolic policy was deployed and tested without retraining in eight healthy adults walking at 1.25 m/s, using indirect calorimetry. Results: In simulation, the Metabolic policy reduced modeled muscle metabolic power by 13.8% relative to no-device walking, versus 8.8% for Metabolic + Power. On hardware, MARL-Exo reduced net metabolic rate by 18.3 $\pm$ 6.6% relative to Passive in all eight participants. Relative to No-Exo, the mean reduction was 7.4 $\pm$ 10.0%; six participants more than offset the 13.1% device-wearing penalty, walking up to 24.2% below their No-Exo cost. Conclusion: A metabolically informed policy co-adapted in simulation transferred to users without per-user tuning and consistently reduced the metabolic cost of walking with the device, while explicit power shaping reduced the simulated benefit. Significance: Unlike per-user human-in-the-loop optimization, this approach yields a single population-level controller and allows assistance objectives to be screened in simulation before human testing.
comment: 14 pages, 6 figures, including supplementary material
★ AuRaSim: A Ray-Tracing Based Automotive FMCW Radar Simulator for Dynamic Scenarios
Modern simulators can generate synthetic camera and LiDAR data with accurate annotations for downstream tasks. In contrast, automotive radar simulation remains less mature because it requires not only a 3D engine to model dynamic objects in the scenes but also electromagnetic material properties, antenna configurations, wave propagation mechanisms, and coherent signal synthesis. Existing frameworks are either proprietary or overly simplified, limiting the scalability of radar raw measurements. This paper presents AuRaSim, a ray-tracing-based automotive frequency-modulated continuous wave (FMCW) radar simulation framework for dynamic scenarios. Rather than implementing all components from scratch, AuRaSim builds on Blender for efficient dynamic-scene construction and Sionna RT for graphics processing unit (GPU) accelerated electromagnetic ray tracing, while introducing radar-specific adaptations for path-resolved propagation with complex coefficients, motion-aware Doppler modeling, and subsequently coherent multi-channel analog-to-digital converter (ADC) cube synthesis based on waveform configurations. Quantitative comparison between simulated and measured range-Doppler maps in digital-twin scenarios achieves a root-mean-square error (RMSE) of 0.147 and a peak signal-to-noise ratio (PSNR) of 16.93 dB, demonstrating the accuracy and fidelity of the synthetic signal. AuRaSim therefore provides a complete and extensible platform for reproducible automotive radar simulation and research. Project homepage: https://wangzhaoze.github.io/aurasim.
★ Learning Quality-Controllable Robot Policies with Iterative Hindsight Relabeling
Robot policy evaluation often relies on a binary success signal for task completion. In real-world deployment, however, desired execution behaviors are also defined by quality attributes such as execution speed and motion smoothness, which need to be dynamically controlled according to task requirements. To address this, we introduce Quality-Controllable Policy (QCP), a robot policy that enables inference-time control over both individual quality attributes and their combinations. To extend the controllable quality range beyond initial demonstrations, we further propose an iterative hindsight relabeling loop: QCP collects online rollouts under target quality labels, adds successful trajectories to the dataset, relabels the accumulated trajectories in hindsight according to their measured quality, and fine-tunes the policy. Experiments on Robomimic and RoboEval demonstrate that QCP consistently improves both task execution quality and success rates across iterations, producing successful rollouts that exceed the quality of the initial demonstrations. Finally, we validate QCP on real-world manipulation tasks, where controlling speed increases throughput in conveyor operations and controlling transport stability improves success when carrying wine glasses, demonstrating that quality-controllable policies yield practical gains in physical deployment.
★ PDRVLA: A Vision-Language-Action Model with Physical Dynamics Representations
Vision--Language--Action (VLA) models generate robot actions from visual observations and language instructions, but their understanding of physical dynamics remains limited. Video world models can capture scene dynamics, yet generating future videos introduces high latency into the control loop. We introduce PDRVLA, whose lightweight Physical Dynamics Module extracts physical tokens from vision--language features to condition action generation without online video generation. The module learns through direct prediction of future RGB, depth, and optical-flow features, with auxiliary supervision through a frozen pretrained video world model. Training relies entirely on publicly available data, expanded through human-to-robot synthesis, subtask annotation, and embodied chain-of-thought (ECoT) annotation. Across four simulation benchmarks, PDRVLA achieves success rates of 99.3% on LIBERO, 89.9% on LIBERO-Plus, 93.6% on RoboTwin~2.0, and 81.5% on SimplerEnv. Ablations show improved future prediction with world-model supervision and evaluate how subtask conditioning, future object-pose supervision, and ECoT training affect action success.
★ VLA-Stem: A Lightweight Safety Mechanism for Vision-Language-Action Models via Reinforcement Learning
Vision-Language-Action (VLA) models integrate high-level reasoning with low-level action execution for end-to-end robotic control. However, most VLA models target static manipulation in unconstrained environments, overlooking spatial constraints and dynamic obstacles. Fine-tuning VLAs on obstacle-avoidance data is a straightforward solution, but collecting such data is costly, especially for dynamic scenarios, and may degrade their original task capabilities. To address these limitations, we propose VLA-Stem, a lightweight, reinforcement-learning-based safety mechanism for VLA systems. Without fine-tuning the underlying VLA policy, VLA-Stem converts its action chunks into motion references and uses trajectory-level guidance to preserve their spatial intent while allowing flexible motion and timing adjustments. Conditioned on these references, robot proprioception, and local obstacle information, the Stem policy adaptively modifies execution at inference time, preserving the original task intent while enabling responsive adaptation to spatial constraints and dynamic obstacles. For robust and sample-efficient training, we combine teacher-student distillation with curriculum learning. We evaluate VLA-Stem on reference following, static spatial constraints, and dynamic obstacle avoidance. Across six constrained simulation settings, VLA-Stem achieves 52.8% average collision-free task success versus 23.7% for the strongest safety baseline, while preserving base-policy performance in obstacle-free settings; real-world experiments further show consistent collision reductions under static and dynamic obstacles. These results demonstrate that VLA-Stem couples high-level VLA task intent with rapid, reflex-like responses to immediate physical risks, providing a brainstem-inspired mechanism for robust robot execution.
comment: 12 pages, 5 figures
★ Generalizable Target-Less Joint Intrinsic-Extrinsic Camera-LiDAR Calibration using Deep Pixel-Point Correspondences
Accurate camera-LiDAR calibration is an essential prerequisite for robust multi-modal perception in robotics. Recent target-less methods based on learned pixel-point correspondences achieve high extrinsic calibration accuracy and generalize across environments. However, they operate on rectified images with known camera intrinsics, restricting their applicability to systems whose cameras have already been individually calibrated. We present a fully automatic, target-less framework that jointly estimates camera intrinsics, radial-tangential distortion, and camera-LiDAR extrinsics from raw images and LiDAR scans. Our method generalizes deep correspondence-based calibration from rectified images to raw cameras through intrinsic- and distortion-aware correspondence learning via synthetic augmentations. During calibration, we iteratively couple correspondence prediction and nonlinear optimization to jointly refine the camera parameters and relative sensor transformation. We evaluate our approach on an unseen camera-LiDAR configuration from the KITTI dataset and on three real robotic platforms in a zero-shot manner. The results demonstrate that joint calibration with our generalized pixel-point correspondence network improves extrinsic accuracy over correspondence-based methods using pre-calibrated cameras, while additionally recovering accurate intrinsics.
comment: 8 pages, 5 figures
★ Tokens of Motion: Primitive Motion Learning for Human-Humanoid Skill Transfer
Vision-language-action (VLA) models provide a promising paradigm for scalable robotic manipulation, yet their reliance on success-only behavioral cloning leaves them brittle; lacking corrective training signals, minor execution errors rapidly compound into unrecoverable, out-of-distribution failures. To address this limitation, we propose Adaptive Failure-Informed Learning (AFIL), an end-to-end framework that leverages failure trajectories as adaptive negative guidance for diffusion- and flow-based VLA policies. AFIL uses a pretrained VLA to generate failure rollouts online, avoiding the need for handcrafted failure-mode design or human-in-the-loop recovery. It then jointly trains Dual Action Generators (DAGs) for successful and failed behaviors while sharing a common vision-language backbone, enabling efficient failure-aware policy learning with limited parameter overhead. During sampling, the failure generator adaptively steers action generation away from failure-prone regions and toward more reliable success modes, with guidance strength determined by the per-diffusion-step distance between success and failure distributions. Experiments across in-domain and out-of-domain robotic manipulation tasks, covering both short- and long-horizon settings, show that AFIL consistently improves task success rates and robustness over existing VLA baselines, demonstrating its effectiveness, efficiency, and generality.
comment: 12 pages
★ Spiking Neuromorphic Control for Periodic Orbits Stabilization in Underactuated Robots: A Greedy Lyapunov-Based Approach
This work investigates the use of event-based spiking controllers, implemented as spiking neural networks, for trajectory tracking in underactuated robots. Specifically, we address the problem of stabilizing a prescribed periodic orbit. It is well known that, by applying the method of transverse linearization to the robot dynamics in a neighborhood of a periodic trajectory, the dynamics can be approximately described by a linear time-varying system with periodic coefficients (a periodic LTV system). The problem of orbital stabilization can therefore be reduced to stabilizing the trivial solution of this LTV system. We show that a previously proposed Lyapunov-based greedy design, originally developed for LTI systems, can also be extended to the corresponding linearized transverse dynamics, which take the form of a periodic LTV system. This enables the construction of a feedback controller for orbital stabilization of the robot. The effectiveness of the proposed method is demonstrated on the trajectory stabilization problem for the butterfly robot.
★ Understanding Human-Robot Collaboration through Joint Action
Human-robot collaboration requires partners to coordinate interdependent contributions toward a shared goal. Joint Action theory provides concepts for understanding this coordination, but how these concepts are instantiated and combined across empirical research remains fragmented. We conducted a systematic review following PRISMA 2020, identifying 83 empirical papers published between 2021 and 2026. Through theory-driven framework analysis, we organized the literature into nine constructs across three capacities: Representation, Prediction, and Integration. We analyze how these constructs are distributed and co-occur across studies, revealing where empirical evidence concentrates and where it is absent. By mapping empirical HRC research onto a unified Joint Action framework, this review offers a coherent overview of HRC studies, identifies gaps in current evidence, and provides guidelines for future research.
comment: 26 pages, 5 figures, 32 tables
★ Vision-Language-Action or World-Action? Just Action Models
The vision-language-action (VLA) and world-action model (WAM) communities debate whether control policies should inherit pretrained knowledge from vision-language models or from visual generators. The debate, however, bundles three separable choices: backbone initialization, the objective applied to future observations, and whether the machinery for that future loss is kept at inference. A gap between a VLA and a WAM therefore cannot simply be attributed to any one of them. We show that decoupling these choices yields a common framework for both families, which we term Just Action Model (JAM): a pretrained visual backbone with an action expert, trained with an action loss and a foresight loss. The foresight loss trains current-observation features to predict frozen encodings of the recorded future through a prediction head discarded after training, so no visual generation is involved in either training or inference, and the backbone can be chosen freely. We first compare objectives within fixed policy architectures, then evaluate the foresight recipe across backbone families. The resulting JAM policies outperform their matched FastWAM and ImageWAM counterparts on LIBERO and LIBERO-Plus, and achieve similar or slightly higher success rates on RoboTwin. Compared with FastWAM, JAM improves the success rates from 97.0%/55.3%/91.2% to 98.2%/64.5%/91.4%, respectively. With cached foresight targets, JAM reduces estimated online training FLOPs per sample per update by 73% relative to FastWAM. Our results show that future supervision improves control without visual generation or additional inference machinery, and strong foresight performance is attainable with either generator-initialized or vision-language backbones. Project Page: https://y0uroy.github.io/JAM
★ KEPT: Conformal Keepouts Kept in Planning for Enclosed-Space Navigation in Degraded Sensing
Autonomous vehicles in enclosed spaces such as tunnels and mines rely on range maps that smoke, dust and fog make inaccurate. Conformal prediction calibrates a keepout around the mapped walls that covers the true unsafe set at a chosen probability. Typically an unmodified planner treats the keepout as an inflated obstacle, and nothing ensures that the vehicle can stop before the keepout as it moves when sensing degrades. In narrow passages this margin and the planner's buffer close the route, and existing remedies recover reach by relaxing the coverage level. This paper conditions the conformal margin on a label-free severity estimate to hold coverage near its target as degradation varies. A planning and control layer enforces one braking-distance condition on the keepout at plan time and at every command, takes the measured tracking residual as its only margin and never alters the keepout. The keepout is validated on ground-robot smoke recordings and the closed loop in simulated quadrotor flights in a foggy tunnel against the standard deployment. In closed-loop simulation the proposed system substantially reduces entries into the calibrated keepout and reaches numerically more goals than the deployment pattern in both calibration regimes, an observed gain of 3.2 percentage points pooled whose per-regime confidence intervals include zero. The per-class keepout holds its coverage near the target on held-out recordings.
comment: 13 pages, 1 figure
★ Retrieve to Act: Motion Primitive Graph Retrieval for Robust VLA Execution
Vision-Language-Action (VLA) policies demonstrate remarkable semantic generalization, yet remain highly vulnerable to out-of-distribution (OOD) visual shifts during execution. While existing test-time methods attempt to suppress spurious visual cues, they fundamentally overlook the rich, uncorrupted kinematic priors embedded in the policy's own training experience. We introduce Retrieve to Act (R2A), a test-time scaling framework that formulates OOD robustness purely as a geometric motion primitive retrieval problem. R2A bypasses fragile visual features by structuring successful training demonstrations into an offline spatiotemporal motion primitive graph. During inference, a query-dependent Graph Neural Network dynamically localizes the robot's ongoing execution within this graph, retrieving structurally compatible future motion primitives. These retrieved geometric priors are then seamlessly harmonized with the frozen VLA's predicted actions via a learned, dimension-wise gating mechanism, adaptively rectifying erroneous spatial trajectories while preserving the policy's local reactivity. Extensive experiments across three state-of-the-art VLA architectures, GR00T N1.7, $π_{0.5}$, and StableVLA, demonstrate that R2A consistently and significantly enhances robustness against diverse contextual perturbations. R2A achieves absolute success rate improvements of up to +12% in simulation benchmarks (LIBERO-Plus, RoboTwin2.0-Plus) and +40% in real-world deployments, without requiring any finetuning of the base policies. Project page: https://retrieve2act.github.io/r2a/
comment: 20 pages, 9 figures, 7 tables
★ Remember Where You Walk: Learning Spatial Memory for End-to-End Legged Robot Exploration
Humanoids and other legged robots operating in complex environments must remember information that leaves their field of view and only becomes relevant much later. Reinforcement learning policies address this using memory networks such as LSTMs that are trained end-to-end with the policy to memorize such task-relevant aspects. Yet, such architectures often fail to preserve sufficient information over long horizons. In this work, we propose a network architecture based on Hadamard memory that specifically targets robot navigation. Our Local Spatial Matrix Memory (LSMM) enables RL agents to store generic task-relevant information at specific, position-based memory cells, enabling flexible and spatially grounded memorization. We evaluate LSMM on the task of autonomous maze exploration, where it matches the maze coverage of strong recurrent and finite-history baselines while being the only architecture that remembers the maze layout, reducing the reconstruction error threefold.
★ A Bi-level Co-design Framework for Lunar Monoped with 2D Jump and Landing Control
Legged robots for lunar exploration must be designed for low gravity, where reduced contact forces and long flight phases make leg morphology and controller choice interdependent. We study this for a planar monoped in lunar gravity ($1.625$ m/s$^2$) using a bi-level co-design framework: a differential evolution outer loop searches over link lengths, and each candidate is evaluated on a vertical jump by either a quadratic program based task-space controller (QP-TSC) or a direct-collocation based trajectory optimization. The two controllers yield different optimal link lengths, and trajectory optimization outperforms the QP-TSC across most of the design space, with 17% lower Cost of Transport (COT) at their respective optima. We then extend to a 2D jump, where trajectory optimization with centroidal angular momentum regulation produces stable in-place and forward jumps, and an LQR controller absorbs impact and stabilizes the robot after touchdown. This is demonstrated on a baseline morphology with both links 0.3 m long.
comment: 8 pages, 13 figures
★ How Much Can Best-of-N Help a World Action Model? Measuring Selectable Headroom and Realizing It in Closed Loop
World action models (WAMs) predict the visual consequences of the actions they generate, which makes test-time selection a natural way to improve a fixed policy: imagine the futures of candidate actions and execute the best one. The benefit of selection is bounded by the selectable headroom, how much better the best candidate truly is, but the standard estimate, which executes each candidate once and keeps the best outcome, conflates candidate quality with the randomness of everything that follows. We introduce a denoised counterfactual protocol, based on bitwise-reproducible branching, repeated continuations, held-out oracles, and common random numbers, that measures the headroom a selector can realize, and use its measurements to build a fail-gated selector. Across four WAMs, two benchmarks, over 100,000 branched rollouts, and pre-registered tests, we find that (i) the standard estimate overstates the selectable headroom of native samples: bounds from 17 continuations per candidate place its selectable share between 24% and 66% for three WAMs; (ii) structured action perturbations create up to 16 points of selectable headroom; and (iii) in both WAMs with action-conditioned imagination, imagined futures carry candidate-outcome information with no detectable loss relative to real futures, information that the value head of Cosmos Policy leaves unused. Deviating from the policy chunk only when a readout of the WAM predicts failure, the selector raises closed-loop task success from 76.9% to 81.6% on held-out episodes of training tasks, with the largest gain where the policy fails most often, and from 62.6% to 69.8% on unseen tasks.
comment: 24 pages, 12 figures, 14 tables
★ Acoustic Signature Optimisation for Bioinspired Flapping-Wing Micro Aerial Vehicles
Flapping-wing micro air vehicles are prized for quiet flight, yet the few existing measurements show engineered wings to be 15-30 dB louder than the animals they imitate. This paper investigates two passive geometric approaches to reduce the acoustic signature of a flapping membrane wing, and applies these to an exemplary wing. A significant sound source in flapping wings is the impulsive click radiated at stroke reversal, when a slack membrane snaps between two curvatures. Our first noise-reduction approach optimises the wing's venation layout, based on an analytical model of the reversal click as a dynamic snap-through of the membrane. The optimised venation pattern was manufactured and resulted in a reduction of up to 8 dB during the reversal snap in experiments. During testing, the venation was found to be sensitive to orientation rather than thickness. Secondly, an additive trailing-edge fringe inspired by hummingbird feathers was shown to further reduce the broadband swoosh by 3-4 dB at no lift cost, matching a cut serration wing without removing lifting area.
comment: 9 pages, 6 figures
★ From 12 to 200 Demonstrations: How ACT, Diffusion Policy, SmolVLA and $π_{0.5}$ Scale on a Low-Cost Arm
We trained four widely used imitation-learning policies, ACT, Diffusion Policy, SmolVLA and $π_{0.5}$, on nested subsets of 12, 25, 50, 100 and 200 teleoperated demonstrations of one pick-and-place task with distractor objects, using one fixed training recipe per policy. Every trained model was then run on a physical SO-101 arm on a fixed set of object placements: 1,340 scored episodes in 126 blocks. ACT, trained from scratch, rose from 1 success in 40 episodes at 12 demonstrations to 37 in 40 at 100. $π_{0.5}$ with its action expert fine-tuned was the best policy when data was scarce (16 of 40 at 12 demonstrations, 28 of 40 at 25) and then stayed near 70%; from 100 demonstrations ACT was ahead, by a margin this study does not fully resolve. SmolVLA and Diffusion Policy peaked near 50%, and both were affected by a failure to start: SmolVLA's arm never started moving in 47% of episodes, and Diffusion Policy hardly ever started until an idle lead-in was trimmed from the demonstrations. For $π_{0.5}$ the fine-tuning recipe mattered more than the amount of data: low-rank adaptation succeeded in 4 of 110 episodes, against 130 of 200 for action-expert fine-tuning. From the joint and camera recordings we place every episode on a four-stage ladder (started, reached the ball, carried it, delivered it) and show that for ACT more data moves failures from reaching, to grasping, to none, while the failures $π_{0.5}$ retains are in grasping. We also measure how weak a ten-episode block is as a measurement (the same checkpoint tested twice differs by 2 successes in 10 on average), separate seed-to-seed variation from that noise, and audit the human scoring against the recordings. Training length was fixed within each recipe, so the amount of data and the number of times it was seen change together; we state which conclusions survive that confound and which do not.
comment: 31 pages, 5 figures, 19 tables
★ FastBEVFusion: Simple Multi-Modal Fusion for 3D Object Detection at 50 FPS
Reliable real-time 3D object detection based on LiDAR and camera data is critical for safety in autonomous driving, where systems must accurately perceive their surroundings with minimal latency. Many state-of-the-art systems achieve high-quality detection results but fail to meet real-time constraints, limiting their practical use. To address this challenge, we propose FastBEVFusion, a simple, fast, and robust fusion system for 3D object detection. Based on modality-specific features, FastBEVFusion projects these features into semi-aligned modality-specific BEV spaces, without relying on a computationally intensive and potentially error-prone depth estimation. Given these BEV spaces, we propose a light-weight, soft, and dynamic fusion module based on cross-attention, which adapts dynamically to the scene context and enables feature fusion across the entire BEV spaces. Experiments on the complex nuScenes dataset show that FastBEVFusion outperforms state-of-the-art real-time systems, and achieves over 50 FPS without deployment libraries. Additionally, our system shows strong robustness to calibration errors, making FastBEVFusion a powerful 3D object detection system for real-world applications.
comment: Accepted at Iberian Robotics Conference (ROBOT2026)
★ MCP-Net: Real-Time Coronary Perception Network and Topological Analysis for Robotic Guidewire Navigation
Coronary guidewire navigation during percutaneous coronary intervention requires continuous interpretation of vessel anatomy, guidewire position, and target stenosis under two-dimensional X-ray fluoroscopy. While existing approaches commonly address vessel segmentation or device localization as separate tasks, a robotic system requires a more structured perception of the coronary scene to support downstream navigation. In this work, we present a simulation-driven framework centered on a Multi-Task Coronary Perception Network (MCP-Net) for robotic guidewire navigation, integrating coronary vessel segmentation, vascular topology extraction, and roadmap generation. Starting from three-dimensional coronary CT angiography, we generate simulated fluoroscopy images containing vascular structures, simulated guidewires, stenotic lesions with varying severity, and diverse vascular geometries, together with simulation-derived ground truth. MCP-Net jointly performs coronary vessel segmentation, guidewire-tip localization, and stenosis localization from a single fluoroscopic frame. These predictions are subsequently converted into a structured vascular graph containing centerlines, bifurcations, endpoints, branch directions, and branch angles. By associating the predicted stenosis with this graph, the framework generates a roadmap from the automatically selected graph root toward the target lesion. The results demonstrate that the proposed framework can jointly recover coronary perception targets and transform them into an interpretable vascular representation suitable for downstream robotic navigation.
★ RAVEN: A Unified Active SLAM System for Rapid Autonomous Visual Exploration with Aerial Robots under Localization Uncertainty
Active simultaneous localization and mapping (SLAM) addresses the problem of planning robot motion to build complete and accurate maps of unknown environments through efficient exploration while maintaining high localization accuracy. This raises a dilemma between exploring unknown spaces to expand coverage and exploiting visited regions to perform active loop closures. Despite numerous prior works on ground robots, aerial robots with high agility and constrained sensor suites face more significant localization drift and thereby demand dedicated active SLAM planning. Moreover, a unified strategy that adapts performance based on task-specific priorities remains an open question. In this article, we introduce RAVEN, a unified active SLAM framework for Rapid Autonomous Visual ExploratioN using quadrotors, which systematically accounts for localization uncertainty. The framework starts with identifying frontiers and active loop closure candidates, from which global tour candidates are generated. These candidates are evaluated using a utility vector that separately quantifies exploration efficiency and localization accuracy, and the final tour is selected according to a prioritization parameter ranging from $[0,1]$. Finally, a perception-aware navigation planner generates smooth execution trajectories that minimize visual state estimation error based on the online Fisher information field that provides visual localization information in real-time. Benchmark evaluations demonstrate that RAVEN outperforms state-of-the-art active SLAM methods, highlighting its advantages in producing a spectrum of performance across prioritizations. Fully onboard field experiments with a customized stereo-inertial quadrotor further validate its practical applicability in challenging environments. The source code will be released to benefit the community.
comment: Conditionally accepted to the IEEE Transactions on Robotics
★ GAWM: A Compact Latent World-Action Model for Multi-Task Robot Manipulation
World-action models (WAMs) incorporate the prediction of future observations into robot policy learning. However, most WAMs build on billion-parameter vision-language or video-generation backbones, which incur high training cost and inference latency. In this letter, we present GAWM (Geometry-Aware World Model), a compact multi-task WAM that predicts future observations in the latent space of a frozen DINOv3 encoder. GAWM compresses the features of each camera view into a small set of tokens. A lightweight predictor, conditioned on the task rather than on actions, forecasts these tokens at two future horizons. The forecast is supervised with the dense patch features of future frames, which requires it to encode the spatial layout of the future scene. An action decoder directly regresses a chunk of future actions from the current and predicted tokens. With fewer than 100M parameters and no robot or video pretraining, GAWM outperforms billion-parameter WAMs on RoboTwin 2.0 and performs comparably to billion-parameter VLAs on LIBERO. On a dual-arm robot, it achieves lower action-prediction error on held-out demonstrations than a fine-tuned billion-parameter VLA, with substantially lower inference latency. Ablation studies show that the latent world model substantially improves performance when training data or training time is limited. Code is available at https://github.com/gaoxiang12/ga-wm.
★ State estimation for a hybrid rigid-continuum space manipulator system
Spacecraft-mounted robotic manipulators can support long-term operability of orbiting assets by conducting tasks such as on-orbit assembly, repair, upgrade, and refueling. In this context, the integration of continuum/soft robotic mechanisms into space manipulator designs promises to reduce stowage volume and mass, as well as to improve operational safety through the inherent mechanical compliance of soft structures. This work presents a novel state reconstruction pipeline for a hybrid soft-rigid space manipulator, in which the distal link is a tendon-driven continuum robot. The estimation problem is formulated as a maximum a posteriori (MAP) inference problem over the hybrid manipulator's configuration, conditioned on discrete, noisy measurements from joint encoders and a vision-based sensing pipeline. The resulting soft-constrained optimization problem is transcribed as a factor graph, providing a compact representation for fusing heterogeneous measurements with the rigid-soft kinematic model. Furthermore, the adoption of the Geometric Variable Strain (GVS) approach yields a minimum-order parameterization of the configuration, a property expected to facilitate model-based feedback control design in future work. The proposed implementation is validated in simulation on a closed-loop trajectory where, for better realism, position measurements for the continuum link are obtained by means of a simulated visual pipeline leveraging synthetic photorealistic images. The state reconstruction accuracy and computational performance confirm the potential of the proposed framework for hybrid soft-rigid robotic applications in orbit.
comment: 10 pages, 9 figures. Version submitted to "Soft Robotics"
★ Structure-Actuation Co-Design of All-Electric Heavy-Duty Robotic Manipulators
This paper presents a mixed discrete-continuous multiobjective bilevel framework for the structure-actuation co-design of all-electric heavy-duty robotic manipulators. The upper level optimizes manipulator geometry and actuation topology, while joint-specific lower-level searches construct feasible archives of discrete motor, gearbox, and screw configurations. A system-level coordinator selects actuator packages according to hardware mass, co-design cost, and loss-aware supply energy, and the resulting actuator masses are fed back to the robot model until a mass-consistent fixed point is obtained. Evaluation on a full-pose SE(3) benchmark with a 100-kg tool payload yields eight feasible nondominated designs, all satisfying the 12 prescribed poses. Actuator-mass coupling converges in three iterations for every reported design. A fixed-design ablation shows that one-pass sequential sizing changes hardware mass by up to 2.13%, co-design cost by 10.20%, and supply energy by 9.40% in magnitude. Payload screening further reveals increasing mass-cost-energy demand and changes in competitive mechanism topology with carried load.
comment: 6 pages manuscript submitted to a conference (ICCMA 2026)
★ Omni-IGN: Onboard Image-Goal Navigation for Drones with Omnidirectional Vision
Image-goal navigation (ImageNav) requires an agent to reach a location specified by a goal image in an unknown environment. For drones, this task demands exploration, goal- directed navigation, and collision avoidance under limited onboard computing resources. Existing aerial ImageNav methods are built on single-view observations, requiring additional rotation and exploration to search for goal-relevant cues outside the field of view. In this paper, we present Omni-IGN, a fully onboard ImageNav system that integrates quad-fisheye perception, an attention-based reinforcement learning (RL) policy, and a collision-aware safety layer. The policy adaptively fuses goal-relevant features across multiple views to provide directional guidance for continuous navigation. The quad-fisheye RGB images support both ImageNav and obstacle avoidance without a dedicated depth sensor. On Habitat benchmarks, Omni-IGN achieves the best performance among continuous-action baselines. Moreover, flight tests in IsaacSim validate the effectiveness of integrating learning-based depth estimation with the action safety layer for collision avoidance. Finally, real-world experiments demonstrate zero-shot sim-to-real transfer without any fine-tuning or external localization. The code and model checkpoints will be released upon acceptance.
★ Keeping Contact: Task-Adaptive Friction Margins for Bimanual Non-Prehensile Manipulation
Maintaining contact in non-prehensile manipulation requires more than nominal friction cone feasibility: a force barely inside the cone and one well inside it are both feasible, but they provide different margins against slip and contact loss. This paper uses signed friction distance to measure friction margin in force space and derives a task-adaptive margin from the force correction needed to balance pivot moment deviations in bimanual tilting. The resulting contact margins are embedded in a sequence of second-order cone programs (SOCPs), each giving a globally optimal contact force solution at a tilt angle. The optimizer balances the object about an environmental pivot while keeping robot and pivot contact forces away from the nominal friction boundary. Optimizer-level results show interpretable force redistribution and adaptation to object property changes. Real-world experiments demonstrate successful execution under object property variations and model mismatch, while fixed margin and no-margin comparisons illustrate the importance of adapting the margin to task demand. These results support signed friction distance margins as a force planning mechanism for maintaining contact in bimanual non-prehensile manipulation. Project page: https://keeping-contact.github.io/
comment: Accepted for publication in IEEE Robotics and Automation Letters (RA-L)
★ ActiveAlign: Aligning Egocentric Manipulation via Active Perception
Real-to-Sim-to-Real (Real2Sim2Real) enables structured world representations for planning, interpretability, and safe execution. Most existing approaches rely on passive perception, which creates real2sim gaps due to imperfect observation. Inspired by how humans perceive actively through actions, we propose ActiveAlign, a framework that leverages active perception to align the Real2Sim2Real gap from interactions. ActiveAlign introduces two complementary modules: (i) ActiveAlign-Waving, which performs hand-eye calibration by exploiting active hand motions and differentiable rendering, and (ii) ActiveAlign-Touching, which refines object alignment through tactile feedback to achieve millimeter-level precision. Together, these capabilities enable strong real2sim alignment. We demonstrate that ActiveAlign-Waving can be used to curate simulation environments for open-sourced uncalibrated robot manipulation datasets, and demonstrate on a real Unitree G1 robot that ActiveAlign enables fine manipulation of millimeter-scale objects such as pressing buttons and moving sliders.
★ When Black-Box Robot Policies Fail: Online Sensor Diagnosis and Repair with a Transparent Surrogate SP
Robot foundation models may generalize across tasks and embodiments, but their actions still depend on sensors that can fail. When the policy is frozen or inaccessible, can an external monitor diagnose and repair corrupted inputs without access to its internals? We test a structured surrogate that separates periodic gait from sensor-dependent corrections to guide input repair around a frozen locomotion policy. On simulated Go2, offline diagnosis raises walking success from 259/640 to 429/640 episodes; online repair reaches 382/640, with no triggers in 64 held-out fault-free episodes. A matched boosted-tree surrogate predicts clean actions similarly well but yields no detected repair improvement over no repair. Separate fits improve walking on Go1 and ANYmal C, although phase-aware gradients perform better. Evidence remains limited to one policy and a partly screened simulated fault suite; robot foundation model and hardware validation remain open.
comment: Accepted at the Conference on Robot Learning 2026 Workshop on the Science of Physical AI Safety (SPAIS)
★ Trace-WAM: Learning to Act via 3D Motion Prediction for World Action Models
World Action Models (WAMs) learn representations for robot control by coupling future visual prediction with action generation. Successful manipulation depends primarily on how the robot and task-relevant objects evolve, whereas future-video prediction must also reconstruct the appearance and background of the entire scene. We therefore propose Trace-WAM, which augments video-action learning with future 3D motion prediction as an auxiliary objective. This objective supervises temporal point correspondence together with image-plane displacement and depth change, directly describing the interaction dynamics and spatial changes involved in manipulation. To this end, we adopt a three-branch Mixture-of-Transformer (Tri-MoT) architecture to jointly learn video prediction, motion prediction, and action generation. The motion expert predicts future trajectories from current visual features, allowing motion supervision to shape the representations used for action generation. Because the action expert does not consume predicted trajectories, inference retains the current-observation policy path: a single visual forward pass followed by action denoising. Experiments on RoboCasa-GR1, LIBERO, and LIBERO-Plus demonstrate strong in-distribution performance and out-of-distribution generalization, with particularly clear gains over Fast-WAM under limited demonstrations and distribution shifts.
comment: 11 pages, 6 figures. Project page: https://trace-wam.github.io/
★ ExecDiffuser: Closing the Sample-to-Execution Gap in Diffusion Planning
Generative planners are often evaluated by the trajectories they can generate, although closed-loop driving ultimately depends on the single candidate committed to control. We identify this mismatch as the sample-to-execution gap and show that improving proposal quality alone can worsen the executed decision. Here, we introduce ExecDiffuser, a recurrent proposal-to-execution framework that couples candidate generation, trajectory selection, and subsequent replanning through shared execution semantics. Intent-Guided Diffusion Proposal (IGD) assigns hypothesis responsibility and shapes trajectories with execution-conditioned supervision. Execution-Grounded Selection (EGS) learns a candidate-conditioned publication utility, commits one trajectory to control, and returns it as context for the next planning cycle. Across nuPlan, InterPlan, and NAVSIM, ExecDiffuser achieves leading closed-loop performance against strong state-of-the-art baselines. Under a unified five-seed protocol, it obtains an average score of 85.62, improving over Diffusion Planner and FlowDrive by 4.02 and 1.31 points, respectively. On NAVSIM, it achieves 92.6 PDMS under the official evaluation pipeline. Matched counterfactual analysis further shows that ExecDiffuser more reliably converts strong candidates into committed decisions, while prefix interventions confirm that this advantage persists after physical execution and subsequent replanning.
★ HC2Robot: Hyper-Connection Transformers with Predicted Foresight for Multi-Terrain Humanoid Parkour
Humanoid parkour requires robots to reason about future footholds and whole-body motions over highly discontinuous terrain. We revisit the information asymmetry in asymmetric actor-critic learning for perceptive humanoid locomotion: the critic exploits privileged simulator signals during training, whereas the deployable actor operates solely from onboard perception. We argue that the key is not exposing privileged information to the actor, but learning a deployable approximation of it to narrow the information gap between training and deployment. We present HC2Robot, a deployable perceptive humanoid parkour framework consisting of a Predicted Foresight Module (PFM) and a Hyper-Connection (HC) Transformer. PFM predicts from deployable observations the policy-conditioned timing, location, and support quality of each foot's next touchdown event, converting privileged supervision into a deployable latent representation that provides the actor with predicted touchdown information without privileged runtime sensing. The HC Transformer further enhances temporal reasoning through learnable multi-stream routing, while a future motion prediction head provides short-horizon motion priors for regularization. Across stairs, gaps, high platforms, and slopes, HC2Robot achieves a mean simulated traversal success rate of 97.60% and consistently higher safe foothold rates than strong perceptive locomotion baselines, supporting improved traversal performance and geometric foothold support under deployable sensing.
★ TAPESTRY: A Single Backbone for Video Generation and Robot Control
Pretrained video generation models encode rich priors about physical interactions, but leveraging them for robot control typically requires either resource-intensive video generation at test time or action heads trained from scratch that do not fully utilize the pretrained backbone. We introduce Tokenizing Actions as Video Patches in a Shared Transformer Policy (TAPESTRY), a world action model that repurposes a pretrained video diffusion transformer into a robotic policy without requiring video generation at inference. Our key idea is to treat robot actions as single-patch tokens in the video model's latent space. A lightweight linear projection maps each action timestep to the patch dimensions of the pretrained VAE, and the resulting tokens are processed by the same patchifying layers and transformer backbone used for video. The shared backbone is optimized through three diffusion objectives sampled per minibatch element: Video diffusion uses VAE latents from robot and human videos to establish dynamics priors. Joint video+action diffusion trains on robot demonstrations using a coupled diffusion timestep and an asymmetric attention mask. Action-only diffusion denoises actions conditioned on clean image latents. At test time, the model generates a compact action token sequence in a single DDIM step from pure noise, producing smooth action chunks at 84 ms per action chunk on consumer hardware and avoiding the cost of full video generation. Across simulation benchmarks and real-robot experiments, TAPESTRY achieves strong performance with substantially lower inference cost than comparable joint video--action policies. More information can be found at https://tapestry-wam.github.io.
comment: Accepted to Conference on Robot Learning (CoRL) 2026
★ Integrated Design and Reproducible Feasibility Analysis of a Four-Wheel Rover with Packet-Commanded Imaging, LiDAR Ranging, and Learning-Assisted Target Recognition
This study develops an integrated, reproducible design assessment for a four-wheel educational rover intended for the high-school remote-control division of the 2027 NASA Human Exploration Rover Challenge. The architecture combines commercial differential drive control, independently sprung wheel carriers, separate drive and task batteries, and a Raspberry Pi-based subsystem that executes received imaging and ranging commands. The analysis reconciles component lists with dimensioned computer-aided design views, derives mobility and electrical constraints, and connects sensor geometry to command execution. At the estimated mass of 10.919 kg, a 215 mm wheel rotating at 200 RPM produces a theoretical rolling speed of 8.105 km/h; consequently, the nominal motor label does not establish compliance with an 8 km/h design ceiling. A steady 20-degree climb requires at least 1.255 newton-meters per wheel under an assumed rolling-resistance coefficient of 0.10 and equal torque sharing. At 50 ft, an ideal 12-inch feature spans approximately 71 pixels across a full-resolution Camera Module 3 Standard image, while a 1-degree pointing error displaces the ranging ray by approximately 266 mm. An executed synthetic benchmark compares color-geometry rules and two linear support-vector classifiers on 2,400 generated board crops with scene-disjoint evaluation. A separate reference command executor is checked using mocked hardware. These software results and analytical sensitivities support a testable development plan; the study does not report physical rover trials. The principal contribution is a traceable connection between component choices, failure mechanisms, quantitative design limits, and reproducible verification tasks.
comment: Research Manuscript for NASA HERC 2027
★ Gradient flow of magnetic inclination and intensity as a GPS-free guidance mechanism
It is well established that some animals, such as sea turtles, sense the Earth's magnetic field and use it as a primary cue for long-distance navigation across the vast and featureless ocean. This remarkable ability has inspired the development of bio-inspired magnetic navigation (BIMN), which seeks to mimic these animals to develop robust and frugal navigation methods. BIMN exploits the Earth's magnetic field, sometimes described as ''Nature's GPS'', as a natural navigation infrastructure, providing a potential alternative to vulnerable satellite navigation systems. However, unlike geographic coordinates, magnetic coordinates extracted from the Earth's magnetic field do not provide a one-to-one mapping to geographic locations. Different locations can exhibit similar magnetic signatures, creating ambiguity and making BIMN inherently more challenging. Despite this fundamental challenge, we demonstrate that, within predefined regions of the Earth, magnetic inclination and intensity provide sufficient information to navigate to any destination. By formulating BIMN as a gradient flow control and guidance problem, we show that the resulting gradient flow act as current that carries the system and can be controlled to guide it through intermediate waypoints toward the desired destination. The resulting framework provides a GPS-free navigation mechanism with potential applications in both engineering and biology. In engineering, it can be used to establish a geomagnetic navigation infrastructure through which prescribed routes can be defined and followed, while in biology, it can be used to analyse the migration routes of animals known to rely on magnetoreception.
★ RynnWorld-Latent: A Cross-Embodiment World Model Conditioned on Universal Latent Actions
Learning controllable world dynamics from heterogeneous interaction videos is challenging because robot embodiments use incompatible action spaces, while human videos lack native control labels. We present RynnWorld-Latent, a cross-embodiment video world model that uses the 3D-grounded latent actions introduced by RynnLAM as a shared conditioning interface. This representation enables joint pretraining on human interaction videos, robot demonstrations, and simulation rollouts without requiring a common native action space. Built on a pretrained video transformer, RynnWorld-Latent combines global action tokens with an identity-initialized local pathway that aligns actions with future video latents. We pretrain on 23 datasets comprising approximately 553.5M frames and adapt the model to target robots through post-training. The framework supports two data-generation routes: digital teleoperation uses target robot commands to generate action-video demonstrations, while latent-action synthesis transfers source motion to an edited initial frame and pairs the resulting rollout with retargeted pseudo robot actions. Experiments show improved video prediction metrics on public benchmarks and two real-robot platforms, alongside qualitative action following when different action sequences share the same initial observation. Synthetic mid-training improves policy success rates on LIBERO and LIBERO-Plus, and both generation routes improve real-world policy success rates over real-demonstration-only training in the evaluated settings. These results support universal latent actions as a scalable interface for learning controllable visual dynamics across heterogeneous embodiments.
comment: Homepage https://alibaba-damo-academy.github.io/RynnWorld-Latent.github.io, Github https://github.com/alibaba-damo-academy/RynnWorld-Latent
★ A fully articulated humanoid dexterous hand with human-level joint structure enabled by linkage-based actuation
Humanoid dexterous hands are a core component of embodied intelligence, enabling robots to translate high-level intent into rich and adaptive physical interaction. However, existing designs remain constrained by a persistent trade-off among anthropomorphic kinematic complexity, mechanical robustness, load capacity and control simplicity, limiting dexterous manipulation in unstructured environments. Here we present Torch-Hand, a fully articulated humanoid dexterous hand that replicates the joint structure and motion hierarchy of the human hand with 21 degrees of freedom (DOF), comprising a highly articulated five-DOF thumb and four DOF in each of the remaining fingers. The differentiated thumb integrates multi-axis CMC motion to provide broad opposition and flexible contact orientation relative to the four fingers. Unlike tendon-driven or hybrid actuation approaches, Torch-Hand adopts an entirely linkage-based transmission architecture. By integrating screw-linkage actuation, the hand achieves high grasping force and substantial payload capacity while maintaining a compact anthropomorphic form. Its actuation architecture is optimized for control tractability, with all actuators integrated within the palm and arranged in a serial-robot-like configuration that establishes a direct correspondence between motor motion and joint kinematics. This architecture mechanically decouples functional DOF to reduce unintended inter-joint motion propagation, while allowing independently addressable DOF to be actively coordinated into coupled multi-joint motion patterns during complex manipulation. Comprehensive experiments assess reachable workspace, coordinated fingertip trajectories, thumb opposition, 33-grasp taxonomy coverage, grasping force, payload capacity and task-level manipulation. These results establish Torch-Hand as a robust hardware foundation for dexterous embodied manipulation.
★ Making Self-Driving Laboratories Auditable:A Unified Framework for Hybrid Actions,Measurements, and Safe Execution
Self-driving laboratories (SDLs) are laboratory systems that couple experimental planning, automated execution, and measurement feedback in a closed loop. Integrating flexible robot control with the typed application programming interfaces (APIs) and continuous measurements exposed by laboratory instruments remains an execution-interface challenge. To address this challenge, we propose a traceable framework comprising four components: LAM (Lab Action Model), OpenSDL, Safe-SDL, and L3-Bench. LAM is a proposed measurement-conditioned policy architecture that extends vision-language-action paradigms to generate both continuous robot trajectories and typed instrument API calls. OpenSDL is an execution substrate with an initial control implementation, designed to map role-level operations to heterogeneous devices, coordinate execution, and record timestamped causal events. The proposed Safe-SDL runtime gate combines pre-dispatch schema and resource checks with signal temporal logic monitoring during execution, including extended waiting periods. L3-Bench (Level-3 Laboratory Benchmark) specifies an episode-level evaluation protocol covering physical outcomes, timing, reproducibility, safety violations, and fault recovery. Together, these components provide an end-to-end design for linking hybrid action generation, monitored device execution, measurement feedback, and experimental evaluation, addressing coordination and traceability challenges in SDL workflows.
★ Learning Side-by-Side Navigation for Personal Mobility Vehicles from Human Drivers
This paper presents an approach to achieve side-by-side autonomous motion with a pair of personal mobility vehicles (PMVs). Given a global path, two PMVs plan cooperatively so as to hold a social formation that lets the passengers hold a conversation, while keeping the ride safe and smooth. Each vehicle runs its own sampling-based model-predictive planner; the planners exchange nothing but the odometry each vehicle could observe of the other, and the joint decision is expressed as a pair of utility matrices over pairs of candidate trajectories, resolved by a common rule. The behavioural parameters are fitted by inverse reinforcement learning from 409 minutes of driving by ten pairs of people in human-driven PMVs in an indoor environment, and the lanes the pair can use are derived offline from the corridor itself rather than fixed by hand. A safety layer the cost cannot override executes a move only if the vehicle can still brake clear, measured from its whole body, and a vehicle that is stuck reverses along its own path and continues. In closed-loop simulation over 24 runs in the conditions the drivers were recorded in and 24 in four generated obstacle layouts, no vehicle entered a wall or a cone. The pair holds conversational formation for 60.5 and 68.9 percent of the time among obstacles (anticlockwise and clockwise) and 81.2 and 91.5 percent in a clear aisle, against the drivers' 34.0 and 36.0 percent, and 52.9 and 56.0 percent, and it matches their speed. It keeps one lateral formation where the drivers alternated between a close diagonal and a wide abreast one, and it varies far less. Experimental results are based on a differential drive kinematic simulator running on ROS 2.
★ On Shaky Ground: Characterising Non-Rigid Base Effects in Mobile Manipulation
Mobile robot manipulators extend contact-task capability beyond the workspace of fixed robot installations, but their bases must also sustain the forces generated by the task. On lightweight platforms, base motion and support compliance can change the force and motion realised at the workpiece. How these non-rigid base effects manifest during sustained contact remains insufficiently characterised. This paper investigates non-rigid base effects in mobile robot manipulators through a sanding case study combining exploratory simulation with experiments on two lightweight, unactuated wheeled testbeds. Joint measurements of contact force, tool motion, and base vibration show that sustained contact coexisted with substantial task error: under low base-ground resistance, only 54-70% of a 100 N command was realised, with tool-centre-point errors reaching 100 mm. Increasing base-ground resistance during contact improved force realisation and arrested further tool displacement, while base vibration did not vary consistently with force-tracking error in the tested conditions. These observations illustrate how non-rigid base behaviour can affect contact-task execution and highlight the value of examining force, motion, and vibration together when characterising lightweight mobile manipulation.
comment: 10 pages, 7 figures, 3 tables. Accepted at the Australasian Conference on Robotics and Automation (ACRA 2026)
★ TOPS: Reshaping Flow Source and Transport for Multi-Task Robot Manipulation
Flow-based policies for language-conditioned multi-task manipulation commonly share a single $\textit{Gaussian}$ source prior and a common linear transport path across tasks. However, reliably executing multiple tasks with distinct motion requirements remains challenging for a shared policy. We introduce $\textbf{TOPS}$ ($\textbf{T}$ask-$\textbf{O}$riented $\textbf{P}$riors and $\textbf{S}$chedules), which adapts the flow source prior and interpolation schedule to each task without modifying the network architecture. TOPS first constructs multimodal task sources from reverse-mapped expert action sequences, allowing language instructions to select where generation begins. Guided by source-target covariance mismatch, it then slows the transition toward expert actions early in denoising and accelerates it later along directions requiring expansion. Under a $\textit{Gaussian}$ endpoint model, our analysis shows that this schedule adjustment can lower an upper bound on velocity-prediction risk. Experiments on LIBERO, LIBERO-Plus, and CALVIN demonstrate improved performance across in-distribution, perturbation and cross-environment settings, alongside better data efficiency. Using one integration step, TOPS raises LIBERO-10 success from $53.44\%$ to $69.67\%$ compared with ten-step Standard Flow Matching (FM), while reducing inference time to approximately one-sixth of the baseline. TOPS also yields consistent gains when transferred to the large-scale flow-based $π_{0.5}$ policy across all four evaluated settings. Real-world experiments further demonstrate improved task success, including an increase from $20\%$ with standard FM to $80\%$ with TOPS on the upper-drawer opening task.
comment: Under review
★ RoboSTD: Zero-Shot Single-to-Dual Transfer via Sagittal Mirroring for Bimanual Learning
Bimanual robot manipulation plays an important role in real-world robotic applications, but the high cost of collecting bimanual demonstrations limits the scalable development of bimanual policy learning. Meanwhile, existing single-arm manipulation data are difficult to directly transfer to bimanual systems due to the Single-to-Dual gap in action spaces, coordination requirements, and arm-side spatial preference. To address this issue, we propose RoboSTD, a zero-shot Single-to-Dual data construction method for bimanual robot learning. RoboSTD first applies sagittal-plane mirroring to observations, proprioception, and actions to construct executable contralateral pseudo-demonstrations. It then uses an LLM to generate structured spatio-temporal coordination constraints that guide the reconstruction of pseudo-bimanual supervision across workspace regions and task stages. We evaluate RoboSTD on the AgileX ALOHA platform in both RoboTwin simulation and real-world settings. Experiments demonstrate the effectiveness of RoboSTD for bimanual policy learning. Without requiring additional bimanual demonstrations, RoboSTD increases the average number of collected cups to 2.11x that of the single-arm baseline in the cup collection task. Project page: https://huahungy.github.io/RoboSTD/
★ From Biological Curves to Soft Modular Robots: Learning-Based Inverse Design with Uncertainty-Guided Refinement
Animals exhibit diverse shape-changing behaviors, from the undulatory swimming of eels to the arching movements of inchworms. Reproducing these biological shapes with modular soft robots offers a promising approach to developing versatile bioinspired robotic systems. However, translating a desired shape into a feasible robotic configuration remains challenging. Existing cable-driven modular soft robots rely on computationally expensive physics-based modeling and combinatorial search for inverse design, making it difficult to efficiently reproduce diverse target shapes. To address this challenge, we present a learning-based real-to-sim-to-real framework that translates biological curves into robotic configurations through surrogate-assisted inverse design and uncertainty-guided refinement. The proposed method combines learned inverse prediction with targeted candidate exploration, using a fast forward surrogate to evaluate and select suitable designs. The learned forward surrogate reduces deformation prediction time from $1.53 \times 10^5$ ms to 0.0008 ms, achieving approximately a $1.9 \times 10^8$-fold speedup. At the same time, the proposed inverse design method reduces design time from $1.99 \times 10^6$ ms to 4.18 ms. On 5,000 test samples generated from previously unseen threading patterns, the method achieves a 93% shape reconstruction success rate under surrogate-based evaluation. Physical demonstrations with four bioinspired robots further illustrate the translation of biological curves into realizable robotic shapes.
comment: 8 pages, 8 figures, 2 tables
★ SWIM: Learning What Matters from Behaviors for JEPA World Models
Joint-embedding predictive architectures (JEPAs) enable world models to plan from images by predicting future embeddings. However, their visual prediction objectives do not explicitly incorporate task-relevant state information, so the physical state that determines task progress may not be reliably captured in the latent space. Supervising world models with simulator state can make planning more informative, but it requires deciding which state features matter. Features are typically selected by hand or weighted uniformly, and a uniform weighting gives every standardized state feature equal influence, so redundant coordinates can overcount the same variation, while irrelevant ones can obscure task-relevant differences. To address this limitation, we propose SWIM (State Weighting with Inverse-Dynamics Model), which learns state weights from offline trajectories by predicting recorded actions from current states and observed future displacements. The learned weights quantify each feature's contribution to action prediction and define a weighted state metric for state-supervised training, which we demonstrate by aligning latent and state distances. The inverse-dynamics model is used only during training, so the resulting world model plans from images with no additional planning-time computation. Across five simulated robotic manipulation tasks, SWIM consistently improves planning success over uniform state weighting.
comment: 25 pages, 1 figure
★ Polarity--Time Interaction and Multi-Level Reliability for Event-Based Spacecraft Pose Estimation
Sparse event responses produce uneven keypoint errors whose effects on spacecraft pose depend on correspondence geometry and the reliability of successive observations. We present a reliability-aware framework connecting polarity--time localization, geometric recovery, event--CAD refinement, and trajectory optimization. A three-branch representation combines positive events, negative events, and recent-event timing through cross-branch attention and gating, followed by spatiotemporal aggregation. An ordered semantic-point head estimates relative correspondence precision for geometrically constrained pose recovery. Event-edge and CAD-projection agreement then guide local pose refinement under keypoint-geometry safeguards. At the sequence level, a ten-dimensional descriptor combines heatmap shape, reprojection consistency, and geometric conditioning to set time-aligned observation scales. On the official SPADES challenge test set, the complete system achieves a combined pose score of 0.0474, approximately 5.4\% below PASAN+IRIS. Validation ablations reduce the score from 0.0772 for base localization with uniform PnP to 0.0514 for the complete cascade; on identical single-frame observations, quality weighting reduces the score from 0.0549 with constant-weight optimization to 0.0514.
★ Mimetic Embodiment Transfer of Human-Humanoid Interaction from In-the-Wild Videos
Enabling a humanoid to interact naturally with human partners requires preserving not only individual motions but also the spatial, semantic, and contact relationships underlying the interaction (i.e., hugging, dancing, hand-shaking, etc.). Furthermore, a practical solution should generalize across various humanoid embodiments with distinct morphologies and kinematic constraints. To tackle this challenge, we propose HumiTransfer, a mimetic embodiment transfer framework that converts in-the-wild human-human interaction videos into physically plausible human-humanoid interactions. HumiTransfer first reconstructs 3D human motion with consistent foot-ground contact, then transfers one participant to a target humanoid embodiment via interaction modeling, using weighted joint-relative directions and anatomical offsets to prioritize interaction-critical inter-body relationships, while post-retargeting optimization enforces balance, contact, and collision avoidance. Subsequently, we apply HumiTransfer to 1,200 in-the-wild videos, producing approximately 500,000 frames of diverse interactions for the 26-DoF Unitree R1 Edu and the 23-DoF HighTorque Mini-hi humanoid robot, resulting in our HumiMimic dataset. Across both robots, HumiTransfer achieves superior motion fidelity, preserves inter-body relationships, and maintains contact accuracy compared with existing baselines. Moreover, we demonstrate that HumiMimic supports motion generation methods for synthesizing interactive humanoid motions following a human partner. Finally, our real robot deployments confirm the physical executability of the transferred motions in real-world settings.
comment: 9 pages, 8 figures
★ Camera-to-Quadruped Robot Pose Estimation from Sequential Images
Quadruped robots are increasingly deployed in hazardous environments such as disaster response and facility inspection, where external-camera monitoring is essential for safe operation. This setting is challenging because quadrupeds are articulated systems with time-varying leg joints, leg symmetry, and frequent self-occlusions, which make keypoint correspondence ambiguous. Moreover, no public benchmark exists, making quantitative comparison difficult. To address these challenges, we propose QuadPose, which estimates the full-body 18-DoF pose (6-DoF base pose + 12-DoF joint angles) from monocular video sequences, and introduce QuadSyn, the first camera-to-quadruped robot pose benchmark. On QuadSyn, QuadPose achieves 92.18% PCK@2.5px, 95.54% PCK-AUC, 85.94% ADD@20cm, 66.98% ADD-AUC, and 3.65 degree mean joint-angle error at approximately 33 FPS.
comment: 17 pages, 8 figures, 6 tables
★ OpenJaws: An Open Hardware UMI Framework with Stereo Localization for Generalist Manipulation
Handheld data-collection interfaces such as the Universal Manipulation Interface (UMI) allow manipulation demonstrations to be gathered in the wild without a robot in the loop, but the quality of the resulting action labels is bounded by the accuracy of vision-based localization. Existing systems recover end-effector pose from monocular visual-inertial odometry, which is subject to scale ambiguity and drift, or delegate tracking to closed-source modules and external infrastructure that reintroduce cost and setup burden. We present OpenJaws, a fully open hardware and software framework for sensorized UMI that improves localization and simplifies deployment across robot platforms. OpenJaws pairs a modular, 3D-printed gripper exposing a standardized ISO adapter with a low-cost stereo camera to recover metric camera trajectories. We further introduce UMI-Loc-Bench-v1, a benchmark quantifying real-world localization accuracy for handheld data-collection devices, and show that stereo localization substantially outperforms monocular alternatives. Finally, we demonstrate that policies trained on data collected with OpenJaws can be deployed on a real robot across multiple tasks. Hardware, software, and data are released under a commercially permissive license. Code, data, and videos are available at https://open-jaws.github.io/
comment: Project page: https://open-jaws.github.io/
★ REACH: An RL-based Hierarchical Humanoid Crawling Framework for Adaptable Confined Space Traversal
Humanoid locomotion in confined spaces requires continuous height adaptation across multiple locomotion modes, yet most existing approaches remain limited to height-adjustable bipedal walking. We present REACH, an RL-based hierarchical framework that enables a Unitree G1 humanoid to traverse height-constrained environments through variable-height bipedal walking and crawling. The learned policies regulate body height over complementary command ranges of 1.27--0.50m in bipedal locomotion and 0.50--0.20m in crawling, extending traversal beyond the clearance limits of crouched walking. To address contact-rich crawling dynamics, we employ a history-conditioned latent representation that captures terrain and interaction context from proprioceptive observations. A bidirectional transition policy further connects crouched bipedal locomotion and crawling across large kinematic and contact discontinuities. These skills are integrated with a LiDAR-based planner that converts point clouds into collision-free three-dimensional paths and locomotion commands, including proactive height targets. Real-world experiments demonstrate accurate height and velocity tracking, robust terrain traversal, reliable mode transitions and confined-space traversal with the proposed pipeline. Code and videos are available at https://aislab-sustech.github.io/REACH/.
comment: To appear at IEEE Humanoids 2026; Project page https://aislab-sustech.github.io/REACH/
★ CARD: Contact-Anchored Retargeting for Dexterous Manipulation
Humans are dexterous, but traditional data collection paradigms such as teleoperation, kinesthetic teaching, and wearable exoskeletons force humans to act through the robot embodiment, bottlenecking their dexterity. Retargeting can avoid this. However, existing methods purely translate kinematics and ignore the real-world contact dynamics essential for task success. We present CARD: Contact-Anchored Retargeting for Dexterous Manipulation, a simulator and object-free framework that retargets human demonstrations to robot actions by explicitly preserving contact force targets. CARD unifies human data sources including video, motion capture, and tactile gloves via a shared hand--contact representation and generates explicit contact force targets in addition to kinematics. To maintain these contact targets during hardware open-loop replay, CARD also introduces a tactile tracking controller. For video which does not contain an explicit tactile source, we present Vid-HACO, our video-to-contact model that sets a new state of the art. Across our video, glove, motion-capture data and the GRAB dataset, CARD maintains contact on $43$--$78\%$ of contact frames, while kinematic and object-aware retargeting methods maintain contact on $<10\%$ and $22.5\%$ of the frames respectively. Further, integrating our formulation into physics-based retargeting improves strict task success from 12\% to 32\% on GRAB dataset. Finally, we show that our tactile tracker is critical to reproduce demonstrated forces during hardware replay. These results establish that contact preservation via CARD is a scalable interface for transferring human dexterity to robots.
comment: Project webpage: https://card-retargeting.github.io/
★ A Trainable CPG Framework for Quadruped Locomotion
We present CPG-ENN, a trainable quadruped locomotion framework that combines explicit neural coordination with learned motion realization. A symmetry-based eight-neuron central pattern generator (CPG) supplies gait rhythms, while a pretrained, leg-shared execution neural network (ENN) maps rhythmic signals and trajectory parameters to joint-angle references without online inverse kinematics. A single feedback policy modulates six physically interpretable trajectory variables. The controller tracks planar velocity and yaw-rate commands across walk, trot, pace, bound, and pronk without gait-specific learned controllers or transitionspecific rewards. Simulation yields 99.39% transition success across all 20 directed gait pairs and all source-target speed combinations from {0.5, 1.0, 1.5, 2.0} m/s. Ablations show high cross-gait transition feasibility without transition exposure; exposure mainly refines gait acquisition and velocity transients. Comparisons with AllGaits and Walk These Ways demonstrate accurate gait-speed regulation, and Unitree Go2 experiments validate five gaits and their transitions.
★ A generalist policy for muscle-driven control across individual musculoskeletal models
Differences in musculoskeletal morphology challenge policy reuse across individual models. Here we present MyoAcross, a framework that combines individual musculoskeletal model construction, morphology-conditioned policy learning and virtual model augmentation to enable muscle-driven reference-motion tracking without individual fine-tuning. Across 31 held-out individual models, MyoAcross achieves 99.24% completion and reduces tracking error by 22.60% relative to an unconditioned multilayer perceptron. Policy reuse extends to models of 22 held-out individuals constructed from single-image shape estimates, measured height and body mass, and, with asymmetry augmentation, to models with unilateral lower-limb shortening. Comparisons with measurements from 120 additional individuals show agreement in vertical ground reaction force waveforms and channel-dependent agreement between simulated muscle activations and electromyography. These results establish a common control framework for comparative simulations of movement across individual musculoskeletal models.
comment: 17 pages, 7 figures. Project website: https://woodater-thu.github.io/MyoAcross/ Bozhao Wang and Yinglei Zhu contributed equally to this work
★ RoboAuto: Autonomous Robot Program Search through Multi-Level Physical Understanding
Recently, LLM-based autonomous agents have shown strong capabilities in iteratively generating, evaluating, and refining executable solutions. In robotic manipulation, Code-as-Policy lets these agents search over modular and editable robot programs. However, effective program search requires physical understanding: agents must understand how objects can be manipulated, how scene geometry constrains actions, and why interactions succeed or fail. Existing methods rely mainly on execution feedback for program refinement, often leaving these physical questions unresolved. We introduce RoboAuto, a framework for robot program search that explicitly organizes physical understanding at the object, scene, and interaction levels through three components. First, a hierarchical knowledge base combines executable skills with object-centric knowledge of affordances, spatial relations, and action effects, learned from embodied data and online task solving. Second, scene grounding connects this knowledge to the current environment by identifying relevant objects and resolving action-critical quantities. Third, multi-stage planning with self-designed experiments checks whether each stage produces its expected physical effects and actively tests competing hypotheses to resolve physical uncertainties. RoboAuto achieves superior performance across five benchmark families, attaining an 89.1% success rate on short-horizon LIBERO-PRO tasks and more than twice the mean success rate of ASPIRE on LIBERO-PRO Long and LIBERO-Plus. It also outperforms ASPIRE and $π_{0.5}$ on three challenging long-horizon real-robot tasks requiring precise manipulation. We further show that the knowledge base transfers across simulators and embodiments, and that program search performance scales with its size. Project page: https://roboauto-ai.github.io/
comment: Project page: https://roboauto-ai.github.io/
★ GRAFT: Geometry-Aware Residual Adaptation for Foundation Trackers
Humanoid behavior foundation models (BFMs) provide reusable whole-body motion priors: given a kinematic reference, they generate coordinated joint commands for motion replay, teleoperation, and VLA control. However, trained primarily on flat-ground motion, BFMs lack an explicit mechanism to adapt foot placement, swing clearance, and contact timing to local terrain. Many terrain-aware BFM approaches address this limitation by retraining or fine-tuning the tracker together with perception, incurring substantial data and computation costs while coupling ordinary tracking to the perception pathway. To bridge this gap, we propose GRAFT, a geometry-aware residual adaptation framework for frozen behavior trackers. GRAFT preserves the BFM parameters and motion-reference interface, while a lightweight adapter adds terrain-conditioned corrections only to lower-body execution targets. The adapter is trained with task and foot-geometry rewards, regularized to suppress unnecessary corrections on flat ground, and distilled from privileged terrain observations to onboard depth. Because the frozen BFM already provides whole-body reference tracking and nominal locomotion, GRAFT reduces terrain learning to a focused execution-adaptation problem. We instantiate the framework on four frozen trackers and evaluate stairs, gaps, and platforms in simulation. We further validate it through real-world deployment on a Unitree G1, where the adapted SONIC tracker traverses terrain with onboard depth and unmodified references.
comment: 8 pages, 6 figures, 3 tables. Project page: https://graft-humanoid.github.io/
★ SVIn+: Multi-Modal Framework for Robust Underwater State Estimation using 3D Sonar, Visual-Inertial, Water-Pressure, and DVL IROS 2026
As expansion and scalability in underwater oper- ations becomes more important and challenging, autonomous underwater vehicles (AUVs) become more relevant and the necessity for reliable autonomy more critical. A fundamental challenge in any autonomous underwater operation, from control to safe navigation and mapping, is state estimation; a notorious hard problem to address, especially in low visibility conditions. In this paper, we tackle this problem, by presenting SVIn+, a multi-sensor framework for robust state estimation of AUVs equipped with an IMU, cameras, a water-pressure sensor, a Doppler Velocity Log (DVL), and a compact 3D sonar. This framework extends an established open-source state-of-the-art visual-inertial SLAM, SVIn, by integrating water-pressure and DVL measurements into a tightly-coupled nonlinear optimiza- tion formulation resulting in improved estimates. To leverage the structural information of recently available 3D acoustic sonars while maintaining computational efficiency, we introduce a hierarchical architecture that incorporates estimates from a decoupled front-end via KISS-ICP, treated as relative pose factors within the factor graph. The proposed framework is validated through field deployments in a shipwreck and coral reef, demonstrating drift mitigation and global consistency. To support further research in underwater multi-modal fusion, the code and the datasets can be found on project website https://cmburgul.github.io/projects/svin+/ https://dartmouthrobotics.github.io/svin-plus
comment: IROS 2026
★ Whole-Body Humanoid Control via Stochastic Motion Tokenization
While velocity-commanded RL policies are effective for controlling quadrupeds, they are a limited interface for humanoids that need to perform complex whole-body motions. Furthermore, RL tracking policies for whole-body motions typically have limited information about the desired future motion, preventing them from performing controlled deviations to achieve long-term task success; such information is particularly relevant when a controller must prepare for transitions between behaviors. We address these challenges using concise motion representations that can be generated by a high-level controller, given a behavior label or text command. These representations can be fed directly into a tracking policy to provide a detailed motion reference. Our method exhibits a high degree of flexibility in performing real-world robot motions such as dancing, fighting, or walking based on conditional information. At matched capacity, our adopted stochastic motion tokenizer reconstructs motions from the Unitree G1 at a level that exceeds state-of-the-art approaches such as finite scalar quantization (FSQ) and the vector-quantized autoencoder (VQ-VAE). The resulting discrete interface conditions whole-body tracking policies that execute generated dance, fight, and walk motions lasting 16.8 s each and a 35 s fight-to-dance sequence on a physical Unitree G1.
comment: Accepted to the 2026 IEEE-RAS International Conference on Humanoid Robots (Humanoids 2026). (c) 2026 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses
★ Ground Sample Distance Constrained k-Coverage Viewpoint Planning via Set Multi-Cover Formulation
Drone-based 3D reconstruction and inspection require viewpoints that satisfy image-quality constraints while observing each surface region multiple times, which overlap-based planning heuristics do not guarantee. We study ground sample distance (GSD)-constrained $k$-coverage viewpoint selection for structures represented by surface voxels. Candidate viewpoints are generated along outward surface directions, screened with distance, viewing-angle, field-of-view, and occlusion tests, and augmented for voxels with insufficient visibility. Selection is then posed as set multi-cover with per-voxel coverage demands. We propose a dynamically weighted greedy heuristic that prioritizes remaining coverage deficits and scarce viewing alternatives, and compare it with sequential overlap-based pruning and an integer linear programming (ILP) formulation on the same candidate pools. Across two structures, four voxel resolutions, and three coverage requirements, the proposed approach achieves full or near-full $k$-coverage with compact viewpoint sets, using only 3.00--16.39\% more viewpoints than ILP while offering an interpretable selection process based on coverage deficits and viewpoint availability.
★ A Singularity Avoiding Track Model for Race Car Optimization
A track fitting pipeline is presented that converts a measured racetrack point cloud into a closed, $C^4$ differentiable, three-dimensional surface model suitable for high-fidelity Modelica race-car optimization. The surface is a Monge patch, parametrised by a curvilinear arc-length coordinate measured along the minimum-curvature line (MCL) and a signed lateral offset, with a height polynomial of user-selectable degree capturing the cross-sectional elevation profile. The choice of MCL as the reference line is deliberate since anchoring the same surface on the geometric centre-line can cause adjacent normals to cross inside the drivable region on tight corners. The resulting non-injective parametrisation assigns multiple curvilinear addresses to the same three-dimensional point and returns invalid road geometry. The MCL is obtained from a quadratic program that minimises integrated curvature subject to track-boundary, periodicity, and jerk-regularisation constraints. By reducing total curvature while remaining within the track bounds, the MCL moves the region of parametrisation breakdown outside the drivable strip on all circuits studied, keeps the normal map injective across the full track width, and yields a more linear time-to-arc-length transformation for minimum lap-time problems. The fitted surface and the associated track widths are exported as periodic piecewise-quintic $C^4$ B-splines, solved as a nonlinear program.
comment: American Modelica & FMI Conference 2026
★ Efficacy of Monocular Depth Estimation in Lunar Environments
Recent advances in monocular depth estimation have demonstrated highly accurate 3D reconstructions within in-distribution terrestrial domains. This study investigates the efficacy of several state-of-the-art approaches to reconstruct out-of-distribution lunar environments using the NASA Ames Polar Optical Lunar Analog Reconstruction (POLAR) stereo dataset and Apollo 17 surface imagery. By leveraging these datasets, we establish a robust analog-to-field validation pipeline that evaluates quantitative performance against ground-truth physical analogs and qualitative performance against historical field data. Our results indicate that select state-of-the-art techniques can successfully generalize to produce high-fidelity relative 3D reconstructions of the unstructured lunar surface.
comment: Accepted at the International Conference on Space Robotics (iSpaRo) 2026
★ Forward-Backward Map Matching for Airborne Magnetic Anomaly Navigation
Airborne magnetic anomaly navigation provides a passive, unjammable alternative to Global Navigation Satellite Systems positioning by matching magnetometer measurements against a lithospheric magnetic anomaly map. Existing map-matching algorithms either assume a Gaussian posterior (extended Kalman filter), require Monte Carlo sampling (particle filter), or discard posterior uncertainty in favour of a single optimal path (Viterbi algorithm). Here we formulate map matching as inference on a chain-structured factor graph and solve it segment by segment using the forward-backward algorithm, which yields a discrete posterior over candidate positions and its Shannon entropy $H_t$ at time $t$. Over the \texttt{Renfrew\_395} map, $10^3$ synthetic trials yield a median position error of $32.7$ m, $91\%$ below the median inertial drift of $362$ m. On these trials and $10^3$ flight paths, the median error is $43\%$ below the extended Kalman filter and particle filter. Across these paths, a threshold on segment-level entropy detects $47\%$ of segments with errors above two map cells and $94\%$ of those above three, at a $10\%$ false-alarm rate. The foregoing results indicate that $H_t$ provides a truth-free diagnostic of map-conditioned navigation ambiguity.
comment: 37 pages, 11 figures, submitted for publication: The Journal of Navigation (CUP)
★ EARS-Info: Information-Aware Event-Driven Reallocation for Multi-Robot Exploration
We introduce EARS-Info, an information-aware extension of Event-Driven Ant Reallocation Search for multi-robot exploration on OpenStreetMap-derived grids. It combines marginal sensing gain, feasible relocation, return-aware routing, sensing reservations, and cached shortest paths. Across 168,000 episodes comparing 30 implementations on eight city maps, EARS-Info achieves the highest mean operational score in every evaluation panel. With eight robots and 3% communication dropout, it improves the score over EARS-Safe by 5.43%, increases weighted discovery from 88.88% to 98.46%, and reduces mean energy use by 2.62%. Score differences are positive in all eight cities ($p=0.0078125$; descriptive 95% city-bootstrap interval $[0.0159,0.0694]$), with no simulated incidents across 5,600 EARS-Info episodes. These results support the complete controller; individual component advantages remain uncertain. Generalization is limited by reused geography, one training replicate, coarse maps, and adapted comparison methods.
★ Learning from Relative Action Credit for Escaping Local Minima
Reinforcement learning holds the promise of robots that learn manipulation skills from interaction alone, without pretrained priors or demonstration data. In practice, on-policy methods such as PPO routinely stall on long-horizon, contact-rich tasks: dense reward shaping carries the policy through stages, yet stitching them together and discovering sparsely rewarded events rarely succeeds. We trace this failure to the missing action discrimination in state-value credit assignment: a state-value critic judges progress only by episode outcomes, so when the policy converges to a partial solution all outcomes coincide, and improvement stops because no action can prove better than another. We therefore extend PPO with an action conditioned value critic which, can compare the executed action with alternatives sampled at the same state and aid the learning signal. Combined with Generalized Advantage Estimation and a bound on exploration noise, we show how the extended algorithm escapes the plateau and completes precise peg insertion within a training budget where PPO with identical exploration fails.
★ METATRON: A Promptable Whole-Body Execution Layer for Humanoids
Humanoid controllers often require task-specific training despite sharing the need for balance and whole-body coordination. We present Metatron, a promptable whole-body execution layer combining a generative base policy with a programmable skill layer. The base policy receives partial goals, collectively called a prompt, that can combine timed body-part targets, velocity commands, and style cues. It produces diverse whole-body motions consistent with these goals. Training combines motion capture, synthetic recovery trajectories, and retargeted scene-interaction motions. After training, the policy is frozen. Stateful programs called skills express task intent as prompts at each control step, drawing rough geometric sketches that the policy turns into coordinated movement. Skills are ordinary code, refined by coding agents that analyze motion data and fit compact rules without task-specific policy training. On the physical Unitree G1, one frozen checkpoint supports walking, recovery, teleoperated object pickup, and sitting on furniture. Comparative evaluations show higher reaching success and push robustness than the tested reference-tracking pipelines. The same recipe produces climbing and acrobatics on a simulated character. Metatron enables coding agents to turn learned motor capabilities into new skills through code, without further policy training.
comment: Project page: https://research.nvidia.com/labs/par/projects/metatron/
★ Self-Labeled Preference Transfer for Steerable Robot Manipulation
Vision-language-action (VLA) policies can learn to follow execution preferences, such as where to grasp an object or how to move around an obstacle, from demonstrations annotated with preference labels. Providing these annotations for every new task, however, is costly. We study whether learned preference control can transfer to new tasks whose demonstrations contain the desired behaviors but lack preference annotations. We find that a policy trained with preference annotations can follow preferences on a new task before adaptation but achieves limited task success. Fine-tuning on target demonstrations without these annotations improves task success while weakening preference control. Motivated by this finding, we introduce Self-Labeled Preference Transfer (SPT). SPT jointly trains a shared vision-language backbone for preference-conditioned action prediction and preference recognition on annotated source demonstrations. It then freezes the recognizer to preserve its cross-task preference recognition capability and uses it to self-label target demonstrations for policy fine-tuning. Across five types of execution preferences in simulation, the recognizer achieves 96.4% accuracy, and SPT reaches 84.8% preference following and 89.2% task success, compared with 59.4% and 88.0% for naive finetuning. We further extend to multiple preferences, achieving 93.8% preference following compared with 42.5% of baselines. In real-world experiments, SPT increases preference following from 52.5% to 92.5% relative to naive finetuning across two task transfers. These results show that SPT can support preference adaptation across tasks and objects while maintaining success, without additional manual preference annotations.
★ Active Constraints for Simultaneous Guidance and Safety in Handheld Micromanipulation ICRA 2027
In micromanipulation, erroneous hand motion limits unaided manipulation accuracy, which can affect task performance and safety in critical applications like microsurgery. Handheld robots (micromanipulators) enable motion accuracy beyond human limits by providing robotic assistance in a low-cost, compact, intuitive form factor. Particularly, active constraints are an effective form of robotic assistance, which aids the operator by restricting the robot's permissible motion based on task-specific requirements or objectives, thereby providing task guidance and enforcing safety. However, existing formulations of active constraints for handheld systems are limited, most notably because of the inability to provide guidance and enforce safety simultaneously. Implementing active constraints with optimization-based methods can address these limitations, but existing implementations are designed for free-standing robots, and neglect that handheld robots actuate relative to the operator's hand with a limited workspace. In this work, we propose an optimization-based active constraint framework for handheld micromanipulation that enables simultaneous guidance and safety enforcement. The formulation was implemented in a handheld robot and demonstrated on a medically relevant artificial task. Results show that the proposed formulation reduced safety violation rate by more than two orders of magnitude relative to baselines, while recovering the threefold accuracy loss from enforcing safety alone.
comment: 8 pages, 3 figures, submitted to ICRA 2027
★ FORGE: Feasible Online Reduced-Order Reference Generation and Whole-Body Execution for Agile Humanoid Locomotion
This paper presents a two-stage learning framework for versatile and agile humanoid locomotion that couples online reduced-order model (ROM) reference generation with whole-body tracking control. In the first stage, we train a command-conditioned whole-body reference generator using single rigid body dynamics combined with full humanoid kinematics. In the second stage, the frozen ROM policy is rolled out online to generate references for training a whole-body tracking policy on the full-order dynamics model. Learning a kino-dynamically consistent reference generator in the simplified ROM space enables us to generate references over a broad command space and over multiple locomotion strategies using a single unified framework. We demonstrate the approach on a Unitree G1 humanoid robot, achieving longitudinal velocities from -2.0 m/s to 2.8 m/s, lateral velocities up to +-1.0 m/s, angular velocities up to +-3.0 rad/s, one-legged hopping, and continuous transitions across the commands, all using one unified reference policy and one whole-body tracking policy.
comment: 8 pages, 6 figures. Accepted to the 2026 IEEE-RAS 25th International Conference on Humanoid Robots (Humanoids)
★ 3D Occupancy-Language Fields from mmWave Radar
Radar systems can recover scene geometry and recognize predefined categories, but these outputs do not provide a persistent spatial representation for open-ended language queries. Such queries require linking observations from different viewpoints to the same physical locations. We present TextRadar, a framework that constructs a scene-level 3D occupancy-language field from panoramic radar observations. The field combines occupancy with scale-dependent language features at surface locations, allowing the reconstructed scene to be queried with new descriptions. A single-frame radar distillation model predicts aligned ranges and language features from each radar observation. Using radar-estimated poses, we register these predictions to reconstruct the field from radar alone in a new scene. Experiments on approximately 20k synchronized observations across seven buildings demonstrate scene-level natural-language querying and prompt-free semantic visualization. Analysis of camera-to-radar concept transferability shows stronger agreement for structures, materials, and human poses than for colors and printed text.
★ What Actually Matters? A Paired Evaluation of Inference Settings for $π_{0.5}$ on BEHAVIOR-1K
Vision-Language-Action (VLA) models offer a promising paradigm for general-purpose household robotics. However, inference and data-collection strategies that have been deployed in smaller-scale settings do not necessarily generalize to large-scale, diverse task suites. To address this gap, we conduct a systematic, paired evaluation of previously proposed inference and data-collection strategies across all 100 tasks. Our evaluation uses a fine-tuned $π_{0.5}$ model based on the 2025 BEHAVIOR Challenge winner, which achieves a Q-score of 0.39 on the public test set. These results show that proposed strategies do not transfer uniformly to new environments and tasks. Project page, videos, and detailed analysis: https://rubenhammele.github.io/behavior2026
★ When More Planning Hurts: Learned Planners for World Models
On PushT, our fixed action updater (1,505 parameters) solves 8 of 16 scenes reserved from our configuration selection. The comparison planners perform as follows: Selected Adam (7 tasks solved, total planning and encoding time 218.440s), adapted iCEM (2 tasks solved, total planning and encoding time 49.309s), released LePlanner (16 tasks solved, total planning and encoding time 5.652s). The total planning and encoding time for our action updater is 59.079s. Note that our action updater does not satisfy the task-cost criterion we fixed before evaluation. We also conduct another matched experiment where we only replay the first ten commands in each context and then allow the next solve 4, 8 or 12 updates (subsequent solves all get 8 updates). Among all 16 contexts (8 scene clusters), the 3 budgets get 9, 8 and 7 successes, respectively. Interestingly, the extra updates did not rescue any failing task in this experiment. All the changes in success/failure happen in one scene cluster, where we observe that the extra updates decrease the costs of the selected latent world-model trajectories for our updater, while the tasks that used to be successes become failures. The code, weights, evaluators, execution traces and cost accounting accompany the paper.
comment: 23 pages, 9 figures, 16 tables, 28 references
★ Echo: Scaling Visual Reinforcement Learning with Off-policy Experience
Scaling visual reinforcement learning (RL) with GPU-accelerated simulation is computationally expensive due to rendering and visual-policy inference costs. Consequently, two-stage teacher--student distillation is commonly used to train visual policies, but introduces an imitation gap. In this preliminary study, we introduce \echomethod{}, \textbf{a single-stage off-policy RL framework that scales experience collection without proportionally increasing visual computation}. Through \textbf{inverse distillation}, a state-based auxiliary policy imitates the evolving visual policy and collects inexpensive, unrendered experience. Echo leverages this experience for off-policy value learning, directly improving the visual policy through a shared privileged critic. We instantiate \echomethod{} with FastTD3 and evaluate it on three visual control tasks and two dexterous manipulation tasks in Genesis. Relative to \sdpgmethod{} under the evaluated configurations, Echo reduces time to reference returns by an average of \textbf{60\% across the three locomotion tasks} and \textbf{56\% across both hand tasks}, while improving peak mean returns by \textbf{62\% and 12\%}, respectively. Both hand tasks also achieve higher mean returns at matched simulator-transition budgets. These initial results suggest that scaling off-policy experience is a promising approach to efficient direct visual RL.
comment: Report. 16 pages
★ GOFish: Full Field-of-View Gaussian Splatting and Mesh Extraction
Robots, autonomous vehicles, and consumer 360° cameras use fisheye lenses for wide fields of view (FoVs). Simulation-ready digital twins require photorealistic rendering across the full FoV and an aligned collision mesh. However, prior fisheye Gaussian Splatting methods only rasterize up to 180° and do not produce meshes, while Gaussian surface reconstruction methods assume pinhole training images. We present GOFish, the first 3D Gaussian Splatting surface reconstruction method to train directly on fisheye images, rasterize FoVs beyond 180°, and extract an aligned mesh from the same scene. Calibrated densification recovers the surface across the entire FoV, while Calibrated Mip Filtering and Wide-Angle Appearance Modeling keep pixel blur and lens falloff out of the scene and mesh. Across the 200° FIORD and 205° FullCircle benchmarks, GOFish improves PSNR over the best prior method by 2.28 and 1.97 dB, respectively. It also outperforms fisheye rendering baselines within their own FoVs and achieves the highest mesh F-score on FIORD and the main ScanNet++ set. Together, the Gaussian scene and mesh form a LiDAR-free, simulation-ready digital twin that renders in real time, ray traces the full FoV, and provides collision geometry.
comment: Project Page: https://3dgofish.github.io/
★ TACOS: Terrain-Aware COntact Supervision for Visual Footstep Planning ICRA 2027
Humans navigate their environment based on their vision system, anticipating how the ground will respond before committing to a step. Self-supervised learning could provide legged robots the same ability, but the supervisory signal is typically chosen on intuition, and its effect on what the model learns is rarely examined. We present Terrain-Aware COntact Supervision (TACOS), which proposes a meaningful and consistent footstep instability score, based on the residual of a factor graph in legged state estimation. It projects the continuous score into preceding images to supervise a dense visual predictor. We propose two variants, a hard score that exploits hindsight and a causal soft score available immediately. Both metrics are compared against state-of-the-art alternatives in two independent evaluations that share no data or labels. On 1716 manually annotated contacts, both variants outperform every evaluated alternative at detecting instability and ranking its severity, improving pooled severity ranking by 28 % over the strongest baseline. On three outdoor test runs, our metric also trains the most accurate visual predictor, improving rank correlation by 58 % over the strongest alternative, and by a further 25 % when supplemented with onboard context. In a real-world experiment over a 50 m forest trajectory, the trained predictor handles non-geometric terrain properties such as mud and vegetation, while stepping around geometric obstacles like fallen branches. Code is available for reproducibility at https://github.com/norlab-ulaval/TACOS.
comment: 9 pages, 8 figures, Submitted to ICRA 2027
★ An Empirical Study of Wrist-Target Construction and Joint-Angle Limits in Humanoid Arm Retargeting ICRA2027
Motion retargeting maps a demonstrated human movement onto a robot to compute its joint motion. This study examines how the construction of the desired wrist position affects whether computed arm joint angles reach their permitted limits on a Unitree H1-2. A reconstructed target is formed from demonstrated arm directions and robot segment lengths, so its shoulder-to-wrist distance depends on the demonstrated angle between the segments even when no posture-matching preference is applied. A reach-normalized target retains that direction while rescaling the distance, using the same processed demonstrations and the same joint-angle method. Across a forward-reach sweep the reconstructed target places a joint angle at a bound in 10 of 20 repetitions and the reach-normalized target in none. Across 25 recordings from one participant the corresponding counts are 8 and 6 on the reaching arm and 14 and 5 on the supporting arm, and a second participant repeats the bricklaying and lifting comparison. In hardware replay a joint is commanded to a bound in 12 of 16 direction-matching arm-records and in none of the 16 normalized records, and within the eligible records both tracking error and executed torque at those frames exceed their values elsewhere in the same run. Cross-body reach and arm-to-arm clearance identify limits of radial adjustment. The findings support treating target construction and posture matching as separate retargeting decisions, and evaluating both alongside the wrist positions and clearance that manipulation requires.
comment: Submitted to ICRA2027 under process
★ DexSuite: An All-in-One Framework for Dexterous Manipulation Across Arms, Hands, and Tasks
Human-level dexterity remains a central open problem in robotics, yet few simulation frameworks cover the full workflow of dexterous manipulation, from setting up a robot to collecting dexterous demonstrations and evaluating policies. We present DexSuite, an all-in-one simulation framework for dexterous manipulation built on the Genesis simulator. DexSuite provides 30 tasks and assembles its 10 arms and 12 end-effectors, from parallel-jaw grippers to dexterous hands, into more than 100 single-arm robots and thousands of bimanual pairs. It also ships a standardized environment API, integrated teleoperation, and simulated tactile sensing. Moreover, we define a benchmark of 16 task-embodiment pairs spanning diverse contact complexities and horizons, and release a dataset of 4,700 successful episodes in simulation from human teleoperation, scripted experts, and learned policies. We release DexSuite as open source and hope others will build on it. Videos and datasets are available at https://dexsuiteorg.github.io
comment: 7 pages, 7 figures, 2 tables. Project page: https://dexsuiteorg.github.io
♻ ★ Quaternion Approximate Networks for Enhanced Image Classification and Oriented Object Detection IROS 2025
This paper introduces Quaternion Approximate Networks (QUAN), a novel deep learning framework that leverages quaternion algebra for rotation equivariant image classification and object detection. Unlike conventional quaternion neural networks attempting to operate entirely in the quaternion domain, QUAN approximates quaternion convolution through Hamilton product decomposition using real-valued operations. This approach preserves geometric properties while enabling efficient implementation with custom CUDA kernels. We introduce Independent Quaternion Batch Normalization (IQBN) for training stability and extend quaternion operations to spatial attention mechanisms. QUAN is evaluated on image classification (CIFAR-10/100, ImageNet), object detection (COCO, DOTA), and robotic perception tasks. In classification tasks, QUAN achieves higher accuracy with fewer parameters and faster convergence compared to existing convolution and quaternion-based models. For objection detection, QUAN demonstrates improved parameter efficiency and rotation handling over standard Convolutional Neural Networks (CNNs) while establishing the SOTA for quaternion CNNs in this downstream task. These results highlight its potential for deployment in resource-constrained robotic systems requiring rotation-aware perception and application in other domains. Code: https://cwru-aism.github.io/ QUANpaper/
comment: IROS 2025 Oral, 8 pages, 5 figures
♻ ★ HGWM: Hierarchical Graph-Guided World Model for Zero-Shot Object Navigation via Graph Matching
Zero-shot object navigation requires an embodied agent to choose where to search while retaining evidence from previous observations. HGWM couples an LLM-generated room-anchor prior with persistent spatial-semantic memory. A coarse stage assigns visual-semantic scores to candidate headings on a 0-10 scale; a fine stage measures symbolic compatibility between observed objects, room context, and the goal prior. Phase-dependent fusion combines these scores with an LLM direction recommendation before local action selection. On HM3D v0.1, the reported HGWM configuration achieves 62.6% success rate (SR) and 31.5% success weighted by path length (SPL), compared with 58.1% and 31.2% for WMNav. On MP3D, the corresponding values are 46.8% and 17.9%, compared with 45.4% and 17.2%. The results indicate higher success point estimates and modest path-efficiency gains. On HM3D v0.2, the full model obtains 74.4% SR and 31.9% SPL. Additional component ablations reduce SR by 0.9-2.3 percentage points and SPL by 0.6-1.5 points. These are author-reported aggregate measurements. The accompanying implementation repairs pass offline tests; episode-level rerun records are not available. Evaluation is confined to simulation.
♻ ★ GARV: Graph-Aware Refinement and Safety Verification for High-Level Robot Plans
High-level robot actions such as goto(region) can hide intermediate traversal, allowing a monitor of the listed actions to accept a plan whose graph-level execution violates a safety rule. We present Graph-Aware Refinement and Verification (GARV), which composes graph-grounded action contracts with a safety monitor to produce a model-relative certificate or counterexample and, when route choice is enforceable, a minimum-cost safe refinement. In a 28-case controlled audit, refinement exposed all 12 targeted hidden hazards and produced the expected verdicts on 16 controls; a separate 14-case SPINE-RoboGuard instantiation yielded two false-safe cases and five action-interface errors. In standalone finite-graph simulation, all 16 designed cases met their expected outcomes, and product verification agreed with exhaustive enumeration on classification, minimum safe hop count, and witness safety in 160 comparisons across 40 graphs. Across 90 graph-contract evaluations on 30 graphs with a hazard deliberately placed on the selected shortest route, GARV found a safe refinement in 72 evaluations and no safe route within the permitted hop bound in 18. For the static forbidden-region rule, synthesis matched a simple hazard-deletion baseline in all 90 evaluations; GARV's guarantees remain conditional on the semantic graph, action contract, safety specification, and enforceability of any selected route.
comment: 7 pages, 2 figures, 1 algorithm, 4 tables
♻ ★ Compact Force Sensor for Dual-UAV Cable-Suspended Payload Transport with Tension-Aware Outer-Loop Control
Cooperative payload transportation using multiple Unmanned Aerial Vehicles (UAVs) poses challenges in stability, coordination, and robustness, especially under external disturbances and unmodeled dynamics. This work proposes a dual-UAV payload transportation framework supported by a compact, custom-designed force sensor measuring the interaction force at the UAV cable anchor point. The sensor design and mathematical model are presented, and its performance is characterized through static and dynamic tests evaluating linearity, hysteresis, repeatability, and crossload. The control architecture follows a cascade structure: fast inner loops handle vehicle stabilization, while outer loops are designed to compensate for the measured forces. The approach is validated through simulations and indoor experiments under position uncertainty. Payload-drop and constrained-space tests assess the proposed sensing and control architecture against literature-based distributed references, showing improved stabilization, coordination, and disturbance rejection. A video of the experiments is available at: https://youtu.be/rIw9-fvV8Qw.
comment: Preprint version for IEEE Robotics and Automation Letters, link to article: https://ieeexplore.ieee.org/document/11743837
♻ ★ Pretraining Knows What, Simulation Teaches How: Compositional Generalization in a Sim-Trained Robot Agent
A robot policy needs to understand what a task is asking for in a scene, and how to move the robot to accomplish that task. To achieve these two skills, we introduce Zero-real Agent Policy (ZAP), a recipe that teaches a small off-the-shelf vision-language model to control a robot via a simple tool harness. We use a pretrained 27B parameter model to bootstrap world understanding and reasoning, followed by supervised fine-tuning and GRPO on simple simulated tasks to teach it how to move the robot. Without observing real-world robot data during training, ZAP reaches 96.7% success on real-world tasks, matching GPT-6 Astra in our harness and outperforming the MolmoAct2 VLA (33.3%), while running 3x faster and 17x cheaper than GPT-6 Astra. Furthermore, ZAP also generalizes to an unseen embodiment as well as new objects, goals and strategies never seen during training. On a single-arm Franka, ZAP reaches 55.9% on RoboLab-120, 4.9 percentage points ahead of the leading VLA/WAM baseline.
♻ ★ MARE: An Open-Source Lunar Rover Simulator with Shadow Ground Truth for Perception and Autonomy
Lunar surface exploration increasingly relies on autonomous rovers that must perceive and traverse unstructured terrain under extreme illumination conditions. Near the lunar poles, low sun elevations produce long, high-contrast shadows that challenge stereo vision, terrain segmentation, and localization. Developing perception and navigation algorithms for these conditions requires large amounts of sensor data with accurate ground truth, which field campaigns and analog facilities can only provide at high cost and with limited variety. This paper presents MARE (Modular Autonomy and Rendering Environment), an open-source simulation suite for lunar rover perception and autonomy built on Unreal Engine 5 and ROS 2. MARE consists of three integrated components. Terrain Studio generates cratered and rocky lunar Digital Elevation Maps (DEMs) to build simulation scenes. A dedicated Unreal Engine project hosts two physics-based rovers with plugins for stereo, lidar, event, and Single-Photon Avalanche Diode (SPAD) cameras, streaming time-stamped sensor data and pixel-accurate ground truth. Lastly, a Guidance, Navigation, and Control (GNC) stack provides web-based mission control. Validation shows a mean Intersection over Union (IoU) of 0.79 between predicted and rendered shadows at low sun elevations, and stereo depth analysis verifies decreased pixel coverage in shadowed regions relative to sunlit terrain.
comment: 10 pages, 16 figures, 3 tables. v2: adds a dataset with segmentation and depth baselines. Code: https://github.com/raulcastar/MARE-Lunar-Simulator. Dataset: https://huggingface.co/datasets/raulcastar/mare-lunar-rover-dataset. Video: https://youtu.be/EI4HgbMFbBE
♻ ★ What Does World Modeling Actually Improve in Robot Action Learning?
World Action Models (WAMs) combine future-video prediction with robot action learning, yet improvements in task success alone do not reveal how predictive supervision manifests in execution behavior or whether its benefits persist under task and environmental shifts. We conduct controlled future-video prediction loss ablations within FastWAM and OpenWAM, keeping model architecture, initialization, robot training data, and training steps fixed within each model family. Under clean conditions on seen tasks, predictive supervision increases success rates by 24.38 and 72.91 percentage points, respectively. Trajectory clustering further reveals higher success rates within several coarse trajectory clusters, including cases where cluster frequencies change little, indicating that the gains are not solely reflected in changes in the prevalence of trajectory patterns. However, these benefits do not consistently persist under distribution shifts: improvements on unseen tasks vary across models and tasks, while both models exhibit low success rates under the evaluated environmental randomizations, without a consistent advantage from predictive supervision. We further introduce future-video prediction as an auxiliary training objective for $π_{0.5}$, where a lightweight Transformer predictor used only during training increases success from 82.94% to 89.19%. These findings support future-video prediction as effective auxiliary supervision for action learning, while distinguishing improved execution on familiar tasks from reliable transfer and showing that such gains need not rely on a large pretrained video diffusion transformer (DiT) in the evaluated setting.
♻ ★ From Language to Navigation Goals: A Vision-Language Approach for Semantic Navigation of Mobile Robots Using RGB-D Perception
Natural language interaction provides an intuitive way for non-expert users to communicate with robotic platforms. However, transforming user requests into executable navigation actions remains a challenging task, requiring the integration of language understanding, environment perception, and autonomous navigation. This work presents a language-driven navigation framework that enables mobile robots to interpret user requests in natural language to move the robot to a destination and autonomously navigate towards it. The framework is composed of modular ROS 2 components that cooperate to transform natural language instructions into navigation actions. Given a natural language request referring to a target in the environment (e.g., "go to the mail box"), the system identifies the referenced object, estimates its position using RGB-D data, and generates a navigation goal, which is then executed through the ROS 2 Nav2 navigation stack. The ROS 2-based implementation facilitates portability across different robotic platforms, requiring only the configuration of the corresponding topics and services. The system is evaluated in both simulation and real-world scenarios using a TurtleBot3 Waffle and a Unitree Go2 robot with a RealSense camera. Experimental results show that the framework successfully interprets both direct commands and contextual requests, generates meaningful natural-language feedback, and navigates towards the desired target. These results demonstrate the feasibility of combining semantic perception and autonomous navigation to provide an intuitive human-robot interaction paradigm. Code will be released as open source upon acceptance.
comment: 7 pages, 7 figures, 4 tables
♻ ★ Informationally Decoupled Trajectory Design for Sim-to-Real System Identification
Sampling-based system identification estimates physically meaningful parameters by tuning a simulator to reproduce the target system dynamics, providing an interpretable approach to improving sim-to-real transfer. Yet when the collected trajectories do not distinguish the effects of different parameters, multiple parameter combinations can reproduce those trajectories, leading to unreliable parameter estimates. To address this challenge, we introduce an Informationally Decoupled Trajectory Design framework (IDTD), which formulates the objective for the exploration policy built on the Schur complement score derived from the Fisher information matrix. To faithfully reflect parameter separability in the exploration objective, IDTD normalizes the score against the per-parameter information, selects the most favorable trajectory segment for each parameter, and aggregates the resulting scores logarithmically. The optimized trajectory is therefore composed of complementary intervals, each exposing a distinct subset of parameters whose contribution to the motion over that interval can be attributed unambiguously. Across diverse simulation environments, ranging from a linear-dynamics system to the Go2 quadruped, G1 humanoid, and Crazyflie quadrotor, IDTD reduces the parameter identification error by 39.6% on average relative to the strongest prior active-exploration baseline and attains improved downstream policy transfer. Furthermore, we validate the proposed trajectory design on a real K1 humanoid, demonstrating that the resulting identified parameters accurately capture the real-system dynamics. Code is available at https://github.com/jsw7460/IDTD.
comment: arxiv_r2
♻ ★ VLA-HWBench: Where the Time Goes When VLA Models Run on Edge Hardware
Vision-language-action (VLA) models increasingly run on edge devices aboard robots, where inference latency constrains closed-loop robot control. However, it remains unclear how to design VLA models, their inference software and edge hardware so that robots can run VLA inference in real time. Most VLA models report only task success, while prior studies cover limited action-generation strategies, divide inference into at most three stages, and lack comparisons of inference optimizations on the same models. Therefore, we introduce VLA-HWBench, a benchmark that measures 18 open VLA models on NVIDIA Jetson AGX Thor and AMD Ryzen AI Max+ PRO 395, with a seven-stage inference breakdown. Our study reveals three insights. First, action-generation strategy matters more than model size. It dominates 14 of 19 Jetson configurations, latency varies by $82\times$ on Jetson and $157\times$ on AMD, and parameter count explains at most 15% of latency variance. Models that generate long action chunks in a few flow-matching steps keep up with control, whereas token-by-token decoding does not. Second, kernel fusion with 4-bit weights helps most, running $π_{0.5}$ and GR00T N1.7 2.6-2.9$\times$ faster than PyTorch with torch.compile, but it exists for only these two models and only on the Jetson. Optimization also moves the time to the stages it shortens least, as CPU preprocessing grows from 12% to 39% of the GR00T N1.7 call. Third, after optimization, memory bandwidth limits only the stages that process few tokens on the Jetson, such as the denoising steps, whereas software limits all other stages, from unfused kernels to serial preprocessing on the host CPU. Overall, low-latency edge VLAs should use few action-generation iterations, inference software should extend kernel fusion and low-bit weights to more models and platforms, and edge hardware needs high memory bandwidth and a fast host CPU.
♻ ★ BiDex-100: A Comprehensive Benchmark for Bimanual Dexterous Manipulation Across 100+ Simulation Tasks
Bimanual manipulation with dexterous hands has attracted growing attention in research on robotic action models. However, few studies have developed benchmarks specifically tailored for dexterous manipulation, leaving the dexterous manipulation capabilities of existing models insufficiently characterized. In this paper, we present BiDex-100, a simulation benchmark for bimanual dexterous manipulation that combines broad task coverage, multi-dimensional evaluation, and high-DoF hand embodiments. Concretely, the benchmark comprises 106 tasks organized along five capability axes: generalization, instruction following, limited demonstrations, long horizon, and memory. Each task is paired with an out-of-domain variant featuring controlled visual and spatial changes, allowing systematic assessment of robustness. In addition, the benchmark supports two embodiments and provides 8,080 success-validated teleoperated demonstrations, totaling 6.25 million frames. We evaluate seven recent Vision-Language-Action models under a unified protocol, establishing baselines for capability-specific performance, robustness, and data efficiency. The website is available at https://robbyant-research.github.io/BiDex-100/.
♻ ★ Multi-robot Learning-based Informative Path Planning Using Spatio-Temporal Gaussian Process Kalman Filter
Multi-robot informative path planning (IPP) for persistent target monitoring requires robots to reason about spatial uncertainty, temporal evolution, and practical sensing and communication constraints. Recent learning-based multi-robot IPP methods use Gaussian Processes (GPs) for target uncertainty, but often rely on simplified sensing models and centralized belief updates. We propose a grid-based spatio-temporal GP-Kalman filtering framework for learning-based multi-robot IPP. Instead of maintaining one GP per target, we represent anonymous target presence as a single latent field over a discrete workspace grid. The proposed recursive update assimilates footprint-level camera measurements and supports arbitrary fields of view and range-dependent noise. A GP-derived temporal process update inflates uncertainty over time to account for moving targets. For decentralized deployment, each robot maintains its own mapper and exchanges fixed-size belief summaries rather than raw measurements. Received beliefs are fused using diagonal covariance intersection to remain conservative under unknown inter-robot correlations. We integrate the mapper with a reinforcement-learning policy for graph-based neighbor selection. In simulation, the centralized deployment improves target visitation and attains about 20% lower posterior target uncertainty than learning-based and classical auction/coverage baselines, a large share of which stems from the belief representation itself. A real-world two-UAV experiment demonstrates transfer to outdoor multi-robot search over a large field (>7000 m^2). Code is available at https://github.com/caomuqing/multirobot-ipp.
comment: IEEE RA-L accepted
♻ ★ FISRF: Flexible Instructional Scaffolding Reflection Framework for Robotics Task Planning with Large Language Model IROS 2026
Large Language Models (LLMs) are widely adopted for reflective error correction in task planning; however, existing approaches primarily rely on autonomous exploration, rendering them prone to accumulating erroneous experiences during early interactions with new environments. To address this limitation, we propose the Flexible Instructional Scaffolding Reflection Framework (FISRF), an agent--mentor architecture that estimates the robot's Zone of Proximal Development (ZPD) in task planning and adaptively selects reflection strategies and mentor guidance, thereby facilitating the accumulation of effective error-correction experience. Experiments in ALFWorld and on real-world robotic platforms demonstrate that FISRF reduces the average number of error-correction rounds by 37% and redundant reasoning by 25.1%, while improving the overall task success rate by 8%. Project webpage : https://fisrfplanning.github.io/
comment: Accepted to IROS 2026. 8 pages, 6 figures
♻ ★ CLEAR-IR: Clarity-Enhanced Active Reconstruction of Infrared Imagery IROS 2026
This paper presents CLEAR-IR (Clarity Enhanced Active Reconstruction of Infrared Imagery), a novel framework for enabling robust robotic perception in dark environments using active infrared (IR) sensing. IR streams are less susceptible to noise than RGB in low light conditions but are dominated by structured light emitter patterns that hinder high level tasks such as object detection, tracking, and localisation. CLEAR-IR addresses this by reconstructing clean, RGB styled images from emitter populated IR inputs using a dual stream architecture inspired by DeepMAO, enabling standard RGB trained vision pipelines to operate without visible light. Evaluation across object detection, ArUco marker detection, and visual SLAM demonstrates that CLEAR-IR produces reliable perception in conditions ranging from well lit to completely dark, where passive RGB based enhancement methods fail entirely due to the physical absence of sensor signal.
comment: Accepted and presented in IEEE/RSJ IROS 2026. Project page: https://ladoszlabs.github.io/clearir/
♻ ★ Investigating Active Sampling for Hardness Classification with Vision-Based Tactile Sensors
Hardness is one of the most important object properties that humans and robots perceive through touch. This paper investigates active sampling strategies for sample-efficient hardness classification with vision-based tactile sensors (VBTS), using information-theoretic measures of model uncertainty. We evaluated three probabilistic classifier models and two active sampling strategies and a random sampling baseline on a hardness classification task, with our own dataset of VBTS samples collected by a robot and a previously published dataset. In addition, we conducted a human study on the hardness classification task with the same objects as our dataset. Our findings indicate that the active sampling approaches, driven by variance and entropy as measures of model uncertainty, perform best with accuracy reaching 90%, compared to human accuracy of 48%, demonstrating the effectiveness of using VBTS for hardness classification. More information can be found at: https://fheller1.github.io/ActiveHardness
comment: Accepted at IEEE-RAS International Conference on Humanoid Robots (Humanoids 2026) Project page: https://fheller1.github.io/ActiveHardness
♻ ★ DeFM: Learning Foundation Representations from Depth for Robotics
Depth sensors are widely deployed across robotic platforms, and advances in fast, high-fidelity depth simulation have enabled robotic policies trained on depth observations to achieve robust sim-to-real transfer for a wide range of tasks. Despite this, representation learning for depth modality remains underexplored compared to RGB, where large-scale foundation models now define the state of the art. To address this gap, we present DeFM, a self-supervised foundation model trained entirely on depth images for robotic applications. Using a DINO-style self-distillation objective on a curated dataset of 60M depth images, DeFM learns geometric and semantic representations that generalize to diverse environments, tasks, and sensors. To retain metric awareness across multiple scales, we introduce a novel input normalization strategy. We further distill DeFM into compact models suitable for resource-constrained robotic systems. When evaluated on depth-based classification, segmentation, navigation, locomotion, and manipulation benchmarks, DeFM achieves state-of-the-art performance and demonstrates strong generalization from simulation to real-world environments. We release all our pretrained models, which can be adopted off-the-shelf for depth-based robotic learning without task-specific fine-tuning. Webpage: https://de-fm.github.io/
comment: Accepted to IEEE Transactions on Robotics (T-RO), 2026
♻ ★ Incremental-Smoothing-Based ARAIM Protection Levels for Tightly Coupled GNSS/IMU Integration in Urban Canyons
Protection levels (PLs) bound position errors under specified fault hypotheses and risk budgets, but advanced receiver autonomous integrity monitoring (ARAIM) is formulated for snapshot GNSS estimation rather than the correlated posterior of tightly coupled GNSS/IMU factor graph optimization (FGO). This paper formulates ARAIM solution separation and PL computation on the FGO posterior. Because current-epoch pseudorange factors enter only the current-state block of the fixed-lag information matrix, the fault-excluded position covariance, solution separation, and nominal-bias propagation follow exactly from the linearized current-state marginal, which re-tains correlations with velocity, IMU-bias, and clock states and with historical information. Woodbury updates with selected-output evaluation compute these quantities for fault-free, single-satellite, constellation-wide, and second-order hypotheses with-out repeated full-state inversion or graph reoptimization. Surrounding monitoring layers screen measurements, handle post-fit outliers, and determine PL validity and availability. In two road experiments, both nominal sequences achieve 100% horizontal and vertical empirical bounding over PL-valid epochs, with a maximum error-to-PL ratio of 0.96. Normalized errors show that the posterior covariance alone does not capture slowly varying ranging biases: a covariance-only bound computed from the same posteriors retains no resolvable margin, and strong temporal correlation limits the empirical evidence to consistency checks rather than validation of the integrity budget. On identical Experiment A posteriors, the proposed method reduces fault-hypothesis evaluation and PL solving time by 65.2% relative to direct full-state inversion with matching integrity decisions, and controlled fault injections demonstrate fail-safe PL invalidation while navigation continues.
comment: 9 figures;removed journal-template header
♻ ★ Temporal Tactile Encoding and Compliance for Intent-Aware Robot-to-Human Bimanual Handover
Reliable robot-to-human handover requires the robot to infer when the person is ready to receive the object and release it safely, comfortably, and at the right time. This is challenging because visual observations alone may not disambiguate clear taking intent from accidental contact, weak grasping, wrong-direction forces, or transient interactions. In this work we treat human-robot handover as an intrinsically multimodal problem. Our approach couples a VLA model with a compliance controller that reduces interaction forces during object transfer. We finetune the VLA model with human demonstrations using RGB observation, temporally encoded tactile feedback and proprioception. We evaluate the complete system in a human-subject study against two baselines: one without tactile feedback and one using tactile feedback without compliance control. We hypothesize that combining compliance and temporal tactile encoding yields the most reliable and comfortable handovers, as compliance facilitates physical interaction while tactile history captures sustained taking intent. Performance is measured through objective metrics and an ad-hoc questionnaire. The results show that the two components provide complementary benefits and substantially outperform the baselines. Project page: https://hsp-iit.github.io/temporal-tactile-bimanual-handover/.
♻ ★ FTP-1: A Generalist Foundation Tactile Policy Across Tactile Sensors for Contact-Rich Manipulation
Despite the success of vision-based generalist robotic policies, existing tactile-based policies remain tied to fixed embodiments and sensor setups. This is because tactile signals are highly heterogeneous across hardware, making cross-sensor generalization difficult. We present FTP-1,the first generalist foundation tactile policy pretrained to acquire transferable tactile manipulation abilities across diverse sensors and embodiments. FTP-1 supports varied tactile inputs, including image-, array-, and state-based signals, by using heterogeneous encoders to project them into unified morphology-aware latent tokens that are jointly modeled by a shared tactile Transformer expert. Pretrained on around 3,000 hours of tactile manipulation data aggregated from 26 data sources, spanning human and robot demonstrations across 21 sensors, FTP-1 learns tactile skills that transfer beyond the sensors seen during pretraining. Across downstream finetuning experiments spanning 5 hardware configurations, FTP-1 improves contact-rich manipulation on seen sensor setups by +17.2% and, surprisingly, transfers to two previously unseen tactile-sensor setups, achieving a +31% gain in success rate. FTP-1 establishes the first unified foundation baseline for tactile manipulation, providing future tactile policies with a shared model-level starting point. Pretrained models, datasets, training code and more visualization at https://ftp1-policy.github.io.
♻ ★ S3KF: Spherical State-Space Kalman Filtering for Panoramic 3D Multi-Object Tracking
Panoramic multi-object tracking is important for industrial safety monitoring, wide-area robotic perception, and infrastructure-light deployment in large workspaces. In these settings, the sensing system must provide full-surround coverage, metric geometric cues, and stable target association under wide field-of-view distortion and occlusion. Existing image-plane trackers are tightly coupled to the camera projection and become unreliable in panoramic imagery, while conventional Euclidean 3D formulations introduce redundant directional parameters and do not naturally unify angular, scale, and depth estimation. In this paper, we present $\mathbf{S^3KF}$, a panoramic 3D multi-object tracking framework built on a motorized rotating LiDAR and a quad-fisheye camera rig. The key idea is a geometry-consistent state representation on the unit sphere $\mathbb{S}^2$, where object bearing is modeled by a two-degree-of-freedom tangent-plane parameterization and jointly estimated with box scale and depth dynamics. Based on this state, we derive an extended spherical Kalman filtering pipeline that fuses panoramic camera detections with LiDAR depth observations for multimodal tracking. We further establish a map-based ground-truth generation pipeline using wearable localization devices registered to a shared global LiDAR map, enabling quantitative evaluation without motion-capture infrastructure. Experiments on self-collected real-world sequences show decimeter-level planar tracking accuracy, improved identity continuity over a 2D panoramic baseline in dynamic scenes, and real-time onboard operation on a Jetson AGX Orin platform. These results indicate that the proposed framework is a practical solution for panoramic perception and industrial-scale multi-object tracking.The project page can be found at https://kafeiyin00.github.io/S3KF/.
♻ ★ RPT*: Global Planning with Probabilistic Terminals for Target Search in Complex Environments
Routing problems such as Hamiltonian Path Problem (HPP), seek a path to visit all the vertices in a graph while minimizing the path cost. This paper studies a variant, HPP with Probabilistic Terminals (HPP-PT), where each vertex has a probability representing the likelihood that the robot's path terminates there, and the objective is to minimize the expected path cost. HPP-PT arises in target object search, where a mobile robot must visit all candidate locations to find an object, and prior knowledge of the object's location is expressed as vertex probabilities. While routing problems have been studied for decades, few of them consider uncertainty as required in this work. The challenge lies not only in optimality ordering the vertices, as in standard HPP, but also in handling history dependency: the expected path cost depends on the order in which vertices were previously visited. This makes many existing methods inefficient or inapplicable. To address the challenge, we propose a search-based approach RPT* with solution optimality guarantees, which leverages dynamic programming in a new state space to bypass the history dependency and novel heuristics to speed up the computation. Building on RPT*, we design a Hierarchical Autonomous Target Search (HATS) system that combines RPT* with either Bayesian filtering for lifelong target search with noisy sensors, or autonomous exploration to find targets in unknown environments. Experiments in both simulation and real robot show that our approach can naturally balance between exploitation and exploration, thereby finding targets more quickly on average than baseline methods.
♻ ★ Goal2Pixel: Grounding Goals to Pixels for Vision-Language Navigation
Vision-language models (VLMs) have become a common foundation for vision-and-language navigation in continuous environments (VLN-CE). Yet most VLM-based methods cast navigation as low-level action prediction, an interface that is ambiguous, tied to short-horizon motion primitives, and inefficient due to repeated VLM querying. We propose Goal2Pixel, a pure pixel-based paradigm that reformulates VLN-CE as navigable pixel grounding. Rather than predicting actions, Goal2Pixel uses the image plane as a unified spatial interface between VLM reasoning and robot motion: the model predicts a visible navigable pixel, which is back-projected into a 3D waypoint for forward navigation. Non-forward actions are encoded by auxiliary left/right/bottom regions for turning and stopping. To support long-horizon navigation, we propose ViKeyMem, a visibility-aware keyframe memory, achieving a favorable performance-efficiency trade-off while reducing training cost by over 50%. To adapt pretrained VLMs to navigable pixel grounding, we introduce semantic embeddings and coordinate-aware auxiliary losses. Among methods trained without external data, Goal2Pixel achieves competitive state-of-the-art performance on R2R-CE and RxR-CE. On R2R-CE Val-Unseen, it achieves 54.1% SR and 52.5% SPL with only 7.75 VLM calls per episode. Compared with direct action prediction, Goal2Pixel improves SR by 21.2 percentage points while requiring 6 times fewer VLM calls. Project Page: https://baobao0926.github.io/Goal2Pixel/.
comment: 8 pages
♻ ★ Learning to Feel the Future: DreamTacVLA for Contact-Rich Manipulation
Vision-Language-Action (VLA) models have shown remarkable generalization by mapping web-scale knowledge to robotic control, yet they remain blind to physical contact. Consequently, they struggle with contact-rich manipulation tasks that require reasoning about force, texture, and slip. While some approaches incorporate low-dimensional tactile signals, they fail to capture the high-resolution dynamics essential for such interactions. To address this limitation, we introduce DreamTacVLA, a framework that grounds VLA models in contact physics by learning to feel the future. Our model adopts a hierarchical perception scheme in which high-resolution tactile images serve as micro-vision inputs coupled with wrist-camera local vision and third-person macro vision. To reconcile these multi-scale sensory streams, we first train a unified policy with a Hierarchical Spatial Alignment (HSA) loss that aligns tactile tokens with their spatial counterparts in the wrist and third-person views. To further deepen the model's understanding of fine-grained contact dynamics, we finetune the system with a tactile world model that predicts future tactile signals. To mitigate tactile data scarcity and the wear-prone nature of tactile sensors, we construct a hybrid large-scale dataset sourced from both high-fidelity digital twin and real-world experiments. By anticipating upcoming tactile states, DreamTacVLA acquires a rich model of contact physics and conditions its actions on both real observations and imagined consequences. Across contact-rich manipulation tasks, it outperforms state-of-the-art VLA baselines, achieving up to 95% success, highlighting the importance of understanding physical contact for robust, touch-aware robotic agents.
♻ ★ FAWAM: Force-Aware World Action Models for Closed-Loop Contact-Rich Manipulation
Force signals provide critical interaction cues for contact-rich robotic manipulation. However, existing methods mostly use force as an additional observation modality, without fully exploiting its role in modeling future interaction dynamics or guiding execution-time feedback correction. In this paper, we propose FAWAM, a force-aware world action model that incorporates force information at three levels: perception, prediction, and closed-loop execution. FAWAM first encodes historical 6-axis force/torque signals to modulate action generation, then jointly predicts future actions and end-effector wrenches to explicitly model contact evolution. It further introduces a residual correction module that uses the predicted wrench trajectory as an execution-time reference to refine actions online based on real-time force feedback. Real-world experiments across multiple contact-rich tasks show that FAWAM improves the average success rate by 36.25% over vision-only baselines and 21.25% over existing force-aware baselines, demonstrating the effectiveness of our force-aware framework for robust contact-rich manipulation.
♻ ★ Robotic Agentic Platform for Intelligent Electric Vehicle Disassembly
Electric vehicles (EV) create an urgent need for scalable battery recycling, yet disassembly of EV battery packs remains largely manual due to high design variability. We present our Robotic Agentic Platform for Intelligent Disassembly (RAPID), designed to investigate perception-driven manipulation, flexible automation, and AI-assisted robot programming in realistic recycling scenarios. The system integrates a gantry-mounted industrial manipulator, RGB-D perception, and an automated nut-running tool for fastener removal on a full-scale EV battery pack. An open-vocabulary object detection pipeline achieves 0.9757 mAP50, enabling reliable identification of screws, nuts, busbars, and other components. We experimentally evaluate (n=204) three one-shot fastener removal strategies: taught-in poses (97% success rate, 24 min duration), one-shot vision execution (57%, 29 min), and visual servoing (83%, 36 min), comparing success rate and disassembly time for the battery's top cover fasteners. To support flexible interaction, we introduce agentic AI specifications for robotic disassembly tasks, allowing LLM agents to translate high-level instructions into robot actions through structured tool interfaces and ROS services. We evaluate SmolAgents with GPT-4o-mini and Qwen 3.5 9B/4B on edge hardware. Tool-based interfaces achieve 100% task completion, while automatic ROS service discovery shows 43.3% failure rates, highlighting the need for structured robot APIs for reliable LLM-driven control. This open-source platform enables systematic investigation of human-robot collaboration, agentic robot programming, and increasingly autonomous disassembly workflows, providing a practical foundation for research toward scalable robotic battery recycling.
comment: 2026 IEEE 22nd International Conference on Automation Science and Engineering (CASE)
♻ ★ When Do Discrete Actions Make Good Continuous Controllers?
Though modern AI models like LLMs predict discrete tokens, today's robotic foundation models represent actions as continuous vectors. In this paper, we challenge this common consensus by asking if continuous action parametrizations are superior to their discrete counterparts. We conduct an extensive comparison of single-step regression and multi-step flow variants of both continuous and discrete policies in both from-scratch single-task and multi-task VLA training, in both simulation and on real hardware. In all regimes, we find that discrete action prediction, even the most naive, uniform-bin quantization, can match or even outperform continuous-action policies in task success rate. Moreover, we find that the challenge to discrete action policies is not action precision, but rather factorization error introduced by product-distribution predictions across time and action dimensions. We show that, when paired with the correct interventions (iterative computation through discrete flows, or distribution sharpening through classifier-free guidance), discrete flow policies can turn this weakness into a strength, enabling more responsive action changes during a predicted action chunk. Lastly, we show that attempts to explicitly inject geometric information into discrete policies, e.g. through corruption processes that emulate continuous flow matching, do not yield consistent improvements. Taken together, our findings question existing assumptions that robotic action prediction requires distinct design decisions compared to other AI domains, and motivate the possibility of discrete action prediction as a compelling alternative to today's continuous action parametrizations. Project page: https://iscoyizj.github.io/discrete-action-website/
comment: 53 pages (12 main text), 19 figures, 12 tables. Project page: https://iscoyizj.github.io/discrete-action-website/
♻ ★ AWARE: Action-Entropy-Weighted and Attention-Guided Retrieval of Observation Evidence for Vision-Language-Action Models
Vision-Language-Action (VLA) models combine multimodal perception with robotic control, demonstrating remarkable potential for generalizable manipulation. Yet, enhancing their execution typically relies on auxiliary sensory inputs (e.g., depth, 3D point clouds) or costly fine-tuning. Inspired by the insight that Feed-Forward Networks (FFNs) act as key-value memory, we propose Action-Entropy-Weighted and Attention-Guided Retrieval of Observation Evidence (AWARE), an effective, training-free, and plug-and-play module for VLAs. Specifically, AWARE tracks layer-wise uncertainty via an element-wise Action Entropy metric, and retrieves observation evidence into the element's FFN through associative memory lookup. Crucially, the retrieved features are weighted by instruction-guided cross-modal attention to isolate task-relevant observation cues, while injection strength is adaptively scaled by the element's entropy and relative network depth. This selectively channels observation evidence to high-entropy action dimensions and deep layers where grounding decays, while leaving converged representations intact. Across LIBERO, SIMPLER, CALVIN, and real-world robot manipulation, AWARE consistently improves diverse VLA architectures with minimal computational overhead, achieving the largest gains on long-horizon tasks without architectural changes or additional training. Project page: https://jiabingyang01.github.io/AWARE/.
comment: Major revision: expanded continuous element-wise formulation (AWARE, evolved from v1 UAOR) and additional benchmarks (LIBERO, SIMPLER, CALVIN, and real-world manipulation)
♻ ★ Towards Accurate End-Effector Localization for UMI-Style Robotic Manipulation Teaching
Robot demonstration learning requires accurate and temporally complete end-effector localization during close-range manipulation and camera occlusion. Existing SLAM benchmarks emphasize navigation motions, whereas manipulation datasets prioritize policy learning over localization evaluation. We introduce MILD, a Manipulation-Interface Localization Dataset with real-world and simulation sequences. The real-world subset provides 86 sensor sequences from Insta360 X5 and Insight9 across 15 repeated tabletop tasks, calibration assets, and a per-execution robot end-effector reference trajectory. The simulation subset, MILD-Sim, extends task coverage in Isaac Sim for controlled manipulation-replay studies. Benchmarking visual-inertial and fiducial-aided systems on instrumented real-world recordings reveals large differences in both TCP-relative trajectory error and temporal coverage, even under the same nominal task. To support marker-augmented teaching workspaces without a pre-surveyed fiducial map, we present AprilVINS, which combines fisheye visual-inertial estimation with sequence-local AprilTag geometry and separates prior admission from guarded export of the jointly optimized state. On Insta360 AprilTag4 recordings, AprilVINS(full) under a unified protocol with sequence-specific profiles reaches millimeter-level SE(3)-aligned TCP-relative APE RMSE with high time completion and lower reported error than the tested routes under their respective protocols, whereas fisheye VIO without tag factors remains at centimeter scale. Ablations separate accuracy from exportability, and a MILD-Sim replay study provides task-specific tolerance references for interpreting those error magnitudes. Together, MILD and AprilVINS provide a diagnostic benchmarking framework for UMI-style demonstration collection. Code, datasets, and evaluation manifests will be released upon acceptance.
comment: 9 pages, 8 figures, 5 tables. Project page: https://mild-web.github.io
♻ ★ Grasp-Then-Plan with Failure Attribution: A Closed Two-Stage Framework for Precise and Generalizable Robotic Manipulation NeurIPS 2026
In robotic manipulation, the tight coupling between grasping and motion planning often obscures the true source of failure, leading to inefficient trial-and-error. To enable efficient long-horizon manipulation, we propose GTP-FA (Grasp-Then-Plan with Failure Attribution), a task-oriented two-stage 'grasp-then-plan' framework that generates grasp candidates and performs downstream motion planning conditioned on the selected grasp. Given a failed manipulation trajectory, we learn a failure attribution model that generalizes to unseen grasps and produces a stable distribution over failure modes for diagnosis-guided optimization. Based on these attribution results, we then optimize both modules in a diagnosis-driven manner: on the grasping side, we inject task-level priors and risk penalties into grasp candidate scoring and optimization to suppress unstable or task-incompatible grasps; on the planning side, we target high-risk initial states through data collection and fine-tuning to address genuine planning bottlenecks. We evaluate the proposed framework in both simulation and real-robot experiments, and show that GTP-FA improves the corresponding base learners across RL, IL, diffusion-policy, and VLA-based settings, achieving substantially higher overall task success rates.
comment: Accepted at NeurIPS 2026. 34 pages. Revised after peer review with additional experiments, updated results, and updated author information. Project page: https://sites.google.com/view/gtp-fa/
♻ ★ Lexicographic Descent: A Gradient-Based Approach to Optimize Totally Ordered Rulebooks via Parametric Scalarization
Safety-critical systems, such as autonomous vehicles, must satisfy conflicting constraints and objectives, collectively called rules, typically ordered lexicographically. Preemptive lexicographic optimization solves a sequence of problems, one per priority level. However, higher levels are optimized without considering lower ones, making the approach susceptible to suboptimal solutions. On the other hand, scalarization optimizes all levels jointly and so avoids this myopia, but a finite weighted sum permits trade-offs across priority levels, and exact scalar representations lack the regularity standard solvers require. We propose Lexicographic Descent, which combines the benefits of both: a single weighted sum whose relative weights decay at priority-dependent rates, recovering the lexicographic order in the limit. We show that Lexicographic Descent converges to constrained stationary points of the highest-priority rule, matching preemptive guarantees. Since the lower-priority weights vanish, we derive a necessary condition for the lower-priority rules to keep guiding the iterates, with the aim of avoiding suboptimal stationary points of the higher-priority rules. We show that Lexicographic Descent can exploit problem structure to reach the lexicographic optima, and outperforms the preemptive approach on an autonomous driving case study.
comment: 8 pages, reviewed,
♻ ★ Context-dependent Runtime Construction of Safe Regions of Attraction from Design-Time Identification for Multi-Candidate Simplex Architectures
Deploying safety-critical cyber-physical systems requires control strategies that reconcile high performance and adaptability with rigorous safety guarantees. Classical control strategies offer stability proofs but are conservative, while advanced optimization- and learning-based controllers achieve high performance but lack formal verification. Runtime Assurance (RTA) architectures bridge this gap by introducing their own (dis-)advantages. Safety filters use online optimization and introduce chattering, while Simplex permissiveness is bounded by the tightness of the certified Region of Attraction (RoA) of its High-Assurance (HA) controller. Targeting this conservatism, we present a two-stage RoA certification procedure leveraging contraction-based stability to establish a target set for Taylor-model-based reachability analysis. We then treat the closed-loop system under HA control as a switched system, enabling: (a) runtime construction of context-dependent safe RoA and (b) safe switching between HA candidates to optimize recovery and maximize the operating envelope of the uncertified High-Performance (HP) controller. We validate this framework on a robotic path-tracking task using a three-stage evaluation: Ordinary Differential Equation (ODE) simulation for theoretical verification, high-fidelity Gazebo simulation under unmodeled dynamics, and experiments on physical hardware. Results show that the Simplex architecture maintains safety under fault injection and successfully shields a behavioral-cloning-trained HP controller. Crucially, the decision module introduces negligible overhead (0.028 ms average, 4.13 ms worst-case), consuming under 10 % of the control period.
comment: 13 pages, 8 figures, 5 tables
♻ ★ BBoE: Leveraging Bundle of Edges for Kinodynamic Bidirectional Motion Planning
In this work, we introduce BBoE, a bidirectional, kinodynamic, sampling-based motion planner that consistently and quickly finds solutions in environments with varying obstacle clutter. The algorithm combines exploration and exploitation while relying on precomputed robot motions, resulting in efficient convergence towards the goal. Our key contributions include: i) a strategy to quickly find solutions in obstacle-rich spaces by leveraging a bundle of precomputed robot motions (i.e., edges); and ii) BBoE, a robust bidirectional kinodynamic planner that utilizes this strategy to produce fast and feasible solutions. Across four systems spanning 3D to 6D state spaces, BBoE is the only planner to achieve $\geq96\%$ success on every system while attaining the lowest planning time on three of four.
comment: Accepted for publication in IEEE Robotics and Automation Letters (RA-L). Project Website: https://brsrikrishna.github.io/BBoE_RAL/
♻ ★ Bilinear World Models: Learning Representations with Structured Dynamics for Efficient Control
World models jointly learn latent representations and dynamics that predict how high-dimensional observations evolve under actions. In this work, we propose a JEPA-style world model in which, rather than learning arbitrary latent dynamics, we restrict them to follow a bilinear parameterization. This structure enables efficient planning and control while shifting the modeling burden onto the encoder, encouraging richer representations that expose the controllable geometry of the system. In particular, this structured parameterization allows us to structurally enforce action recoverability, thereby preventing representation collapse by construction. Although prescribing a bilinear parametrization may appear restrictive, we show that a broad class of nonlinear dynamical systems admits a transformation under which the dynamics become bilinear. Empirically, we show across standard 2D and 3D control tasks that representations with bilinear-parameterized dynamics can be learned directly from high-dimensional observations, reducing planning time by nearly three orders of magnitude while retaining or even improving control accuracy. We also propose more demanding regimes of longer-horizon planning and real-time control, and demonstrate that our method succeeds in both, moving JEPA-style world models beyond short-horizon offline planning.
♻ ★ GlanceWAM: Sparse Test-Time Imagination for World-Action Models
Video generative models provide rich physical priors for robot learning, yet existing world-action models (WAMs) face a fundamental trade-off: synchronous video generation at control rate is latency-prohibitive, while abandoning test-time visual imagination sacrifices task success. We show that visual imagination achieves both real-time inference and superior success rates when generated asynchronously off the critical path and consumed directly in latent space. We introduce GlanceWAM, which decouples imagination from control on a single shared video DiT backbone: an asynchronous proposer glances ahead on a slow clock to imagine a single lookahead frame seconds into the future in the background, while an action head decodes action chunks at control rate (48 ms) purely in latent space without blocking. Enabled by a non-interfering attention mask that isolates video representations and staleness-robust horizon training that accommodates asynchronous lookahead aging, GlanceWAM breaks the speed-success dilemma. Trained purely on demonstrations, it attains 72.2% on the 24-task RoboCasa kitchen benchmark (vs. 67.1% for synchronous Cosmos Policy) and 99.0% on LIBERO while cutting per-chunk control latency 24x relative to synchronous world-action models (48 ms on one A100). In single-arm and bimanual real-robot manipulation, it achieves higher average success than $π_{0.5}$ without any robot-data pretraining. Code is available at https://github.com/linhanwang/GlanceWAM and real-robot videos at https://linhanwang.github.io/glancewam/.
Multiagent Systems
★ Can LLMs Uncover Hidden Physical Laws through Free Exploration?
Scientific discovery often begins with an unexpected observation that prompts a new research question. In a world with altered physical laws, can large language models (LLMs) notice unexpected behavior, such as heavier objects falling more slowly, and discover the law through experiments without being told what to investigate? To evaluate this ability, we introduce LongChang, an automated framework that creates simulated worlds with altered physics, where one or more laws differ from ordinary physics. The evaluated LLMs act as explorers, manipulating objects and building tools to investigate their own questions without research tasks or hints about these laws. Separate LLM agents generate and review scenes, review tool designs, and evaluate reports for correctness and experimental support using the altered physical laws and exploration records. These records guide scene evolution for further exploration, allowing the full cycle to run without human intervention. Among 12 LLMs, GPT-6-Astra achieves a discovery rate of 16.7%, while the other 11 score 0%. The exploration records reveal uninvestigated anomalies and only partial identification of the altered physical laws.
comment: https://zhenyangcs.github.io/LongChang-website/
★ Rare, Relevant, and Still Wrong: How Memory Fails LLM Group Decisions
When an LLM group's discussion is compressed into a short memory for a final decision maker, what should the memory keep? Two natural heuristics keep the rarest claims or the claims relevant to the decision criteria. On hidden-profile tasks both look attractive, because how widely a fact is held is confounded with whether it helps. We introduce CrossProfile, a diagnostic benchmark of 60 templates with automatically verified answer keys, in which the ownership of supporting, decoy, and irrelevant facts is crossed while their text is held fixed (480 tasks). Across Qwen2.5-7B, Llama-3.1-8B, and two DeepSeek models, both heuristics fail in predictable places. Selecting by relevance readmits decoys, because decoys are on topic: even a perfect relevance filter loses 23 to 31 points to rarest-first selection in the classic hidden-profile configuration. Rarest-first selection is no better than random selection on average and wins only where the rare facts happen to be the useful ones. In both cases accuracy follows the balance of evidence the memory keeps, not the policy that kept it. The decision model adds a third failure, a form of social proof: it gives extra weight to facts attributed to many agents. Removing the attribution tags, or reassigning them at random with line order fixed, raises accuracy in the classic configuration by 13 to 24 points. In natural discussions the evidence is lost before any memory acts: the complete transcript yields lower accuracy than no notes at all.
★ TANDEM: Balancing Centralized and Decentralized Critics to Address the Information Gap in Complex Multi-Agent Environments
Cooperative multi-agent reinforcement learning under centralized training with decentralized execution (CTDE) shapes decentralized policies with a centralized critic that observes the global state, then removes the critic at execution. The policies are optimized to rely on information they will not have when they act, an information gap between training and execution. It widens under partial observability, observation noise, and openness, where the set of agents and tasks present changes during an episode. Existing methods transfer centralized guidance in one direction only, removing or distilling the critic once, so the policies never rehearse acting without the critic while they can still be corrected. We propose TANDEM, which keeps one centralized critic and reads it two ways at each training step: a centralized read from the global state, and a decentralized read restricted to the information each agent will have at execution. A parameter-free gate compares the two reads for each agent and trains the policy on the decentralized read where they agree, reinstating the centralized read only where the local view is misleading. A shared target pulls the decentralized read toward the centralized one. We formalize the setting as a task- and agent-open decentralized partially observable Markov decision process (TAO-Dec-POMDP) and evaluate TANDEM on three cooperative domains, each isolating one form of the information gap: discovery for partial observability, sampling for observation noise, and openness in wildfire suppression. We compare against six CTDE baselines and ablate the gate and the shared target; the supplementary material holds the full variant sweep, diagnostics, and learning curves. We measure return at execution, once the centralized critic is removed. TANDEM is the best method on eight of the nine setups and never significantly worse than any baseline.
★ Strategyproof Deep Reinforcement Learning for Online Resource Allocation
Online resource allocation faces two coupled challenges: (1) allocating scarce resources efficiently under uncertain future demand, and (2) ensuring that agents report task valuations and requirements truthfully (strategyproofness). Reinforcement Learning (RL) adapts allocation decisions through experience but does not necessarily ensure strategyproofness. Existing strategyproof RL uses a linear allocation rule, which limits the resource interactions it can represent. Neural mechanism design methods offer greater expressiveness, but provide only approximate incentive guarantees. Against this background, we introduce the first strategyproof deep RL mechanism for online resource allocation. It integrates three novel components: (i) a monotonic neural network that learns nonlinear interactions by construction (higher reported value cannot reduce acceptance, while a longer reported duration or larger resource requirement cannot increase it); (ii) a generalised critical payment: the lowest accepted value, with all other inputs fixed; and (iii) training by either RL or imitation of an existing rule. Together, monotonicity and the critical payment ensure strategyproofness and individual rationality, while the neural rule provides flexibility and imitation improves training efficiency. Empirically, on a four-resource stream with two partner pairs that a linear difference cannot represent, the neural rule achieves 97.8% of an offline welfare upper bound, compared with 93.8% for a linear learner and 89.0% for first-come, first-served, closing 65% and 80% of the remaining optimality gap, respectively. Moreover, imitation reaches 97.5% in only 261 seconds, about 89 times faster than RL.
comment: 15 pages, 2 figures
★ Concordat: Typed Sessions for LLM Agents
Agents built on large language models are increasingly allowed to take consequential actions: cancelling bookings, issuing refunds, changing accounts. The rules that govern those actions, such as "obtain the customer's explicit confirmation before changing anything", are usually stated in the prompt and followed only as reliably as the model chooses to follow them. We present Concordat, a Python library that treats an agent as an object whose interface is a finite-state protocol, in the spirit of typestate and session types, and enforces that protocol on the model at run time. On every turn the model is offered only the tools that are legal in the current protocol state, calls outside that set are rejected before they have side effects, decisions reserved for humans can only be requested, and token and tool-call budgets are charged and split linearly across handoffs. Protocols are checked when they are defined, and a subclass of an agent is rejected if it could bypass its parent's protocol, a behavioural form of Liskov substitution. We evaluate a Concordat agent against the standard agent of tau-bench on its 50 airline tasks, adding simulated customers who pressure the agent to skip confirmation. The standard agent made changes without the required "yes" in 13 of 50 tasks with gpt-4.1 and in 29 of 128 task-runs with Claude Haiku 5.5; the Concordat agent made none, with no significant difference in task success. tau-bench's own reward did not detect most of these violations. Concordat is open source.
comment: 11 pages, 1 figure, 1 table. Code, benchmark harness and results: https://github.com/miladsikaroudi/concordat
★ Test-Time Communication in Trading Agent Swarms
Communication between agents has mostly been studied on tasks with a shared objective and fast, clear feedback. Trading has neither: each agent manages its own account, with decisions being scored by markets late and noisily. A trading swarm is thus a population of independent decision-makers rather than a team, for which convergence is a risk and not the goal. We ran 200 agents from four model families in groups of ten under five communication regimes, from isolation to message boards and shared notes, over seven US trading days (1 to 9 October 2026), logging everything each agent read before every decision. In two of the four families, communicating groups came to hold far more of the same stocks than the isolated group. A group's return was largely the return on what its members held in common. Seeing a post from a groupmate made an agent 7 percentage points more likely to reuse an eight-word phrase from it. Agents that started too close together to see each other's posts shared such a phrase in 14% of cases, the rate that common inputs produce. Seeing the post did not detectably change that morning's trades. Case studies traced the adoption of positions and decision rules over subsequent days, and communicating groups sold in step more often than isolated ones. Because isolated groups also converged on the same stocks, sometimes completely, assessments of herding need to measure excess convergence relative to a no-peer baseline.
comment: 31 pages, 10 figures, 10 tables
★ Agent-Conditioned Routing: Diverse Reasoning Paths Within One Frozen Model NeurIPS 2026
Generating several solutions can expose alternative methods, but additional trajectories often paraphrase one dominant approach. We investigate whether the conditional capacity of a frozen sparse mixture-of-experts (MoE) model can provide such diversity from one problem prompt, without user-assigned roles, prescribed methods, fine-tuning, or model copies. Agent-Conditioned Routing (ACR) assigns $m$ parallel trajectories persistent, untrained shared/private expert-eligibility masks while preserving the learned, input-dependent token router. On a screened 126-question multi-mode mathematics and physics benchmark, with model judges reading the reported derivations in full, we count the number of problems yielding at least two distinct primary methods among soft-correct solutions in four trajectories. ACR reaches approximately twice as many such problems as the strongest temperature setting under GLM 5.3 (37.3 versus 18.7), and 1.67 times as many under GPT-5.6 Sol (45.00 versus 27.00), at matched output budgets. A blinded four-expert audit on a stratified sample of 30 problems also finds more multi-method solutions with ACR on both backbones, with lower correctness. Matched-eligibility ablations show modest multi-method count advantages for ACR, while token-resampled masks attain higher correctness. An informed sequential prompt elicits more diversity still, using previous solutions and explicit requests for alternative methods. ACR adds about 160 KiB of mask state. In a three-problem microbenchmark capped at 1,024 tokens, its latency and peak VRAM usage remain within 3% of batched temperature sampling.
comment: 17 pages, 4 figures, 11 tables. Extended version of a paper accepted at the NeurIPS 2026 Workshop on Resource-Aware Agentic AI (non-archival)
★ Aligning Institutional Power and Operational Ability
To be socially sustainable, Agentic AI systems require normative designs that align responsibilities with the ability to fulfil them. Identifying incongruences such as unsatisfiable expectations or privilege escalation is crucial in avoiding normative pitfalls. In this study, we discuss design dimensions suitable for describing multi-agent systems in light of this norm-ability alignment requirement. We propose a formal framework centred on process-oriented modelling and institutional power for the analysis of a given normatively relevant process, discuss strategies for repairing it, and elaborate on how the symmetrical relation between operational process and institutional state affects the lifecycle of normative positions.
★ Evaluating Priority Dispatch for Premium Membership in On-demand Taxi-sharing
In this paper, we analyze the dispatch-level feasibility of introducing premium membership into an on-demand taxi-sharing service using a sequential optimal insertion method. We propose two dispatch-level methods: deadline shortening for premium users, and deadline shortening with dedicated premium vehicles. We compare them with a baseline dispatch method through SUMO-based simulations under three demand scenarios: steady uniform demand, time-varying demand with a peak period, and urban-focused demand with concentrated peak traffic. Our results show that the suitable priority mechanism depends on operating conditions and premium ratio, suggesting that condition-dependent method switching may provide premium benefits while keeping non-premium-user burden smaller for overall optimization.
★ A Serious-Game Testbed for Self-Governing Human-Machine Institutions
AI agents no longer act alone: they cooperate, coordinate and compete within and between groups, increasingly alongside humans who are in, on or out of ``the loop''. Whether such mixed groups can govern themselves is an open question about machine behaviour and human-computer interaction that needs a testbed for hybrid institutions. We present Megabike, a multi-agent simulator and serious game with interdependent arenas for common-pool resource management, cooperative survival and constitutional choice. Riders sharing a megabike must agree their arrangements for steering, pedalling and distributing resources, and their political regime, while negotiating with rival bikes and avoiding an existential threat. Every rule is enforced by the simulator, every action is logged, and any seat can be taken by a human or a machine, so governance can be studied under well-defined metrics. We demonstrate Megabike with forty sessions in which language-model riders with cooperative, lazy, subversive or opposing dispositions chose regimes, elected leaders and expelled one another. The results show that different combinations of agent archetypes could produce unexpected and counterproductive outcomes. We conclude with extensions for systematic experiments mixing human and computational agents.
★ Feedback autocratic strategies in stochastic games
A stochastic game is an optimal control problem for multiple agents in a stochastic environment. Stochastic games contain important practical problems, related to environmental issues or traffic flows. If the number of environmental states is one, such stochastic games are called repeated games. In repeated games, several strategies which unilaterally control payoffs have been proposed, and these strategies work in a mechanism similar to feedback control. In this paper, we generalize these strategies as feedback strategies in stochastic games, and investigate their properties. In particular, we provide a necessary and sufficient condition for such feedback strategies to work as controllers of payoffs under some assumptions. We demonstrate the performance of such feedback strategies in several examples related to social dilemmas. Our findings connect the theory of payoff control in repeated games to the theory of feedback control in stochastic games.
comment: 19 pages, 2 figures
★ Remembering Is Not Enough: Refining Memories and Learning When to Use Them in Multi-Agent Systems
Memory helps LLM-based multi-agent systems reuse experience, but stored advice can be flawed or applied in the wrong situation. Team-level success or failure alone cannot reliably show whether a memory needs correction or should not have been used. We propose DiscernMem, a training-free framework that improves both memory quality and memory use. To improve quality, Defect-Aware Memory Refinement (DMR) compares team executions with a single memory and with no memory, diagnoses rule defects, and tests repairs on current and earlier tasks to guard against losing previously useful behavior. To guide appropriate use, Boundary-Learning Meta-Memory (BLM) records each memory's past uses and effects, learning when its owner role should apply or avoid it based on task context and incoming messages. Together, these components help teams correct flawed experience while retaining useful rules that were applied incorrectly. Experiments with Qwen3-4B-Instruct-2507 on four benchmarks show that DiscernMem achieves the highest average score on AutoGen, CAMEL, and MacNet, exceeding the strongest reported baselines by 0.51-4.32 points. It leads on code generation and symbolic planning across all three systems, including a PDDL score of 37.71 versus 28.48 on CAMEL.
★ GarmentSegAgent: Autonomous Garment Segmentation for Web Fashion Images via Textual Self-Refinement
Web fashion images support applications such as visual product search and consumer-to-shop matching, yet their visual heterogeneity makes reliable garment segmentation challenging. Despite advances in promptable segmentation, autonomously identifying and correcting garment-specific errors without human-provided spatial prompts remains difficult. We propose GarmentSegAgent, an autonomous framework that integrates semantic planning, spatial execution, and visual evaluation through iterative textual self-refinement. At its core, we introduce CAST (Constrained Attribution-guided Selective TextGrad), which connects visual error attribution with targeted textual optimization. CAST identifies instruction components responsible for segmentation errors, selectively updates them through textual gradients, and validates the resulting instructions against garment-specific constraints. This enables image-specific error correction at inference time while keeping the underlying foundation models frozen. Experiments on DeepFashion2, ModaNet, and Fashionpedia demonstrate consistent improvements in region overlap and boundary accuracy, alongside more reliable refinement than conventional TextGrad. By enabling visual feedback to guide controlled instruction updates, GarmentSegAgent offers a promising direction for autonomous garment understanding, with potential applications in scalable fashion content analysis, visual retrieval, and consumer-to-shop matching on the Web.
comment: 12 pages
★ Cooperative EV Charging Assignment Using Multi-Agent Reinforcement Learning: A Comparison of QMIX and Independent Q-Learning Under Urban Traffic Uncertainty
The allocation of electric vehicles to public charging stations is increasingly constrained by the fact that several heterogeneous facilities often serve the same stream of requests while observing only partial local information. An accept/reject decision that is defensible for one station may nevertheless reduce the efficiency of the charging district as a whole. We formulate this operational problem as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP) and examine whether Centralized Training with Decentralized Execution (CTDE), instantiated through QMIX, improves assignment performance relative to a cooperative Independent Q-Learning baseline (IQL). The evaluation uses a six-station urban network simulated with SUMO, five independent random seeds, 200 training episodes, and four traffic flux scenarios ranging from 100 to 250 vehicles/hour. Under this protocol, QMIX reaches a mean reward per evaluated decision of +0.776, versus +0.578 for IQL (delta = +0.198, seed-paired Cohens dz = 2.98, exact two-sided Wilcoxon p = 0.0625, the minimum attainable value at this sample size), and raises the acceptance rate from 72.4% to 94.0% (delta = +21.6 pp, dz = 5.24, Wilcoxon p = 0.0625). A five-variant ablation suggests this gain is primarily attributable to the CTDE architecture when holding cooperative reward terms fixed, though not formally isolated with a single seed. By contrast, cooperative reward shaping and experience sharing show no clear descriptive benefit and may slightly reduce performance. These findings support the view that, for multi-station charging assignment, coordination should be embedded in the learning architecture rather than added only through heuristic cooperation terms.
comment: 14 pages, 7 figures. Presented at ICATH 2026 (Rabat, Morocco, 15-17 July 2026); accepted for publication in MDPI Engineering Proceedings (in press)
★ EARS-Info: Information-Aware Event-Driven Reallocation for Multi-Robot Exploration
We introduce EARS-Info, an information-aware extension of Event-Driven Ant Reallocation Search for multi-robot exploration on OpenStreetMap-derived grids. It combines marginal sensing gain, feasible relocation, return-aware routing, sensing reservations, and cached shortest paths. Across 168,000 episodes comparing 30 implementations on eight city maps, EARS-Info achieves the highest mean operational score in every evaluation panel. With eight robots and 3% communication dropout, it improves the score over EARS-Safe by 5.43%, increases weighted discovery from 88.88% to 98.46%, and reduces mean energy use by 2.62%. Score differences are positive in all eight cities ($p=0.0078125$; descriptive 95% city-bootstrap interval $[0.0159,0.0694]$), with no simulated incidents across 5,600 EARS-Info episodes. These results support the complete controller; individual component advantages remain uncertain. Generalization is limited by reused geography, one training replicate, coarse maps, and adapted comparison methods.
★ Who Else Is In? Multi-Agent Collusion Initation Via Strategic Vagueness
Multi agent LLM systems raise the risk of collusion among agents pursuing objectives hidden from their overseers. When potential partners identities are unknown, collusion first requires initiation: agents must discover one another, often through a public channel observed by ordinary agents and monitors. Strategic vagueness offers a way to navigate this mixed audience, conveying intent to a prospective partner while preserving a plausible ordinary interpretation. We introduce a mutual identification game in which two schemers must identify each other through public interaction, while jointly solving an ordinary task. We find that some models discover successful handshake strategies without explicit signalling guidance, using task compatible exchanges that monitors often fail to flag at low false positive rates. To characterize the communicative capabilities underlying these exchanges, we study a controlled coordination setting. We find an asymmetric response to message explicitness: as messages become more indirect, monitor punishment can fall faster than partner coordination, creating a profitable intermediate regime of strategic vagueness. Together, these findings illustrate a Blackstone style evidential asymmetry: communication sufficient for coordination between agents may remain insufficient for confident detection by monitor, opening a new surface of collusion.
★ Solitary movement style predicts collective foraging while whom to follow emerges only in company
How far can behaviour inside a group, and the group's performance, be predicted from members measured while alone? We tested this on three public datasets of human collective foraging (621, 128 and 1,133 participants). A low-dimensional individual representation estimated from private-phase movement was mapped onto behaviour inside the group through a mapping learned on other participants. Private information processing was stable between the two halves of a session (split-half reliability, the correlation between the halves, 0.60--0.83) and predicted behaviour inside the group with correlations of 0.30--0.66. Selective social learning, following those who have succeeded or stopped, showed stable individual differences inside the group (reliability 0.45--0.59) yet was not predicted at all from the same person's private behaviour. Group performance was predicted with a correlation of 0.55 in all three datasets by simulations never fitted to group outcomes or by the mean of individually measured performance, a prediction carried by mobility and individual performance rather than by social weights. We found no credible evidence for an effect of how members were combined (equivalence bound 0.06). Whom to follow thus emerges only in company, which delimits what solitary measurements can predict and compose.
★ GARDEN: Generalist Autonomous Reward Design Through Evidence Navigation for Cooperative Multi-Agent Reinforcement Learning
Cooperative multi-agent reinforcement learning (MARL) enables teams of agents, such as robots and autonomous vehicles, to learn coordinated behavior, but what such teams learn is largely determined by their reward function. Because task success alone provides too little guidance for learning, intermediate rewards are usually designed by hand, and this effort must often be repeated whenever the environment or the team changes. Large language models (LLMs) can automate the effort by writing reward functions as code, yet the resulting rewards are typically designed and tested in a single setting. To enable their reuse across settings, GARDEN (Generalist Autonomous Reward Design through Evidence Navigation) is proposed, in which one reward program is developed for a family of related scenarios and transferred, unchanged, to scenarios that were not available during its design. During development, environment code, training records, and agent behavior are investigated by an LLM reward engineer, and draft rewards are tested in trial training runs, while the learning algorithm and the rule for selecting the final reward are held fixed. On unseen scenarios of the Overcooked and SMAX benchmarks, task returns were increased by 58.2% and 52.5% over the default rewards on two kitchen layouts, and the mean win rate in two combat scenarios was raised from 51.0% to 89.9%. Under matched candidate and probe budgets, this exceeded the 84.2% and 79.2% obtained with fixed diagnostic feedback and Eureka-inspired reward reflection. These results show that a single autonomously engineered reward can be reused effectively across related cooperative scenarios.
comment: 8 pages, 3 figures, 5 tables. Code: https://github.com/DoganUrgun/GARDEN
★ Multi-Agent Reinforcement Learning for Global Climate Policy Cooperation NeurIPS 2026
International climate policy requires coordination among countries whose mitigation decisions impose local costs but generate global benefits. Classical game-theoretic and integrated assessment approaches can study these incentives, but typically rely on prescribed behavioural assumptions or equilibrium concepts. Recent work has therefore explored Multi-Agent Reinforcement Learning as a way to model climate-policy cooperation as an emergent outcome of repeated strategic interaction. We extend an existing MARL climate framework with a multi-stage negotiation mechanism in which agents propose agreements, accept or reject commitments, and face penalties for non-compliance. This allows us to study how agreement design affects learned mitigation behaviour, compliance, and climate outcomes. In an idealized setting with four homogeneous agents, we find that non-binding agreements do little to overcome free-riding, while moderate non-compliance penalties improve cooperation and reduce warming. However, excessively strong penalties discourage ambitious commitments, revealing a trade-off between enforcement and participation. We position the framework as a methodological contribution for testing agreement design rather than a predictive model of real negotiations.
comment: Tackling Climate Change with Machine Learning: workshop at NeurIPS 2026
★ Measuring Where Intent Gets Lost in Agentic Smart Contract Synthesis
AI agents can now write software from a plain-English description, but code that runs is not the same as code that does what was asked. This gap matters most for smart contracts, the programs behind Web3 applications that move money on a blockchain and cannot be fixed once deployed. We ask how to check whether an AI-written contract behaves as it should, and where the requested behavior gets lost. We design an evaluation built on execution rather than opinion. We replay the same transactions on an AI-written contract and on the original it should reproduce and measure how often they agree. We test every grading tool on planted mistakes, compare against an answer key written without the description, and audit our own scorer until known wrong answers cannot pass. Common methods, such as AI judges and standard analysis tools, miss obvious mistakes, and scoring choices matter too. Checking only the functions every system includes hid about half the difference between pipelines. We then propose an elicitation-first architecture. Before writing code, the agent asks the client a short checklist of questions, checks the contract against an independently written executable answer key, and returns disagreements to the client instead of silently rewriting code. Asking first matters far more than reviewing or patching afterwards, across three AI models, while automated security fixes often break requested behavior. A few well-chosen questions recover much of what a short description omits, and their value can be estimated without the original contract. To our knowledge, this is the first study in smart contract synthesis to gather missing requirements before generation and check the result against an independently written executable model. The lessons extend to any setting where AI agents turn requests into programs. We release our tools, 600 tasks and all results.
comment: ACM TheWebConf2027 8 pages main content 4 pages appendix
★ Selective and Hybrid-State Communication for Latent Multi-Agent Systems NeurIPS 2026
Efficient communication remains a major bottleneck in large language model multiagent systems. Latent communication reduces repeated natural-language exchange by transferring internal model states between agents, but these transfers remain costly and are typically designed around conventional KV-cache representations. We address this problem from two complementary perspectives. We introduce SelectKV, a boundary-aware method that selectively communicates KV states while protecting inherited context, and HyLaMAS, which extends latent collaboration to hybrid linear-attention models by communicating both KV caches and recurrent states through a shared workspace. Across six benchmarks, SelectKV reduces communicated KV positions and logical KV payload by 9.7-9.9%, with accuracy unchanged on two benchmarks, within one percentage point on three, and six points lower on HumanEval+. Across Qwen3.5 model scales, HyLaMAS reduces decoded output tokens by 53-80% and achieves 1.16-2.78x speedups relative to text-based multi-agent communication. Together, these independently evaluated methods represent two complementary approaches to efficient latent-state communication across conventional Transformer and hybrid model architectures.
comment: Accepted at the NeurIPS 2026 Workshop on Resource-Aware Agentic AI (RAAAI)
♻ ★ Annotrace: A Workbench for Step-Level Annotation of Agent Traces AAAI 2027
Every study of why LLM agents fail begins by labelling traces by hand, and every such study builds its own throwaway tool. Observability platforms record agent traces but offer no structured labelling; annotation tools label text but have no notion of a step, a delegation or a tool call, leaving it unclear what the smallest unit of execution to label should be. Annotrace reads six trace formats, including OpenTelemetry GenAI, LangSmith and Langfuse, and turns each run into an ordered, nested list of steps. The annotator writes their own labelling codebook; a frontier model proposes a starting set from a sample of their own traces, answers first, and is asked exactly the questions the human is asked, making annotation a joint human-AI process. Annotrace measures agreement between any two annotators, human or AI. On a multi-agent question-answering corpus it reproduces a previous manual effort's 235 annotation units exactly, and two models labelling the same traces agree 91% of the time at $κ= 0.82$.
comment: 4 pages, 4 figures, 1 table. Under review at the AAAI 2027 Demonstration Program
♻ ★ HGWM: Hierarchical Graph-Guided World Model for Zero-Shot Object Navigation via Graph Matching
Zero-shot object navigation requires an embodied agent to choose where to search while retaining evidence from previous observations. HGWM couples an LLM-generated room-anchor prior with persistent spatial-semantic memory. A coarse stage assigns visual-semantic scores to candidate headings on a 0-10 scale; a fine stage measures symbolic compatibility between observed objects, room context, and the goal prior. Phase-dependent fusion combines these scores with an LLM direction recommendation before local action selection. On HM3D v0.1, the reported HGWM configuration achieves 62.6% success rate (SR) and 31.5% success weighted by path length (SPL), compared with 58.1% and 31.2% for WMNav. On MP3D, the corresponding values are 46.8% and 17.9%, compared with 45.4% and 17.2%. The results indicate higher success point estimates and modest path-efficiency gains. On HM3D v0.2, the full model obtains 74.4% SR and 31.9% SPL. Additional component ablations reduce SR by 0.9-2.3 percentage points and SPL by 0.6-1.5 points. These are author-reported aggregate measurements. The accompanying implementation repairs pass offline tests; episode-level rerun records are not available. Evaluation is confined to simulation.
♻ ★ Streaming Communication in Multi-Agent Reasoning
Multi-agent reasoning systems adopt a generate-then-transfer paradigm that forces end-to-end latency to scale linearly with pipeline depth. We introduce StreamMA, a multi-agent reasoning system that streams each reasoning step to downstream agents as soon as it is generated, pipelining adjacent agents and thus reducing latency. Surprisingly, this pipelining also improves effectiveness: because multi-step reasoning quality is non-uniform and early steps are more reliable than later ones, working with these reliable early steps instead of the full chain prevents error-prone late steps from misleading downstream agents. We formalize both advantages with the first closed-form joint analysis of stream, serial, and single protocols, deriving the effectiveness ordering, speedup upper bound, and cost ratio. Across eight reasoning benchmarks spanning mathematics, science, and code, three frontier LLMs (Claude Opus 4.6, GPT-5.4, and GLM-5.2), and three topologies (Chain, Tree, Graph), StreamMA outperforms both baselines (avg.+7.3pp, max +22.4pp on HMMT2026; Claude Opus 4.6-High). Beyond these contributions, we discover a step-level scaling law: increasing per-agent steps consistently improves both effectiveness and efficiency, a new scaling dimension orthogonal to and composable with agent-count scaling.
comment: project page: https://zhenyangcs.github.io/StreamMA-website/
♻ ★ PIVOT: Bridging Planning and Execution in LLM Agents via Trajectory Refinement
Large language model (LLM)-based agents frequently generate seemingly coherent plans that fail upon execution due to infeasible actions, constraint violations, and compounding errors over extended horizons. PIVOT (Plan-Inspect-eVOlve Trajectories) addresses this plan-execution misalignment through a self-supervised framework that treats trajectories as optimizable objects iteratively refined via environment interaction. The framework comprises four stages: PLAN generates candidate trajectories; INSPECT executes them and computes structured losses with textual gradients encoding plan-execution discrepancies; EVOLVE applies these signals to produce improved trajectories; and VERIFY performs a final global check against task constraints. A monotonic acceptance process ensures a non-decreasing solution quality. Empirical evaluations on DeepPlanning and GAIA demonstrate state-of-the-art performance: with human-in-the-loop (HITL) feedback, PIVOT establishes a strong upper bound up to 94% relative improvement in constraint satisfaction, while its fully autonomous variant retains substantial gains, showing that the core trajectory-refinement mechanism remains effective without external supervision. At the same time, PIVOT remains computationally efficient, requiring up to 3x to 5x fewer tokens than competing refinement methods. These findings establish that (self- or human-supervised) feedback-based trajectory optimization is a principled methodology for mitigating plan-execution gaps in autonomous agent systems.
Systems and Control (EESS)
★ Pushing Dexterity and Safety for In-Hand Object Pose Reaching That Works Remarkably Well
Dexterous manipulation with multi-fingered hands is the next frontier of robotics, yet learned hand skills remain limited to simple grasps or basic repeated motions, such as rotating an object. As robots attempt more demanding in-hand maneuvers, safety, such as not dropping the object, becomes critical, yet it is rarely measured. We study in-hand object pose reaching, a task that demands diverse and difficult grasps: moving an object to almost any position and orientation within reach of the fingers, at any wrist orientation, through long sequences of consecutive goals. We present a recipe for training such policies for a LEAP hand in simulation without human data, built on a range of insights, from diverse resets and goals via generated grasp datasets to an adaptive friction curriculum. With careful system identification, the policies transfer zero-shot to the real hand, using either motion capture or a single wrist-mounted depth camera, and prove remarkably capable and robust. On top of the task policy, we add a learned residual safety filter with adjustable conservativeness, letting us tune the balance between performance and safety and greatly reduce object drops. In 13.3 hours of real-robot evaluation over 4,800 unseen goals across 4 objects and diverse wrist orientations, the state-based policy reaches 95.2% of goals within 2 cm and 0.4 rad, and the filter cuts its drops by 75% while still reaching 91.5% of goals. The policies can also spin a lightbulb held in an arbitrary pose, reach out-of-distribution goals, withstand fast arm motion and perturbations, and, needing only a wrist camera, serve as portable skills such as screwing in a lightbulb. We release the full recipe as a platform for studying performance, reliability, and safety in high-dimensional dexterous systems. Videos and more are available at https://ian-chuang.github.io/dexsafe/.
comment: 26 pages, 30 figures, 14 tables. Project page: https://ian-chuang.github.io/dexsafe/
★ A heuristic for optimal coordination of autonomous vehicles: real-time algorithms
When coordinating connected and automated vehicles at an unsignalized intersection, the crossing order and the vehicle trajectories must be decided jointly. For a fixed crossing order, the problem can be formulated conveniently in a bi-level form where each vehicle solves its own optimal control problem, whose optimal value is a function of its entry and exit times, and a coordinator optimizes these times subject to precedence constraints. We compute the first and second order sensitivities of lower-level problems, which are required to solve the upper-level problem, by means of algorithmic differentiation using the real-time software package \texttt{acados}. This implementation exploits tailored algorithms to obtain a computationally efficient implementation of the optimal coordination algorithm. We validate our implementation in numerical simulations that demonstrate both real-time feasibility and effectiveness of our algorithm.
★ An Open-Source Hybrid Ray-Tracing and 3GPP Channel Simulator for Non-Terrestrial Network
This paper proposes a hybrid channel simulation framework that combines site-specific ray tracing (RT) with 3GPP-compliant statistical channel simulation for non-terrestrial networks (NTNs). The proposed approach uses RT to generate dominant propagation paths and augments them with statistically generated paths and subpaths to provide richer multipath realizations under practical simulation complexity. To provide a complete simulation framework, we integrate Sionna RT, 3GPP, and hybrid RT-3GPP simulators within a unified workflow. Simulation results based on key channel statistics and power delay profiles show that the proposed method preserves site-specific propagation characteristics while providing sufficiently rich multipath components, thereby addressing the complementary limitations of conventional RT-based and purely statistical NTN channel simulators. The developed simulator is also released as an open-source project to support reproducibility and further NTN research.
★ Computing GR(1) Assume-Guarantee Contracts
Assume-guarantee contracts provide a set of algebraic operations that enable the efficient compositional design and analysis of complex engineering systems. The explicit computation of these operations at component interfaces is currently supported only for some logics lacking a notion of time. This paper develops theory and algorithms to compute these operations when the contracts are expressed in the GR(1) modulo theory. The key challenge is the computation of two quantifier elimination problems for temporal formulas. We develop context-aided variable elimination procedures on GR(1)-modulo formulas, establish their soundness, characterize subproblem-wise optimal solutions within an admissible class of formulas, and use the procedures to compute contract operations, including composition and quotient. We implement the resulting framework in Pacti, automating compositional reasoning over GR(1) contracts. Finally, we demonstrate the framework on an automated emergency-braking case study, where we compute system-level contracts and expose compatibility conditions arising from the interaction of component specifications. Experiments on synthetic families show that the cost is governed by the number of eliminated variables and liveness conjuncts, and a comparison with exact automata-based elimination shows complementary scaling.
comment: 55 pages, 10 figures
★ Coordinated and Uncoordinated Channel Tracking for High-Rate IoT in Distributed Massive MIMO With Low-Resolution ADCs
This paper investigates dynamic channel tracking for high-rate Internet of things (IoT) in distributed massive multiple-input multiple-output (mMIMO) with low-resolution analog-to-digital converters (ADCs), where pilot and data activities are decoupled, pilots are reused, and mobile channels evolve across access slots. We hereby develop a quantization-aware joint coordinated Kalman filter (Q-JCKF) for known pilot activity and a quantization-aware distributed probabilistic data association filter (Q-DPDAF) for uncoordinated access. For uplink analysis, we derive a closed-form statistical use-and-then-forget (UatF) expression for Q-JCKF and mixture-aware, pilot-conditioned, and long-term statistical UatF formulations for Q-DPDAF, together with the large-N pilot-reuse ceiling and a cross-access point (AP) association term with an O(epsilon) posterior-concentration bound. The proposed framework recovers JC-KF/PDAF in the ideal-ADC co-located limit, and quantifies the conditional loss of Kalman measurement information caused by finite-resolution quantization. We further characterize computational, memory, and fronthaul scaling: Kronecker-structured Q-JCKF reduces covariance-processing complexity, while groupwise Q-DPDAF limits hypothesis enumeration from 2^K to 2^{K_r} and enables parallel AP-local processing. Simulations corroborate the analytical and complexity outcomes, quantify ADC-resolution and pilot-access effects, and demonstrate improved lower-tail spectral efficiency, robust mobility tracking, near-ideal 4-bit performance, reliable Q-DPDAF activity inference, and substantial runtime reductions over legacy implementations and processing frameworks.
★ Autonomous fixed-wing interception using reinforcement learning
Medium altitude, long endurance unmanned aerial systems such as Shahed-type strike drones are an unanswered threat in modern conflicts. Conventional air defense systems are too expensive to provide a feasible large-scale solution. Quadcopter interceptors are a cheap, battle-proven alternative, but their extremely limited flight time restricts them to low altitude targets at short range, whereas fixed-wing platforms have the endurance to reach higher altitude targets from longer distances. However, such interceptors are still operated manually. Recent developments in learning-based controllers show potential for automation. In this work, a fixed-wing interceptor is presented that is controlled by a neural network that maps filtered monocular vision observations directly to actuator commands. The policy was trained in a custom Python environment using reinforcement learning and proximal policy optimization. The policy was then deployed on a fixed-wing platform and tested in real interception scenarios. As a baseline, two classical guidance controllers were also developed and tested in simulation. The policy is able to intercept targets both in simulation and in real flight. In simulation, the policy outperforms the classical controllers under most conditions. The interceptor was ultimately successfully validated in real flight by intercepting maneuvering fixed-wing targets. This novel system shows the potential of artificial intelligence for autonomous control of fixed-wing interceptor drones.
comment: MSc thesis, Faculty of Aerospace Engineering, Delft University of Technology, 2026. 24 pages, 29 figures, 4 tables
★ Electric-Field Induction Motor with Discrete Circuit Elements and Improved Rotor Dynamics
Electric-field-based actuators and machines lead the design space at the nanoscale and in biological systems, while having the potential to be used in favorable macroscale applications which prioritize minimal mass, performance in a vacuum, or performance in high temperatures. Electric-field induction motors consist of a set of stator electrodes which induce opposite charges onto a movable rotor surface. In earlier work, rotor surfaces for electric-field induction motors have been made of single continuum materials, which are limited in possible resistivity, permittivity and circuitry characteristics. To enhance design freedom, we develop a rotor for electric-field-based machines which may be constructed out of discrete circuit components between a finite number of conductive vanes. Magnetic-field induction motors, which are common in machinery, often utilize ``deep bar'' rotors, consisting of conducting bars which decrease the rotor time constant with increasing frequency using the skin depth effect. We propose, analyze, design and demonstrate a macro-scale electric-field induction motor which uses circuitry on the rotor surface to emulate the ``deep bar'' design, providing improved torque over a wide range of speeds.
★ Joint Antenna Positioning and Pilot Power Allocation for Uplink Fluid Antenna-Aided Cell-Free Massive MIMO Systems
This paper investigates joint fluid antenna (FA) position optimization and pilot power allocation for uplink cell-free massive MIMO networks supporting ultra-reliable low-latency communication (URLLC) traffic under pilot contamination. By continuously adjusting FA elements within a prescribed aperture, we adapt the spatial correlation structure to the varying propagation environment. We formulate a joint problem to minimize the average normalized mean-squared error (NMSE) subject to QoS constraints governed by finite blocklength and Markov-modulated Poisson process queueing analyses. To ensure tractability, we establish an analytical quadratic relationship between the NMSE and the effective signal-to-interference-plus-noise ratio through eigenstructure alignment factors. This mapping translates complex latency requirements into explicit NMSE thresholds, maintaining a median absolute approximation error below 0.03 dB and less than 10% relative error for over 90% of practical-regime samples. Given the nonconvex nature of the formulation, we propose an alternating optimization framework. Antenna positions are iteratively updated via successive convex approximation using analytical gradients and closed-form Euclidean projections. Pilot powers are allocated through a block coordinate descent scheme that employs safeguarded Newton steps to guarantee monotonic convergence. Furthermore, we introduce an auto-calibrated regularization variant to navigate the Pareto frontier between estimation accuracy and energy consumption. Numerical results show the proposed positioning yields up to 22% NMSE improvement over fixed uniform arrays while operating 7-8x faster than particle swarm optimization. The power allocation achieves within 0.5% of the global optimum at over two orders of magnitude lower computational cost, providing up to 39% pilot energy reduction with negligible estimation penalty.
★ Privacy-Preserving Over-the-Air Federated Learning over Cell-Free Massive MIMO System with User-Centric Clustering
This paper investigates cross-layer design for differentially private federated learning over cell-free massive MIMO networks with over-the-air (OTA) aggregation. Distributed access points (APs) collaboratively serve user equipments (UEs) through user-centric clustering, where clipped local model updates are perturbed with Gaussian noise and simultaneously transmitted over a shared uplink for OTA averaging at a central processing unit. Exploiting channel hardening, we show that thermal noise inherent in wireless reception can act as a privacy-masking resource, reducing or even eliminating the need for artificial noise. Based on this observation, we formulate a joint optimization problem over AP-UE associations, UE transmit powers, and artificial noise variances to minimize aggregation variance subject to per-UE zero-concentrated differential privacy constraints derived from an ergodic privacy accounting framework. Analysis reveals a regime-dependent structure: stringent privacy requirements favor low transmit powers and sparse AP associations to exploit thermal noise for privacy, whereas relaxed requirements favor larger clusters to improve aggregation quality. The resulting mixed-integer nonconvex problem is solved via alternating optimization combining difference-of-convex power allocation and greedy coordinate-descent clustering. Numerical results show up to 89% lower aggregation variance than conventional baselines and enable 97% test accuracy on MNIST with consistent gains on CIFAR-10. Under strict privacy budgets, competing schemes remain near random-chance performance, while under relaxed budgets they continue to lag significantly behind the proposed design.
★ A New Parametric Uncertainty Approach to Systems with Time-Varying Delay and Feedback Stabilization
Delay Differential Equations (DDEs) with an uncertain time-varying state delay are often modeled as a feedback interconnection of a nominal time-invariant system with a time-varying uncertainty operator. However, typical choices of uncertainty operators do not admit tight characterizations of input-output properties. In this paper, we propose a new representation of a DDE with time-varying delay system as a feedback interconnection of a nominal fixed delay DDE with a time-varying multiplicative parametric uncertainty operator, where the parameter is bounded in $L_\infty$ norms -- yielding an operator norm bound on the uncertainty. We then provide a modified uncertainty representation which uses the $L_2$ norm of the parameter to bound the operator norm of the uncertainty. Finally, we show that by representing the nominal system using Partial Integral Equations (PIEs), these characterizations of the uncertainty operator can be used for stability analysis or stabilizing feedback controller synthesis. The effectiveness of the approach is illustrated with simple numerical examples.
★ Probabilistic Sequential Safety for Stochastic Dynamical Systems
In this letter, we investigate a notion of probabilistic safety for a stochastic dynamical system. Standard safety notions are based on the idea that an unsafe region of the state-space must be avoided at each instant. In contrast, we study a sequential safety notion which is based on avoiding a tuple of target regions of the state-space. The proposed safety notion is inspired by many practical systems, ranging from modeling jailbreak attacks in large language models (LLMs) to predicting hazardous chemical reactions. In solving the sequential safety problem, we solve a sequential reachability problem for a stochastic dynamical system.
★ Spiking Neuromorphic Control for Periodic Orbits Stabilization in Underactuated Robots: A Greedy Lyapunov-Based Approach
This work investigates the use of event-based spiking controllers, implemented as spiking neural networks, for trajectory tracking in underactuated robots. Specifically, we address the problem of stabilizing a prescribed periodic orbit. It is well known that, by applying the method of transverse linearization to the robot dynamics in a neighborhood of a periodic trajectory, the dynamics can be approximately described by a linear time-varying system with periodic coefficients (a periodic LTV system). The problem of orbital stabilization can therefore be reduced to stabilizing the trivial solution of this LTV system. We show that a previously proposed Lyapunov-based greedy design, originally developed for LTI systems, can also be extended to the corresponding linearized transverse dynamics, which take the form of a periodic LTV system. This enables the construction of a feedback controller for orbital stabilization of the robot. The effectiveness of the proposed method is demonstrated on the trajectory stabilization problem for the butterfly robot.
★ A heuristic for optimal coordination of autonomous vehicles: dynamic problems
The coordination of autonomous vehicles at unsignalized intersections requires deciding both the trajectories of the vehicles and their crossing order, with the additional difficulty that the set of vehicles to be coordinated changes as traffic flows through the intersection. Since the resulting mixed-integer optimal control problem is hard to solve online, we propose an arrival-triggered receding-horizon scheme built around a schedule-refinement heuristic: for a fixed crossing order, a single nonlinear program (NLP) optimizes all trajectories and crossing times, and the first-order sensitivities of the cost with respect to the vehicle schedule are used to select adjacent swaps that are likely to reduce the overall cost. Replanning is triggered only when a new vehicle arrives, the new vehicle is appended at the tail of the current crossing order, and a memory of evaluated swaps that persists across replanning steps avoids redundant computations. Simulations of traffic scenarios involving streams of a few hundred heterogeneous vehicles show that the proposed scheme recovers most of the optimality gap incurred by first-in-first-out scheduling.
★ A heuristic for optimal coordination of autonomous vehicles: static problems
We consider the coordination of autonomous vehicles at unsignalized intersections, where trajectory planning and crossing-order decisions are intrinsically coupled, thereby making their simultaneous optimization intractable in practice. To this end, we propose a heuristic based on a hierarchical structure consisting of a series of nonlinear programs (NLP) in the lower level, which optimizes trajectories of each vehicle for a given crossing order, and a high-level heuristic to design new crossing schedules that exploits sensitivity information from the lower-level NLPs. Extensive numerical simulations across a broad range of traffic scenarios demonstrate that our strategy achieves substantially improved scheduling with respect to first-in-first-out ordering while preserving computational tractability.
★ A Bi-level Co-design Framework for Lunar Monoped with 2D Jump and Landing Control
Legged robots for lunar exploration must be designed for low gravity, where reduced contact forces and long flight phases make leg morphology and controller choice interdependent. We study this for a planar monoped in lunar gravity ($1.625$ m/s$^2$) using a bi-level co-design framework: a differential evolution outer loop searches over link lengths, and each candidate is evaluated on a vertical jump by either a quadratic program based task-space controller (QP-TSC) or a direct-collocation based trajectory optimization. The two controllers yield different optimal link lengths, and trajectory optimization outperforms the QP-TSC across most of the design space, with 17% lower Cost of Transport (COT) at their respective optima. We then extend to a 2D jump, where trajectory optimization with centroidal angular momentum regulation produces stable in-place and forward jumps, and an LQR controller absorbs impact and stabilizes the robot after touchdown. This is demonstrated on a baseline morphology with both links 0.3 m long.
comment: 8 pages, 13 figures
★ Structure-Actuation Co-Design of All-Electric Heavy-Duty Robotic Manipulators
This paper presents a mixed discrete-continuous multiobjective bilevel framework for the structure-actuation co-design of all-electric heavy-duty robotic manipulators. The upper level optimizes manipulator geometry and actuation topology, while joint-specific lower-level searches construct feasible archives of discrete motor, gearbox, and screw configurations. A system-level coordinator selects actuator packages according to hardware mass, co-design cost, and loss-aware supply energy, and the resulting actuator masses are fed back to the robot model until a mass-consistent fixed point is obtained. Evaluation on a full-pose SE(3) benchmark with a 100-kg tool payload yields eight feasible nondominated designs, all satisfying the 12 prescribed poses. Actuator-mass coupling converges in three iterations for every reported design. A fixed-design ablation shows that one-pass sequential sizing changes hardware mass by up to 2.13%, co-design cost by 10.20%, and supply energy by 9.40% in magnitude. Payload screening further reveals increasing mass-cost-energy demand and changes in competitive mechanism topology with carried load.
comment: 6 pages manuscript submitted to a conference (ICCMA 2026)
★ Noise Steering: Exploiting Wake Steering and Acoustic Directivity for Noise-Constrained Wind Farm Control via Reinforcement Learning
Existing noise mitigation strategies for wind farms usually consist in reducing turbines power output, without accounting for the wake interactions between them and the noise emission directivity. This leads to over-conservative policies and unexploited wind power. Yet, turbine yaw can simultaneously affect wake interactions and the direction of noise emission, offering an additional degree of freedom for farm-level noise control. We therefore propose Noise Steering, a model-free reinforcement learning (RL) strategy that jointly controls turbine yaw angles and power setpoints to maximise farm power while satisfying a local noise constraint in a defined region. To enable training at a reduced computational cost, a low-fidelity acoustic model is coupled with the steady-state FLORIS simulator. The RL agent effectively learns how to redirect wakes and noise directivity away from the protected region while redistributing power across the farm. On a six-turbine test farm, the policy satisfies the 40 dB noise constraint throughout the full range of turbulent winds, with an average power deficit of only 2% relative to greedy unimpeded control, and up to 7.7% at high wind speeds. In comparison, the industry-standard noise-reduced operation (NRO) baseline here generates an average power loss of 17%. When deployed in the higher-fidelity FAST.Farm simulator, the policy largely retains its performance under steady and turbulent inflow, despite differences in simulation fidelity. These results demonstrate the potential of combining yaw control and power derating through RL to improve the power-noise trade-off in wind farm operation.
comment: 17 pages, 11 figures
★ Co-Optimization of Network Topology and Variable Impedance Devices under Dynamic Line Ratings Using Iterative McCormick Relaxation
Grid Enhancing Technologies (GETs), such as Network Topology Optimization (NTO), Dynamic Line Rating (DLR), and Variable Impedance Devices (VIDs) are increasingly deployed in power transmission grids to address congestion and improve grid utilization. However, their growing adoption necessitates coordinated operation to fully exploit their benefits. This paper presents a unified co-optimization framework that integrates NTO, DLR, and VID to alleviate congestion and minimize operational costs. NTO is modeled using a node-breaker formulation to provide enhanced switching granularity, while DLR incorporates weather-dependent parameters, including wind speed and ambient temperature, to dynamically adjust transmission limits. The resulting formulation is a mixed-integer nonlinear program due to power flow constraints with bilinear VID terms, and discrete topology decisions. To improve tractability, we propose using a McCormick relaxation approach to linearize bilinear terms. Furthermore, we propose an iterative refinement of the McCormick relaxation scheme to enhance solution accuracy. The proposed framework is validated through detailed analyses on the IEEE 118-bus system, along with additional case studies on other benchmark systems to assess scalability. Results demonstrate that the proposed approach improves solution accuracy compared to standard McCormick relaxation, achieves solution quality close to the nonlinear formulation, and maintains computational efficiency.
★ Gradient flow of magnetic inclination and intensity as a GPS-free guidance mechanism
It is well established that some animals, such as sea turtles, sense the Earth's magnetic field and use it as a primary cue for long-distance navigation across the vast and featureless ocean. This remarkable ability has inspired the development of bio-inspired magnetic navigation (BIMN), which seeks to mimic these animals to develop robust and frugal navigation methods. BIMN exploits the Earth's magnetic field, sometimes described as ''Nature's GPS'', as a natural navigation infrastructure, providing a potential alternative to vulnerable satellite navigation systems. However, unlike geographic coordinates, magnetic coordinates extracted from the Earth's magnetic field do not provide a one-to-one mapping to geographic locations. Different locations can exhibit similar magnetic signatures, creating ambiguity and making BIMN inherently more challenging. Despite this fundamental challenge, we demonstrate that, within predefined regions of the Earth, magnetic inclination and intensity provide sufficient information to navigate to any destination. By formulating BIMN as a gradient flow control and guidance problem, we show that the resulting gradient flow act as current that carries the system and can be controlled to guide it through intermediate waypoints toward the desired destination. The resulting framework provides a GPS-free navigation mechanism with potential applications in both engineering and biology. In engineering, it can be used to establish a geomagnetic navigation infrastructure through which prescribed routes can be defined and followed, while in biology, it can be used to analyse the migration routes of animals known to rely on magnetoreception.
★ Verifying Control Invariance of H-Polytopes is coNP-Complete
We prove that verifying whether an H-polytope is control invariant for a discrete-time linear system with polytopic input constraints is \textsc{coNP}-complete. This establishes a fundamental computational complexity barrier for set-theoretic control. As an immediate consequence, the exact computation of maximal control invariant subsets of H-polytopes is \textsc{coNP}-hard. These hardness results hold regardless of whether the input constraint set is given in H- or V-representation.
★ Whittle Index Policy for Three State Collapsing Bandits
Partially observable restless multi-armed bandit models have been studied for dynamic resource allocation problems under uncertainty. This paper analyzes three-state restless multi-armed bandits with partial observability. These are called collapsing bandit problems if the state of an arm (bandit) is perfectly observable when it is scheduled for allocation. The state is not observable when the arm is not scheduled. The planner has a budget constraint on the scheduling decisions in each time slot. The planner's goal is to maximize expected cumulative long term discounted reward. We study the Whittle index policy. To analyze index policy, first the $N$-armed collapsing bandit problem is decoupled into $N$ single arm problems, and convexity and monotonicity of the optimal value function in belief vector are derived using monotone likelihood ratio (MLR) ordering on state transition dynamics and monotone reward structure assumption. We next show that the optimal policy is of threshold type in belief under sufficiency conditions on the model. These sufficiency conditions depend on Lipschitz bound on the value function in the belief. We further prove the indexability for an arm using Lipschitz bound on subsidy sensitivity with minimal restrictive assumption on the transition probability model and discount factor $β.$ Our results generalize to three state and strengthen existing results on threshold policy and indexability with weaker model assumptions. We provide index computation algorithm and we present comparison of the Whittle-index, myopic policy and round-robin policies in the simulation study.
★ Feedback autocratic strategies in stochastic games
A stochastic game is an optimal control problem for multiple agents in a stochastic environment. Stochastic games contain important practical problems, related to environmental issues or traffic flows. If the number of environmental states is one, such stochastic games are called repeated games. In repeated games, several strategies which unilaterally control payoffs have been proposed, and these strategies work in a mechanism similar to feedback control. In this paper, we generalize these strategies as feedback strategies in stochastic games, and investigate their properties. In particular, we provide a necessary and sufficient condition for such feedback strategies to work as controllers of payoffs under some assumptions. We demonstrate the performance of such feedback strategies in several examples related to social dilemmas. Our findings connect the theory of payoff control in repeated games to the theory of feedback control in stochastic games.
comment: 19 pages, 2 figures
★ Retrofitting centralized air compressor stations with end-use-aware predictive set-pressure control using a physics-constrained pressure-drop model
Compressed air systems usually run at a constant set pressure well above the end-use requirement; the excess margin is wasted energy. A lightweight hierarchical nonlinear model predictive control (NMPC) scheme, retrofitted to the sequencing controller of a centralized compressor station typical of small and medium-sized plants, lowers the set pressure as far as the end-use requirement allows. It uses only two pressure gauges (receiver and one critical end-use point) and compressor data, without flow meters or piping surveys. The end-use pressure is predicted by a low-order pressure-drop model identified by least squares under stability, damping and steady-state equilibrium constraints, which serve use rather than accuracy: they let the measured pressure drop be held over the horizon with unit gain. Validated in three stages (plant data, a pipe-network simulator checked against measurements, and the actual controller, at most 1.41 s per update), the NMPC meets the required pressure under real demand and reduces the estimated annual energy consumption by 2.8 % relative to the current constant operation (9.3 % relative to 0.70 MPa). A PID cascade with the same gauge saves about as much only when tuned by a standard rule at its recommended setting from a constant-load step test, difficult in an operating plant; the NMPC needs no such test and keeps its performance with its initial model and default weights after plant expansion and network split. The margin becomes a design parameter instead of a fixed excess, and the controller's signals show where the supply system lacks headroom.
comment: 31 pages, 17 figures
★ Thermal-Mass Coupled Modeling and Optimal Scheduling of Aluminum Smelting Loads for Safe Demand Response Participation
Aluminum smelting loads (ASLs) offer substantial flexibility potential, yet their actual participation in demand response remains conservative owing to the risk of operational failures. Power adjustments disturb the electrolyte temperature and superheat, which may lead to electrolyte freeze-out or yield loss. Existing ASL demand response methods often rely on oversimplified dynamic models of the electrolytic cell's thermal behavior, such as linear models and the Equivalent Thermal Parameter (ETP) model, which may either violate temperature constraints or underestimate the available flexibility. To address this limitation, this paper proposes a Thermal-Mass Coupled (TMC) dynamic model. The model captures the essential physical mechanisms of electrolytic cells, including alumina feed dynamics, ledge phase-change dynamics, and state-dependent current efficiency. The resulting nonlinear relations are reformulated into a tractable mixed-integer linear programming (MILP) model using Taylor approximations and McCormick-envelope reformulations. In the illustrative PBDR and IBDR case studies, the nonlinear reference-model assessment shows that the TMC model achieves net economic improvements 122.4% and 325.1% higher, respectively, than the ETP model. The proposed framework can therefore support deeper demand response participation by aluminum smelters while explicitly assessing adherence to operational safety constraints.
comment: Accepted for publication in IEEE Transactions on Smart Grid. DOI: 10.1109/TSG.2026.3736640. This is the author-accepted manuscript
★ A Topology-Aware Neural Network for Fault Location With Missing Measurements
This paper proposes a topology-aware neural network-based fault locator (TANFL) for radial distribution feeders, which uses voltage and current phasors obtained after the decay of fault induced transients, as well as feeder connection relationships and line parameters, as inputs. The shared bus encoder, bus readout module, and gated line readout module work together to identify the faulty bus or line and estimate the continuous fault location on the selected line. During training, explicit missing data indicators and random bus data block masking are introduced, enabling a single trained model to handle both complete measurement sets and scenarios where any permitted bus measurement device fails. On the IEEE 15 bus feeder, this paper compares TANFL with a conventional multilayer perceptron (MLP) and the formula-based localization method under identical data, measurement availability scenarios, and evaluation metrics. On the test partition, TANFL outperforms the conventional MLP; with complete measurements, its line fault location performance is slightly lower than that of the formula-based method, but in the average results for scenarios with one missing bus measurement block, its accuracy is the highest among the three methods at a 5% tolerance.
★ Phase-Coordinated Model Predictive Control for AI Training Power Smoothing
Periodic power fluctuations from artificial intelligence (AI) training workloads can excite oscillations in power systems. This paper proposes a model predictive control (MPC) framework that exploits short start-time delays to mitigate power fluctuations at the computing source. By coordinating the relative phases of independent training jobs, the proposed method reduces the superposition of oscillatory power components without interrupting running jobs or requiring additional power-smoothing hardware. To obtain a tractable optimization formulation, cosine and sine projections are employed to quantify the dominant oscillatory power components, and their magnitudes are used to construct a convex quadratic power-smoothing objective. The receding-horizon framework updates launch decisions as new workload information becomes available while maintaining scheduling feasibility under limited computation time. Numerical results demonstrate reductions of 74.39\% in mean-square power fluctuations within the 0.2--2-Hz band and 46.6\% in power standard deviation, with an average launch delay of 2.188~s. All launch deadlines are satisfied, and the maximum end-to-end computation time is 0.390~s, below the 0.50-s MPC update period.
★ A Singularity Avoiding Track Model for Race Car Optimization
A track fitting pipeline is presented that converts a measured racetrack point cloud into a closed, $C^4$ differentiable, three-dimensional surface model suitable for high-fidelity Modelica race-car optimization. The surface is a Monge patch, parametrised by a curvilinear arc-length coordinate measured along the minimum-curvature line (MCL) and a signed lateral offset, with a height polynomial of user-selectable degree capturing the cross-sectional elevation profile. The choice of MCL as the reference line is deliberate since anchoring the same surface on the geometric centre-line can cause adjacent normals to cross inside the drivable region on tight corners. The resulting non-injective parametrisation assigns multiple curvilinear addresses to the same three-dimensional point and returns invalid road geometry. The MCL is obtained from a quadratic program that minimises integrated curvature subject to track-boundary, periodicity, and jerk-regularisation constraints. By reducing total curvature while remaining within the track bounds, the MCL moves the region of parametrisation breakdown outside the drivable strip on all circuits studied, keeps the normal map injective across the full track width, and yields a more linear time-to-arc-length transformation for minimum lap-time problems. The fitted surface and the associated track widths are exported as periodic piecewise-quintic $C^4$ B-splines, solved as a nonlinear program.
comment: American Modelica & FMI Conference 2026
★ Electric Vehicle Scheduling and Carbon-Aware Energy Dispatch for Charging Stations
Electric vehicle (EV) charging stations integrating photovoltaic (PV) generation, battery storage, and bidirectional grid exchange must coordinate access to limited chargers with energy supply while considering both energy cost and carbon emissions. This paper proposes a solver-free greedy heuristic (SFGH) for dynamic vehicle scheduling (VS) and a twin delayed deep deterministic policy gradient (TD3) controller for carbon-aware energy dispatch (ED). SFGH assigns EVs according to reductions in service delays, and TD3 controls battery operation to balance energy costs and carbon emissions while meeting the resulting charging demand. The model allows both PV and battery exports, accounting for export revenue and carbon credits. Simulations on year-long case studies show that SFGH--TD3 achieves the lowest accumulated service delay, energy cost, and carbon emissions among the evaluated heuristic and multi-agent deep reinforcement learning (DRL) configurations.
★ Neural Compression of Long ADMM Trajectory for Multiparametric Quadratic Program NeurIPS 2026
Solving large-scale multiparametric quadratic programs (mpQPs) in real time often requires thousands of iterative optimization updates, creating a major computational bottleneck in model predictive control and learning-enabled systems. This paper proposes TQPNet, a self-supervised neural compression framework that learns to compress long ADMM optimization trajectories into a compact neural warm start followed by a few refinement iterations. We derive a low-dimensional ADMM scheme operating on the primal-dual variables $(x,β)$ and show that its iteration admits an equivalent ReLU-layer representation, termed ADMM(ReLU). Building on this insight, TQPNet combines a multilayer perceptron with ReLU activations (MLP(ReLU)) and a small number of unrolled ADMM(ReLU) refinement layers. The network is trained offline using an optimality-condition-informed residual loss, eliminating the need for solver-generated labels. The proposed residual-based framework enables both self-supervised training and online solution certification through adaptive ADMM(ReLU) refinement. Experiments on large-scale mpQPs and real-time MPC applications demonstrate that TQPNet achieves high accuracy, strong generalization, and substantially faster inference than conventional optimization-based methods.
comment: Accepted at NeurIPS 2026. Code: https://github.com/SOLARIS-JHU/TQPNet
★ State-Dependent Delay Modeling of Wind-Farm Wake Dynamics
Turbines convert wind to electrical power. In the wake of a turbine, wind speed is reduced in proportion to the amount of power extracted. In a wind farm, downstream turbines may be located in wake of an upstream turbine, requiring coordination to yield maximum power generation. This paper creates a state-dependent delay model for a two turbine wind farm. The upstream turbine creates a wake that diminishes the wind speed at the downstream turbine. Changing the upstream turbine operation will affect the downstream turbine after a delay that depends on the wind-propagation time between them. This delay depends on the upstream power extraction and it can destabilize the combined dynamics. Conventional models of this effect rely on nonlinear partial differential equations (PDEs). Control design and analysis with such models is difficult with established methods. We first show that the PDE model can be represented by an Abel-type state-dependent delay differential equation. The equation has an implicit solution. To obtain an explicit representation, we then introduce a simplified state-dependent delay model. Both models predict destabilizing interactions between the two turbines. Numerical examples illustrate delay-induced oscillations using the simplified model and verify their effectiveness in simulation.
★ Trajectory-Planning Attitude Control System for Satellites with Magnetic Attitude Control and One Reaction Wheel
Selecting attitude actuators for spacecraft in low earth orbit (LEO) often involves a trade between reaction wheels and magnetorquers. Reaction wheels generally have higher size, weight and power as well as cost, compared with magnetorquers, but have performance advantages. We consider a hybrid system relying on a single reaction wheel and three magnetorquers, using a trajectory planning control method that focuses on meeting pointing goals and minimizing the need to desaturate the reaction wheel. The use of planning allows for intelligent use of the reaction wheel and accommodates time-changing goals and orbit-varying magnetic fields. Preliminary results show trajectories with fractional-degree-accuracy, as well as management of reaction wheel momentum and avoidance of stiction. We also compare to a magnetorquer-only planning-based control system and show better convergence and faster slews.
comment: 19 pages, 26 figures in 14 groups, AAS/AIAA Astrodynamics Specialist Conference
★ GARDEN: Generalist Autonomous Reward Design Through Evidence Navigation for Cooperative Multi-Agent Reinforcement Learning
Cooperative multi-agent reinforcement learning (MARL) enables teams of agents, such as robots and autonomous vehicles, to learn coordinated behavior, but what such teams learn is largely determined by their reward function. Because task success alone provides too little guidance for learning, intermediate rewards are usually designed by hand, and this effort must often be repeated whenever the environment or the team changes. Large language models (LLMs) can automate the effort by writing reward functions as code, yet the resulting rewards are typically designed and tested in a single setting. To enable their reuse across settings, GARDEN (Generalist Autonomous Reward Design through Evidence Navigation) is proposed, in which one reward program is developed for a family of related scenarios and transferred, unchanged, to scenarios that were not available during its design. During development, environment code, training records, and agent behavior are investigated by an LLM reward engineer, and draft rewards are tested in trial training runs, while the learning algorithm and the rule for selecting the final reward are held fixed. On unseen scenarios of the Overcooked and SMAX benchmarks, task returns were increased by 58.2% and 52.5% over the default rewards on two kitchen layouts, and the mean win rate in two combat scenarios was raised from 51.0% to 89.9%. Under matched candidate and probe budgets, this exceeded the 84.2% and 79.2% obtained with fixed diagnostic feedback and Eureka-inspired reward reflection. These results show that a single autonomously engineered reward can be reused effectively across related cooperative scenarios.
comment: 8 pages, 3 figures, 5 tables. Code: https://github.com/DoganUrgun/GARDEN
★ Robust Dynamic Safety Margins are Robust Control Barrier Functions
This paper establishes robustness properties for control barrier functions (CBFs) constructed using dynamic safety margins (DSMs). We show that DSMs can act as robust CBFs by leveraging input-to-state stability for Lyapunov-based constructions, and contractivity for trajectory-prediction-based approaches. Additionally, we derive closed-form DSM-based robust CBFs (RDSM-RCBFs) specifically for linear systems. Simulations highlight the improved robustness of the proposed RDSM-RCBFs compared to nominal DSM-CBF baselines.
comment: 15 pages
★ Efficient Joint Trajectory Optimization and Resource Allocation for Secure UAV Networks: A Deep Reinforcement Learning Approach
Unmanned Aerial Vehicles (UAVs) are increasingly deployed in wireless networks; however, ensuring secure communication and efficient energy usage remains challenging. This paper addresses the joint trajectory optimization and resource allocation problem in UAV networks in the presence of a tethered UAV flying center and a malicious ground eavesdropper. In the proposed model, UAVs are equipped with energy harvesting capabilities and can adaptively switch between transmitting, jamming, and relaying roles to deliver data to a tethered UAV center for optimized network performance. An optimization problem is formulated to maximize a utility function that balances secrecy rate, energy consumption, and interference. To address the high non-convexity of the problem and its real-time decision requirements, a Proximal Policy Optimization (PPO)-based Deep Reinforcement Learning (DRL) algorithm is proposed to solve the problem and learn the network dynamics. Extensive simulation results demonstrate that the proposed approach achieves superior secrecy performance, lower energy consumption, and reduced interference compared to conventional and DRL-based methods.
comment: This paper has been submitted for possible publication as a journal paper and is currently under review
★ A Community Impact-Centered Network Reduction Method for Power Systems Resilience Planning
Power systems resilience planning requires evaluating many hazard scenarios and investment portfolios, often requiring network reduction for computational tractability. The resulting aggregation, however, can obscure community impact data needed for equitable planning. We present a network reduction method that jointly preserves network physics and the spatial distribution of community data. We embed a household-level scarce resource score that characterizes outage impact within optimization-based Kron reductions with two formulations: a scarcity-weighted method that prioritizes vulnerable nodes, and a scarcity-constrained method that bounds data dissimilarity within aggregated clusters. Both are evaluated on a realistic distribution feeder of Galveston Island, Texas and benchmarked against multiple approaches. The scarcity-constrained method performs best, achieving 86% reduction with worst-case voltage errors below 0.01p.u. while preserving the spatial distribution of community data. By explicitly controlling data heterogeneity within and across clusters, the proposed method yields reduced networks that retain decision-relevant community information without sacrificing electrical accuracy.
♻ ★ Revisiting Certainty Equivalence: The Structural Coupling Between Estimation and Control in Underactuated Nonlinear Systems
The certainty equivalence (CE) principle underpins a wide range of control architectures by enabling the separation of estimation and control design. While this property holds for linear systems, its validity in nonlinear settings remains limited and often implicitly assumed. This paper revisits CE from a nonlinear perspective, showing that estimated states induce an intrinsic coupling between estimation and tracking dynamics. By analyzing the closed-loop system in tracking-error coordinates, we demonstrate that nonlinear state dependence gives rise to higher-order interaction terms during aggressive transients. Motivated by this limitation, we propose an estimation-aware (EA) control paradigm that incorporates estimation quality into the feedback law to isolate estimation-induced loops. The formulation remains filtering-agnostic while preserving general applicability to smooth, underactuated nonlinear systems. We derive analytical conditions guaranteeing bounded tracking under uncertainty, validating the framework under high-fidelity quadrotor flight simulation along complex 3D trajectories at speeds up to 57.6 km/h. Frequency-domain evaluations demonstrate that the EA law extends tracking bandwidth by 39% and improves stability margins by up to 55%, effectively mitigating severe cross-couplings to offer a robust alternative to classical CE-based designs.
comment: 18 pages, 21 figures
♻ ★ Stochastic MPC under Heavy-Tailed Disturbances: An Extreme Value Theory Approach
Safety-critical control systems must contend with disturbances whose extreme deviations occur far more frequently than classical light-tailed models predict. Existing stochastic MPC (SMPC) formulations tighten constraints using an assumed distribution, a moment bound, or a finite scenario sample, each of which degrades under an unknown heavy-tailed disturbance. This paper develops an SMPC formulation for linear systems under heavy-tailed disturbances that is only assumed to be regularly varying, which allows an explicit extreme value theory (EVT) characterization of the tube error tail that is asymptotically exact. We further show that closed-loop dynamics induce temporal clustering of rare excursions across the prediction horizon, and characterize this clustering through a closed-form extremal index estimable from data. The resulting clustering-corrected constraint bounds the probability of a rare-event episode over the horizon, rather than only the marginal per-step exceedance probability. Simulations of a linear two-CSTR-in-series process and of a nonlinear unicycle navigating past an obstacle under Student-$t$ disturbances validate both approaches and demonstrate reduced frequencies of safety constraint violations.
comment: 29 pages, 6 figures, submitted to Journal of the Franklin Institute
♻ ★ Smoothing the Ramp, Not the Peak: Scheduling-Induced Power Dynamics of LLM Inference and Their Grid-Scale Consequences
Large language model inference serving is a fast-growing electricity load whose power dynamics remain uncharacterized from a grid-planning perspective. Using real, measured GPU power traces, we show that chunked prefill scheduling, a latency-motivated technique already deployed by default in production LLM serving, is a controllable knob that regulates power ramp rate without touching peak power. Contrary to the intuitive hypothesis that splitting a long prompt's computation into smaller steps should flatten its power spike, peak power stays relatively the same while mean ramp rate falls substantially. Critically, this ramp-rate benefit is not a fixed property of the policy: it grows monotonically with system saturation, and we confirm this along two independent axes: concurrency and long-prompt request load (from near zero at low whale incidence to 39.6% at high whale fraction/size). We translate this single-GPU mechanism into an operational grid quantity, regulation-reserve procurement, posed and solved as a chance-constrained problem using a model-free bootstrap directly resampling real measured power traces. For independently operating servers, chunked prefill reduces the fast-ramping reserve requirement by an estimated 24% at heavy load (95% CIs within 13-34%) and 12% at moderate load, consistently across reliability levels from 95% to 99.9%. Repeated with 100 trials per policy on two-GPU tensor-parallel deployments, the reduction is about 36% for a dense 14B model and 10% for a sparse mixture-of-experts model. The benefit holds for ramps resolved at up to 2 s and persists at a reduced 16-20% when servers' load changes are synchronized. Together, these results identify a no-cost lever for fast-timescale demand shaping that is available today, whose benefit is largest when data centers run hottest.
comment: 17 pages, 9 figures. A shorter version focused on dense model is accepted at PES-IM 2027
♻ ★ SD-DPC: Sparse Dictionary Differentiable Predictive Control
We present sparse dictionary differentiable predictive control (SD-DPC), a framework for learning sparse, interpretable feedback policies for nonlinear systems from data. A prediction model is first identified by rollout-based sparse identification of nonlinear dynamics (SINDy), building on gradient-based and multistep formulations. The policy is then parameterized as a sparse combination of dictionary functions and trained by differentiating a constrained finite-horizon predictive-control objective through this model, so that its terms are selected by closed-loop performance rather than by imitating a previously trained controller. The result is an explicit feedback law with only a handful of terms. Across three benchmark control problems, SD-DPC satisfies the constraints in almost all test scenarios, outperforms a policy distilled onto the same terms by up to an order of magnitude, and requires orders of magnitude less memory and online computation than an optimization benchmark, while admitting explicit sensitivity bounds.
comment: Revised the experimental setup details in Table I and the evaluation reporting, corrected the numerical results in Table II, and clarified the training and validation protocols. The main conclusions are unchanged
♻ ★ Action-Increment-Constrained Policy Optimization for Smooth Online Adaptive Critic Flight Control with Actuator Rate Constraints and Low Computational Complexity (extended version with Ultimate Uniform Boundedness analysis)
Approximate Dynamic Programming (ADP) enables online control-law adaptation through adaptive critic methods. A safety challenge in applying ADP to angle-of-attack (AoA) tracking control is that an overly sensitive neural network (NN) controller may generate oscillatory or saturated control actions under nonlinear dynamics, potentially compromising flight stability and increasing actuator stress. To address this challenge, this paper proposes two innovations: (1) the actuator rate constraints are enforced within a an Action-Increment-Constrained Policy Optimization (AI-CPO) framework inspired by safe reinforcement learning (RL), in which a primal-dual approach adaptively adjusts the penalty weight to address actuator rate constraints. This optimization is solved online using adaptive Temporally Smoothed Incremental Model-Based Heuristic Dynamic Programming (TS-IHDP) for smooth and computationally efficient online adaptive control; and (2) a low-pass filter integrated into the cascaded NN architecture to attenuate high-frequency pitch-rate commands while preserving the Markov property during online learning by exploiting the hierarchical actor structure. Compared with the IHDP baseline, simulations using a longitudinal air vehicle dynamics model demonstrate that TS-IHDP improves control smoothness and tracking performance, reduces system oscillations, and enhances the trained policy's robustness to model uncertainties.
comment: This version extends the preliminary manuscript by providing a uniform ultimate boundedness (UUB) convergence analysis of the proposed TS-IHDP with constant penalty weight
♻ ★ A model predictive control framework with robust stability guarantees under unbounded disturbances
To address feasibility issues in model predictive control (MPC), most implementations relax state constraints by using slack variables and adding a penalty to the cost. We propose an alternative strategy: relaxing the initial state constraint with a penalty. Compared to state-of-the-art soft constrained MPC formulations, the proposed formulation has two key features: (i) input-to-state stability and bounds on the cumulative constraint violation for unbounded disturbances; (ii) close-to-optimal performance under nominal operating conditions. The idea is initially presented for open-loop asymptotically stable nonlinear systems by designing the penalty as a Lyapunov function, but we also show how to relax this condition to: i) Lyapunov stable systems; ii) stabilizable systems; and iii) utilizing an implicit characterization of the Lyapunov function. In the special case of linear systems, the proposed MPC formulation reduces to a quadratic program, and the offline design and online computational complexity are only marginally increased compared to a nominal design. Numerical examples demonstrate benefits compared to state-of-the-art soft-constrained MPC formulations.
comment: Extended version of accepted paper in Automatica. Appendix contains additional details
♻ ★ Stealthy bias injection attack detection based on Kullback-Leibler divergence in noisy linear systems
This paper studies the design of observer gains to enhance the detectability of stealthy bias injection attacks (BIAs) in linear systems driven by Gaussian noise, where adversaries compromise a subset of sensors and exploit system dynamics and noise to remain stealthy. We formulate the observer-gain design as a max-min optimization problem that enhances the detectability of the least detectable BIAs, with detectability measured by the Kullback-Leibler divergence (KLD) subject to a prescribed attack impact. To reduce the computational complexity of the derived non-convex design problem, we consider attack onset, one-step, and steady-state detectability. We show that the Kalman filter is optimal at attack onset regardless of the subset of attacked sensors. For the one-step and steady-state cases, the observer gain design problems are approximated by bi-convex optimization problems, which can be efficiently solved using alternating optimization (AO) and alternating direction method of multipliers (ADMM). Moreover, more tractable linear matrix inequality (LMI) relaxations are developed. Finally, the effectiveness of the proposed stealth-aware detection framework is demonstrated through an application to a thermal system.
comment: 26 pages, 11 figures
♻ ★ Control and Stability of a Multilevel Power System for a Future Distribution Network
The growing integration of renewable energy sources into distribution networks poses significant challenges to frequency and voltage stability due to their intermittent nature and low-inertia dynamics. This paper proposes a multilevel control framework for a future decarbonized power system, using energy storage systems as power buffers to mitigate frequency and voltage fluctuations. A nonlinear interconnected model is formulated to describe the multilevel aggregated dynamics of the distribution network. To reduce the operational complexity and communication overhead of the control system, a distributed linear quadratic regulator strategy is developed based on bottom-up information exchange, where each level implements local feedback control over a short time horizon. Local stability conditions are established for the open-loop nonlinear model, and the closed-loop stability is characterized by the Hurwitz property of the system matrix. Performance bounds are also derived to quantify the gap between the distributed and centralized control strategies.
♻ ★ Manifold meta-learning for reduced-complexity neural system identification
System identification has greatly benefited from deep learning techniques, particularly for modeling complex, nonlinear dynamical systems with partially unknown physics where traditional approaches may not be feasible. However, deep learning models often require large datasets and significant computational resources for both training and inference due to their high-dimensional parameterizations. To address this challenge, we propose a meta-learning framework that discovers a low-dimensional manifold within the parameter space of an over-parameterized neural network architecture. This manifold is learned from an abundant meta-dataset of input-output sequences generated by a simulator of the class of systems of interest, and thereafter enables efficient model training while preserving the network's expressive power for the considered system class. Unlike bilevel meta-learning approaches, our method employs an auxiliary neural network to map datasets directly onto the learned manifold, eliminating the need for costly second-order gradient computations during meta-training and generally reducing the number of first-order updates required in inference, which could be expensive for large models. Finally, we derive a probabilistic bound that quantifies the performance gap between training on the learned low-dimensional manifold and the full parameterization. We validate our approach on a family of Bouc-Wen oscillators, which is a well-studied nonlinear system identification benchmark. We demonstrate that we are able to learn accurate models even in small-data scenarios.
♻ ★ Deep Behaviour Cloning of Model Predictive Control for Real-Time Operation of a Hydrogen-Diesel Dual-Fuel Engine
Hydrogen-diesel dual-fuel (H2DF) combustion engines offer a promising pathway for decarbonising hard-to-electrify transport sectors, yet their highly nonlinear dynamics and coupled process variables demand constraint-aware control strategies. Model Predictive Control (MPC) meets these requirements but requires an online optimisation in every combustion cycle, which limits deployment on low-cost embedded hardware. This paper trains a feedforward deep neural network (DNN) by behaviour cloning (BC) to imitate an MPC expert, using 86,000 engine cycles of demonstration data collected at 1500 min-1 on a modified Cummins QSB 4.5-litre hydrogen dual-fuel engine. Two variants, one with process feedback and one without, are validated experimentally. Both track unseen fast-transient load steps (3-8 bar indicated mean effective pressure, IMEP) with normalised root mean square error (NRMSE) values of 7.80% and 9.03% against the expert's 8.01%, while keeping mean NOx and particulate matter emissions at or below those of the expert. Inference takes 2 ms or less on a Raspberry Pi 400 (ARM Cortex-A72 at 2.2 GHz), including 1 ms communication latency, compared to up to 7 ms for the MPC expert, a 3.5x speedup. Open-loop profiling on a low-cost ESP32 microcontroller at 180 MHz gives 4.3 ms per inference, 4x faster than required for the 18 ms cycle window. Beyond the training range the cloned policy saturates its controls but exceeds the pressure-rise-rate limit. To the authors' knowledge, this is the first experimental BC controller for cycle-to-cycle combustion control of an internal combustion engine (ICE), trained from demonstrations recorded on the engine itself.
comment: Submitted to Control Engineering Practice
♻ ★ Perturbed Gradient Descent Algorithms are Small-Disturbance Input-to-State Stable
This article investigates the robustness of gradient descent algorithms under perturbations. The concept of small-disturbance input-to-state stability (ISS) for discrete-time nonlinear dynamical systems is introduced, along with its Lyapunov characterization. The conventional linear Polyak-Lojasiewicz (PL) condition is then extended to a nonlinear version, and it is shown that the gradient descent algorithm is small-disturbance ISS provided the objective function satisfies the generalized nonlinear PL condition. This small-disturbance ISS property guarantees that the gradient descent algorithm converges to a small neighborhood of the optimum under sufficiently small perturbations. As a direct application of the developed framework, we demonstrate that the LQR cost satisfies the generalized nonlinear PL condition, thereby establishing that the policy gradient algorithm for LQR is small-disturbance ISS. Additionally, other popular policy gradient algorithms, including natural policy gradient and Gauss-Newton method, are also proven to be small-disturbance ISS.
comment: 16 pages
♻ ★ Data-driven Wasserstein distributionally robust generalized Nash equilibrium problems with heterogeneous uncertainty
We study stochastic generalized Nash equilibrium (GNE) problems with heterogeneous, stochastic uncertainty in the cost functions. To address the distributional uncertainty, we let each agent construct an individual Wasserstein ambiguity set around an empirical distribution obtained from private data. We establish finite-sample robustness guarantees for any distributionally robust (DR) GNE. Focusing on variational equilibria, we derive a DR variational inequality (VI) reformulation and establish asymptotic consistency with the nominal stochastic game. Under mild conditions, we derive an exact finite-dimensional generalized Nash equilibrium reformulation. We then quantify how heterogeneous worst-case distributions affect monotonicity, and derive conditions for uniqueness and equilibrium sensitivity. To compute a variational DR GNE, we propose an inexact projected method and establish convergence under vanishing oracle errors, an asymptotic error bound under bounded oracle errors, and linear convergence under exact oracle evaluations.
♻ ★ LLM-Driven, Datasheet-Aware Automated Hardware Compatibility Verification for Early-Stage, Pre-Schematic Embedded System Design
We present an LLM-driven, datasheet-aware framework for early-stage hardware compatibility verification that identifies documentation-level interface incompatibilities based on hardware datasheets and high-level component connectivity descriptions. It does not require, and can therefore be used, before detailed schematic simulation and implementation. We view trustworthy LLM-assisted design automation not as directly generating answers from documents, but as transforming engineering information through traceable verification stages. Given hardware datasheets and high-level component connectivity descriptions, the framework constructs a design graph that captures device connectivity and shared interaction domains, retrieves only the engineering properties required by explicit, domain-oriented verification criteria , and generates deterministic scripts for compatibility evaluation. By decomposing compatibility analysis into modular stages and preserving intermediate results, the framework reduces context overhead, improves transparency and tractability, enables scaling, and avoids reliance on LLMs for numerical computation. Evaluated on seven embedded-system designs comprising 34 datasheets, our framework achieves 97.5% compatibility-verification accuracy and an 8.6 times reduction in input context size compared with ``upload-and-query'' workflows. These results demonstrate the feasibility of LLM-assisted, specification-based hardware compatibility verification at an early design stage, as well as the need for, and substantial benefits of, modular task decomposition, formalized verification criteria, and task-aware compact context construction.
♻ ★ Context-dependent Runtime Construction of Safe Regions of Attraction from Design-Time Identification for Multi-Candidate Simplex Architectures
Deploying safety-critical cyber-physical systems requires control strategies that reconcile high performance and adaptability with rigorous safety guarantees. Classical control strategies offer stability proofs but are conservative, while advanced optimization- and learning-based controllers achieve high performance but lack formal verification. Runtime Assurance (RTA) architectures bridge this gap by introducing their own (dis-)advantages. Safety filters use online optimization and introduce chattering, while Simplex permissiveness is bounded by the tightness of the certified Region of Attraction (RoA) of its High-Assurance (HA) controller. Targeting this conservatism, we present a two-stage RoA certification procedure leveraging contraction-based stability to establish a target set for Taylor-model-based reachability analysis. We then treat the closed-loop system under HA control as a switched system, enabling: (a) runtime construction of context-dependent safe RoA and (b) safe switching between HA candidates to optimize recovery and maximize the operating envelope of the uncertified High-Performance (HP) controller. We validate this framework on a robotic path-tracking task using a three-stage evaluation: Ordinary Differential Equation (ODE) simulation for theoretical verification, high-fidelity Gazebo simulation under unmodeled dynamics, and experiments on physical hardware. Results show that the Simplex architecture maintains safety under fault injection and successfully shields a behavioral-cloning-trained HP controller. Crucially, the decision module introduces negligible overhead (0.028 ms average, 4.13 ms worst-case), consuming under 10 % of the control period.
comment: 13 pages, 8 figures, 5 tables
♻ ★ Bilinear World Models: Learning Representations with Structured Dynamics for Efficient Control
World models jointly learn latent representations and dynamics that predict how high-dimensional observations evolve under actions. In this work, we propose a JEPA-style world model in which, rather than learning arbitrary latent dynamics, we restrict them to follow a bilinear parameterization. This structure enables efficient planning and control while shifting the modeling burden onto the encoder, encouraging richer representations that expose the controllable geometry of the system. In particular, this structured parameterization allows us to structurally enforce action recoverability, thereby preventing representation collapse by construction. Although prescribing a bilinear parametrization may appear restrictive, we show that a broad class of nonlinear dynamical systems admits a transformation under which the dynamics become bilinear. Empirically, we show across standard 2D and 3D control tasks that representations with bilinear-parameterized dynamics can be learned directly from high-dimensional observations, reducing planning time by nearly three orders of magnitude while retaining or even improving control accuracy. We also propose more demanding regimes of longer-horizon planning and real-time control, and demonstrate that our method succeeds in both, moving JEPA-style world models beyond short-horizon offline planning.
♻ ★ Response time central-limit and failure rate estimation for stationary periodic rate monotonic real-time systems
Real-time systems consist of a set of tasks, a scheduling policy, and a system architecture, all constrained by timing requirements. Many everyday embedded systems, within devices such as airplanes, cars, trains, and spatial probes, operate as real-time systems. To ensure safe failure rates, response times-the time required for the exection of a task-must be bounded. Rate Monotonic real-time systems prioritize tasks according to their arrival rate. This paper focuses on the use of the central limit of response times built in \cite{zagalo2022} and an approximation of their distribution with an inverse Gaussian mixture distribution. The distribution parameters and their associated failure rates are estimated through a suitable re-parameterization of the inverse Gaussian distribution and an adapted Expectation-Maximization algorithm. Extensive simulations demonstrate that the method is well-suited for the approximation of failure rates. We discuss the extension of such method to a chi-squared independence test adapted to real-time systems.
comment: Accepted to IEEE Transactions on Computers
♻ ★ Input Design for Angles-Only Initial Relative Orbit Determination Using Active Learning
Angles-only initial relative orbit determination (IROD) is a challenging estimation problem, as the lack of direct range information results in scale ambiguity and weak practical observability. To address these limitations, this paper introduces an input-design approach to angles-only IROD, in which excitation maneuvers are systematically designed within an active-learning framework. From a dual-control perspective, the resulting control inputs promote exploration of the output space to improve initial-state estimation while accounting for station-keeping requirements and control effort. Angular measurements collected under the resulting maneuver sequence are processed by a batch IROD algorithm extended to handle generic input sequences. A first-order analytical covariance matrix of the initial-state estimate is derived to quantify the uncertainty induced by measurement noise and to support the transition from batch estimation to sequential filtering. Numerical simulations and Monte Carlo sensitivity analyses show that the proposed input-design strategy improves initialization accuracy and conditioning compared with baseline excitation methods and supports autonomous rendezvous in the tested scenarios.
Robotics
★ The Language of Deformation: Semantic Failure Analysis as a Prerequisite for Dexterous Manipulation of Deformable Objects
Dexterous manipulation of rigid objects has reached a level of maturity where grasp stability, control, and planning strategies are successfully applied in industry and logistics. In contrast, manipulating deformable objects (cables, plastic pieces, cloth) remains far more challenging, as rigid-object methods do not account for their additional complexity. This paper identifies semantic failure analysis as a key step toward mature deformable-object manipulation and supports this claim with proof-of-concept experiments. Unlike rigid-object manipulation, where goals are well-defined in terms of criteria such as force balance and slippage, deformable-object manipulation involves a broader variety of failure modes arising from differences in materials, geometry, and behaviour. Typical failures include buckling, plastic deformation, pressure variations, and tearing. These diverse and often hard-to-detect events require distinct corrective strategies, making semantic failure analysis necessary. We define its key components and challenges and present experimental results supporting its potential as a research direction.
comment: 9 pages, 2 figures. Accepted at the European Robotics Forum (ERF) 2026
★ Uncertainty Evaluation to Support Safety of the Intended Functionality Analysis for Determining the Applicability of Performance Targets Across Triggering Conditions in ML-Based LiDAR Object Detection
For safety of the intended functionality (SOTIF) analysis according to ISO 21448, a machine learning object detector for light detection and ranging (LiDAR) point clouds can meet a performance target, a limit on the mean per-frame share of wrong detections among accepted detections, by rejecting detections above an uncertainty threshold. A threshold selected on data of one condition carries a statistical guarantee only for data distributed like its selection data, whereas SOTIF analysis covers every triggering condition. This paper presents an uncertainty evaluation in which the threshold is transferred into a triggering condition, a test decides at a stated error probability whether the target is applicable there, not applicable or not determined, and a lower bound counts the independent units an applicable outcome requires. For PointPillars ensembles on KITTI and nuScenes, counted in recorded sequences, as the dependence between frames requires, neither dataset held enough independent units for any target limit, and no transfer had a not applicable outcome. Counted in frames, the same data gave not applicable outcomes at limits fixed in advance after transfers into far distance, into rain and between two separately trained detectors, thresholds that met the target limit after a transfer rejected most correct detections, and in rain the area under the receiver operating characteristic curve did not decrease. A performance target met under one condition therefore counts as evidence under a triggering condition only after a test there, reported with its share of accepted correct detections and its number of independent recorded sequences.
comment: 24 pages, 6 figures. Extended version of the VEHITS 2026 conference paper (doi:10.5220/0014967200004030). Submitted for journal publication. Code and recorded results: https://doi.org/10.5281/zenodo.23495356
★ Zero-HLM: Generalizable Humanoid Loco-Manipulation without Real Robot Data
Learning generalizable humanoid loco-manipulation remains bottlenecked by costly real-world demonstrations. Simulation provides a scalable alternative, but requires visually faithful, physics-ready scenes and executable whole-body motions. We present Zero-HLM, a real-to-sim-to-real framework that transforms smartphone captures and simulated source demonstrations into scalable training data for humanoid loco-manipulation. Gaussian-guided agentic real-to-sim reconstruction aligns visual and physical scenes; motion-environment co-adaptation uses rollout feedback to correct whole-body references and interaction geometry. We generate 4,800 physically validated demonstrations for zero-shot transfer across three Unitree G1 tasks. Simulation-trained GR00T N1.7 achieves 66.7% mean full-task success, versus 46.7% with real-data training. On generalized configurations, expanded simulation training raises pooled success from 23% for base-trained real-data policies to 69%, without additional real demonstrations. Project page: https://young-bit.github.io/zero-hlm/
★ PushAround: Collaborative Path Clearing via Physics-Informed Hybrid Search
Large vehicles in cluttered environments are often blocked by movable obstacles, motivating mobile robot teams to clear traversable corridors. Navigation Among Movable Obstacles (NAMO) approaches often abstract obstacle displacements while neglecting robot dimensions, obstacle mass, contact geometry and forces, yielding physically infeasible plans. PushAround is a physics-informed hybrid search that couples geometric connectivity reasoning with short-horizon simulation to construct executable clearing plans. Repeated W-clearance connectivity graph (WCCG) queries expose frontier gaps requiring widening and rank them by estimated downstream costs. The search generates direct or recursive pushing candidates, selecting obstacles, directions, contacts and forces, and admits physics-evaluated candidates. Numerical experiments validate WCCG against standard connectivity checks and demonstrate scaling to denser clutter and teams of up to four robots, with robustness to simulated execution-side physical-parameter mismatch.
comment: Accepted for publication in IEEE Robotics and Automation Letters (RA-L). 8 pages, 11 figures. Supplementary video and evaluation records are included as ancillary files. Video: https://youtu.be/HkQIcVWiIuQ
★ 3D-Force Estimation for Dual-arm Manipulation of Deformable Linear Objects via Factor Graphs
Estimating forces acting on Deformable Linear Objects (DLOs) is challenging due to their underactuated nature. However, reliable contact estimation is crucial to precisely control them and enable safe manipulation tasks. Differently from the various approaches presented in the literature, this work adopts a probabilistic formulation where the DLO state and sensor data are related through conditional distributions. In particular, factor graphs are used to determine the magnitude, direction, and location of a 3D force acting on a DLO, while simultaneously reconstructing its shape. Since DLOs are not actuated and are typically thin, like cables and wires, strain gauges and actuation readings are usually impractical. Therefore, to overcome this issue, this work is tested using one type of sensing information only, thus developing a generic technique suited for a wider variety of tasks. Realistic tests are performed by introducing noise into the measurements while the DLO follows a prescribed path in the presence of external forces that are accurately estimated.
★ VISTA: Active Exploration for Self-Improvement in Embodied Agents
Agentic embodied intelligence plays a foundational role in open-ended, dynamic physical environments. Rather than treating policy learning as an end-to-end mapping from instructions to actions, recent studies reveal that embodied systems depend not only on pre-trained foundation models, but also on harness. Prior approaches largely treat rollouts as static logs for post-hoc reflection, producing harness updates without autonomously testing and refining them through subsequent embodied interaction. To address this challenge, we propose VISTA, a meta-agent framework that transforms post-hoc rollout reflection into active self-exploration for embodied harness improvement. Specifically, VISTA maintains a harness graph that records the lineage of proposed harnesses, which enables the meta-agent to retrieve prior hypotheses, build on effective mechanisms, and avoid repeatedly exploring failed directions. Each iteration begins from a retained harness version and its execution evidence. Rather than directly committing an update from these historical trajectories, the meta-agent actively explores in the environment. It autonomously generates new rollouts, probes execution failures and recovery behaviors, and iteratively modifies the harness based on the resulting evidence. The refined harness is then evaluated afresh and added to the harness graph. We evaluate VISTA on the LIBERO-Pro, RoboCasa, and RoboTwin benchmarks. Experimental results demonstrate that VISTA progressively improves task success and is more rollout-efficient than static reflection, achieving higher success rates with fewer simulator rollouts.
comment: code: https://github.com/AgenticEI/VISTA, project page: https://agenticei.github.io/VISTA-page/
★ DriveUniTok: A Unified Multimodal Tokenizer for Driving World-Action Models
World models are emerging as a fundamental component of policy learning for real-world agents, enabling agents to predict future world states and plan actions within a learned representation of the dynamic environment. Existing approaches, however, often model appearance, metric geometry represented by projected LiDAR depth, and ego motion either partially or independently, overlooking the fact that these modalities are different projections of a shared underlying world state. This separation can lead to information loss and cross-modal inconsistency, ultimately limiting downstream performance. Motivated by this unified view of the world, we propose DriveUniTok, a unified multimodal tokenizer for driving world-action models (WAMs) that encodes these modalities in a single decodable continuous latent. We freeze DriveUniTok and adapt a pretrained diffusion model, DriveUniWAM, to predict future unified latents; the tokenizer then decodes them into future appearance, projected LiDAR depth, and ego trajectories. Experiments on NAVSIM demonstrate that our unified design achieves better future prediction and planning performance at lower downstream computational cost than separate-stream alternatives. With reinforcement learning post-training, DriveUniWAM achieves a PDMS of 91.54, reaching state-of-the-art performance among the evaluated WAM-based planners.
comment: 31 pages, 16 figures, 8 tables, including appendices
★ Context-dependent Runtime Construction of Safe Regions of Attraction from Design-Time Identification for Multi-Candidate Simplex Architectures
Deploying safety-critical cyber-physical systems requires control strategies that reconcile high performance and adaptability with rigorous safety guarantees. Classical control strategies offer stability proofs but are conservative, while advanced optimization- and learning-based controllers achieve high performance but lack formal verification. Runtime Assurance (RTA) architectures bridge this gap by introducing their own (dis-)advantages. Safety filters use online optimization and introduce chattering, while Simplex permissiveness is bounded by the tightness of the certified Region of Attraction (RoA) of its High-Assurance (HA) controller. Targeting this conservatism, we present a two-stage RoA certification procedure leveraging contraction-based stability to establish a target set for Taylor-model-based reachability analysis. We then treat the closed-loop system under HA control as a switched system, enabling: (a) runtime construction of context-dependent safe RoA and (b) safe switching between HA candidates to optimize recovery and maximize the operating envelope of the uncertified High-Performance (HP) controller. We validate this framework on a robotic path-tracking task using a three-stage evaluation: Ordinary Differential Equation (ODE) simulation for theoretical verification, high-fidelity Gazebo simulation under unmodeled dynamics, and experiments on physical hardware. Results show that the Simplex architecture maintains safety under fault injection and successfully shields a behavioral-cloning-trained HP controller. Crucially, the decision module introduces negligible overhead (0.028 ms average, 4.13 ms worst-case), consuming under 10 % of the control period.
comment: 13 pages, 8 figures, 5 tables
★ JevVLA: Jev-Inspired Energy-Based Adaptation for Vision-Language-Action Policies
Vision-language-action (VLA) policies can select among discrete candidates for pick-and-place manipulation or generate continuous action chunks for motion control. Improving both forms of decision making through a common mechanism while retaining the capabilities of a pretrained policy is an important adaptation problem. Inspired by Jev's structured, probabilistic decision interface, we propose JevVLA, a lightweight energy-based adaptation framework for VLA policies. JevVLA instantiates this inspiration by modeling the relative quality of action proposals: with the base policy frozen, it learns a differentiable scalar energy conditioned on observations, task context, and actions, using execution feedback, expert corrections, or demonstration-derived preferences. JevVLA uses this single learned action preference function in two ways: its values compare discrete candidates, while its action gradients correct continuous actions or guide their generation. Candidate selection and continuous control thus share one trainable decision core. Each benchmark uses independently trained weights and fixed input/output encodings, while retaining the same core network with approximately 80,000 trainable parameters. On the held-out test sets, JevVLA improves task success over the respective matched base-policy controls by 1.28, 2.31, 1.60, 1.52, and 1.28 percentage points on MetaWorld MT50, VIMA, LanguageTable, LIBERO Plus, and CLIPort, respectively. These results support the feasibility of a lightweight, Jev-inspired energy model for adapting both discrete action selection and continuous action guidance. The JevVLA implementation will be open-sourced at https://github.com/ZJU4EmbodiedAI/JevVLA.
comment: 30 pages, 8 figures
★ Learning Direct Control from Optimization for Dynamic Target Tracking
Dynamic target tracking requires a mobile robot to maintain target visibility while navigating obstacles with smooth, feasible controls. Online trajectory optimization explicitly encodes these objectives but requires repeated numerical solving, whereas waypoint-based neural policies separate geometric prediction from control execution. We introduce LECO, a framework that transfers tracking optimization into offline learning of a direct-control policy. LECO predicts control sequences and maps them to trajectories through a differentiable kinematic model, allowing trajectory-level objectives to directly guide control learning. A decaying expert-guidance curriculum initializes effective tracking behavior and progressively shifts learning toward these task objectives. At deployment, a predictive safety lock checks short-horizon obstacle clearance and invokes a classical planning and control fallback when the predicted motion is unsafe or the target prediction is invalid. Across three simulated obstacle densities, LECO reduces root-mean-square linear jerk by 14.1--31.2\% relative to the best competing method for this metric and achieves the highest tracking success rate, with per-cycle computation times of 4.25--5.63\,ms. Experiments on two robot platforms further demonstrate tracking through indoor narrow passages and outdoor forest environments. These results demonstrate that learning direct control from optimization enables smooth and robust dynamic target tracking.
★ Quantile-Based Command-Reserve MPC for Current-Command Gimbal Pointing
This paper studies the constrained line-of-sight pointing control problem of a two-axis current-command gimbal subjected to mismatched mechanical disturbances. Friction, torque ripple, current-response lag, model mismatch, and limited drive commands make accurate pointing more challenging yet interesting. A quantile disturbance predictor is proposed to compensate predictable acceleration residuals and describe their remaining uncertainty, with the aid of learning median and outer quantiles from state, command, and reference histories. A tube-inspired model predictive controller is proposed to reserve command headroom according to prediction uncertainty, with the aid of calibrated radii and stage-dependent input margins. The median improves the nominal prediction, while the radius adapts the command reserve without adding optimization variables. A finite-horizon tube inclusion result is established for a specified linear error model and fixed ancillary policy; its conditions are distinguished from the approximations used in hardware. Native command bounds and a final command limiter keep the sent commands within their prescribed range. On a two-axis embedded platform, four controller configurations are compared in seven repeated trials each. The proposed configuration achieves a two-dimensional pointing root-mean-square error of $3.94 \pm 0.10$ mrad with no command-limit samples in the evaluation windows. Its performance supports the combined use of residual prediction and adaptive command margins in the tested operating condition. Solver times and logged loop overruns are reported separately for the nominal 100 Hz implementation.
comment: 10 pages, 4 figures. Supplementary hardware trial data and recomputation scripts included as ancillary files. Submitted to Journal of the Franklin Institute
★ MMGNet: Multi-Modal 6-DoF Grasp Synthesis for Cluttered Scenes
Grasping is a foundational capability in robotic manipulation. Scene-level 6-DoF grasp synthesis relies on point clouds for metric geometry, yet real-world point observations are often noisy, incomplete, and over-smoothed. RGB images provide object-region, boundary, and appearance cues, yet two bottlenecks limit their effective use in scene representation and local grasp reasoning. We present MMGNet, a Multi-Modal Grasp Synthesis Network that addresses these bottlenecks through hierarchical image-to-3D feature injection and rotational graspness modeling. Hierarchical injection integrates multi-scale image features throughout 3D representation formation, while the rotational graspness jointly predicts the grasp approach direction and in-plane rotation before organizing local image-informed features through grasp-shaded grouping. Ablation studies show that these two modules provide complementary gains. MMGNet achieves a mean AP of 60.34 on GraspNet-1Billion with RealSense, surpassing all compared methods. In real-world experiments with adversarial grasp objects, MMGNet improves the grasp success rate over AnyGrasp by 8.46 percentage points. \emph{Code, models, and data will be released upon publication.}
comment: Under Review
★ Localization-Uncertainty-Aware UAV Swarm Search for GNSS-Denied Maritime Surveillance
This paper studies UAV swarm search in GNSS-denied maritime environments, where inter-UAV ranging drives cooperative localization and uncertainty propagates along the multi-hop reference chain, affecting connectivity, collision avoidance, and search effectiveness. We propose an uncertainty-aware control barrier function (CBF) framework approximating cascading localization covariance through Fisher information matrices. First, the cascading covariance model captures how ranging noise and neighbor uncertainty accumulate along the localization chain. Second, uncertainty-tightened CBFs incorporate the covariance model and its rate into margins for connectivity maintenance and collision avoidance, with conditional invariance and graceful-degradation bounds. Third, a hierarchical Voronoi-target strategy with formation keeping supports efficient forward-looking coverage. The distributed controller is validated with a standard unbiased extended Kalman filter in the loop. The framework is demonstrated on several polygon shapes and swarm sizes from 6 to 30 sUAVs. An estimator-in-the-loop run completes the search in 302.5 s, 20 Monte Carlo deployments confirm repeatable completion with all local QPs feasible and containment in at least 98.97% of UAV-frames, and six MBZIRC runs provide estimator-in-the-loop evidence. The source code is publicly available.
comment: 14 pages
★ Physics-Informed and Data-Driven Composite Disturbance Separation and Predictive Control for Airborne Electro-Optical Gimbals with Multiple Constraints
This paper studies the constrained pointing control problem of airborne electro-optical gimbals subjected to composite periodic and aperiodic disturbances. Under carrier vibration, input delay, and actuator constraints, achieving line-of-sight stabilization and tracking is challenging. A disturbance-separated model predictive control (MPC) scheme is proposed to compensate these heterogeneous disturbances according to their physical characteristics. An adaptive periodic observer predicts multi-harmonic vibrations over the MPC horizon using linear-regression frequency estimation. A linear observer estimates the aperiodic disturbance, and a bounded long short-term memory (LSTM) network predicts its change over the input delay. Embedding periodic predictions into the MPC model while applying aperiodic compensation outside the optimizer limits amplification of aperiodic estimation errors and improves pointing accuracy under frequency changes and parameter mismatch. On the stated headroom envelope, input-set tightening provides feasible commands within the prescribed limits, and a regional practical tracking bound is established under explicit input-sequence and predicted-state margin conditions. A UAV-mounted two-axis gimbal platform is assembled for comparative indoor hovering experiments, complemented by simulations under frequency changes and parameter mismatch. In the reported trials, the proposed method achieves yaw and pitch relative-angle root-mean-square errors (RMSEs) of 0.770 degrees and 0.350 degrees, reducing tracking errors by 25.1% and 45.9% compared with linear active disturbance rejection control.
comment: 20 pages, 8 figures. Includes supplementary derivations and proofs. Demonstration video included as an ancillary file
★ Behavioural Quality Is Not Learning Utility: Objective Mismatch in Human Behaviour Models for Robot Learning
Human behaviour models are increasingly used to provide simulated human interactions for training robots via reinforcement learning (RL). Existing approaches primarily evaluate these models according to how closely their generated behaviours resemble real human data, while their usefulness as training partners is less commonly assessed. This creates a potential objective mismatch: a model that performs well according to behavioural evaluation metrics may not necessarily provide useful training experiences for downstream robot policy learning. Across two human-robot interaction (HRI) domains, highway driving and object handover, we evaluate different types of human behaviour models using common generative-model evaluation metrics. We consider multiple human trajectory representations and compare the resulting evaluations with the downstream performance of RL policies trained using the corresponding models. We find that existing behavioural evaluation metrics do not consistently predict downstream policy performance across domains and trajectory representations, while their evaluations are sensitive to the choice of trajectory representation. We also show that even when the true testing behaviour distribution is known, training against a different distribution may yield better downstream performance than training directly against the true distribution.
comment: Project page: https://objective-mismatch-hri.github.io/supplementary/
★ ISB-NMPC: Interception-Oriented Spherical-Bearing Nonlinear Model Predictive Control for Quadrotors
Quadrotor interceptors equipped with strapdown cameras offer a low-cost solution for physically countering unauthorized UAVs. However, this configuration inherently couples vehicle maneuvering and camera pointing, creating a conflict between rapid closing and maintaining target visibility. We present interception-oriented spherical-bearing nonlinear model predictive control (ISB-NMPC), which couples spherical-bearing prediction with nonlinear quadrotor dynamics to jointly optimize target-directed closing motion and visibility. An attitude-compensated extended Kalman filter on S^2 estimates target bearing motion, compensates for perception delay, and predicts target bearings to guide scale-adaptive detection. Across 30 paired simulation trials, ISB-NMPC achieves an interception success rate of 86.7%, the highest among the evaluated controllers, with a mean per-trial target-visible time of 99.11%. Outdoor flights demonstrate acquisition of a 30 cm target at approximately 150 m, with interceptor speed reaching 17 m/s in the long-range flight and approximately 14 m/s while pursuing a quadrotor-towed maneuvering target. The software and hardware are open source at https://github.com/HOLYNOTBAD/ISB-NMPC and https://github.com/HOLYNOTBAD/holydrone, respectively.
comment: 9 pages, 10 figures, and 1 table. Submitted to IEEE/ASME Transactions on Mechatronics
★ SceneAgent: An Agentic Pipeline from 3D and 4D Captures to Simulation Environments
Visual captures provide detailed scene appearance, but converting them into interactive simulation environments requires object reconstruction, physical modeling, and correction of intermediate errors. We present SceneAgent, an agentic pipeline that coordinates these steps through simulator-based visual inspection and repair. The pipeline processes 3D Gaussian Splatting (3DGS), photogrammetry, LiDAR captures, AI-generated 3DGS scenes, and 4D videos of dynamic scenes, and combines object segmentation, physical parameter estimation, part decomposition, articulation, and deformable simulation. To support policy learning, SceneAgent generates diverse object variants ("digital sisters") and scripted demonstrations. On a cluttered bin scene from GraspClutter6D, SceneAgent raises F1 at 1 cm for every coding agent with scorable outputs both with and without its skills, and its reconstructions are better than SimFoundry: all agent configurations exceed SimFoundry's F1 score, with a best of 60.75% versus 6.9%. On three dynamic scenes from 4DCodeBench, SceneAgent raises the mean dynamic IoU of Gemini 4 Argon from 0.86% to 99.09%. We further demonstrate diverse simulated interactions and preliminary policy transfer to a real robot.
★ Networked Humanoid Intelligence: Communication, Coordination, Collaboration, and Social Interaction
Humanoid robots are moving from laboratory platforms toward connected fleets that share workspaces, infrastructure, tasks, and knowledge with other robots and humans. As deployments grow, the operation and improvement of each humanoid depend on information exchanged across the network. However, existing research has largely studied humanoids as individual systems. This survey examines networked humanoid intelligence through four functions of information exchange: communication, coordination, collaboration, and social intelligence with human interaction. We first establish the foundations of networked humanoid systems, define the question addressed by each function, and connect them through a common factory scenario. We review the literature along six dimensions: interaction architecture, exchanged information, physical coupling, operating timescale, system scale, and embodiment. We separate results obtained with physical and simulated humanoids from those based on other robotic platforms or disembodied agents, because the latter do not fully capture humanoid-specific constraints such as balance, whole-body motion, contact, and physical interaction. We examine hardware, datasets, and benchmarks, and distinguish risks handled locally from those networking introduces, such as stale information or unsafe knowledge propagation. Finally, we identify open problems in control-aware networking, compact body-centric messages, physically grounded coordination, network- and embodiment-aware knowledge sharing, fleet-level social consistency, and accountable human oversight.
comment: 35 pages, 6 figures
★ VLA-HWBench: Where the Time Goes When VLA Models Run on Edge Hardware
Vision--language--action (VLA) models increasingly run on edge devices aboard robots, where inference latency constrains closed-loop robot control. However, it remains unclear how to design VLA models, their inference software and edge hardware so that robots can run VLA inference in real time. Most VLA models report only task success, while prior studies cover limited action-generation strategies, divide inference into at most three stages, and lack comparisons of inference optimizations on the same models.benchmarking 18 open VLA models across released and alternative backends on NVIDIA Jetson AGX Thor and AMD Ryzen AI Max+ PRO 395, with a seven-stage inference breakdown. Our study reveals three insights. First, action-generation strategy matters more than model size. It dominates 14 of 19 Jetson configurations, latency varies by $82\times$ on Jetson and $157\times$ on AMD, and parameter count explains at most 15% of latency variance. Models that generate long action chunks in a few flow-matching steps keep up with control, whereas token-by-token decoding does not. Second, kernel fusion with 4-bit weights helps most, running $π_{0.5}$ and GR00T N1.7 2.6--2.9$\times$ faster than PyTorch with \texttt{torch.compile}, but it exists for only these two models and only on the Jetson. Optimization also moves the time to the stages it shortens least, as CPU preprocessing grows from 12\% to 39\% of the GR00T N1.7 call. Third, after optimization, memory bandwidth limits only the stages that process few tokens on the Jetson, such as the denoising steps, whereas software limits all other stages, from unfused kernels to serial preprocessing on the host CPU. Overall, low-latency edge VLAs should use few action-generation iterations, inference software should extend kernel fusion and low-bit weights to more models and platforms, and edge hardware needs high memory bandwidth and a fast host CPU.
★ Resource-Performance Trade-offs in Intra-Aisle Robotic Picking: A Discrete-Event Simulation Comparison of Shuttle Configurations With and Without an Onboard Tote Lift
This study compares two implemented intra-aisle robotic-picking configurations through discrete-event simulation. Without an onboard tote lift, two level-captive shuttles serve the two tested levels; with the lift, one shuttle remains on its horizontal level and its onboard mechanism moves the order tote to allow picks from the adjacent upper level. Thirty configuration-task-size experiments (n = 1-15 locations; ten replications each) yield 300 replication-level cycle-time observations. Two-sided Welch tests are applied to the replication means, with Holm adjustment across the 15 comparisons. The onboard-tote-lift configuration has longer mean cycle time at all tested task sizes. At n = 15, the means are 299.477 and 333.231 s/cycle, respectively: a 33.753 s increase (11.27%). The corresponding cycle-time-derived item rates are 180.314 and 162.050 items/h (-10.13%). These rates are calculated as 3600n divided by the scenario-mean cycle time; they are not measurements of the combined concurrent capacity of two level-captive shuttles. Importantly, task locations are sampled from different eligible-level sets in the two configurations, so the result concerns the implemented configuration-and-task-generation packages, not the isolated effect of a five-second tote-lift operation under identical picked locations. The study quantifies a cycle-performance difference alongside a reduction from two dedicated shuttles to one within the tested two-level block; it does not quantify system-level maximum throughput, investment cost, or reliability.
★ Benchmarking Obstacle-Conditioned Safety in Household Robot Navigation
Safe household navigation requires robots to regulate clearance and speed according to what an obstacle is and how close it is, without unnecessary conservatism. Yet existing evaluations do not test whether semantic safety distinctions are reflected in continuous motion. To that end, we introduce OSMA-Bench, an Obstacle-Conditioned Safe Motion Adaptation Benchmark built on spatially matched scenario sets that differ only in obstacle type. We also propose the \emph{Semantic Sensitivity Index} (SSI), which measures whether clearance and distance-dependent speed and smoothness follow the caution ordering of obstacles. Evaluating VLM-based and vision-language-action (VLA) policies on the benchmark, we find that neither consistently adapts its motion to obstacle safety, even with explicit safety prompts. Our analysis shows that the policies can correctly distinguish obstacles by safety relevance but fail to reflect this distinction in the motion parameters or actions they produce. For VLM-based policies, which recognize the obstacle and then generate a navigation program, adding an explicit parameter-inference step between these two stages improves adaptation. These findings expose a gap in \emph{semantic-to-motion grounding}, highlighting the need to translate semantic caution into appropriately safe motion.
★ Video World Models Need Predictable Latents
Video world models trained on large-scale robot data promise a safe and scalable approach for learning and evaluating policies that act in the physical world. However, they still struggle with long-horizon rollouts and out-of-distribution generalization, and they remain computationally demanding at inference time. Typically, video world models make predictions in the latent space of a reconstruction-trained autoencoder, which however is not directly optimized for dynamics modeling. On the other hand, world models designed for latent planning usually optimize latents to be predictable, but they forgo accurate prediction of observations in pixel space. In this paper, we show that it is possible to achieve the best of both worlds: a latent geometry optimized to make future prediction as simple as possible without sacrificing accurate prediction of future observations. More precisely, we find that temporal objectives for representation learning, paired with a past-conditioned decoder architecture, lead to consistent improvements in generative video world models at the same training budget. The gains include up to 5x cheaper inference, over 7x faster training, and 15-24% lower pixel and perceptual errors out of distribution and over long horizons. Our world models also agree more closely with real-world policy evaluation when using a vision-language model to judge task success from their pixel-space predictions. Overall, our study indicates that temporal predictability is an important design criterion for the latent space of video world models, and provides concrete guidance on how to incorporate it into the training of autoencoders.
comment: 54 pages, 25 figures, 14 tables. Code: https://github.com/mattia-scardecchia/predictable-latents
★ Dream to Assist: Intent-Conditioned World Models for Shared Vehicle Control
Assistive agents in shared-control settings face a dual challenge: acting proactively to align with the human's intentions while remaining legible enough for the human to comprehend and fluently respond in real time. We present Dream2Assist, a model-based reinforcement learning framework for tightly-coupled shared decision making that addresses both sides of this challenge. Our approach extends a recurrent state-space world model with intent inference, enabling an assistive policy trained via fictitious co-play to anticipate and support discrete human objectives without overriding human agency. Beyond the computational framework, we describe the full deployment pipeline needed to translate a world-model-based policy into a safe, human-centered system in a resource-efficient way. Crucially, we address the sim-to-real-human gap via policy down-selection and reward restructuring within an iterative human-machine interface design. We validate the framework through three human-subjects studies and a preliminary on-vehicle deployment spanning two qualitatively distinct domains: high-speed racing (N=30) and urban driving with richer intent structure (N=10). Our results demonstrate that intent conditioning yields measurable improvements in torque alignment and collision rate. We surface key design insights and practices for deploying learning-based assistive agents in safety-critical, real-time human-robot teaming.
comment: 32 pages, 18 figures, 16 tables
★ RobotAgentBench: Towards Open-world Robot Autonomy
Recent advances in vision-language-action (VLA) and world-action models (WAMs) have significantly improved robotic manipulation, yet deploying robots in the open world demands a higher level of autonomy beyond executing predefined tasks. Robot agents offer a promising paradigm for this transition, integrating task discovery, contextual reasoning, physical execution, and adaptive recovery into a closed interaction loop. However, existing benchmarks largely evaluate isolated capabilities, failing to capture the complexity of open-world robot autonomy. We introduce RobotAgentBench (RAB), a comprehensive benchmark for evaluating robot agents in realistic, open-world household environments. Through an agent-assisted, human-in-the-loop real-to-sim pipeline, RAB constructs 50 diverse household layouts with interactive objects and physically grounded tasks, encompassing 90 atomic tasks, 18 short-horizon tasks, 8 long-horizon tasks, and 4 open-ended scenarios. RAB provides a multi-level evaluation of robot agent capabilities: (1) task-level assessments systematically examine five complementary dimensions: task discovery, context reasoning, manipulation, failure recovery, and robustness; (2) scenario-level evaluation assesses how agents integrate these capabilities to autonomously fulfill high-level user needs. Furthermore, RAB enables systematic comparisons of standalone VLA/WAM models and diverse robot agent architectures under a unified planning-execution-verification framework. Extensive experiments reveal substantial gaps between manipulation and autonomy, demonstrate the advantages of agentic architectures and recursive self-improvement, and highlight the remaining challenges toward open-world robot autonomy.
comment: Project page: https://aetherlabsai.github.io/RobotAgentBench/
★ Learning Force-Reactive Humanoid Loco-Manipulation
Many everyday tasks require controlling contact force as well as motion. On a humanoid, this force must often be coordinated across both hands and the whole body, and cannot be specified by kinematic trajectories alone. Therefore, we present FLORE, a framework for learning force-reactive humanoid loco-manipulation. We collect robot-free demonstrations using two force-sensing UMI-style handheld grippers for hand motion and contact wrenches, and wearable trackers for body motion. A force-reactive diffusion policy takes vision, proprioception, and measured wrenches as input, then predicts actions and per-hand force commands. A hybrid compliance whole-body controller with force tracking executes these commands while maintaining balance: an integral of the force error measured by real force/torque (F/T) sensors lets each hand track force accurately along the commanded direction and remain compliant along the others. In simulation, the controller tracks per-hand force commands of varied magnitude and direction, with little loss in whole-body motion accuracy. Across three contact-rich tasks on the robot, the force-reactive policy succeeds in 80% of rollouts, compared with 26.7% for a position-only policy trained on the same demonstrations and executed by the same controller. For more details, please visit our website: https://louis-leee.github.io/FLoRe/.
comment: 9 pages, 6 figures
★ Differentiable Contact Dynamics for Legged Locomotion: Policy Learning and Contact-Implicit Trajectory Optimization
This paper proposes a smooth analytic gradient for frictional contact and applies it to policy learning and contact-implicit trajectory optimization for legged locomotion. The exact contact sensitivity is available in closed form but describes changes within the current contact mode, providing no first-order information toward neighboring contact modes. The proposed gradient replaces this sensitivity only in the backward pass, leaving the hard-contact forward solve unchanged and the realized rollout feasible. A central-path relaxation of the de Saxcé-corrected second-order-cone complementarity problem for Coulomb friction smooths the conditions defining the separating, clamping, and sliding modes. The relaxed sensitivity is evaluated at a point on the central path obtained by a Newton iteration initialized from the hard-contact solution. The sensitivity remains continuous through contact-mode transitions and converges to the exact sensitivity as the relaxation vanishes. Using 100,000 samples and less than 2 min of training on a desktop PC, policy learning with the proposed gradient produces a policy that transfers to hardware without fine-tuning. On hardware, the robot under a trotting policy withstands pushes and traverses debris and slippery ground. In trajectory optimization, the proposed gradient reaches contact-rich solutions missed by exact and Signorini-relaxed gradients. Project page: https://diffcontact.github.io
comment: Submitted to Transactions on Robotics
★ Nonprehensile Robotic Catching Using Virtual Constraints
The problem of nonprehensile robotic catching is addressed using virtual nonholonomic constraints (VNHCs); the motion of the robot end-effector is designed and controlled directly in terms of the states of the object before it is caught on the surface of the end-effector. In contrast to contemporary approaches, our approach avoids the need for object trajectory prediction, and consequent time-varying robot trajectory generation. With a focus on the planar case, we demonstrate that our VNHC approach can be used to catch an object on an end-effector at a desired height, with minimal relative normal velocity at the contact instant. Moreover, the VNHC function depends only on the desired catching height, and the same VNHC can be used for a range of initial conditions. We use model predictive control (MPC) to bring the object and end-effector to rest at a desired location after contact is made. The approach is validated in simulation.
comment: 9 pages, 6 figures; an abridged version of this article has been submitted to the 2027 American Control Conference (ACC)
★ OSWAM: An Effective World Action Model with One-Step Action Generation
Direct-policy World Action Models (WAM) have demonstrated the benefits of decoupling action generation from explicit future visual prediction for efficient inference in autonomous driving and embodied control. However, they still require tens of denoising steps to achieve competitive action quality, incurring substantial latency and hindering real-time deployment. Simply reducing the number of denoising steps will collapse the nonlinear multi-step denoising trajectory into coarse updates, substantially degrading action quality. To reduce the denoising steps while enabling real-time action generation, we present an effective world action model with one-step action generation, namely OSWAM, for autonomous driving and embodied control. We argue that it is the absence of structured action priors in random noise initialization that forces direct-policy WAMs to recover coherent action sequences through iterative denoising. Hence, we construct imagination tokens to initialize the diffusion process, providing physical priors and action-conditioned representations. We further introduce privileged information distillation with future conditioning to enhance future-aware action generation. As a result, OSWAM can efficiently and effectively generate high quality actions in just one diffusion step. Our experiments demonstrate that OSWAM achieves comparable or even better results in both autonomous driving and embodied control than direct-policy WAM and Joint-WAM that require tens of steps. It obtains approximately $3\times$ speedup over direct-policy WAM baselines and supports real-time compiled inference with latency as low as 40ms on an NVIDIA RTX 5090 GPU. The project page is available at https://gwenzhang.github.io/OSWAM/
comment: 18 pages, 6 figures
★ Profiling a Frontier Agent with EBench: From Direct Control to Harness with Policy
Frontier multimodal large language models (MLLMs) can control robots without robot-specific training, but how their capabilities compare and interact with post-trained vision-language-action (VLA) and world action model (WAM) policies across manipulation demands remains to be thoroughly evaluated. In this report, we evaluate GPT-6-Astra on EBench as the core of an agent that controls a robot through an agent harness under the same visual and proprioceptive observation protocol as post-trained policies. We study two agent configurations: direct control, in which the agent generates waypoint commands directly, and harness with policy, in which it can also invoke a post-trained VLA or WAM policy for action execution. Under direct control, the agent achieves competitive overall performance relative to post-trained policies. It leads on short-horizon tasks and exhibits the least performance variation across generalization dimensions, whereas it underperforms post-trained policies on long-horizon tasks, tabletop manipulation, and high-precision contact-rich tasks. In the harness with policy, invoking a post-trained VLA or WAM combines the complementary strengths of both components, raising overall task success to the level of the strongest compared post-trained policies while substantially reducing token cost. However, performance remains below the per-task upper bound of the standalone components due to suboptimal policy invocation, fixed-instruction execution without subgoals, and grasp disruption when the agent resumes control. Overall, our study characterizes the capability boundary of a frontier MLLM agent and suggests directions for improving future agent harness design. Project page: https://internrobotics.github.io/EBench-Astra-Tech-Report/
comment: Project page: https://internrobotics.github.io/EBench-Astra-Tech-Report/
★ Pretraining Knows What, Simulation Teaches How: Compositional Generalization in a Sim-Trained Robot Agent
A robot policy needs to understand what a task is asking for in a scene, and how to move the robot to accomplish that task. To achieve these two skills, we introduce Zero-real Agent Policy (ZAP), a recipe that teaches a small off-the-shelf vision-language model to control a robot via a simple tool harness. We use a pretrained 27B parameter model to bootstrap world understanding and reasoning, followed by supervised fine-tuning and GRPO on simple simulated tasks to teach it how to move the robot. Without any real-world robot data during training, ZAP reaches 96.7% success on real-world tasks, matching GPT-6 Astra in our harness and outperforming the MolmoAct2 VLA (33.3%), while running 3x faster and 17x cheaper than GPT-6 Astra. We demonstrate that ZAP also generalizes to completely new objects, goals, strategies, and language by seating a test tube in a rack (and choosing a different hole in the rack when prompted), handing a test tube between arms mid-air, drawing an X on a tic-tac-toe board, and playing chess. These goals and types of manipulation don't appear in our training data. Furthermore, ZAP also transfers to an unseen embodiment. On a single-arm Franka, ZAP reaches 55.9% on RoboLab-120, 4.9 percentage points ahead of the leading VLA/WAM baseline.
★ Multi-Objective Reinforcement Learning Using Routed Ensembles and Trajectory Attribution NeurIPS 2026
Multi-objective reinforcement learning is a fundamental problem for modern AI, with applications spanning post-training, reasoning models, and robotic control. A central challenge is to learn policies that balance competing objectives, such as helpfulness versus honesty in safety and alignment, or speed versus energy in robotics. The optimal policy can vary sharply with objective-weighting combinations, and the extent of this variation depends crucially on the degree of conflict among objectives, making per-preference policy learning sample-inefficient. In this paper, we introduce objective-conflict coefficients and derive near-optimal regret bounds for online settings using this measure, thereby quantifying the impact of misalignment on learning efficiency. Then, we design routed ensemble policy learning, an algorithm that maintains a small set of basis policies and dynamically routes trajectories to the policy with the highest potential reward. Finally, we consider trajectory selection as another approach to mitigate objective conflict, and design a kernel surrogate trajectory attribution algorithm that estimates how feedback signals from a set of trajectories influence policy learning on other trajectories. The kernel can capture nonlinear interactions such as synergy and antagonism between trajectories. We evaluate our approach across diverse environments and benchmarks: routed ensembles improve average performance by 4.3% over baselines for online environments from Meta-World and MO-Gymnasium, and kernel surrogate attribution improves correlation with leave-one-out analysis over existing baselines by 50%. When applied to trajectory selection, kernel-based attribution yields a 27% improvement in reward over baseline attribution methods.
comment: 30 pages. To appear at NeurIPS 2026 (extended version)
★ SPECOLA: A Tool for Specification Driven Robot Evaluation
Simulation benchmarks are a useful way to measure progress in robotics; typically they provide success rate as the dominant measure of performance. However, success rate reports only whether a rollout completed the task, hiding by what margin it succeeded, how sensitive the success is to goal parameter changes, requirements it may have violated, and whether success persisted. Signal Temporal Logic (STL) provides a formal language for capturing specifications; it can be used to define requirements, and measure how close a robot behavior is to satisfying or violating them. Using STL monitors in a simulator, however, requires exposing simulator state and mapping it to specification-relevant signals. We present Specola, a tool for evaluating robot behaviors in simulation with respect to STL specifications. Specola provides an interface for writing specifications over simulator signals, and creates and manages the relevant monitors while a policy executes. We provide a Python API and command-line interface for integrating monitoring into existing policy execution loops. We provide an adapter interface for connecting new simulators, and provide adapters for nine simulation environments and benchmarks, including LIBERO, RoboCasa, Isaac Lab, and RoboDojo. We provide case studies that illustrate the nuanced evaluation that Specola enables. Code and adapters are available at https://github.com/specola-eval/specola.
★ TrySee: Modular Perception-Enhanced VLM Grounding for Closed-Loop Robot Task Planning in Specialized Domains
Vision Language Models (VLMs) have shown impressive capabilities in high-level reasoning and task planning for robots. However, specialized domains remain a challenge for VLMs' image interpretation and grounding abilities. We introduce TrySee, a modular closed-loop robot task planning pipeline integrating general-purpose VLM reasoning with domain-specific perception and established robotics components. The system consists of an inherently reactive cyclic planning pipeline, augmented with domain-specific modular vision preprocessing, to provide online grounding. Finally, a dynamic preprocessing selection mechanism contributes to the scalability of the pipeline. The system requires neither VLM fine-tuning nor end-to-end demonstrations and adapts through replaceable perception modules and motion planning. The approach is evaluated on planning tasks across 5 widely different domains, including a real-world robot disassembly task. The field tests in a brownfield environment demonstrate the approach's practicality in integrating VLM planning into specialized robotic systems with limited domain-specific training data.
comment: To be published in the proceedings of the 2026 IEEE International Conference on Robotics and Biomimetics (ROBIO)
★ FISRF: Flexible Instructional Scaffolding Reflection Framework for Robotics Task Planning with Large Language Model
Large Language Models (LLMs) are widely adopted for reflective error correction in task planning; however, existing approaches primarily rely on autonomous exploration, rendering them prone to accumulating erroneous experiences during early interactions with new environments. To address this limitation, we propose the Flexible Instructional Scaffolding Reflection Framework (FISRF), an agent--mentor architecture that estimates the robot's Zone of Proximal Development (ZPD) in task planning and adaptively selects reflection strategies and mentor guidance, thereby facilitating the accumulation of effective error-correction experience. Experiments in ALFWorld and on real-world robotic platforms demonstrate that FISRF reduces the average number of error-correction rounds by 37% and redundant reasoning by 25.1%, while improving the overall task success rate by 8%. Project webpage : https://fisrfplanning.github.io/
★ Continuous-Time Safe Planning and Control for Agile Quadrupeds
Agile quadruped navigation in dynamic environments demands trajectories that are both dynamically feasible and provably collision-free. Most real-time planners achieve tractability through a convex reduction that requires simplified kinematic models, discarding motion that the robot can readily execute, and certify safety only at discretization points, leaving open possible inter-sample collisions with moving obstacles. We present a real-time motion planning framework that retains the fully-actuated $SE(2)$ model of a quadruped, with its anisotropic, heading-coupled velocity bounds, and accepts the resulting nonconvexity. By parameterizing trajectories as composite Bernstein polynomials, we exploit the convex hull property to certify collision avoidance and the input bounds over the entirety of each planning horizon. Solved at online replanning rates, the optimization runs in closed loop, commanding body velocities to the locomotion controller. We validate the approach in simulation against ablations and external baselines, and in hardware experiments on a quadruped navigating among unknown moving obstacles, recovering both lateral and heading-coupled motion that simplified models exclude.
★ Environment Edits as a Corrective Signal for Reward Learning
Robot corrections allow people to adjust a robot's behavior, but a single correction changes many features. Also, preferences about object attributes and relationships between objects cannot be expressed by moving the robot. We propose environment edits as a complementary form of feedback. The robot's motion is held constant as the human edits the scene, so the only explanation for the edit is the moved object. We introduce Feature Learning from Environmental and Robot-Relational Edit Traces (FERRET), which learns preferences from both robot and environment corrections. We show that environment edits are able to disambiguate features and learn preferences that robot corrections alone cannot.
comment: Accepted to CoRL 2026 Workshop on Learning from Corrections
★ VPR-Space: A Benchmark for Visual Place Recognition in Space Analog Domains
Visual Place Recognition (VPR) is an important means for loop closure detection in visual Simultaneous Localization and Mapping (SLAM). While state-of-the-art VPR and SLAM approaches excel in urban and feature-rich natural environments, the vast and feature-poor environments of extraterrestrial surfaces like on the Moon or Mars pose severe challenges for existing approaches. This paper presents a novel benchmark, VPR-Space, to evaluate VPR capabilities on extraterrestrial surfaces. There are three contributions: (1) We collected sixteen VPR database-query datasets based on existing data from Space Analog Domains (SAD), terrestrial environments that resemble properties of the Moon or Mars. The collection includes a range of challenges regarding environment, viewpoint change, and illumination conditions. (2) To address the ill-posed nature of VPR, we propose a novel approach to generate the ground-truth information in VPR benchmarks. Instead of defining that a query and datbase image show the same place if and only if their camera poses are within fixed position and viewing direction thresholds, we propose to make the decision dependent on whether the match would be valuable in the downstream SLAM task. To avoid the overhead and restrictions of evaluating VPR in-the-loop within SLAM systems, we provide VPR ground-truth (GT) matrices based on the performance of an ensemble of state-of-the-art Relative Pose Estimators (RPE). Under this GT we require a VPR method to identify exactly those image pairs for which a subsequently applied RPE is expected to provide a valuable pose estimate in a SLAM context. (3) We evaluated nine state-of-the-art VPR approaches. The results show considerable room for improvement and indicate directions for future research. We provide a pre-release version of the dataset on github: $\texttt{https://github.com/rfechner/VPR-Space}$
★ Adapt for Planning, Not just Prediction: Optimistic Test-Time World Model Adaptation
Latent world models are typically pre-trained offline and kept frozen at test time, so planning with them can degrade under distribution shift. Test-time adaptation (TTA) can mitigate this degradation by updating the model during deployment. Unlike pre-training on fixed data, TTA for planning is closed-loop: the planner uses model rollouts to select actions, and the resulting transitions drive the next update. This raises the question: which objective should guide TTA for planning? Under a prediction-only objective, the model is fit only to the outcomes of its own actions, so it can remain wrong about the actions that would reach the goal, and low prediction error need not translate into better planning. Inspired by reward-biased maximum likelihood from adaptive control, we introduce Optimistic Test-Time Adaptation (OpTTA). Its objective has two terms, the prediction loss on observed transitions and the planner's goal-reaching cost on its candidate rollouts, both minimized over the model parameters. The planning term biases adaptation toward models that are optimistic about reaching the goal, without any additional environment interaction. Across visual and environment shifts on manipulation, reaching and navigation tasks, OpTTA raises final planning success over the frozen model by 13.5 percentage points on average, compared with 6.9 for the prediction-only objective (AdaJEPA), when only the predictor is adapted. When the encoder is adapted as well, the gains are 13.5 and 7.5 points. These experiments span three JEPA world models with different encoders and training objectives and, for temporal straightening, both CEM and gradient-based planners. Our results suggest that test-time adaptation of world models for planning benefits from an optimistic, task-aware term in addition to the prediction loss.
comment: Accepted at the CoRL 2026 Workshop on Pretrain to Adapt
★ Revisiting Direct Control: A Field Evaluation of Sensor Fusion Efficacy in Teleoperating Full-Scale Autonomous Vehicles via Bird's Eye View IROS
In autonomous driving, robot teleoperation serves as a critical safety fallback mechanism, requiring operators to assume direct control in the event of system failure. While recent research has pivoted toward high-level supervisory control or simulation-based studies, these approaches often fail to capture the physical dynamics and sensory demands of driving a full-scale vehicle. To address this gap, we evaluated the impact of spatial data representation on operator performance using a real car-sized vehicle in a field experiment with $N = 18$ participants. We compared three visualization interfaces: (i) a standard camera feed baseline, with two bird's-eye views (BEV) from (ii) a raw LiDAR point cloud, and (iii) a sensor fusion of LiDAR and camera data streams. Contrary to prior passive observation studies suggesting that sensor-fusion BEV may hinder awareness, our active-control results demonstrate that such representation significantly outperforms the standard LiDAR BEV in perceived usability ($p \leq 0.0001$) and cognitive task load ($p = 0.01$, NASA-TLX score: 34.58/100) while also being significantly better than the Camera-only view ($p = 0.043$). The fused interface also led to statistically significant reductions in task completion time ($p = 0.0461$) and superior usability (SUS: 77.9/100) compared to those of the Camera-only and LiDAR-only views. By synthesizing objective performance data with comprehensive subjective assessments, we demonstrate that BEV fusion maximizes safety margins while minimizing the operator's mental demand. These findings challenge existing teleoperation conventions and provide an empirically validated approach for designing robust remote piloting systems.
comment: Accepted at 2026 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
★ From Language to Drives: A Multimodal Framework for Aligning Autonomous Robots with Human Purposes
This work explores a multimodal approach to the alignment of the behavior of open-ended learning autonomous robots with human purposes through the use of large language models as cultural enablers. While autonomous robots can generate their own skills and goals through interaction with the environment, they often lack mechanisms to ensure that these remain relevant and aligned to those of human users. The proposed approach makes use of the general and cultural knowledge contained in LLMs and VLMs to establish a multimodal interactive alignment process in which human purposes, expressed in natural language, are transformed into motivational components of a robot architecture in the form of robot missions and drives that guide the behavior of the robot. The system is structured into three layers for semantic alignment, mission generation, and drive modeling. This design enables a bidirectional interaction loop: human inputs define high-level purposes, robots expand them into executable missions making use of multimodal information relayed by LLMs and VLMs, and feedback maintains alignment with human intent. Experiments in simulation and with a physical robot demonstrate that the process produces valid missions and behaviors, supporting effective human-agent alignment in interactive scenarios with autonomous robots.
★ Robot-Aligned 3D Gaussian Splatting for Precise Reaching with Wrist-Camera VLA Policies
Efficient robotic data collection remains a major bottleneck for vision-language-action (VLA) policy learning, especially for tasks requiring accurate end-effector positioning and camera-action alignment. This challenge is particularly severe in electronic-waste disassembly, where robots operate in cluttered assemblies of densely packed, visually similar components. Full digital-twin simulation is difficult because many parts are thin, reflective, or mechanically complex, making accurate object-level meshes and contact models difficult to obtain. In this work, we focus on pre-contact reaching and propose a 3D Gaussian Splatting (3DGS)-based data generation pipeline for wrist-camera VLA adaptation. Our method reconstructs the real workspace with 3DGS and renders diverse wrist-camera observations from robot-aligned poses. These observations are paired with smooth, structured reaching trajectories to provide robot-aligned visual-action supervision for VLA fine-tuning. We further augment the VLA policy with an instruction-conditioned reach verifier that estimates target proximity and provides reliable termination before physical interaction. Experiments in motherboard disassembly demonstrate precise VLA-based reaching using wrist-camera observations alone, without an external side-view camera. With $π$0.5, 1,500 generated trajectories achieve 80.0\% precise-reaching success across three targets, compared with 58.7\% using 150 teleoperated demonstrations, while requiring less than 10 minutes of active human capture. Project website: https://001-wang.github.io/PrecisionReach-GS/
★ Context-Aware Generation and Modulation of Expressive Motion Behavior using Multimodal Foundation Models
Expressive robot motion positively impacts human-robot interaction by improving user engagement, likability, or task performance. We present a novel approach to automatically generate and modulate complex, expressive full-body gesture sequences from a multimodal (text, audio, video) context description. Based on a unified mathematical implementation of the Principles of Animation using Dynamic Movement Primitives, we use multimodal foundation models to generate such sequences along with parametric motion variations that are highly context-aware. This method extends the state of the art in terms of flexibility and generality by being interpretable, composable, and working across different robot morphologies. Moreover, we integrate the system into a continuous control framework and leverage knowledge distillation to learn a much smaller model, significantly improving token efficiency and system latency. Results from a user study with a human-like platform indicate that participants judged our system's motions to be better aligned with the interaction context than motions produced under a non-modulated or API-level condition. We also demonstrate how the system generates expressive motion for robots with different kinematics, showcasing its versatility. Paper webpage: https://gen-mod-expressive-motion-behavior.github.io/.
★ RoboDoctor: Training-Free Recursive Self-Improvement for Robot Policies
Generalist robot policies and language-model agents provide complementary capabilities, but completing a task also requires adapting their use to the consequences of physical interaction. We study execution as a locus of adaptation: feedback guides the current action and supplies evidence for revising subsequent decisions under fixed model weights. We introduce RoboDoctor, a training-free framework that connects diagnosis, grounded action, and persistent experience across three revision targets. The harness allocates control between policy and agent; intervention determines when to assist while preserving useful policy behavior; and operation refines physical methods through observed effects. Linked experience records retain the reason for a decision, the attempted method, its outcome, and conditions for reuse. On a curated archive of 54 RoboDojo-Sim task entries, RoboDoctor achieves a five-dimension mean score of 57.10, with a native success rate of 49.44%. Recorded interactions illustrate strategy changes, selective correction, and grasp revision. These findings characterize execution-time adaptation in the evaluated system; controlled trials remain necessary to isolate the contribution of cross-episode experience.
comment: Project Page: https://robodoctor-rsi.github.io/
★ Distributed Predictive Flocking for a Swarm of Miniature Vibration-Driven Robots in a Confined Arena
Coordinating many miniature robots in a confined arena, with no leader, central planner, or global map, is a core challenge for swarm robotics. As the swarm grows denser, collision avoidance becomes the limiting factor, and the number of safely coexisting robots depends on the formulation of the inter-robot safety constraint. We present a distributed predictive flocking controller in which each robot solves its own nonlinear model predictive control (NMPC) problem, optimizing swarm cohesion and alignment costs, inter-robot and obstacle safety, and a task objective, while exchanging only predicted trajectories with local neighbors. Within this generalized controller architecture, solved in real time, we compare six barrier and control-barrier-function (CBF) safety formulations on a digital twin of an open-source vibration-driven bristlebot in a $0.64\,\mathrm{m^2}$ arena. A look-ahead discrete-time CBF performs best overall, with the fewest collisions and the widest inter-robot margins across nearly every swarm size tested. We deploy it on phototaxis, phototactic orbiting, and cohesion without a stimulus tasks, using swarm sizes starting from ten robots and up to the geometric packing limit. The swarm holds well: collisions stay low in small swarms, rising steeply as the packing limit approaches, while coordination and cohesion degrade gently rather than collapsing. The swarm is found not to trade safety for order, keeping both across the confined arena, with local alignment high at every swarm size.
comment: Submitted to Robotics and Autonomous Systems
★ Preemptive Instruction Following with Resumption
Long-horizon embodied navigation may be interrupted by a higher-priority instruction before the current task sequence is complete. The agent must finish its current atomic task, execute the inserted task, and resume the remaining sequence from the updated environment state, where its viewpoint and available navigation candidates may have changed. To evaluate this capability, we introduce PIFR benchmark, a novel task for runtime task preemption and resumption in embodied navigation. PIFR includes single, repeated, and nested interruption settings,with stage-specific metrics for inserted-task completion, resumption of the original task, and completion of the full sequence. Evaluations show that recovery performance varies across interruption conditions, while aggregate success can obscure differences in interruption handling and resumption. We further propose a lightweight candidate-ranking baseline to improve target re-grounding after interruption. Together, PIFR and the baseline provide a systematic framework for studying interruption handling and task recovery in long-horizon embodied navigation.
★ High-resolution tactile observation makes hidden racket-control mechanisms learnable
Experts can master fine hand-object manipulation skills, yet limited conscious perception of rapid tactile interactions hinders their ability to articulate the underlying control mechanisms. Badminton racket control requires rapid, fine hand-force coordination, providing a demanding testbed for uncovering these mechanisms. An 864-node tactile-inertial glove with synchronized 100-Hz acquisition recorded over 14,000 strokes from more than 70 participants across five skills. Inspired by sensorimotor control, our mechanics-grounded two-layer framework inferred object-driving and grip-stabilizing control from tactile history, while resolving their spatiotemporal force realization. We found that higher proficiency was associated with selective rather than extensive control: greater skill specificity, anticipatory regulation, focal distal modulation and lower geometric torque cancellation, while spatiotemporal coordination showed a non-monotonic acquire-refine pattern. We translated these mechanisms into personalized guidance targeting force control and force-motion coordination. In a randomized cohort, mechanism-guided learners gained 0.72 expert-rated skill levels versus 0.05 with level-only feedback, making control mechanisms actionable.
comment: 58 pages, including 14 pages of main text and 44 pages of supplementary information. 5 main figures and 19 supplementary figures
★ Enhancing Deep Reinforcement Learning for Robotic Motion Planning via Task Decomposition
Training robotic manipulation policies in simulation is widely adopted due to safety and cost efficiency; however, tasks involving obstacles remain under-performing and often converge slowly under Deep Reinforcement Learning (DRL). To address this challenge, we propose a task decomposition framework for DRL that integrates Transfer Learning (TL) with Reward Redesign (RR) to improve learning efficiency and robustness in obstacle environments. The TL strategy selectively transfers only the first-layer critic parameters across structurally related tasks, since that layer captures general state--action representations while deeper layers encode reward- and obstacle-specific value geometry; RR decomposes tasks into staged sub-objectives to provide denser optimization signals. Experiments on the PyBullet Panda platform show consistent gains in obstacle settings. On Push with obstacle, our method achieves $0.83 \pm 0.05$, outperforming the strongest baseline (Truncated Quantile Critics (TQC), $0.77 \pm 0.06$) by approximately $7.8\%$. On Slide with obstacle, performance reaches $0.95 \pm 0.02$, exceeding Soft Actor-Critic (SAC) ($0.84 \pm 0.06$) by about $11\%$ success rates. In addition to higher final performance, forward transfer reduces convergence time by up to $70\%$ in obstacle tasks, and RR shortens sparse-reward Pick-and-Place training from roughly $0.75$M to $0.20$M steps ($\approx73\%$ reduction). The best policy is deployed on a real Franka Emika FR3 robot as a qualitative feasibility demonstration.
comment: Accepted to IEEE MSN 2026. 9 pages
★ DEX-PHD: A Dexterous Portable Handheld Device for Multimodal Data Collection
Existing manipulation data collection methods typically offer either accurate robot-state acquisition through kinematic-chain-based teleoperation or convenient, portable operation through vision-based approaches. We present DEX-PHD, a portable handheld device that combines these advantages for dexterous manipulation by integrating a dexterous robot hand, continuous per-finger control, and multimodal logging. The system records proprioceptive information, fingertip contact data, and wrist-camera video, while the operator directly controls the robot hand through joint-level mapping. Its untethered design enables data collection across diverse environments. We demonstrate DEX-PHD on contact-rich tasks requiring coordinated multi-finger manipulation and deploy the collected demonstrations through open-loop replay on a physical humanoid robot. The results show that open-loop replay preserves the main temporal patterns of finger motion and task-relevant contact. Therefore, DEX-PHD provides a practical platform for portable, multimodal dexterous manipulation data collection.
★ RoKON: Efficient and Adaptive Data-Enabled Predictive Control for Robotic Manipulation
Data-enabled predictive control has recently emerged as a promising control paradigm, enabling a transparent optimal-control formulation without the need to learn an explicit dynamics model. This paper presents a number of extensions paving the path towards its use in real-time robotics tasks. Following recent work, we utilize a nonlinear predictor based on Gaussian kernels to represent the relationship between input and output data. We propose a damped Gauss-Newton method that enables real-time control by efficiently exploiting the least-squares structure of the control problem. In addition, we introduce a hybrid buffer management strategy that incorporates informative online data while retaining relevant offline data, allowing the predictor to account for changes in the dynamics. We demonstrate the tracking performance and computational suitability of the proposed method in MuJoCo using a Franka Research 3 arm pushing a block across surface, without prior knowledge of its changing friction values.
comment: 8 pages, 3 figures
★ Model-Aware Generation of Safety-Critical Scenes for Robust End-to-End Driving via Distilled Planner Surrogates
End-to-end driving models perform well on standard benchmarks but collide frequently in safety-critical scenarios. We present a failure-mode analysis on NeuroNCAP which shows that 95.9% of DiffusionDrive's collisions are planning failures despite reliable detection, pinpointing the bottleneck in planning rather than perception. We attribute this to a fundamental data imbalance, as safety-critical interactions are rare in real driving logs, so planners are hardly ever trained on them. We close this gap with a novel model-aware framework that generates safety-critical training scenes against a specific planner. Each planner is attacked at the bird's-eye-view (BEV) level through a differentiable surrogate distilled from its own planning head. This keeps the expensive video renderer out of the attack loop. We extract 50 scenes per planner, each pairing an attack with a safe escape trajectory that serves as the fine-tuning label. A fully automated quality-diversity search generates both the attacks and the escapes, so that their variety is not specified by hand. Only these scenes are rendered to fine-tune the planner, which reduces the computational cost significantly. Fine-tuning on these scenes roughly halves the closed-loop collision rate of two planners on NeuroNCAP (SparseDrive 28.0% to 12.2%, DiffusionDrive 37.5% to 19.2%) while largely preserving normal driving. A cross-planner transfer experiment shows that the gain comes from targeting the specific planner. Each planner improves most on the scenes generated against itself, and DiffusionDrive fine-tuned on SparseDrive's scenes retains only 6.7 of its 18.3-point collision-rate reduction.
★ MATE-BT: A Behavior-Tree-Based Framework for Hierarchically Decentralized Mission Autonomy in Heterogeneous Multi-Robot Teams
Multi-robot teams executing predefined missions must adapt to runtime changes and sustain mission management despite robot failures. This paper proposes MATE-BT (Mission, Allocation, and Task Execution with Behavior Trees), a hierarchically decentralized framework that separates mission execution, allocation, and task execution. An active onboard leader manages mission progression and team-level task assignment, with authority transferable to another leader-capable robot after failure. Interchangeable decentralized methods resolve target assignments, while each robot adapts its local behavior through planning and acting with behavior trees. Webots simulations and real-robot experiments demonstrate mission continuity despite new targets and robot failures, including active-leader failure. In the physical experiments, all MATE-BT computation runs onboard. Hardware-in-the-loop resource measurements further support onboard execution on Jetson Orin Nano and Raspberry Pi 5. By updating only the affected layers without full-plan regeneration, MATE-BT maintained a response time of approximately 130 ms with 30 robots and 120 targets. Comparisons of four target-assignment plugins reveal scenario-dependent trade-offs between mission completion time and total travel distance.
★ EmbodiedLab: Benchmarking Embodied Agents That Act to Know
Embodied agents are expected to accomplish real-world goals by deciding for themselves how and when to interact with their surroundings. For many such goals, changing the physical state of the world is the means rather than the end: the agent changes the state of a system through interaction, observes the feedback, and explores until it either achieves the goal itself (e.g., discovering physical laws) or learns what the goal requires; it acts to know. Manipulation benchmarks stop at acting to change, with the target state given; scientific discovery benchmarks evaluate knowing but replace physical experimentation with symbolic actions or numerical queries. We introduce EmbodiedLab, a benchmark in which a robot must act to know. Quantitative science experiments make this setting controllable: the parameters of the system are resampled for every instance, so the result cannot be known in advance, yet it can be scored exactly. Across 30 experiments in five disciplines on 14 apparatus platforms, the agent receives an objective and the apparatus, chooses its own procedure, and operates a physics-based simulated laboratory, reading the instruments through its cameras. Evaluation checks what the agent came to know, grounded in readings recorded during the episode, as well as the physical state the task requires; every experiment is verified to be solvable. We evaluate six frontier models in their developers' coding agents with native observation and control, with added perception and motion planning, and with manipulation skills distilled from the strongest agents. With native control, even the strongest agent succeeds less than half the time; for every agent, most failures arise in interacting with the apparatus, and manipulation skills help weaker agents more than perception and planning. The benchmark and its evaluation code are available at https://github.com/HexaFuture/EmbodiedLab.
comment: Technical report. 19 pages, 10 figures, 9 tables. Project page: https://hexafuture.ai/benchmarks/embodiedlab/
★ VisGeo-PipeTrack: A Visual Geometric Servo Control Framework with MPC-RCAPID Control for UAV-Based Dynamic Pipes Tracking
While unmanned aerial vehicles (UAVs) are increasingly deployed for pipeline inspection, prevailing research predominantly assumes static pipes. In real-world industrial environments, pipes often exhibit dynamic movements, such as vibrations. This imposes more stringent requirements on UAVs' adaptive control and computational efficiency than in static scenarios. Therefore, this paper proposes a servo control framework named VisGeo-PipeTrack, which utilizes geometric constraints to enable UAV tracking of dynamic pipes. To achieve efficient UAV positioning, YOLOv5 is employed to detect labels on the pipe. The positions are derived from visual geometric constraints based on these detected labels to enable precise localization. To address tracking control challenges, a model predictive control based real-time corrected adaptive PID (MPC-RCAPID) controller is proposed. This controller employs the Recursive Least Squares method for real-time model estimation and integrates rolling horizon optimization to dynamically adjust PID parameters. Experiments conducted on both the AirSim simulator and the Tello UAV platform validate the effectiveness of the proposed framework. Simulation comparisons with standard PID, fractional-order PID, adaptive PID, and fuzzy PID demonstrate that MPC-RCAPID achieves the shortest settling time (3.26 s) and the lowest average tracking error (3.52%). In physical experiments on the Tello platform, compared with standard PID control, MPC-RCAPID reduces the average tracking error by 55.52%, 26.72%, and 26.52% across the three test dimensions. These results demonstrate that VisGeo-PipeTrack facilitates efficient autonomous tracking in dynamic pipe environments.
comment: Submit to IEEE Transactions on Intelligent Transportation Systems IEEE TITS
★ Reward-Guided RRT* for Budgeted Task and Motion Planning
Robotic missions involving spatially distributed tasks often require jointly selecting rewarding locations and planning feasible motions under limited travel budgets. This paper proposes Reward-Guided RRT*, a new sampling-based framework that constructs a budget-feasible mission through a sequence of locally joint task-selection-and-motion-planning stages, evaluated directly in the continuous state space, without requiring a precomputed task-transition graph. The method incrementally builds a mission using reward-guided RRT* searches, while a return-feasibility procedure ensures that each selected task can be visited without violating the mission budget. Experiments with increasing obstacle complexity and number of tasks show solutions comparable to VNS-PRM* while requiring substantially fewer sampled states. We also extend the framework to Dubins steering and evaluate it in CoppeliaSim with a scaled car-like robotic platform, demonstrating its ability to incorporate vehicle kinematic constraints.
★ Labference: Benchmarking the Reasoning-Action Connection in Robotic Laboratories
Realizing Embodied AI for Science requires robots to translate experimental goals into reliable physical procedures, reducing hazardous and repetitive human work while generating evidence for scientific discovery. Although language-based planning and learned manipulation provide complementary capabilities, their integration requires robots to adapt subsequent operations to actual execution outcomes. For example, an unsuccessful placement changes what must be done next, even when the experimental goal remains unchanged. Existing benchmarks assess planning, manipulation, and laboratory workflows through task completion and reasoning or execution metrics; however, aggregate performance leaves the connection between outcome-dependent choices and physical execution unresolved. To address this evaluation gap, we introduce Labference, a benchmark for evaluating how observed physical outcomes guide subsequent choices and how those choices translate into robotic laboratory actions. Its evaluation protocol is designed to test outcome-dependent behavioral adaptation through controlled evidence variations, while independently measuring operation selection, physical execution, and task completion. Matched Oracle-assisted continuations from shared checkpoints then assess which forms of support improve subsequent choices and task completion. Initial weighing evaluations reveal target-selection errors in the tested VLA policies despite successful object transfers, whereas GPT-driven control achieves correct selection but encounters execution failures and substantial computation costs. Together, these findings highlight the need to connect task-appropriate reasoning with reliable, timely action. By making this connection measurable and diagnosable, Labference supports targeted improvements toward autonomous experimentation for Embodied AI for Science.
★ Thermal-Headroom Routing for Efficient Robot Perception in Extreme Heat
Adaptive robot perception systems must balance detector accuracy, inference latency, and energy consumption, particularly when high ambient temperatures cause processors to thermally throttle. We propose Thermal-Headroom Routing (THR), a lightweight model-selection policy that forecasts processor temperature using a closed-form RC thermal model and excludes detector configurations predicted to cross a throttling threshold. We evaluate THR with four YOLO11 configurations across seven real-world perception traces, eight ambient temperatures, and 20 randomized seeds, comparing it with latency-aware routing, fixed-threshold thermostats, duty caps, and a matched-compute-budget control. At ambient temperatures of 45-55 degrees C and a tight inference deadline, THR eliminates throttling in simulation, reduces deadline misses by 3.9-7.5 percentage points, and reduces energy per on-time frame by approximately 12-13% relative to a latency-belief router. A separate experiment using deadline-paced, nonoverlapping frame execution yields miss-rate reductions of 6.2-9.5 percentage points. A tuned thermostat or duty cap can match THR at a fixed operating point; predictive routing is most useful when ambient conditions or calibrated thermal parameters differ from those used to tune static policies. Measurements on an Apple M5 support the single-node heating approximation but do not validate hardware throttling. The results characterize the conditions under which thermal forecasting improves the efficiency and timeliness of resource-constrained robot perception.
comment: 9 pages, 5 figures, 11 tables. Code and experimental outputs: https://github.com/ImSpxrsh/thermal-headroom-routing {Submitted IEEE Big Data}
★ RetroNav: Scene-Grounded Retrospection and Retry for Self-Evolving Lifelong Navigation
Existing robot navigation methods primarily focus on isolated tasks, overlooking the need to accumulate knowledge and experience across repeated navigation episodes. Although recent lifelong navigation methods have explored cross-task knowledge accumulation, they mainly focus on scene knowledge while neglecting historical navigation experience and its connection to scene evidence, limiting the ability to reflect on past mistakes and improve future decisions. To address this limitation, this paper presents RetroNav, a training-free self-evolving navigation framework that integrates navigation, memory accumulation, and failure reflection. Specifically, we develop a unified memory system that connects geometric and semantic scene knowledge with historical navigation experience, distinguishing objective observations from task-dependent interpretations. After each navigation attempt, an independent evaluator determines whether a retry is necessary. If so, a Retrospection Agent revisits historical decisions using accumulated scene knowledge and navigation records, generating an evidence-grounded retry brief to guide the next attempt. RetroNav achieves state-of-the-art performance across five navigation benchmarks. Our method will be open-sourced at https://github.com/3dv-casia/RetroNav .
★ How Tuning, Reference Preview, Actuator Modeling, and Fallback Policies Change Quadrotor Controller Comparisons: A Reproducible Simulation Study
Quadrotor controller comparisons commonly report a single tracking error, although the resulting ranking also depends on tuning effort, reference information, actuator prediction, optimizer fallback, and command timing. We quantify these effects for cascade PID, hover LQR, boundary-layer sliding-mode control (BL-SMC), linear MPC with LQR fallback (LMPC-F), a preview-enabled LMPC-F, and a geometric $SO(3)$ controller on a common nonlinear plant with rotor gyroscopic coupling, motor lag, and saturation. A primary study (556 episodes at 50~Hz) combines matched twelve-candidate tuning, 400 paired evaluation episodes, and a 48-cell MPC factorial; a confirmation study (1462 episodes) repeats the evaluation on 50 fresh realizations with tuning budgets of 12, 24, and 48 candidates. A matched twelve-candidate tuning protocol changed the mean-RMSE ordering obtained with the fixed baseline settings, moving BL-SMC from last to first; the new ordering recurred on the fresh realizations, and a 48-candidate budget left it unchanged. Reference preview substantially reduced MPC tracking error and, on the figure-eight task, removed fallback activity. On an abrupt step, the outcome depended on the interplay between fallback policy, predictor, and actuator authority. Command timing also changed the outcome: replaying measured command-call delays terminated three of four nominal MPC configurations on a desktop computer and all four on a Raspberry Pi~5, whereas capping the solver at 100 iterations removed every deadline miss in the tested nominal timing profiles. Controller rankings therefore depend not only on tracking accuracy but also on the tuning protocol, reference information, fallback behavior, and implementation timing. We propose a reporting checklist covering tracking accuracy, tuning effort, completion and failure rates, fallback activity, applied-input variation, and command timing.
comment: 20 pages, 5 figures, 11 tables. Reproducible simulation study of quadrotor controller benchmarking
★ A 2-DOF MultiBody Track Joint for Race Car Optimization
TrackJoint is a Modelica MultiBody component that turns a three-dimensional road surface into a joint with two degrees of freedom. The two joint coordinates are the distance travelled along a reference line and the sideways offset from that line. Together they give the position, orientation and angular velocity of a single frame_b connector. All road geometry reaches the joint as a small set of periodic piecewise-quintic splines, so one component can represent any circuit whose splines have been fitted beforehand. The road constraint is closed with d'Alembert's principle. The two generalized forces that belong to the joint coordinates are set to zero, so no Lagrange multipliers are needed. The joint is connected to a multibody race car in Dymola and the pair is exported as one FMI~3.0 FMU. The same FMU is then used for time-domain simulation and for CasADi/IPOPT minimum lap-time optimization, with no change to the model in between. This works because the joint keeps the arc-length coordinate as a real state and publishes its time derivative, which is the quantity that lets the problem be rewritten with distance We report results for minimum lap time simulations on the Singapore Formula~1 circuit.
comment: American Modelica & FMI Conference 2026
★ HIDRA: Hierarchical Dual-Routing Attention for Replay-Free Lifelong Imitation Learning NeurIPS 2026
Robotic agents operating in real-world environments must adapt continuously from a stream of multimodal demonstrations, often under constraints that preclude storing or revisiting past data. In this replay-free, single-pass lifelong imitation learning setting, sequential updates can progressively distort the representation space, making expert-based approaches particularly sensitive to routing interference, causing expert selection to degrade over time. We propose HIDRA, a structured routing framework that mitigates this issue through a hierarchical dual-attention mechanism, which decouples instruction-level expert retrieval from context-dependent refinement. To further stabilize routing under representation drift, HIDRA introduces a key-level geometric regularization that enforces both alignment within tasks and separation across tasks. The resulting approach enables reliable expert selection while supporting an expandable set of experts for continual adaptation. We evaluate HIDRA on several LIBERO benchmark suites, including more challenging variants with heterogeneous task sequences and paraphrased language instructions. Our approach consistently outperforms both replay-free and replay-based baselines, improving AUC and forward transfer while maintaining low forgetting, with stronger gains under high interference and language variation. The code is available at: https://github.com/yfqi/HIDRA.
comment: This is the authors' preprint, accepted to the NeurIPS 2026 Conference. The final version is available in the NeurIPS 2026 proceedings
★ SG^2SLAM: Spatial Geometry-Informed 3D Gaussian Splatting SLAM for Monocular Outdoor Navigation
Monocular outdoor navigation requires robust camera motion estimation and a dense, navigable map, where 3D Gaussian Splatting SLAM serves this task through an explicit differentiable parameterization and closed-form projection. However, typically, the geometry is recovered by associating 2D pixels with 3D points, so a modular SLAM system absorbs tracking error into the map, leading to cascading degradation. To address these gaps, we present SG^2SLAM, which couples streaming spatial geometry with a ray-conditioned 3DGS kernel. Streaming spatial geometry replaces the per- frame cross-dimensional correspondence with a chunk-level metric prior that supervises tracking and mapping from one source. A ray-conditioned kernel extends each primitive with a viewing-direction subspace that senses the monocular camera viewpoints. The joint optimization over spatial geometry and poses ensures a globally consistent map and trajectory outcome. Experiments on the KITTI and Waymo Open datasets show that our SG^2SLAM surpasses the per-frame association baselines in the monocular outdoor navigation task. Code will be made publicly available upon acceptance.
comment: 9 pages
★ Bridging Exploration and Loss of Plasticity in On-Policy Reinforcement Learning
Deep reinforcement learning (RL) algorithms have been observed to lose the ability to adaptively learn from new data, a phenomenon termed Loss of Plasticity (LoP). While many metrics have been shown to correlate with it, the causal mechanisms and the geometric conditions from which it can arise have not been clearly identified. We analyze LoP in Proximal Policy Optimization with isotropic Gaussian policies, the state-of-the-art formulation for many continuous action spaces in RL. We report multiple metrics which characterize the effect that the policy standard deviation $σ$ has on LoP, and we also suggest related explanations for the effectiveness of widely adopted LoP mitigation techniques. We observe a significant effect of $σ$ on the extent of LoP, and accompany this with both theoretical and experimental supporting arguments. Our experimental validation consists of LoP benchmark evaluations against standard baselines as well as real-world policy deployment on a 7-DoF robotic arm.
★ Compatibility and future-proof testing for multi-agent systems
Independently developed autonomous agents, such as autonomous vehicles, must guarantee joint compatibility, such as safety and task-completion requirements, when they operate in a shared environment. Existing methods require a centrally designed specification, or interaction mechanism before any agent is deployed. However, such centralized design is unrealistic when agent designs are confidential to their developers, and it is not flexible enough to accommodate future agent designs. We study a bi-level structure in which a regulator coordinates developers by publishing seed agents, and each developer independently tests its agent to ensure that it is compatible with every seed-compatible agent, a property we call future-proofness. Under this structure, joint compatibility is guaranteed for any agents that are designed and tested independently, without any coordination with each other. In this work, we first formulate compatibility for general multi-agent systems and show why it is not trivially implied by seed-compatibility. We then characterize the future-proof agents as the closure of the seed and show that this closure is monotone in the seed. Finally, we study a class of executable tests that involve interaction with the seed alone, and we ask to what extent such tests can distinguish the future-proof agents.
★ ASPEN: Adaptive Semantic Reasoning via Command Persistence Prediction for Efficient Navigation VLAs
Navigation vision-language-action (VLA) policies repeatedly invoke costly semantic reasoning even when the next command remains unchanged. Command reuse can reduce computation but risks missing instruction-dependent turns or stops. We introduce ASPEN, an event-triggered scheduler based on command persistence prediction: estimating whether the frozen base policy would reissue its previous command. A lightweight multimodal predictor combines temporal observations, language instructions, execution history, and cached VLA confidence. At eligible command boundaries, sufficient predicted persistence permits command reuse. Otherwise, ASPEN triggers fresh semantic reasoning from the VLA. Evaluated with frozen NaVILA on cross-dataset RxR-CE val-unseen, ASPEN reduces VLA calls by 9.5% and estimated inference time by 8.1%, including scheduling overhead. It maintains comparable navigation performance and preserves identical baseline action sequences on 92.1% of episodes.
★ Learning Multi-Phase Robot Loco-Manipulation via Phase-Conditioned Policy Optimization
Complex whole-body loco-manipulation requires executing diverse maneuvers, such as crouching, grasping, and locomotion, within a single long-horizon task. While performing simple pick-and-place behaviors is well-established, scaling to complex, long-horizon tasks remains difficult. Currently, this challenge is addressed by relying on human motion data and correspondingly humanoid platforms, which limits platform independence and introduces substantial data processing overhead. Instead, we extend proximal policy optimization (PPO) to train a single policy for multi-phase tasks across diverse robotic hardware, leveraging prior task structure to shape behavior. Our method adapts PPO using a phase-conditioned critic, phase-gated reward functions, and a multi-phase reset curriculum to guide exploration. To support varied hardware, we employ a decoupled hierarchical control architecture that separates lower-level balancing dynamics from high-level phase execution. We demonstrate multi-stage loco-manipulation skills on hardware on both the Unitree G1 and a Boston Dynamics Spot equipped with a manipulator arm, and validate our design choices through ablations. Our approach enables the acquisition of complex loco-manipulation behaviors purely through environment interaction while directly incorporating high-level task structure into policy learning.
★ cuArms: GPU-Accelerated Conflict-Based Planning for Multi-Arm Manipulation
Multi-arm manipulation has the potential to dramatically improve the throughput, efficiency, and dexterity of many manipulation tasks. Recent work coordinates multiple robot arms in a shared workcell by allocating and scheduling tasks for greater parallelism, jointly planning multi-arm motions, or directly learning multi-arm policies. So far, these approaches are either too slow, rely on domain-specific assumptions, or scale poorly with the number of robots and task horizon. Thus, we introduce cuArms, a conflict-based planner co-designed with GPU parallelism that efficiently decomposes the search space using reusable multi-modal roadmaps for individual robots and resolves conflicts and deadlocks through a lazy search. To jointly optimize task assignment, schedules, motions, and manipulation skills, we integrate a high-level scheduler that proposes candidate skeletons for cuArms to ground into coordinated manipulation plans. Experimental results show that our method establishes a new state of the art, with real-time initial solutions and anytime improvements in both motion quality and scheduling, without days of domain-specific training required by learning approaches. Videos of both simulated and real-world industrial tasks are available on https://cuarms.github.io/.
comment: 8 pages, 8 figures
★ Program-Level Adversarial Co-Adaptive Self-Play for Robotic Security
Coding agents can adapt robotic software by revising executable programs, but successful responses to an evolving opponent may not remain effective. We present a framework for program-level adversarial co-adaptive self-play in which coding agents generate attacks and defenses from tested opponent code and mission feedback, without model fine-tuning. Environment-specific interfaces support external programs for autonomous driving and SDK-level defenses for humanoid locomotion. Two eight-generation case studies reveal concrete software responses: a driving defender introduces direct-actuation recovery, while a humanoid defender revises incoming-state validation. These responses provide local task recovery but do not establish sustained robustness. The framework makes such adaptation and limitations observable through executable artifacts and task-level evidence.
comment: 8 pages, 5 figures
★ Closed-Loop Aware Planning via Imitation-Free Speed Scoring for End-to-End Autonomous Driving WACV 2027
End-to-end autonomous driving has predominantly relied on open-loop imitation learning for training, which presents two limitations. First, expert demonstrations show only one behavior per scenario, so the model cannot learn alternative valid behaviors and, in turn, cannot adapt to user-specified preferences at test time. Second, noise or ambiguity in these demonstrations can produce inconsistent, unreliable supervision for the end-to-end models. Driving is inherently multi-modal: multiple different behaviors can be valid responses to the same scene, and this diversity arises largely from speed rather than path. We introduce a planning paradigm that learns the path through imitation while scoring a set of candidate speeds across multiple criteria with a learned function supervised by physically grounded safety constraints. We realize this paradigm as Closed-Loop Aware Planning for Autonomous Driving (CLAP-AD), a plug-in planning head that integrates with existing end-to-end driving models and, to our knowledge, is the first to enable explicit test-time control over their speed selection without speed-conditioned training. On Bench2Drive-Speed, CLAP-AD sets a new state of the art Speed-Adherence Score of 82.53, topping TFv6, the strongest baseline on this metric, by 12.9 points while retaining strong closed-loop driving performance. On Bench2Drive, it improves driving score by +28.5 and +32.7 over UniAD and VAD, and on NAVSIM v2, built on real-world driving data, EPDMS by +6.1 over its backbone. Project page: https://clap-ad.github.io
comment: Accepted to WACV 2027. 17 pages (8-page main paper, references, and supplementary material), 5 figures, 7 tables. Project page: https://clap-ad.github.io. \c{opyright} 2027 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media
★ Rollback: Experience Backtracking in World Models for Policy Improvement
Action-conditioned world models offer a scalable alternative to costly real-world interaction for improving robot policies. However, existing world-model-based methods largely treat world models as forward-only surrogates of the physical environment, leaving untapped a capability no physical environment affords: rewinding to past decision points to explore counterfactual futures. We introduce Rollback, a policy improvement paradigm that turns world models from passive simulators into counterfactual engines: it rolls back to critical decision points, branches into alternative actions, and contrasts their outcomes to attribute success to individual decisions. This concentrates imagination where it matters most and distills counterfactual outcomes into fine-grained, action-level supervision without a separately learned value model. We build a scalable training system that unifies distributed world-model search with asynchronous real-robot execution, forming a self-reinforcing cycle: real-world experience continually sharpens the world model, and a sharper world model in turn drives further policy improvement. We term the world-model side of this cycle test-time physics grounding, a capability that hand-built physics simulators lack: while the world model serves as the training environment, real transitions from the policy it trains correct its dynamics through a lightweight LoRA pipeline. Experiments on three challenging real-world tasks show that Rollback improves success rates over the strongest baseline by an absolute 10.67%, 11.11%, and 11.85% on block extraction, ball tossing, and tabletop curling, respectively, while achieving more stable improvement throughout training. Although initialized from teleoperated demonstrations far below human level, Rollback surpasses the average success rate of human players on both ball tossing and tabletop curling within 8 hours of training.
★ E0.5: From Reconstructive to Future-Predictive 3D Foundations for Robotic World-Action Modeling
World-action models (WAMs) offer a promising framework for embodied intelligence, yet mainstream approaches remain centered on video foundation models or vision-language models. Even when geometric features extracted by 3D models such as VGGT are incorporated, they are typically passed into a video model or VLM that serves as the primary backbone, leaving 3D geometry as an auxiliary input or prediction target rather than the native foundation for world modeling. We present E0.5, a geometry-native world-action model built directly on 3D foundation models. E0.5 introduces two key innovations. First, it adopts a 3D foundation model as the primary backbone, making 3D geometry the native representation for both world modeling and action prediction. Second, it explicitly predicts future 3D states together with actions, enabling the model to capture geometric world dynamics beyond action-only modeling or future video prediction. Experiments on RoboTwin 2.0 first demonstrate the effectiveness of E0.5 without embodied-data pretraining, achieving success rates of 89.90\% and 50.09\% under clean and randomized settings, respectively. Building on these results, we further pretrain E0.5 on more than 160k robot trajectories from AgiBotWorld-Beta, jointly learning robot actions and future geometry within the pretrained 3D foundation. This embodied pretraining improves mean real-world manipulation success rates by approximately 15 and 7 percentage points on AgileX Piper and Zerith H1, respectively. Together, these results demonstrate the potential of geometry-native foundations to acquire robot interaction knowledge and transfer it to physical manipulation. E0.5 embodies this geometry-native approach as a member of the VWA family of world-action models.
♻ ★ One-shot Human Video-Guided Adaptation for Vision-Language-Action Models
Vision-language-action (VLA) models have recently achieved remarkable progress by leveraging large-scale robot datasets and foundation models for general-purpose robot manipulation. However, their ability to rapidly adapt to novel manipulation tasks beyond the training distribution remains limited. Existing approaches often require task-specific robot demonstrations and additional training, which are labor- and compute-intensive, limiting the scalability of VLA adaptation to open-world scenarios. To reduce this dependence on target-task robot data, we propose CapPro, a framework that enables a VLA policy to adapt to previously unseen manipulation tasks using only a single human demonstration video. CapPro consists of two components. First, a phase-aware human video representation module constructs a compact video representation through learned visual token selection and learns task-relevant query representations under phase-caption supervision. Second, a video-conditioned adaptation module jointly incorporates the compact video and query-derived representations into the action-conditioning context of the pretrained VLA policy. % This design introduces task-specific guidance while largely preserving the original VLA action-generation backbone. In this way, the human demonstration provides task-specific execution guidance rather than an embodiment-aligned trajectory to imitate, without requiring target-task robot demonstrations or target-task fine-tuning. Experiments across source and unseen target tasks show that CapPro improves average performance across unseen target groups while maintaining comparable performance on the source tasks. These results demonstrate the effectiveness of human-video guidance for cross-task VLA adaptation.
♻ ★ Reasoning as a Distribution: Generative Plan Latents for Vision-Language-Action Models
Vision-Language-Action (VLA) policies increasingly reason before acting, but existing reasoning models compress this intermediate decision into a single representation. When several plans are valid, this commitment hides alternatives that could otherwise be selected or revised. We introduce the Generative Plan Latent (GPL), which represents the reasoning state as a low-dimensional conditional distribution over plans, such as which object, grasp, or route to use. This makes the reasoning state addressable: without retraining the VLA, plans can be sampled (coverage), selected on request (steering), and reweighted by deployment outcomes (outcome adaptation). We test all three on SimplerEnv manipulation with $π_0$, and extend the analysis to RoboCasa, LIBERO, and a maze domain. Steering the plan raises the fraction of executions that follow the request from 0.48 to 0.94, while the same guidance on the native action distribution shows no reliable gain. Finally, we show that an explicit plan is most useful when it carries decision information not already determined by the observation.
♻ ★ Illumination-Aware Active Perception for Spacecraft Inspection
Harsh and rapidly varying orbital illumination introduces failure modes during spacecraft inspection. Existing planners rely on line-of-sight inspection metrics which perform occlusion checks against a known target and conflate geometric visibility with task-relevant information. These metrics overlook illumination effects including eclipse, self-shadowing, and specular reflection from spacecraft materials and tend to declare components as inspected despite the corresponding images lacking sufficient information. We propose an inspection-planning framework that replaces geometric visibility with a learned measure of camera information derived from a classifier variational autoencoder (VAE) that operates directly on the captured image. The encoder model is trained to classify spacecraft surface components from rendered patches, and its total latent variance is used to learn the varying illumination of said components, replacing binary visibility checks with a continuous, image-grounded quantity. We evaluate the approach by implementing a Guidance, Navigation, and Control architecture in a photorealistic physics simulator, and vary lighting and viewing geometries with multiple orbits. We show that, although all approaches successfully maximize their own objective, our approach results in better illumination: a mean brightness of $0.57$, with $81.6\%$ of observations above $0.25$, compared with $0.02$ ($3.0\%$) for geometric planning with an illumination band and $0.04$ ($6.0\%$) for purely geometric planning.
♻ ★ LLM-Guided Future Hypotheses for Horizon-Aware Exploration in Multi-Step Robot Manipulation
Multi-step robot manipulation requires acting under uncertainty about how the scene will evolve, making exploration and policy adaptation challenging. We study whether short-horizon, task-consistent future videos can provide useful structured priors for control and reinforcement-learning fine-tuning. We formalize this idea through Future-Experience Conditioning (FEC), a simple interface that conditions closed-loop policies on a latent representation of a short future video. In our simulation setup, future clips are generated in three stages, an LLM reasoner operating over a task ontology initialized from the current scene state, a robot-free digital-twin rollout of the intended object motion, and a mask-free video diffusion model that synthesizes a robot-consistent future clip without requiring segmentation at inference. We instantiate this future-conditioning interface primarily with BC and BC+RL, and compare against a future-conditioned Streaming Flow Policy (SFP) baseline on RoboCasa and CALVIN under NoFuture, GTFuture, GenFuture, and WrongFuture. Generated futures improve performance over no-future conditioning, while mismatched futures degrade it, and our BC+RL instantiation achieves the strongest overall results. An average BC+RL learning-curve analysis across 8 CALVIN tasks further shows that GTFuture improves fastest, GenFuture improves earlier and to a higher level than NoFuture, and WrongFuture remains at zero throughout training. These results suggest that short-horizon future videos can serve as useful structured priors for exploration and policy adaptation under imperfect future predictions. https://enact2026.github.io/
♻ ★ CoMo3R-SLAM: Collaborative Monocular Dense SLAM with Learned 3D Reconstruction Priors for Outdoor Multi-Agent Systems
Outdoor robot teams need a shared dense map despite limited overlap, independent reference frames, and uncertain monocular scale. Collaborative dense SLAM systems typically resolve this with depth sensors, which add payload, power, and calibration cost. We present CoMo3R-SLAM, a collaborative monocular dense SLAM system that places learned feed-forward 3D reconstruction priors at the center of the multi-agent problem: their dense pointmaps anchor scale across agents and supply correspondences strong enough to verify inter-agent links geometrically. Each agent tracks and fuses its own keyframes from a single RGB stream, while a coordinator retrieves cross-agent keyframes over the prior's encoder features, verifies them by bidirectional dense pointmap matching, synchronizes the independent similarity gauges in closed form, and refines every keyframe in one unified multi-agent sim(3) graph. Finally, a pose-depth alternation over geometry-aware segments lets inter-agent observations constrain dense structure as well as trajectories. Requiring neither measured depth nor supplied intrinsics, CoMo3R-SLAM attains the lowest trajectory error on three of four Tanks and Temples scenes, and competitive accuracy on Waymo driving sequences, while running at approximately 6-8 FPS on RTX 3080 Ti. A long-horizon traversal, independently captured day and night streams, and teams of up to four agents further map its operating range.
comment: Code and project website: https://como3r-slam.github.io
♻ ★ PlayPolicy: Dexterity from Supervised Play
How can robots autonomously collect the contact-rich experience needed to learn dexterous manipulation? We present PlayPolicy: Dexterity from supervised play (DSP), a framework in which a robot learns a play policy from minutes of human play through behavior cloning, then deploys it autonomously to collect hours of interaction data to train a world model. During inference, the same play policy then guides model-predictive control (MPC) by proposing candidate action sequences for evaluation under the learned world model. To accelerate inference, we also introduce a method for distilling a closed-loop, goal-conditioned policy through self-play within the learned world model. We evaluate this framework on underactuated pushing and any-to-any in-hand pose reorientation with a multifingered hand, in simulation and real world. In real world experiment, our method outperforms hindsight behavior cloning by 69.6% in success rate and achieves comparable success to reinforcement learning with 490 times less interaction data. These results demonstrate that supervised play can autonomously collect interaction data, guide model-based planning, and enable policy learning without any task-specific demonstrations or sim-to-real transfer. Project page: https://play-policy.github.io/
comment: Project page: https://play-policy.github.io/
♻ ★ Prepare to Turn: Learning Anticipatory Maneuvering for Fast Sequential Quadrupedal Navigation
Fast quadrupedal navigation requires preparing for the next turn while approaching the current goal. Policies trained to reach and remain at a single target can instead produce stop-and-go motion when applied to sequential goals, compromising its agility potential. We introduce PrepTurn, a reinforcement learning framework for anticipatory maneuvering in sequential local navigation. PrepTurn maps the current and next pose goals directly to joint actions, using the upcoming goal to inform how the robot should approaches the current one. Its persistent-credit reward encourages timely completion of the whole sequence by continuing to reward reached goals while moving on and preparing subsequent maneuvers, and an automatic curriculum that gradually increases goal difficulty is critical for learning stable turning behaviors. Together, these components enable the policy to coordinate approach speed and reorientation across goal transitions without prescribed trajectories or arrival times. Simulation and onboard Unitree Go2 experiments demonstrate fast, efficient sequential traversal with high completion and reduced stop-and-go motion. We further show that when the goal-reaching tolerances are more relaxed, the policy learns to exploit the freedom allowed by the task with emergent behaviors, such as turning to the next goal in advance and cutting corners within the permitted regions behaviors, to traverse goal sequences more efficiently.
♻ ★ RoboGene: Diversity-Driven Agentic Task Generation Framework for VLA Pre-training
The pursuit of general-purpose robotic manipulation is hindered by the scarcity of diverse, real-world interaction data. Unlike large-scale web vision-language data, robotic data collection incurs prohibitive physical costs. The key challenge is thus not merely to collect more robot data, but to obtain informative training data that broadens behavioral coverage across objects, scenes, and skills. Yet automatically curating task proposals to maximize the utility of collected data remains under-explored. We introduce RoboGene, an agentic framework for generating diverse, informative manipulation tasks across single-arm, dual-arm, and mobile robots. RoboGene combines (1) diversity-driven sampling for broad task coverage, (2) agentic self-reflection for task-quality assessment, and (3) human-verified feedback memory for execution-grounded validation and continual refinement. Through quantitative analysis and large-scale real-world experiments, we collect 18k trajectories and introduce metrics for task feasibility and diversity. RoboGene significantly outperforms frontier foundation models, including GPT-4o and Gemini 2.5 Pro. VLA models trained on RoboGene-generated data achieve higher success rates and stronger generalization, highlighting the value of high-quality task generation. Project page: https://robogene-boost-vla.github.io/. Datasets: Hugging Face (https://huggingface.co/datasets/X-Humanoid/RoboGene) and ModelScope (https://modelscope.cn/datasets/X-Humanoid/RoboGene_tasks).
comment: Accepted at the 10th Conference on Robot Learning (CoRL 2026). 36 pages
♻ ★ OneDemo: Unified Policy Learning and Data Generation from a Single Demonstration
Learning robot policies from demonstrations is often limited by the cost of collecting diverse, high-quality expert data. While an appealing alternative is to expand a single successful demonstration into a diverse set of successful trajectories, existing augmentation methods typically rely on manually designed transformation rules or complex trajectory planning, requiring substantial task-specific engineering and human effort. We introduce OneDemo, a reinforcement learning framework that learns to edit a single successful demonstration for new task configurations. We formulate demonstration editing as a structured reinforcement learning problem, in which an editor learns, from sparse task-success rewards, both when to transition between object-centric trajectory segments and how to geometrically adapt each segment. The learned editor serves both as a task policy and as a reusable demonstration generator. Across 15 Meta-World tasks, OneDemo reaches a 92.7% success rate within 53.7k environment interactions on average. Across 9 DexMimicGen tasks, it generates successful demonstrations with a 98.4% average success rate, and BC-RNN policies trained on the generated data achieve 99.6% average success at matched dataset scale. These results show that learning to edit a single successful demonstration provides a structured and effective basis for both policy learning and scalable demonstration generation.
♻ ★ ActECC: Action Chunk Temporal Harnessing for Edge-Cloud Collaborative VLA Inference System
VLA models have emerged as the mainstream for embodied intelligence, which decouples reasoning and execution via action chunks. To enable real-time deployment of VLA models, existing studies investigate edge-cloud collaborative inference, which pose two challenges to the action chunk mechanism. (1) Edge-cloud collaboration complicates system latency components, hindering the determination of valid ranges for action chunks. (2) Latency fluctuations in cross-edge update links under edge-cloud scenarios make it difficult to schedule replanning requests. To address these, we analyze the key factors affecting action chunk usable ranges and replanning request timing, revealing phase dependence and edge-cloud latency constraints. Based on these insights, we propose ActECC, which enables Action Chunk Temporal Harnessing through kinematics-guided usable range assessment and delay-aware replanning scheduling. Experiments show that ActECC reduces per-action latency by up to 27.4% and cloud calls by up to 45.1%, while lowering average execution error compared with existing edge-cloud collaborative.
♻ ★ Rethinking Densification in Gaussian SLAM: A Stationary Sampling Perspective
Gaussian Splatting enables high-fidelity differentiable SLAM by representing scenes as anisotropic Gaussian primitives. Most systems rely on residual-driven densification, which couples refinement with representation growth: accuracy saturates at a moderate primitive count, yet memory grows throughout the trajectory, degrading throughput. We introduce TopoGS-SLAM, a stationary-sampling formulation that decouples refinement from capacity through coverage-triggered Gaussian initialization and bounded-topology optimization, turning mapping into a bounded-dimensional problem with stable scaling. On standard indoor benchmarks, TopoGS-SLAM matches or surpasses densification-based pipelines in localization and reconstruction accuracy while substantially improving memory scalability and throughput, showing that iterative topology mutation is not required for accurate Gaussian SLAM in bounded-area static indoor scenes with sufficient multi-view coverage. Additional details and resources are available at this URL: https://breeze1124.github.io/topogs-slam-project-page/
comment: 10 pages, 6 figures
♻ ★ A Shared Control Framework for Mobile Robots with Planning-Level Intention Prediction
In mobile robot shared control, effectively understanding human motion intention is critical for seamless human-robot collaboration. This paper presents a novel shared control framework featuring planning-level intention prediction. A path replanning algorithm is designed to adjust the robot's desired trajectory according to inferred human intentions. To represent future motion intentions, we introduce the concept of an intention domain, which serves as a constraint for path replanning. The intention-domain prediction and path replanning problems are jointly formulated as a Markov Decision Process and solved through deep reinforcement learning. In addition, a Voronoi-based human trajectory generation algorithm is developed, allowing the model to be trained entirely in simulation without human participation or demonstration data. Extensive simulations and real-world user studies demonstrate that the proposed method significantly reduces operator workload and enhances safety, without compromising task efficiency compared with existing assistive teleoperation approaches.
comment: Accepted by IEEE Transactions on Robotics
♻ ★ ArrayTac: A Closed-Loop Piezoelectric Tactile Platform for Continuously Tunable Rendering of Shape, Stiffness, and Friction
Human touch depends on the integration of shape, stiffness, and friction, yet existing tactile displays cannot render these cues together as continuously tunable, high-fidelity signals for intuitive perception. We present ArrayTac, a closed-loop piezoelectric tactile platform that simultaneously renders these three tactile dimensions with continuous tunability on a 4 by 4 actuator array. Each unit integrates a three-stage micro-lever amplifier with end-effector Hall-effect feedback, enabling up to 5 mm displacement, an array refresh rate greater than 500 Hz, and a closed-loop bandwidth of 123 Hz. In psychophysical experiments, naive participants identified three-dimensional shapes and distinguished multiple stiffness and friction levels through touch alone without training. We further demonstrate image-to-touch rendering from an RGB image and remote palpation of a medical-grade breast tumor phantom across a distance of more than 1,000 km, in which all 11 participants correctly identified the number and type of tumors with a sub-centimeter mean localization error. Together, these results establish ArrayTac as a programmable multidimensional tactile rendering platform that bridges digital representations, physical environments, and human tactile perception, enabling intuitive tactile communication between humans and machines.
comment: Project website: https://arraytac.github.io/
♻ ★ LocoManiLab: A Unified Testbed for Data, Policies, Controllers, and Deployment in Humanoid Loco-Manipulation
Humanoid robots must coordinate locomotion and manipulation in everyday environments, yet progress is difficult to assess across differing policies, controllers, and task settings. LocoManiLab provides an integrated testbed spanning data collection and generation, policy training, evaluation, and physical deployment. Its task suite covers 234 atomic and compositional tasks across five household scene categories, supported by 51.5 hours of simulated teleoperation. The evaluation covers twelve policies with a shared controller across task and scene splits and perturbation levels, two policies across five controllers, and studies of demonstration scale and data-source mixtures. Success rates fall on unseen tasks and scenes and generally decline as visual perturbations increase, with differences in generalization and robustness across policies. For data generation, a humanoid motion generation model is fine-tuned for loco-manipulation to study the alignment of its outputs with task execution requirements and its integration into the data generation pipeline. Three simulated loco-manipulation tasks closely match real-world setups, enabling evaluation of three policies in both domains. Similar policy rankings support the reliability of simulation-based evaluation.
♻ ★ JRDB-AVR: An Active Visual Reasoning Benchmark for Embodied Agents in Real-World Environments NeurIPS 2026
In complex embodied visual reasoning scenarios, an agent often has only a limited field of view, and the evidence needed to answer a question may be distributed across time, viewpoint, and interacting objects. A model may therefore give a plausible answer without ever observing the relevant object, time, or view that supports it. Current visual reasoning benchmarks largely evaluate passive observations and final answers, overlooking settings that require active reasoning and evidence acquisition. We introduce JRDB-AVR, a benchmark derived from existing real-world JRDB robotics data through a structured question-generation engine that turns this gap into an explicit evaluation: an embodied agentic system receives a visual reasoning question, requests bounded observations by timestamp and viewing angle, and is evaluated on both the final answer and the grounded visual evidence supporting it. The benchmark contains diverse questions over multiple real-world environments involving temporal search, viewpoint selection, and human-oriented compositional reasoning. We also introduce JRDB-AVR-Agent, a reference active reasoning agentic method that maintains an explicit observation-grounded graph-based world model and answers through solving. Experiments reveal a substantial gap between answer accuracy and evidence accuracy in current baselines, showing that current VLMs can produce unsupported correct answers and that active evidence-aware evaluation is necessary for embodied visual reasoning. Code and benchmark are available at https://github.com/ControlNet/JRDB-AVR.
comment: NeurIPS 2026
♻ ★ FlatVPR: Plug-and-play Geo-linear Residual Adapter for Geometric Rectification of Foundation Model Feature Manifolds
This paper proposes ``FlatVPR,'' a novel geometric rectification paradigm that effectively bridges the trade-off between map lightweightness and localization accuracy in visual place recognition (VPR) by enforcing a feature manifold structure where any descriptor between two adjacent anchors $\mathbf{z}_A$ and $\mathbf{z}_B$ can be accurately reconstructed via linear interpolation $\hat{\mathbf{z}}_{pseudo} = (1-t)\mathbf{z}_A + t\mathbf{z}_B$, where $t \in [0,1]$ denotes the relative position. While state-of-the-art foundation models such as DINOv2-ViT-S/14 provide robust semantic features, their latent manifolds exhibit prominent curvature, projecting uniform linear motion in physical space onto highly non-linear trajectories in the feature space, which hinders reliable reconstruction under sparse anchor conditions. To enable the aforementioned interpolation-based reconstruction, we introduce a residual transformation $\hat{\mathbf{z}} = \mathbf{z} + \text{Res}(\mathbf{z})$ to the raw foundation features $\mathbf{z}$, where $\text{Res}(\cdot)$ represents a learnable adapter. Our method explicitly suppresses manifold curvature using a mathematically grounded Pullback Flatness Loss that minimizes the deviation of intermediate features from the linear segment connecting adjacent anchors, thereby minimizing the intrinsic curvature of the manifold. Through this spatial flattening, map construction is formulated within an Expectation-Maximization (EM) framework, decoupled into a continuous M-step for manifold adaptation and a conceptual E-step for optimal anchor selection guidelines. Experiments on the NCLT dataset demonstrate that the application of our adapter leads to significant performance improvements even under extremely sparse anchor conditions with 100m intervals and extreme seasonal changes.
comment: 25 pages, 7 figures, technical report; v3: Added supplementary materials for downstream tasks and included a supplementary roadmap in Section I
♻ ★ Expert-Play Contouring Control: Faster-than-Demonstration Planning from Slow Expert and Fast Play
Expert demonstrations often specify what a robot should do, but not how fast it can do it. Imitation Learning (IL) inherits demonstration timing, while directly accelerating the learned motion can fail when faster execution changes the robot-object dynamics. We study faster-than-demonstration execution as a dynamics-aware control problem and introduce Expert-Play Contouring Control (EPCC), which combines slow expert demonstrations with fast, non-expert play. Expert demonstrations train a latent trajectory generator whose predictions are reparameterized into a time-independent contour of successful task progression, while play trains a world model (WM) of fast-action outcomes. At deployment, our proposed planner uses the WM to optimize actions that makes maximize progress along the expert-derived contour while penalizing deviation from the intended task evolution. Averaged across three visuomotor manipulation tasks, EPCC achieves a $2.0\times$ the throughput of the IL baseline, including $2.2\times$ that of the throughput of the strongest acceleration baseline on a task with interaction-sensitive object dynamics. Our analysis shows that the gains concentrate where faster execution changes robot-object evolution. Together, our results highlight a simple yet effective principle: demonstrations provide task intent, while play data provides the dynamic coverage needed to execute that intent faster.
♻ ★ No Eyes, No Problem: What Does Touch Contribute to Blind Baoding Ball Manipulation?
Humans manipulate objects within their hands through touch alone with remarkable ease, but for robots, achieving sustained real-world manipulation without vision remains an open challenge. Although reinforcement learning (RL) can generate sophisticated blind manipulation policies in simulation, transferring them to a real dexterous hand remains hindered by continuously changing contacts, motion-induced tactile information drift, and the gap between simulated and physical dynamics. We study this problem through Baoding ball rotation on the Shadow Dexterous Hand Lite, using only proprioception and binary tactile feedback as observations. We achieve successful zero-shot transfer through a combination of a sensor-aligned, coupling-aware embodiment in simulation, tactile calibration, and two novel episodic domain randomisations: slew-rate action limits and STuck-AT Taxels. On the physical hand, the policy achieves sustained blind Baoding rotation, completing $\textbf{225 consecutive $\mathbf{180^\circ}$ rotations (112.5 full rotations) at $\sim0.36 \;\text{rot/s}$}$ in a single continuous trial without any vision or object-state information. Furthermore, we evaluate the contribution of tactile sensing relative to proprioception alone and identify the hand regions where tactile information is most important. This is the strongest reported hardware result for Baoding ball manipulation in both rotation count and speed, irrespective of the sensing modality used. By effectively bridging the sim-to-real gap, our framework sets a new benchmark for sustained tactile-driven dexterous manipulation on physical hardware.
comment: 9 Pages, 8 Figures
♻ ★ AutoWorld: Learning Multi-Agent Traffic Simulation with Self-Supervised World Models
Simulation with realistic traffic agents is essential for validating autonomous driving systems. Existing data-driven simulators learn agent behavior from higher-level abstractions such as 3D bounding boxes and polylines, inferred by upstream perception pipelines. These lossy abstractions discard sensory context that directly shapes agent behavior, limiting the distributional realism that simulation aims to reproduce. To address this limitation, we propose AutoWorld, a traffic simulation framework that grounds agent behavior in raw sensor observations through a self-supervised world model trained on LiDAR occupancy data. Given world model samples, AutoWorld constructs a coarse-to-fine predictive scene context as input to a multi-agent motion generation model. Furthermore, we designed a motion-aware latent supervision objective that enriches AutoWorld's latent representation of scene dynamics. To better exploit this latent space during inference, AutoWorld employs a cascaded Determinantal Point Process framework to guide diversity-aware sampling across both the world model and motion model. Experiments on nuPlan and Waymo demonstrate that AutoWorld improves generation realism, with larger gains in partially observed scenarios where trajectory abstractions are most limited, reducing collision rates by 36%. AutoWorld also remains physically plausible under planner-controlled ego behavior and supports controllable, diverse safety-critical scenario generation.
♻ ★ RigidDriveGS: Object-Centric Vehicle Reconstruction from Fixed Multi-Camera Drive-Through Captures WACV 2027
High-fidelity 3D reconstruction of vehicle exteriors improves buyer confidence in online automotive marketplaces, but fixed multi-camera drive-through capture departs from standard Structure-from-Motion (SfM): static cameras observe a moving vehicle amid clutter, rotating wheels, fisheye distortion, and specular reflections. We present RigidDriveGS, an object-centric pipeline combining motion-gated tracking, bifurcated masks that exclude wheels for SfM while optionally retaining them for rendering, mask-guided dense correspondences on raw distorted images, rig-constrained global SfM, and clean object-centric neural rendering with trimap-aware opacity or density supervision and native-alpha white compositing. We evaluate 25 vehicles from 10 dealership installations and six phone-recorded vehicles, each captured from 14 independent handheld positions. Under a staggered per-camera temporal-block holdout, 3DGUT+MCMC achieves 30.42 dB PSNR, 0.90 SSIM, and 0.20 LPIPS, improving PSNR by 7.78 dB over undistorted 3D-GS; independently recorded reference LiDAR scans yield full/body Chamfer-L1 of 0.026/0.013 m and F1@5 cm of 0.703/0.885. These results support clutter-free, object-centric novel-view synthesis without controlled studio infrastructure.
comment: 8 pages, 6 figures, Accepted at IEEE/CVF WACV 2027
♻ ★ Visibility-Aware Mobile Grasping in Dynamic Environments
We consider mobile grasping in unknown and dynamic environments, where a robot must approach and grasp a target object using a camera with a limited field of view. This setting introduces two visibility constraints. Incomplete environmental observations can create collision risks along the whole-body trajectory, while occlusions can hinder object pose refinement during grasping. Previous systems often assume known or static environments and decouple perception from motion, leaving safety unresolved around unobserved obstacles. We present a unified system that enforces both visibility constraints at runtime through three core components. A velocity-aware gaze policy directs perception toward fast-moving regions of the planned swept volume, a hierarchical subgoal policy samples grasp and intermediate configurations that maintain target visibility, and a real-time whole-body planner replans over a receding horizon as the map updates. Across 400 randomized simulation scenarios and real-world trials on a Fetch mobile manipulator, our system achieves success rates of 70.4% in unknown static environments and 57.9% in dynamic environments, outperforming the decoupled navigation- and-manipulation strategy by 16.3 and 9.3 percentage points, respectively, while significantly reducing collisions.
♻ ★ SHRIMP: Iterative Refinement of Robot Task Plans
As collaborative robots have entered domains such as manufacturing, agriculture, and healthcare, programming or adapting robot behavior typically requires robotic expertise that most end users lack. Natural language lowers this barrier. Recent advancements in large language models (LLMs) have made it feasible to translate natural language into robot task plans. However, language-based task specification suffers from semantic ambiguity, and generative models lack transparency for how language instructions become robot actions, making it difficult for users to validate the plan before execution. To address these issues, we introduce SHRIMP, a system that allows users to automatically generate a hierarchical robot primitive plan using natural language and iteratively revise their plan through re-prompting and explicit correction. At each revision, SHRIMP allows users to validate their plan in simulation, and once satisfied, execute it on the physical robot. Through a user study involving participants planning tabletop kitchen tasks (n=35), we validate that SHRIMP improves perceived control and enhances robot transparency. System videos and source code are available at https://wisc-hci.github.io/SHRIMP.
comment: 11 pages, 8 figures, The 39th Annual ACM Symposium on User Interface Software and Technology (UIST '26)
♻ ★ An Efficient Parallel Solver for Linear-Equality-Constrained LQR Problems
We present a parallel solver for finite-horizon linear-quadratic regulator (LQR) problems with arbitrary stagewise linear equality constraints, using linear work and logarithmic depth in the horizon for fixed local dimensions. The solver uses a two-part reduction that separates feasibility from optimization. First, an associative suffix scan computes the states from which each remaining horizon is feasible. Second, we use these feasible-continuation sets to eliminate the constraints independently at each stage, producing an unconstrained LQR problem with smaller or equal, possibly stage-varying dimensions. A parallel unconstrained-LQR solver completes the primal solution. Finally, we recover valid multipliers from the original KKT stationarity equations using a logarithmic-depth contraction-expansion procedure, even when the multipliers are not unique. Sequential and parallel implementations are validated against dense KKT solutions and original residuals. We release a C++/CUDA implementation with JAX bindings under the MIT license.
comment: Accepted for publication in IEEE Control Systems Letters (L-CSS) on October 25th, 2026
♻ ★ Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models NeurIPS 2026
A fine-tuned Vision-Language-Action (VLA) policy can easily place the alphabet soup in the basket on command, just as it saw during fine-tuning. However, if the basket is shifted five centimeters to the left, it drops the soup off the table. We developed an interpretability pipeline to investigate what such a policy is computing. Injecting task A's activations into the action expert while the policy looks at task B's scene makes pi0.5 execute task A's motor trajectory in 99.6\% of episodes ($n{=}1{,}968$). The injected program is bound to absolute workspace coordinates and ignores the visible scene, which provides a mechanistic account of the perturbation brittleness that concurrent benchmarks report. The binding holds on a physical UR10e: directed replay redirects the commanded task in 26/26 rollouts in one direction and 18/26 in the other, with no-injection and random-direction controls at the floor, and once the source object is moved 10--15\,cm the arm grasps and releases at the recorded coordinates in 50/50 trials, picking up the wrong block when one sits there and closing on empty air when none does. Applying the same interventions to six VLAs on LIBERO, MetaWorld, SimplerEnv, and ALOHA over 420{,}000+ rollouts yields three further findings: every architecture encodes the instruction, and four of the five weight it below vision whenever the scene alone identifies the goal; under the standard replacement write-back the SAE pooling that preserves behavior is architecture-specific (pi0.5 per-token, SmolVLA indifferent), and goal selection and motor generation separate into VLM and expert pathways wherever the two are distinguishable. We release \textbf{Action Atlas} (https://action-atlas.com), an interactive platform covering all 520+ trained SAEs, 79 identified concepts, and the scored UR10e rollouts. Code: https://github.com/CWRU-AISM/action-atlas
comment: NeurIPS 2026 Poster, ICLR Multimodal Intelligence Workshop Oral, 44 pages, 14 figures
♻ ★ WorldArena 2.0: Extending Embodied World Model Benchmarking on Modality, Functionality and Platform
World models have emerged as a central paradigm for embodied intelligence, enabling agents to predict action-conditioned future and reason about environmental dynamics. However, existing embodied world model benchmarks are still largely confined to vision-only prediction, offline embodied applications, and simulator-based evaluation, making them insufficient for assessing increasingly comprehensive world models. In this work, we introduce WorldArena 2.0, an expanded benchmark that systematically broadens embodied world model evaluation along three dimensions: modality, functionality, and platform. Along the modality dimension, WorldArena 2.0 extends evaluation from vision-only to visuotactile modalities, enabling assessment of multimodal perception and prediction. Along the functionality dimension, it extends beyond policy evaluation and planning to assess world models as interactive RL environments for policy optimization. Along the platform dimension, it moves beyond simulator-only evaluation to a diverse suite of simulated and real-world robotic settings across multiple embodiments. Under a standardized protocol, WorldArena 2.0 comprehensively evaluates perceptual quality, interactive utility, and cross-platform performance, providing a comprehensive testbed for tracking progress toward embodied world models. The project page including codes and demos is available at: https://world-arena.ai/.
♻ ★ A Novel Path-Tracking Algorithm for Automated Tractor-Trailer Forward and Backward Maneuvers
Fully autonomous tractor-trailer systems are increasingly deployed in logistics, agriculture, and industrial environments, where precise and robust path-tracking capabilities are essential. However, the articulation between the tractor and the trailer introduces additional nonlinearities and significantly complicates lateral and longitudinal control, particularly during reversing maneuvers. This paper introduces a novel path-tracking algorithm specifically designed for articulated vehicles with a single trailer. The proposed method combines a lateral control law applied at the trailer level with a short-horizon predictive adjustment of the tractor steering angle, ensuring stable convergence toward the desired path in both forward and backward motion. The approach is geometry-based and requires no per-vehicle calibration or training. Simulation studies in a high-fidelity physics simulator show steady-state or final lateral errors of $0.02$--$0.05$~m on straight-line, circular, and reverse docking trajectories, which is of the same order of magnitude as, or lower than, the errors reported in the literature. Real-world experiments on a small-scale tractor--trailer robot, with the whole controller running on a low-cost microcontroller, further confirm the feasibility of the approach, with lateral errors below 13~mm during repeated reverse docking maneuvers despite a pose detection latency of about 200~ms.
♻ ★ Learning Action-Conditional and Object-Centric Gaussian Splatting World Models for Rigid Objects
World models enable intelligent agents to predict the consequences of their actions on the environment. In this paper, we propose Multi Rigid Object Gaussian World Model (MRO-GWM), a novel model that learns action-conditional dynamics of rigid objects in 3D. Using object-centric Gaussians, we can represent arbitrary object shapes and multi-object scenes. We develop a novel temporal-spatial transformer architecture that predicts future rigid body motion from a history of object Gaussians and future actions. Objects are represented by Gaussians in an object-centric frame, which allows to describe object motion as a rigid body transformation. Our model is trained on reconstructions from multiple viewpoints, which requires the model to also handle partial observations of objects due to occlusions. We analyze prediction performance of our approach on synthetic datasets composed of typical household objects with multi-object dynamics and interactions by a robot end effector. We also evaluate our model in model-predictive control for non-prehensile manipulation in simulation and for image-based tracking and prediction of object motion in simulated and real-world scenes.
♻ ★ DWPP: Dynamic Window Pure Pursuit Considering Velocity and Acceleration Constraints
Pure pursuit and its variants are widely used for mobile robot path tracking owing to their simplicity and computational efficiency. However, many conventional approaches do not explicitly account for velocity and acceleration constraints, resulting in discrepancies between commanded and actual velocities that result in overshoot and degraded tracking performance. To address this problem, this paper proposes dynamic window pure pursuit (DWPP), which fundamentally reformulates the command velocity computation process to explicitly incorporate velocity and acceleration constraints. Specifically, DWPP formulates command velocity computation in the velocity space (the $v$-$ω$ plane) and selects the command velocity as the point within the dynamic window that is closest to the line $ω= κv$. Experimental results demonstrate that DWPP avoids constraint-violating commands and achieves superior path-tracking accuracy compared with conventional pure pursuit methods. The proposed method has been integrated into the official Nav2 repository and is publicly available (https://github.com/ros-navigation/navigation2).
comment: 51 pages, 18 figures. Accepted for publication in Robotics and Autonomous Systems
♻ ★ HoloBoxing: A Simple Recipe for Humanoid Boxing with 7 Minutes of Single-GPU Training
HoloBoxing presents a simple recipe for learning humanoid boxing by building on a pretrained whole-body controller. The recipe separates reusable motor skills from task decisions: a compact planner requests future body-point targets, while the frozen controller handles joint-level execution. A learned decoder maps 32-dimensional actions to structured target windows, and measured-state imitation initializes the planner before reinforcement learning. Task learning then refines interaction behavior while reusing the decoder and controller, avoiding the need to relearn whole-body control. This combination keeps task optimization focused on a compact upper policy and supplies an initial behavior before reward-driven exploration. Starting from that available controller, the recipe accounts for about seven minutes of additional training, including reused representation fitting and a single-GPU task run. This excludes controller pretraining, data preparation, evaluation, and search. Simulated boxing experiments and component controls evaluate the resulting behavior and the roles of imitation initialization and the target representation.
♻ ★ AffordSim: A Scalable Data Generator and Benchmark for Affordance-Aware Robotic Manipulation
Many everyday robot manipulation skills are affordance-dependent, with success determined by whether the robot contacts the functional object region required by the subsequent action. Current simulation data generators obtain contacts from generic grasp estimators or per-object manual contact annotations, but generic estimators rank stable grasps without task semantics and often select contacts that are misaligned with the downstream action, while manual contact annotations must be rewritten for each new object and task. To solve these challenges, we introduce AffordSim, a scalable data generator and benchmark that integrates open-vocabulary 3D affordance prediction into simulation-based trajectory generation. Given a natural-language task description, AffordSim synthesizes a task-relevant scene, emits affordance queries, grounds them on object surfaces, samples region-conditioned grasps, and selects executable candidates with motion planning. It further randomizes object pose, texture, lighting, image noise, and cross-viewpoint backgrounds for sim-to-real transfer. We instantiate AffordSim as a 50-task benchmark across diverse manipulation skills, five robot embodiments, and 500+ rigid and articulated objects. AffordSim achieves 93% of the trajectory collection success rate of manual contact annotations on affordance-critical tasks and 89% on hard composite tasks. Vision-language-action policies trained on AffordSim data transfer zero-shot to a real Franka FR3, reaching 24% average success.
♻ ★ Distributed Swarm Morphogenesis with Neural Cellular Automata
Swarm morphogenesis requires robots to assemble desired shapes through local interactions. We present a modular framework integrating neural cellular automata (NCA) with existing site assignment and collision-aware control. Within this framework, we introduce a decentralized attachment-site discovery mechanism. Robots already in the formation update and share local NCA states to identify new attachment sites, enabling progressive growth without a predefined list of sites during deployment. To bridge the mismatch between conventional NCA growth and gradual physical assembly, we additionally propose offline sparse-retention fine-tuning that simulates partial attachment, together with a false-positive loss that penalizes invalid site activations. Simulations demonstrate formation of diverse lattice morphologies, scaling to several hundred robots, and substantial improvements in morphology accuracy from the proposed fine-tuning. Experiments with Crazyflie quadrotors validate the integrated execution pipeline using robot-relative observations reconstructed from external tracking.
♻ ★ FlowHOI: Flow-based Semantics-Grounded Generation of Hand-Object Interactions for Dexterous Robot Manipulation NeurIPS 2026
Recent vision-language-action (VLA) models can generate plausible end-effector motions, yet they often fail in long-horizon, contact-rich tasks because the underlying hand-object interaction (HOI) structure is not explicitly represented. An embodiment-agnostic interaction representation that captures this structure would make manipulation behaviors easier to validate and transfer across robots. We propose FlowHOI, a two-stage flow-matching framework that generates semantically grounded, temporally coherent HOI sequences, comprising hand poses, object poses, and hand-object contact states, conditioned on an egocentric observation, a language instruction, and a 3D Gaussian splatting (3DGS) scene reconstruction. We decouple geometry-centric grasping from semantics-centric manipulation, conditioning the latter on compact 3D scene tokens and employing a motion-text alignment loss to semantically ground the generated interactions in both the physical scene layout and the language instruction. To address the scarcity of high-fidelity HOI supervision, we introduce a reconstruction pipeline that recovers aligned hand-object trajectories and meshes from large-scale egocentric videos, yielding an HOI prior for robust generation. Across the GRAB and HOT3D benchmarks, FlowHOI achieves the highest action recognition accuracy and a 1.7$\times$ higher physics simulation success rate than the strongest diffusion-based baseline, while delivering a 40$\times$ inference speedup. We further demonstrate real-robot execution on four dexterous manipulation tasks, illustrating the feasibility of retargeting generated HOI representations to real-robot execution pipelines.
comment: Accepted by NeurIPS 2026. Project Page: https://huajian-zeng.github.io/projects/flowhoi/
♻ ★ CognitiveReality: Robot-Agnostic Semantic Gaussian Mapping with an LLM Agent for Immersive Collaborative VR Teleoperation
A photorealistic 3D view tells a teleoperator where a robot is, but not what the scene contains, how well each object has been observed, or how to turn pointing and speech into robot action. CognitiveReality turns a robot's RGB-D stream into a live, semantically indexed Gaussian-TSDF map shared by an operator in virtual reality and a tool-using language agent. One mapper binary serves any platform through configuration alone: it ingests poses from robot SLAM, joint kinematics, motion capture or an inline visual tracker, bridges localization outages with a shadow tracker and keyframe-anchored PnP, and maintains open-vocabulary instance identities with per-object quality at 2 Hz. Speech and controller rays are grounded against persistent scene objects through validated typed tools and operator-confirmed robot actions. In the controlled agent evaluation, the deployed local Qwen3-VL-8B router reaches 81.24\% tool exact match, while merge-aware replay correctly redirects 101 absorbed object identifiers. On robot data CognitiveReality exceeds a Gaussian-plus-SDF baseline by 2-8 dB; pose error through 5-40 s SLAM outages stays within 1-8 cm. Deployed live on two quadrupeds, the agent executed 26 of 30 navigation requests and 20 of 20 re-observation requests, raising object quality by 2-5 dB.
♻ ★ Video Prediction Policy 2: Predict Better, Act Better
World action models (WAMs) have emerged as an important class of generalist robot policies, aiming to transfer video prediction priors to action learning. However, we find that existing WAMs frequently produce incorrect motion predictions in open-ended environment, leading to erroneous actions. We attribute this limitation to two factors: (1) base video models are not optimized for manipulation, and (2) naively incorporating action components into video models can substantially degrade their generalization capabilities. We introduce Video Prediction Policy 2 (VPP2), a WAM that enables strong zero-shot generalization in both video prediction and action generation. First, we curate a large-scale, diverse dataset of manipulation videos to continue pretraining the base video foundation model. We annotate video clips with detailed captions and perform \textit{event-level} video pretraining to promote generalization across open-ended manipulation tasks. Second, we post-train and distill the video model into a single-step visual planner with fixed prediction horizon. Finally, we introduce action module via a mixture-of-transformers (MoT) architecture to learn implicit inverse dynamics model. Experiments demonstrate three key results: (1) VPP2-14B outperforms Cosmos3-64B by 11.0\% points in video prediction instruction-following success rate on open-ended tasks; (2) VPP2 surpasses the strongest baseline by 18.5\% points in success rate on real-world zero-shot ALOHA manipulation tasks; and (3) following benchmark-specific post-training, VPP2 achieves the highest success rates among evaluated methods on the challenging LIBERO-Pro, LIBERO-OOD, and RoboDojo benchmarks.
comment: Project Page: https://robert-gyj.github.io/video-prediction-policy-2
♻ ★ Learning Loco-Manipulation From SMPC Demonstrations With Sparse Offline-to-Online RL
Integrating locomotion and manipulation is essential for robot autonomy, but scaling standard Reinforcement Learning (RL) to complex tasks is severely bottlenecked by the slow, manual process of dense reward shaping. To bypass this limitation, we leverage Sample-based Model Predictive Control (SMPC) entirely in simulation as an automated, rapidly tunable expert to generate massive offline datasets. Because this data solves the fundamental exploration problem, we can train an off-policy RL agent using purely sparse task rewards, drastically reducing the time required to learn new skills and eliminating the need for manual tuning. Integrating this high-level agent with a low-level dynamic stability controller yields more optimal behaviors that strictly align with true task objectives, ultimately allowing the learned policies to surpass the original optimal control teacher. We validate the robustness of this sim-to-real framework by successfully deploying complex loco-manipulation skills across different morphologies, including an arm-equipped Spot quadruped and a G1 humanoid.
comment: CoRL 2026 Camera-ready version
♻ ★ Crazyflow: An Accurate, GPU-Accelerated, Differentiable Drone Simulator in JAX
High-quality, large-scale synthetic data from simulations is becoming a cornerstone for pushing robots' capabilities. While aerial robotics simulators have evolved to support specialized needs such as fidelity, differentiability, and swarms independently, a unified platform that synthesizes data across all these domains is missing. In this work, we propose Crazyflow, a simulator designed to push the frontier of aerial robotics algorithm development, supporting model-based and data-driven methods, gradient-based and sampling-based approaches, single- and multi-agent systems, and state- and vision-based policy learning. Crazyflow combines high-fidelity dynamics with JAX for speed and differentiability, scaling to millions of environments or thousands of swarms with thousands of drones on single CPUs and GPUs, and beyond on clusters. Real-world experiments show Crazyflow supports both analytical-gradient-based policy learning, achieving high-accuracy tracking without domain randomization, and sampling-based obstacle avoidance at speeds exceeding half a billion steps per second. Breaking the traditional train-then-deploy paradigm, Crazyflow's performance enables in-flight reinforcement learning; we demonstrate this by throwing a physical drone into the air and training a recovery policy from scratch in 0.37 seconds, catching it before it hits the ground. By pushing accuracy, speed, and differentiability simultaneously, Crazyflow serves as an open-source resource for synthetic data generation, with emerging capabilities for large-scale parallelization for online, in-execution learning and optimization, opening the door to novel algorithm development.
comment: Revision
♻ ★ Graphs of Trajectory Bundles for Nonlinear Motion Planning
We present the Graphs of Trajectory Bundles (GTB) method for motion planning of systems with nonlinear and black-box dynamics, combining sampling-based motion planning with trajectory optimization and graph search. Based on a graph-of-convex-sets decomposition of the workspace, GTB iteratively constructs a graph of trajectory bundles that captures the local dynamics of the system at any vertex. By solving a shortest-path problem on this graph, we find maximally dynamically feasible iterates that satisfy the system's dynamics in the limit and any constraints such as collision avoidance or additional safety or specification constraints by construction of the underlying graph. As a result, GTB can handle highly nonlinear, unstable, and even black-box dynamics with complex collision and specification constraints, making it a powerful tool for motion planning of complex systems.
comment: 9 pages, 4 figures
♻ ★ Active Perception of Upper-Limb Functional Mobility through Human-Robot Handovers
The reintegration of people in the workplace after an upper-limb injury is often a long process. Collaborative robots and humanoids operating in shared workspaces must adapt their behaviour to the physical capabilities of their human partners. Achieving this requires robots to autonomously estimate task-relevant human mobility during interaction, without relying on external clinical assessments. In this work, we formulate the estimation of upper-limb functional mobility as an active perception problem. We model task-available upper-limb functional ranges of motion through impairment coefficients that parameterise reductions of healthy joint limits, and integrate this representation into a human model used by the robot during interaction. Based on this formulation, we propose an online active perception pipeline in which the robot actively probes the user through repeated handover requests while progressively updating a Bayesian belief over the impairment coefficients. This enables the robot to continuously refine its internal model of the user's joint-level capabilities through interaction rather than passive observation. The proposed framework was evaluated through simulation studies and a proof-of-concept physical validation with a robotic platform and enacted human impairments, demonstrating that the pipeline can accurately recover the impairment coefficients during real human-robot handovers.
♻ ★ Gradient-Free Neural Hamilton-Jacobi Reachability for Scalable Safety-Critical Control
Hamilton-Jacobi (HJ) reachability provides a principled framework for synthesizing safety certificates and robust controllers for safety-critical robotic systems. However, applying reachability analysis to high-dimensional nonlinear systems remains challenging: classical grid-based solvers suffer from the curse of dimensionality, continuous-time neural solvers require accurate spatial value gradients, and reinforcement-learning-based approaches often suffer from weak boundary anchoring and non-stationary adversarial policy optimization. We propose a discrete-time neural reachability framework for control-disturbance-affine systems that learns backward reachable tubes (BRTs) and backward reach-avoid tubes (BRATs) through Bellman-Isaacs value propagation. Our key idea is to combine equation-driven self-supervision with structured policy learning: rather than computing explicit PDE-gradients, we exploit the bang-bang structure of optimal safety interventions to construct approximate teacher actions from gradient-free value probes, converting adversarial actor learning into supervised policy learning. To stabilize long-horizon value propagation, we leverage the learned actor to train the value function backward from the terminal boundary using a windowed temporal curriculum, where each window is used as the boundary condition for the next window. Across benchmark problems up to 80 dimensions, our method learns accurate reachability value functions while improving stability over existing learning-based solvers. We further demonstrate observation-space scalability on F1-tenth racing with over 16,000-dimensional egocentric inputs. The learned safety filter generalizes zero-shot to unseen tracks and transfers to a physical RC car, achieving real-time robust collision avoidance.
♻ ★ When Does Test-Time Physical Diagnosis Pay?
When a robot faces unfamiliar physical conditions, a common approach is to collect evidence about what changed and adapt. For such diagnosis to improve behavior, six ordered empirical conditions must hold: a meaningful reference, identifiability of the physical condition, use of the acquired evidence, decision value, selection value over a fixed alternative, and safe realization. We test this chain in controlled and public environments. In our controlled environments the frozen policy chooses correctly on every decision, but which evidence it buys is fixed by how the panels assign decisions to evidence strata. On three mechanisms absent from training, the frozen decoder fails its registered transfer gate. On all 96 decisions of the transfer panel, its full-trace input lies far outside its training range, and on such inputs it returns one answer whatever the trace contains. All 15 of its errors fall in the 30 decisions from which cheaper evidence was removed by design, a split hidden by aggregate accuracy. None of three primary public tasks passes its registered qualification gates. Before attributing a failure to how a frozen model uses evidence, evaluation should therefore check whether its deployment inputs lie inside its training range.
comment: 12 pages, 4 figures
♻ ★ Where Small Mistakes Matter: Quantifying State Criticality for Imitation Learning SC
Imitation learning typically treats every demonstration state as equally important, optimizing a behavior-cloning objective uniformly across the dataset. Yet not all states are created equal: in a small number of critical states, even a slight deviation from the expert action can lead to a failure, while in many other states such errors are recoverable and inconsequential. We formalize state criticality in imitation learning as the worst-case degradation in downstream value induced by a bounded perturbation of the expert action at a given state. We then propose SCRIM, a practical State CRIticality Measure that estimates this quantity by training an adversarial disturbance policy and a value critic around a fixed imitation policy. SCRIM assigns each demonstration state a score reflecting how vulnerable the task outcome is to small, policy-relevant action errors. We show that these scores can be used to reweight behavior-cloning losses, improving imitation at consequential states and increasing overall task performance. We further show that SCRIM enables data curation by selecting compact, high-value subsets of demonstrations that preserve performance while substantially reducing the amount of training data. We evaluate SCRIM across various simulated and real-world robotic manipulation tasks, and demonstrate that SCRIM improves policy training, enables efficient data selection, and provides interpretable insights into which moments in a demonstration are most critical for task success. Project website: https://sia-lab-git.github.io/SCRIM/
comment: Accepted to the Conference on Robot Learning (CoRL 2026). 18 pages, 10 figures, 8 tables. Project website: https://sia-lab-git.github.io/SCRIM/
♻ ★ LOPAL: Local Performance-Aware Active Learning from Imperfect Demonstrations
Learning from Demonstration (LfD) enables intuitive robot skill acquisition by allowing robots to learn directly from human task demonstrations. However, current methods often fail to address the fact that due to suboptimal and inconsistent human behavior, the quality of the demonstration can vary within each demonstration. Therefore, we introduce LOPAL (LOcal Performance-aware Active Learning), an active learning approach that leverages this local demonstration quality information. Our approach consists of two synergistic components. First, a local performance-driven LfD method uses a Gaussian Mixture Model (GMM) to encode both the demonstrated trajectories and their associated local quality assessments. This enables the generation of trajectories that outperform the imperfect demonstrations by utilizing complementary local data of high performance. Second, active data acquisition allows to improve beyond the imperfect demonstrations by collecting additional informative samples. In areas missing good data, the user is actively requested to provide corrections through a shared autonomy (SA) mechanism, while the robot autonomously executes the learned behavior. The efficacy of LOPAL was validated in both a simulation and a real-world experiment. The results from a real-world pipe inspection task showed that the proposed approach can achieve up to 27.31 % improvement in task performance while also reducing the effort required to collect the demonstrations.
comment: Accepted for publication in IEEE Robotics and Automation Letters (RAL), 2026. Project page: https://jheiders.github.io/LOPAL/
♻ ★ From Grasps to Dexterity: Large-Scale Grasp Pretraining for Dexterous Manipulation
Large-scale dexterous grasp datasets encode rich priors over hand-object interaction, but their use has largely been confined to grasp generation and pick-and-place manipulation. We study whether such data can instead support functional dexterity in articulated tool use, where a robot must acquire a tool, maintain contact, and operate its functional moving parts. We adapt a hierarchical imitation learning framework that combines high-level hand sub-goal prediction with a low-level goal-conditioned controller. We construct a 355k-trajectory grasp-pretraining dataset from large-scale dexterous grasp annotations and use it to pretrain the low-level controller. The controller is then fine-tuned on downstream task demonstrations. To evaluate this setting, we introduce DexCraft, a simulation benchmark with six articulated tool-use tasks requiring coordinated finger motion. Across simulation and real-world experiments, our approach outperforms end-to-end diffusion policy baselines and hierarchical policies trained from scratch. In the real world, it improves full-task success by 33.3 percentage points over DP3. These results show that grasp datasets can serve not only as resources for grasp synthesis, but also as scalable pretraining data for contact-rich dexterous manipulation. Videos are shown on https://yingyuan0414.github.io/grasp2dexterity/ .
comment: Project page: https://yingyuan0414.github.io/grasp2dexterity/
♻ ★ Semantic World Models
Planning with world models offers a powerful paradigm for robotic control. Conventional approaches train a model to predict future frames conditioned on current frames and actions, which can then be used for planning. However, the objective of predicting future pixels is often at odds with the actual planning objective; strong pixel reconstruction does not always correlate with good planning decisions. This paper posits that instead of reconstructing future frames as pixels, world models only need to predict task-relevant semantic information about the future. For such prediction the paper poses world modeling as a visual question answering problem about semantic information in future frames. This perspective allows world modeling to be approached with the same tools underlying vision language models. Thus, vision-language models can be trained as "semantic" world models through a supervised finetuning process on image-action-text data, enabling planning for decision-making while inheriting many of the generalization and robustness properties from the pretrained vision-language models. In this paper, we introduce this new method and validate it on simulated robotic tasks leading to improvements over typical paradigms of reconstruction-based action-conditional world modeling. Website available at https://weirdlabuw.github.io/swm.
♻ ★ Legislating World-Model-Based Planning with Legal Reasoning
As robotic systems grow more general, legal norms are needed to integrate them into society. This paper extends the isomorphism problem of aligning legal source texts with their encodings, and measures two key challenges to robot normative control: (1) the grounding isomorphism gap, where perception error grounds false atoms for legal reasoning, and (2) the ontological isomorphism gap, where one legal conclusion admits many faithful translations into planning constraints. The paper introduces a legal planning stack that employs Defeasible Deontic Logic (DDL) to constrain a motion planner. The stack leverages learned world models to plan and to provide legal context, enabling ex ante governance that intervenes before an illegal action is executed. It was deployed on a simulated robot arm pushing a cube across a 3x3 grid. The findings were (1) the legislated agent abided substantially more often than the non-legislated one, and modeling perception uncertainty lifted abidance even further, (2) the legal reasoning ran efficiently at runtime and its verdicts were auditable, and (3) the stack adapted to exogenous signals and endogenous rule changes. Both gaps were measured: (4) world model and probe error corrupted the factual input for the DDL reasoner, and (5) a single law admitted several faithful metric interpretations yielding drastically different abidance. Thus, ex ante legislation functions as intended, and closing these gaps with a standardized mapping from the law to runtime constraints and improved fact grounding from perception will yield robust laws that align robot behavior with society's norms. Project page: https://dylanwaldner-cail.github.io/Legislated-Planner/.
comment: Project page: https://dylanwaldner-cail.github.io/Legislated-Planner/. Accepted at JURIX 2026
♻ ★ Boreas Road Trip: A Multi-Sensor Autonomous Driving Dataset on Challenging Roads
The Boreas Road Trip (Boreas-RT) dataset extends the multi-season Boreas dataset to new and diverse locations that pose challenges for modern autonomous driving algorithms. Boreas-RT comprises 60 sequences collected over 9 real-world routes, totalling 643 km of driving. Each route is traversed multiple times, enabling evaluation in identical environments under varying traffic and, in some cases, weather conditions. The data collection platform includes a 5MP FLIR Blackfly S camera, a 360 degree Navtech RAS6 Doppler-enabled spinning radar, a 128-channel 360 degree Velodyne Alpha Prime lidar, an Aeva Aeries II FMCW Doppler-enabled lidar, a Silicon Sensing DMU41 inertial measurement unit, and a Dynapar wheel encoder. Centimetre-level ground truth is provided via post-processed Applanix POS LV GNSS-INS data. The dataset includes precise extrinsic and intrinsic calibrations, a publicly available development kit, and a live leaderboard for odometry and metric localization. Benchmark results show that many state-of-the-art odometry and localization algorithms overfit to simple driving environments and degrade significantly on the more challenging Boreas-RT routes. Boreas-RT provides a unified dataset for evaluating multi-modal algorithms across diverse road conditions. The dataset, leaderboard, and development kit are available at www.boreas.utias.utoronto.ca.
comment: 23 pages, 15 figures, 12 tables, submitted to The International Journal of Robotics Research (IJRR)
♻ ★ Semantic-Geometric Task Representations for Bimanual Manipulation from Human Demonstrations to Robot Action Planning
Learning structured task representations from human demonstrations is essential for bimanual manipulation, where action ordering, object involvement, and interaction geometry vary significantly across executions. A key challenge lies in jointly capturing the discrete semantic task structure and the temporal evolution of object-centric geometric relations in a form that supports reasoning over task progression. We introduce a semantic-geometric graph-based task representation that jointly encodes object identities, inter-object semantic relations, and per-object motion histories, via a Message Passing Neural Network (MPNN) encoder and a Transformer-based decoder. The encoder operates solely on the temporal scene graph, producing structured representations decoupled from action labels. The decoder then conditions on action-context to forecast future actions, associated objects, and object motions. This decoupling learns task-agnostic representations, enabling encoder reuse across embodiments through decoder-only finetuning on a small robot dataset. Across eleven bimanual tasks from two datasets, we find that the benefit of structured semantic--geometric representations over simpler sequence-based models grows with task variability in action ordering and object involvement. At deployment, a planner couples the action and motion predictions with learned Probabilistic Movement Primitives, achieving full task success on two real-robot bimanual tasks and outperforming graph ablations, Transformer, decoder-only, and finetuned vision-language model baselines.
comment: 9 pages, 7 figures, preprint version of paper accepted at IEEE RA-L 2026
♻ ★ RePO-VLA: Recovery-Driven Policy Optimization for Vision-Language-Action Models NeurIPS 2026
Vision-Language-Action (VLA) models remain brittle in long-horizon, contact-rich manipulation because success-only imitation provides little supervision for execution drift, while failed rollouts are often discarded. We introduce RePO-VLA, a recovery-driven policy optimization framework that assigns distinct roles to success, recovery, and failure trajectories. RePO-VLA first applies Recovery-Aware Initialization (RAI), slicing each corrective suffix into a new episode so corrective actions depend on the current adverse state rather than the preceding failure. It then learns a Progress-Aware Semantic Value Function (PAS-VF), aligning spatiotemporal trajectory features with instructions and successful references. The resulting labels salvage useful failure prefixes via reliability decay, while low-value labels mark drift and terminal breakdowns, teaching differences among nominal, failed, and corrective actions. The data engine turns adverse states into planner-generated or human-collected corrective rollouts, teaching recovery to the success manifold. Value-Conditioned Refinement (VCR) trains the policy to prefer high-progress actions. At deployment, a fixed high-value token biases actions toward the learned success manifold without online failure detectors or heuristic retries. We introduce the Failure-Recovery Benchmark (FRBench), with standardized error injection and recovery-focused evaluation. Across simulated and real-world bimanual tasks, RePO-VLA improves robustness, raising adversarial success from 20% to 75% on average and up to 80% in scaled real-world trials. Code and video results are available at https://1018weijia.github.io/RePO-VLA-Web/.
comment: Accepted at NeurIPS 2026. Weijia Liufu and Xiaoyu Guo contributed equally. Xiaoyu Guo is the project leader. Liang Lin and Xiaodan Liang are co-corresponding authors. Updated to the camera-ready version
♻ ★ MO-Playground: Massively Parallelized Multi-Objective Reinforcement Learning for Robotics
Multi-objective reinforcement learning (MORL) is a powerful tool to learn Pareto-optimal policy families across conflicting objectives. However, unlike traditional RL algorithms, existing MORL algorithms do not effectively leverage large-scale parallelization to concurrently simulate thousands of environments, thus facing vastly increased computation time. Ultimately, this has limited the application of MORL towards complex multi-objective robotics problems. To address these challenges, we present 1) MORLAX, a new GPU-native, fast MORL algorithm, and 2) MO-Playground, a pip-installable playground of GPU-accelerated multi-objective environments. Together, MORLAX and MO-Playground approximate Pareto sets within minutes, offering 26-271x speed-ups compared to legacy CPU-based approaches and up to 19x speed-ups over prior GPU-based approaches whilst learning superior Pareto front hypervolumes. We demonstrate MO-Playground's versatility by implementing a custom BRUCE humanoid robot environment and learning Pareto-optimal locomotion policies across 6 practical objectives in simulation, such as smoothness, efficiency and arm swinging. Our toolbox is available at https://dynamicmobility.github.io/moplayground/.
comment: 8 pages, 4 figures, 2 tables
Multiagent Systems
★ Guarded Agent-in-the-Loop Residual Adaptation for Evolving Wind Power Regimes
Wind power forecasting models are often deployed as fixed temporal backbones, but the errors they make can vary across meteorological regimes such as cold waves, heat-and-calm periods, productive typhoon, and destructive storms. We present a guarded Agent-in-the-loop residual adaptation framework that leaves the temporal backbone unchanged and adapts only a small set of scene-conditioned residual experts. An Agent layer interprets external scene information, identifies the current regime, selects residual experts, and applies a correction only when recent observed errors support it. New experts enter the pool after they outperform their initializations on a validation set, keeping model growth under explicit control. Backbone-only and Agent-enhanced forecasts are compared at lead times of 4, 8, 12, 24, and 48 across nine time-series backbones. Across these runs, the guarded framework lowers both MAE and RMSE for all 45 backbone-horizon combinations, with average MAE reductions ranging from 11.21% to 20.26% across backbones. Online-selection baselines, component ablations, and routing analyses identify the roles of residual experts, scene-aware routing, and the online acceptance rule. Candidate-scene runs evaluate the promotion path under controlled conditions; long-term unattended deployment was not evaluated.
★ ClinVerse: A Realistic Multi-Agent Diagnostic Environment for Training and Evaluating Medical Agents
Disease diagnosis is a key application of large language model (LLM) agents in healthcare. However, existing medical agents are primarily trained and evaluated with static question-answering data, providing limited opportunities to learn active information acquisition, examination decision-making, and dynamic clinical interaction. To address this limitation, we present ClinVerse, an interactive, real-data-grounded multi-agent environment designed for training and evaluating medical diagnostic agents. Built on ClinPatient, a scalable dataset comprising 150k diagnostic cases derived from real-world electronic health records, ClinVerse supports coordinated interactions among six specialized clinical roles. Specifically, a doctor agent starts from an initial patient complaint, progressively gathers clinical evidence through patient interviews and examination requests, and ultimately produces a diagnosis. For each interaction trajectory, a rule-based scoring framework quantifies information acquisition, examination ordering, and diagnostic accuracy. These trajectory-level scores serve as reward signals for agent training while also enabling scalable evaluation. In experiments, we evaluate various general-purpose and medical-domain models and show that nine self-evolution methods across three categories consistently improve agents' diagnostic capabilities within ClinVerse. We further analyze the interplay among model capability, environment feedback, and self-evolution methods.
★ Collective Emergent Alignment: Measuring and Amplifying Interaction-Induced Reasoning Alignment in Multi-Agent LLMs
Multi-agent large language model (LLM) systems can exhibit emergent behaviors through interaction, yet whether a similar phenomenon exists for reasoning alignment remains largely unexplored. We study whether agent interaction produces additional reference-grounded reasoning alignment beyond matched individual self-refinement, and introduce REMAP (Reference-Entailment-Matrix for Agentic Process), a framework that quantifies this effect through Semantic Coverage and interaction-specific alignment gains. Across diverse model families and two multi-hop reasoning benchmarks, we observe broadly positive interaction gains, particularly on reasoning content that depends on integrating evidence distributed across agents. These gains are consistently associated with independently measured peer-information recovery, while further analysis reveals composition-dependent heterogeneity effects and asymmetric information transfer across agent capability levels. Motivated by these findings, we further introduce a fact-level multi-agent communication framework that replaces full reasoning-trace exchange with compact, gap-aware, and selectively routed information. Our experiments show that compact fact exchange improves reasoning alignment and task performance while reducing communication overhead, whereas more selective communication exposes a trade-off between acquiring distributed information, retaining local content, and communication efficiency. Overall, our results show that collective emergent alignment can be quantified, linked to information-transfer patterns, and used to guide more effective multi-agent communication.
comment: Preprint - Under Review
★ SlowLab: Benchmarking Autonomous AI Agents on a Year of Experiments They Cannot Undo
AI agents are increasingly proposed as autonomous experimenters, yet the benchmarks that test them mostly offer cheap experiments that answer at once. Experiments in physical facilities are different: they take months, occupy scarce equipment and cannot be undone. We present SlowLab, a benchmark in which an agent runs a one-year campaign in four simulated greenhouse compartments on a site whose crop, facility and prices are hidden. The agent decides when to plant which climate policy, whether to stop a crop and which measurements to read while crops grow, and at the end recommends one policy, which a private evaluator scores in weather the agent never saw. In a preregistered study on 48 unseen sites, three low-cost language-model agents ended near a standard grower setting and clearly below a Bayesian-optimisation baseline that starts from that setting. They ran the facility as two synchronous batches, so measurements taken during the season never changed what they planted, and these measurements improved no method's result. Letting other methods re-read the evidence a campaign had collected did not yield significantly better recommendations, which points to where the models start and which experiments they try. Code is released under the MIT license.
comment: 16 pages (9 main text), 7 figures, 5 tables. Code: https://github.com/dingningpei/slowlab-benchmark
★ Preference-Guided Cooperation in Multi-Objective Multi-Agent Reinforcement Learning
Multi-objective multi-agent reinforcement learning (MOMARL) addresses agent coordination and objective trade-offs simultaneously while optimising overall system performance. Existing work tends to treat multi-agent cooperation and multi-objective optimisation as largely independent components of a learning problem, which amounts to an exorbitant complexity except in trivial cases. We study here whether the structure provided by multiple objectives can itself be exploited to facilitate cooperation, rather than being treated as an additional optimisation challenge. We formulate MOMARL at both the team and individual levels and introduce objective-wise counterfactual credit assignment to evaluate each agent's contribution across the team objectives. We propose Meta-Gradient Informed Multi-Objective Multi-Agent Reinforcement Learning (MIRL), with static (MIRL-S) and dynamic (MIRL-D) variants. By incorporating meta-gradient information, MIRL enables agents to learn how to prioritise objectives individually while collectively optimising the system-level objective, allowing the multi-objective structure to directly guide cooperation. Experiments on a multi-objective multiple particle environment show that our methods improve Pareto-front quality over baselines, with the dynamic variant achieving the strongest performance.
comment: 14 pages, 4 figures, 1 table (plus 3 figures, 3 tables, and 2 algorithms in the appendix)
★ Multi-Objective Reinforcement Learning Using Routed Ensembles and Trajectory Attribution NeurIPS 2026
Multi-objective reinforcement learning is a fundamental problem for modern AI, with applications spanning post-training, reasoning models, and robotic control. A central challenge is to learn policies that balance competing objectives, such as helpfulness versus honesty in safety and alignment, or speed versus energy in robotics. The optimal policy can vary sharply with objective-weighting combinations, and the extent of this variation depends crucially on the degree of conflict among objectives, making per-preference policy learning sample-inefficient. In this paper, we introduce objective-conflict coefficients and derive near-optimal regret bounds for online settings using this measure, thereby quantifying the impact of misalignment on learning efficiency. Then, we design routed ensemble policy learning, an algorithm that maintains a small set of basis policies and dynamically routes trajectories to the policy with the highest potential reward. Finally, we consider trajectory selection as another approach to mitigate objective conflict, and design a kernel surrogate trajectory attribution algorithm that estimates how feedback signals from a set of trajectories influence policy learning on other trajectories. The kernel can capture nonlinear interactions such as synergy and antagonism between trajectories. We evaluate our approach across diverse environments and benchmarks: routed ensembles improve average performance by 4.3% over baselines for online environments from Meta-World and MO-Gymnasium, and kernel surrogate attribution improves correlation with leave-one-out analysis over existing baselines by 50%. When applied to trajectory selection, kernel-based attribution yields a 27% improvement in reward over baseline attribution methods.
comment: 30 pages. To appear at NeurIPS 2026 (extended version)
★ EvoRouter: Cost-Efficient Multi-Agent Routing via Insight-Guided On-Policy Distillation
Serving web queries at scale with multi-agent systems requires jointly configuring large language models, collaboration modes, agent counts, interaction rounds, and skills. However, the resulting configuration space grows combinatorially, while sparse execution rewards provide limited fine-grained supervision for interdependent planning decisions. We propose EvoRouter, a self-evolving multi-agent routing framework driven by insight-guided on-policy distillation. A dedicated expert periodically synthesizes performance-cost feedback from configuration trajectories across tasks into reusable conditional routing insights about capability requirements, returns from resource scaling, and interactions among configuration choices. A privileged teacher combines these evolving insights with current execution feedback to provide dense token-level supervision for the router's own planning trajectories through on-policy distillation. Cost-aware reinforcement learning jointly optimizes task performance and computational cost. Together, these training signals internalize execution insights into the routing policy. The updated policy generates new trajectories that support subsequent insight extraction, forming an execution-driven self-improvement loop. We evaluate EvoRouter on 19 in-domain datasets spanning five domains and two out-of-domain benchmarks. EvoRouter achieves an average in-domain accuracy of 68.69%, outperforming the strongest routing baseline by 7.23 percentage points while reducing inference compute cost by 50.95%. Our code is open-sourced at https://github.com/CenYangtze/EvoRouter.
comment: 13 pages, 12 figures, 4 tables
★ GEAR: Feature-Based Configuration and Code Generation for Agentic Systems
Engineering multi-agent systems (MAS) requires choosing agents, orchestration strategies, communication mechanisms, reasoning approaches, and memory configurations. These choices depend on framework-specific capabilities and abstractions, requiring developers to learn different APIs when exploring alternative implementations. We present GEAR, an open-source tool for configuring MAS and generating framework-specific implementations from a common specification. GEAR uses a feature-based representation to make configuration options and their constraints explicit, enabling users to validate specifications before code generation. Through connectors for ten agentic frameworks, users generate projects from the same specification and can then inspect reports describing framework-specific adaptations. A preliminary assessment of mapping feasibility identifies translation strategies for 271 of 320 evaluated feature-framework pairs (84.69%). GEAR supports developers prototyping agentic systems and researchers exploring MAS configurations across frameworks. GEAR is available as open-source software at https://github.com/brellsanwouo/gear-framework and a video demo is available at https://youtu.be/iwNi-oC2tCw.
★ Simulation-Efficient Double-Flow Neural Posterior Estimation for Agent-Based Models
Neural posterior estimation (NPE) enables Bayesian inference of the posterior parameter in agent-based models (ABMs) with intractable likelihoods, but simulating training data can be computationally expensive. We propose Double-Flow Neural Posterior Estimation (DF-NPE), which combines learned time-series summaries with two conditional normalizing flows for synthetic-data generation and posterior estimation. The framework supplements direct simulations with synthetic training pairs and supports both pooled and ordered training strategies. Experiments in two settings of the Brock-Hommes asset-pricing model show that DF-NPE can reduce direct simulations by 90 percent while achieving lower mean posterior discrepancies and runtime than standard NPE. Performance varies with the allocation of directly simulated and synthetic data and the training strategy. These results establish learned summary augmentation as a practical approach to reducing simulation costs for Bayesian posterior inference of parameters in ABMs.
★ Belief-Conditioned Multi-Agent Actor Critic for Decentralized LLM Collaboration NeurIPS 2026
Recent work has explored optimizing LLM collaboration through Multi-Agent Reinforcement Learning (MARL), but many approaches still rely on predefined workflows, limiting efficiency and scalability. Decentralized LLM collaboration is challenging because agents execute independently with limited communication, share a joint objective, and must infer useful information from communication traces. In this paper, we propose BCMAAC, a Belief-Conditioned Multi-Agent Actor-Critic method that maintains beliefs over contextual information and uses them to facilitate training. Experiments on Minecraft game-playing, web searching and tool-using tasks show that BCMAAC improves task performance and collaborative behavior compared to naive actor-critic baselines. The improvements are particularly pronounced in multi-turn tasks with richer contextual information, where belief modeling provides a clear advantage.
comment: Accepted at the NeurIPS 2026 Workshop PTA: From Pretrained Representations to Acting Agents. 18 pages, 3 figures
★ Knowing When to Escalate: Disagreement Patterns for Uncertainty-Aware Content Moderation CIKM 2026
Agentic content-moderation systems increasingly rely on collective judgments from LLM agents, while aggregation methods commonly treat inter-agent disagreement as a reliability problem to be removed through consensus or voting. This framing is inappropriate for value-laden tasks, where disagreement may reflect irreducible value pluralism - a distinction standard uncertainty decompositions cannot recover, since they operate on output distributions and discard the reasoning behind each judgment. We propose Reasoning Trace Divergence Analysis (RTDA), which embeds model-generated justifications from perspective-differentiated agents and classifies each case by justification similarity and conclusion agreement. The resulting four-state taxonomy isolates convergent disagreement, where agents reasoning similarly but concluding differently, as a candidate signature of value-laden conflict. On 1,000 items from the Measuring Hate Speech corpus, this structural signal predicts human annotator disagreement better than magnitude-only and output-only baselines. We contribute a design principle: condition human escalation on the structure of disagreement, not its magnitude alone, with RTDA serving as a post-processing layer for systems that already generate reasoning traces.
comment: Accepted at CIKM 2026, November 7-11, 2026, Rome, Italy
★ Bayesian Replicator Dynamics and the Inversion of Strict Dominance in Bimatrix Games
Replicator dynamics, the central model of evolutionary game theory, describe populations evolving under selection and agents learning by trial and error. They also admit a third use: continuous-time Bayesian inference, in which an agent planning against an opponent accumulates evidence for each candidate strategy. Here we show that this use divides all two-player games into two classes. With the standard expected-utility payoff, the dynamics can be expressed as Bayesian updating only in games (with any number of actions) where no player has a dominant strategy. The logarithmic payoff of Ay and Erb admits the Bayesian representation in every game. Coordination, anti-coordination, and cyclic games share qualitative dynamics under both payoff definitions, with equilibria displaced and selection thresholds shifted. In contrast, the flows of every game with a dominant strategy are reversed between the payoff definitions, with the strictly dominant strategy eliminated under logarithmic payoffs. Cooperation therefore emerges in the Prisoner's Dilemma, and games of common interest converge to mutual defection. For two-action games the classification is complete and rests on an exact Hamiltonian structure shared by both payoff definitions. The structure gives a behavioural test: cycles in the classic Matching Pennies game have amplitude-independent period (isochronous) under logarithmic payoffs, but not under expected utility. Logarithmic evidence integration thus determines which strategic outcomes dynamical selection can reach, wherever learning approximates Bayesian updating.
comment: 18 pages, 3 figures. The following article has been submitted to Chaos
★ Ontology-guided multi-agent reasoning for explainable differential diagnosis
Effective differential diagnosis depends on well structured and unambiguous patient case descriptions, since incomplete or ambiguous clinical information readily leads to diagnostic errors and inconsistent reasoning. Large Language Models (LLMs) have recently shown promising capabilities in medical question answering and clinical reasoning; however, even when achieving competitive performance on medical benchmarks, they often rely on pattern recognition learned from training data rather than grounded clinical evidence, and can generate plausible yet unsupported diagnostic conclusions. Their reasoning remains largely opaque, making it difficult for clinicians to assess the evidence and rationale behind a given recommendation. To address this gap, we propose an ontology-guided, multiagent differential diagnosis framework that replicates the iterative reasoning process of physicians. The framework grounds diagnostic inference in a structured, expert-curated medical knowledge base, enabling interpretation of symptoms, clinical findings, or disease relationships, while decomposing diagnosis into explicit, auditable reasoning steps. This ontology-based grounding constrains inference to evidence-backed associations, reducing hallucination and improving diagnostic transparency, traceability, and consistency. The system then identifies the most discriminative missing symptoms and generates targeted follow-up questions to reduce uncertainty through a process that mirrors to clinical inquiry. Together, these components improve the reliability, explainability, and diagnostic accuracy of AI-assisted differential diagnosis. The source code is publicly available.
comment: preprint
★ Compatibility and future-proof testing for multi-agent systems
Independently developed autonomous agents, such as autonomous vehicles, must guarantee joint compatibility, such as safety and task-completion requirements, when they operate in a shared environment. Existing methods require a centrally designed specification, or interaction mechanism before any agent is deployed. However, such centralized design is unrealistic when agent designs are confidential to their developers, and it is not flexible enough to accommodate future agent designs. We study a bi-level structure in which a regulator coordinates developers by publishing seed agents, and each developer independently tests its agent to ensure that it is compatible with every seed-compatible agent, a property we call future-proofness. Under this structure, joint compatibility is guaranteed for any agents that are designed and tested independently, without any coordination with each other. In this work, we first formulate compatibility for general multi-agent systems and show why it is not trivially implied by seed-compatibility. We then characterize the future-proof agents as the closure of the seed and show that this closure is monotone in the seed. Finally, we study a class of executable tests that involve interaction with the seed alone, and we ask to what extent such tests can distinguish the future-proof agents.
★ The Inbox as a Coordination Primitive for LLM Agent Societies
Language-model agents increasingly coordinate in large groups, and current agent protocols focus on delivering messages. For these agents, however, the scarce resource is not delivery but reading: every message costs tokens, and the context is finite. Current infrastructure works by push: the sender decides when a message enters the recipient's context, so the recipient reads in arrival order and drops the oldest messages once its context is full. We study the alternative, pull, in which messages wait in an inbox outside the context and the recipient's own rules decide what it reads and keeps. We argue that the inbox is a coordination primitive: each agent sets its rules privately, and those rules decide whether other agents' requests reach it. In simulated societies of up to 16,000 agents, with every decision an LLM judgment, we report three findings. First, push fails as an agent takes part in more coordination at once: an agent leading 20 negotiations with a full memory proposes options already rejected 97% of the time, because it has dropped the oldest replies. Second, an inbox is necessary under heavy, noisy traffic. Where first come, first read misses every request, an inbox that first reads messages carrying a refundable deposit still finds about 60% of the agents a request needs, at every size from 1,000 to 16,000. On a real email network, losses compound: when half the agents adopt such an inbox, forwarded search recovers only 5% of full adoption, against 42% for one-hop search. Third, an inbox comprises four decisions (what to read, what to keep, what a sender pays, and what a reply may say), and the best rule for each depends on the traffic and the agent's role. Multi-agent coordination under load requires an inbox layer designed for the traffic each agent receives.
★ cuArms: GPU-Accelerated Conflict-Based Planning for Multi-Arm Manipulation
Multi-arm manipulation has the potential to dramatically improve the throughput, efficiency, and dexterity of many manipulation tasks. Recent work coordinates multiple robot arms in a shared workcell by allocating and scheduling tasks for greater parallelism, jointly planning multi-arm motions, or directly learning multi-arm policies. So far, these approaches are either too slow, rely on domain-specific assumptions, or scale poorly with the number of robots and task horizon. Thus, we introduce cuArms, a conflict-based planner co-designed with GPU parallelism that efficiently decomposes the search space using reusable multi-modal roadmaps for individual robots and resolves conflicts and deadlocks through a lazy search. To jointly optimize task assignment, schedules, motions, and manipulation skills, we integrate a high-level scheduler that proposes candidate skeletons for cuArms to ground into coordinated manipulation plans. Experimental results show that our method establishes a new state of the art, with real-time initial solutions and anytime improvements in both motion quality and scheduling, without days of domain-specific training required by learning approaches. Videos of both simulated and real-world industrial tasks are available on https://cuarms.github.io/.
comment: 8 pages, 8 figures
★ EREBOTS 4.0: A framework for conversational assistants with A2A agents and MCP tools
In recent years, multi-agent systems have shifted from rule-based, symbolic agents to sub-symbolic agents built on language models. However, several limitations remain unresolved. The new frontier is neuro-symbolic agents, which pair language models with deterministic tools and explicit protocols. Without a common framework to standardize and compose them, each study rebuilds the same infrastructure, and results are hard to replicate. This paper presents EREBOTS 4.0, an open-source, hybrid-AI, multi-agent framework that fills this gap for conversational assistants. Language-model agents plan and route requests, while deterministic tools compute exact results over the Model Context Protocol (MCP). An orchestrator selects one or more specialist agents per turn and calls them over the Agent2Agent protocol (A2A). A new agent, local or remote, is added by listing its URL in a configuration file. The framework also manages identity, user context with consent flags, task states and traces. Its A2A server passed 97 tracked requirements in the official A2A Technology Compatibility Kit, with no failures.
★ Replication or Corroboration? An Axiomatic Framework for Evidential Dependence in Multi-Agent Debate
In multi-agent debates, matching reports can arise for two different reasons: agents may repeat the same evidence (replication), or they may provide separate evidential roots that support the same claim (corroboration). A judge must distinguish these two cases. Counting report occurrences gives extra weight to relays, paraphrases, alias resubmissions, and excerpt splits, while merging all matching content avoids this only by also removing genuine corroboration. We formalize this tension as a minimal judgment game in which acquiring new evidence is costly, but replication is cheap. We show that root replication invariance (RRI) is exactly threshold-uniform replication immunity. Under RRI, replication-only deviations cannot change the verdict, and positive-cost replication is strictly dominated, whereas occurrence-sensitive and content-keyed judges admit profitable deviations. We also establish an information boundary: if a judge's observation interface cannot distinguish a replica from a matched fresh root, no judge can both ignore the replica and respond strictly to the fresh root. To preserve this distinction, we introduce the lineage quotient, which removes wording, identity, occurrence multiplicity, and arbitrary root names while keeping distinct roots separate. On orbit-connected domains, it is the coarsest representation sufficient for every deterministic judge invariant to replication and root-label renaming. We further lift this construction to quantitative bipolar argumentation frameworks (QBAFs), characterize strict fresh-root response, prove its propagation along sign-consistent paths under DF-QuAD, and derive a tight closed-form response modulus. Exhaustive audits over 28,224 direct-star profiles, 2,052 multi-layer DAGs, with matched interventions on 277 AVeriTeC-derived profiles, show the predicted separation between replication and corroboration under a supplied partition.
★ PolicyMediator: Multi-Regional Policy Learning through Mediation Agents with Evolving Skills in OPC Ecosystems
One-person companies (OPCs) are emerging enterprises operated by individual entrepreneurs through digital platforms and LLM-based AI agents. These highly skilled entrepreneurs are characterized by being digital nomads, with limited reliance on local ecosystems. Governments in China use distinct regional support policies to attract mobile OPCs, turning local support into a cross-regional coordination problem. OPCs compare regions to maximize their own utility, but imperfect policy perceptions may cause them to reject otherwise suitable locations. Existing policy-learning methods optimize resources under fixed perceptions, while consensus-formation methods aggregate views without feeding cognitive changes back into policy learning. This disconnect produces mismatched location choices and inefficient regional policy adjustment. We introduce PolicyMediator, a cognition-constrained leader-multi-follower Stackelberg framework with bilateral mediation. The mediator explains verified policies to OPCs to build awareness and trust, and reports their needs and unresolved concerns to governments for policy learning. Its self-evolving skills retain, revise, or prune strategies using verification, allocation, capacity, and objective trajectories rather than mediation success alone. Across six-region simulations in China with five OPC types, five scenarios, and three OPC-simulator backbones, PolicyMediator ranks first in all 15 OPC-backbone--scenario combinations. Over the eight-baseline mean, it improves the aggregate government objective, combining economic performance, employment benefit, and policy-access equality, by 42.4% and average OPC utility by 40.9%.
comment: 12 pages, 5 figures
★ Balancing Mobility Efficiency and Environmental Exposure During Mega Events: A Multi-agent Routing Framework
Mega events can generate concentrated travel demand, intensifying congestion around venues and increasing travelers' environmental exposure. Existing traffic management strategies focus on travel efficiency, with limited attention to exposure risks. Taking mega-event travel as a short-term high-demand scenario, this study develops an exposure-aware multi-agent routing framework integrating road-network conditions, event-induced demand, background traffic, and environmental data within a unified simulation environment. Motorized travelers optimize travel efficiency, whereas micro-mobility travelers consider travel time and dynamic exposure to traffic-related PM2.5 and noise. We compare the framework with shortest-distance, congestion-aware, and exposure-aware routing baselines. Results show that, under high demand, the SARSA-based framework reduces vehicle travel time by 34.59% and cumulative environmental exposure by 58.65%, while micro-mobility travel time remains nearly unchanged. Exposure reduction is jointly driven by vehicle-side congestion mitigation and low-exposure routing. The framework provides quantitative support for vehicle diversion, low-exposure route guidance, and health-oriented traffic management in short-term high-demand scenarios.
♻ ★ Not Every Agent Needs the Strongest Model: Unified Prompt-Model Optimization for Heterogeneous LLM-based Multi-Agent Systems
Large language model (LLM)-based multi-agent systems (MAS) rely on role-specific prompts and execution models to solve complex tasks collaboratively. However, prompt optimization typically keeps model assignments fixed, leaving differences in role requirements and their associated inference costs outside the search. We present UniMASO, a Unified framework for heterogeneous Multi-Agent System Optimization through the joint adaptation of prompts and model assignments. UniMASO evaluates candidate updates through local, downstream, and global feedback, using cost-aware Pareto search to retain useful quality-cost trade-offs. Topology-aware co-evolution then coordinates these updates according to agent roles and dependencies. Cross-level trajectory memory further supplies historical successes and failures to guide subsequent optimization. Experiments on five benchmarks and five execution topologies show that UniMASO achieves competitive task performance at substantially lower deployment cost than the strongest homogeneous configurations. Under centralized execution, it improves the average score from 69.79 to 70.40 while reducing token-weighted deployment cost by 85.2% relative to DeepSeek-V4-Pro with MASPO. These results support joint prompt-model adaptation as an effective approach to efficient multi-agent collaboration. The code is available at https://github.com/Achilles008/UniMASO_main.
♻ ★ OpenAgenet / OAN Yellow Paper: Technical Architecture for Trust-Governed Resource Identity and Discovery
This yellow paper specifies the technical architecture and verification semantics of OpenAgenet (OAN), a protocol-neutral trust layer for heterogeneous Agent resources. The design combines did:oan decentralized identifiers (DIDs), verifiable credentials (VCs), on-chain infrastructure governance, and Root-verified resource packages. It defines the objects and lifecycle transitions that connect subject-control verification, Registrar admission, Root acceptance, authorized Discovery, and signed invocation. Infrastructure authorization requires both active governance state and a valid Root-issued credential; resource discovery additionally requires package integrity, current-version state, and authorized-domain coverage. Local OAN Identity signing material remains separate from public protocol objects. Integration profiles carry resource identity into the Model Context Protocol (MCP) and Agent-to-Agent (A2A) protocol through Skill, Runtime, Sidecar, and Gateway interfaces without redefining native task semantics. The paper distinguishes protocol requirements from implementation choices and sets out security properties, persistence and recovery expectations, and conformance criteria. Together, these rules provide a technical basis for independently implemented registration, discovery, and identity-protected interaction within an OAN trust domain.
♻ ★ OpenAgenet / OAN White Paper: Open Infrastructure for Trusted Agent Interconnection
OpenAgenet (OAN) is an open infrastructure project for trusted Agent interconnection across independently operated networks. It addresses the verification of resource identity, governance state, discovery authorization, and freshness before interaction. OAN combines decentralized identifiers (DIDs), verifiable credentials (VCs), and on-chain infrastructure governance with a routing-aware did:oan registration and discovery model. Registrars verify onboarding evidence; the Root service accepts resource identity versions and publishes verifiable packages; authorized Discovery nodes return signed results within their governed scope. Root also governs the semantic tag tree shared by registration and discovery. Public DID Documents and credentials remain separate from local OAN Identity signing material and from controller authorization. Skill, Runtime, Sidecar, and Gateway integration paths allow existing Agents to adopt this identity model progressively, including through the Model Context Protocol (MCP) and Agent-to-Agent (A2A) protocol, while retaining their native task and workflow semantics. Prototype evaluation and a public pilot support examination of the trust lifecycle and its operational bottlenecks. This white paper presents the architecture, governance model, ecosystem adoption paths, and standardization opportunities for OAN as a shared identity and trust infrastructure for heterogeneous Agent resources.
♻ ★ Distributed Swarm Morphogenesis with Neural Cellular Automata
Swarm morphogenesis requires robots to assemble desired shapes through local interactions. We present a modular framework integrating neural cellular automata (NCA) with existing site assignment and collision-aware control. Within this framework, we introduce a decentralized attachment-site discovery mechanism. Robots already in the formation update and share local NCA states to identify new attachment sites, enabling progressive growth without a predefined list of sites during deployment. To bridge the mismatch between conventional NCA growth and gradual physical assembly, we additionally propose offline sparse-retention fine-tuning that simulates partial attachment, together with a false-positive loss that penalizes invalid site activations. Simulations demonstrate formation of diverse lattice morphologies, scaling to several hundred robots, and substantial improvements in morphology accuracy from the proposed fine-tuning. Experiments with Crazyflie quadrotors validate the integrated execution pipeline using robot-relative observations reconstructed from external tracking.
♻ ★ Adapting Neural Posterior Estimators for Simulation-Based Inference in Agent-Based Models
Agent-based models (ABMs) are widely used in economics and social sciences. They represent heterogeneous personal behaviors and their interactions by using simulations, whereas their likelihoods are often intractable for parameter estimations. Therefore, most traditional statistical inference techniques are hindered. Driven by recent advances in probabilistic machine learning, recent works have applied neural networks to achieve likelihood-free inference in large-scale ABMs. Specifically, Neural Posterior Estimation using normalizing flows provide amortized Bayesian inference from simulations, yet a pre-trained estimator may become inaccurate when the target task changes. In this work, we investigate the practicality of adapting a pre-trained conditional normalizing flow through a unified sequential neural posterior estimation test-time training (SNPE-TTT) framework. The framework includes four adaptation strategies: full-parameter adaptation, low-rank adaptation, and two update spaces identified from target-task gradients. We evaluate all four methods across economic simulation models with both tractable and intractable likelihoods. These benchmarks demonstrate that the sequential neural posterior estimation test-time training framework can adapt pre-trained posterior estimators to related inference tasks using a limited budget of target simulations. Posterior recovery and computational benefits depend on the task and adaptation strategy, with lower target-stage cost in some settings but no consistently superior posterior accuracy.
♻ ★ Crazyflow: An Accurate, GPU-Accelerated, Differentiable Drone Simulator in JAX
High-quality, large-scale synthetic data from simulations is becoming a cornerstone for pushing robots' capabilities. While aerial robotics simulators have evolved to support specialized needs such as fidelity, differentiability, and swarms independently, a unified platform that synthesizes data across all these domains is missing. In this work, we propose Crazyflow, a simulator designed to push the frontier of aerial robotics algorithm development, supporting model-based and data-driven methods, gradient-based and sampling-based approaches, single- and multi-agent systems, and state- and vision-based policy learning. Crazyflow combines high-fidelity dynamics with JAX for speed and differentiability, scaling to millions of environments or thousands of swarms with thousands of drones on single CPUs and GPUs, and beyond on clusters. Real-world experiments show Crazyflow supports both analytical-gradient-based policy learning, achieving high-accuracy tracking without domain randomization, and sampling-based obstacle avoidance at speeds exceeding half a billion steps per second. Breaking the traditional train-then-deploy paradigm, Crazyflow's performance enables in-flight reinforcement learning; we demonstrate this by throwing a physical drone into the air and training a recovery policy from scratch in 0.37 seconds, catching it before it hits the ground. By pushing accuracy, speed, and differentiability simultaneously, Crazyflow serves as an open-source resource for synthetic data generation, with emerging capabilities for large-scale parallelization for online, in-execution learning and optimization, opening the door to novel algorithm development.
comment: Revision
♻ ★ The Theory of Strategic Evolution: Three Principles of Strategic Replicators
Strategic replicators combine evaluated choice and reproductive transmission. An agent chooses among alternatives; the consequences of those choices influence continuation or reproduction; and successors receive organization that shapes later choices. Artificial intelligence makes this combination salient when agents have authority to create, configure, or retire software instances. This paper organizes the general logic around three principles: situated choice, reproductive transmission, and institution-mediated coupling. The principles are constitutive rather than predictions of growth or improvement. A common representation distinguishes carriers, inherited records, decision owners, and the mechanisms that allocate consequences. Actual parent-descendant attribution yields a selection-transmission identity. Separate compatibility results characterize when utility-maximizing choices also maximize a reproductive criterion. Gode and Sunder's zero-intelligence traders reveal the role of institutions: enforced properties can hold across different decision procedures, while aggregate efficiency alone determines neither individual gains nor descent. A worked market-rent-entry model connects institutional profits to actual offspring, with its occurrence-level ancestry bridge checked in Lean. A direct-deployment example then shows evaluated preferences undergoing selection without presupposing that every chooser values reproduction. Formal appendices develop games with endogenous players, deployment-induced tipping, endogenous electorates, and the production of institutions, with models, proofs, and examples. The paper and its supporting formal artifacts are a prototype kernel with a generative function: an instrument for constructing models of strategic replication. The results are formal; their application to deployed populations and their empirical implications remain to be tested.
comment: 59 pages. Rewritten around Three Principles; expanded applications to endogenous players, tipping, and endogenous electorates. Selected finite foundations and ancestry bridge checked in Lean; appendices analytical; executable ecology tested. Source and evidence: https://github.com/selfreferencing/TSE_Paper/tree/three-principles-2026-10-13. Companion to Agentic Capital
♻ ★ Fig: Automated Multiagent Behavior Elicitor
Many social evaluations for multiagent systems study behaviors as they emerge from environments instead of systematically intervening in the environment to elicit such behaviors. We introduce Fig (Finding Interactions in Games), an agentic system that performs targeted elicitation interventions on game environments. It takes a natural language behavior description, decomposes it into sub-behaviors, generates new social games or selects existing ones, and executes single or combined interventions to the games' engine, opponents, instructions, and context. To evaluate model behavior, it combines built-in mechanical checks and synthetic rubric judgments. We validate the interventions, generated games, and evaluation mechanisms via human review. Models attempt the target behavior in 45.2% of episodes on average for valid scenarios generated by Fig for six test behaviors -- cooperation, exploitation, deception, covert coordination, betrayal, and sabotage. For game-behavior pairs with available controls, Fig's interventions increase attempt rates by 10.9% on average compared to matched control games. Fig shows that multiagent environments can serve as programmable test beds for intentional, controllable, and reproducible behavior evaluation. Code is available at https://github.com/fleet-ai/fig.
Systems and Control (EESS)
★ Moving bits instead of watts: Accelerating global AI data-centre deployment at the energy-digital-water-land nexus
Artificial intelligence is driving an unprecedented expansion of data centres. However, this growth is increasingly stalled by grid transmission constraints that trap new data centres in years-long connection queues. Unlike electricity, computation can be moved across the globe through fibre-optic networks, but whether this spatial transfer can bypass constrained grids and accelerate global AI data-centre deployment remains unknown. Here we develop a global framework that places future data-centre capacity in energy-abundant regions and routes computational workloads through fibre-optic networks rather than transmitting electricity. We evaluate the framework through more than 11 million hourly operation simulations at subnational resolution, integrating constraints on energy systems, water, digital connectivity, land and financing risk. Results show that across multiple data-centre development and climate-policy scenarios, leveraging global data transfer allows new data centres to be sited where these conditions are favourable, lowering construction costs by 44-47% and carbon emissions by 38%. By treating the global fibre-optic network as an energy-system asset, it avoids 34-67 GW of high-voltage transmission capacity, comparable in magnitude to Italy's national peak electricity demand. Crucially, this methodology could bring more than 95% of connection-delayed data centres online in 2030, with Canada, the Nordic countries and the Arabian Peninsula emerging as preferred locations for data-centre development. Our findings establish computational mobility as an alternative to energy-system expansion, informing the coordinated global development of energy and artificial-intelligence infrastructure.
★ Joint Dynamic Lane Reversal and Variable Speed Limit Control: A Graphormer-Based Approach
Connected and automated vehicles could allow reassigning lane directions and speed limits on every road frequently, but the benefit hinges on how traffic reroutes, and one-way roads can isolate junctions. We formulate network-wide dynamic lane allocation and directional speed limit control with one-way roads, lane clearing before each reversal, and a navigable graph kept strongly connected at all times. An action-resolution algorithm grounded in graph orientation theory admits a proposal closing a direction only if the network stays strongly connected and otherwise relaxes or reverts it, keeping all junctions mutually reachable within the model, even during clearing. All road controllers share one transformer whose attention, Global Endpoint Relation-Aware Spatio-Temporal Self-Attention (GERASTSA), spans all road-time tokens, biased by the endpoint relations between roads. It is pretrained on action-conditioned traffic forecasting and fine-tuned with per-road proximal policy optimization under a mixed reward that also credits neighboring roads and the network. On a 6 x 6 grid with commuter rush hours, the learned policy makes about two thirds of roads one-way in the peak direction and cuts daily delay by 90% against the initial configuration held fixed and by 85% against a tuned two-way load balancer. Restricted to two-way roads at test time, it gains only about as much as this balancer, whereas its learned speed limits add delay. Pretraining roughly halves the updates needed to surpass the fixed configuration. Of the ablations, only removing the relational bias or the spatial embedding clearly matters, multiplying delay about fivefold.
comment: 21 pages, 10 figures, 3 tables, 1 algorithm
★ Context-dependent Runtime Construction of Safe Regions of Attraction from Design-Time Identification for Multi-Candidate Simplex Architectures
Deploying safety-critical cyber-physical systems requires control strategies that reconcile high performance and adaptability with rigorous safety guarantees. Classical control strategies offer stability proofs but are conservative, while advanced optimization- and learning-based controllers achieve high performance but lack formal verification. Runtime Assurance (RTA) architectures bridge this gap by introducing their own (dis-)advantages. Safety filters use online optimization and introduce chattering, while Simplex permissiveness is bounded by the tightness of the certified Region of Attraction (RoA) of its High-Assurance (HA) controller. Targeting this conservatism, we present a two-stage RoA certification procedure leveraging contraction-based stability to establish a target set for Taylor-model-based reachability analysis. We then treat the closed-loop system under HA control as a switched system, enabling: (a) runtime construction of context-dependent safe RoA and (b) safe switching between HA candidates to optimize recovery and maximize the operating envelope of the uncertified High-Performance (HP) controller. We validate this framework on a robotic path-tracking task using a three-stage evaluation: Ordinary Differential Equation (ODE) simulation for theoretical verification, high-fidelity Gazebo simulation under unmodeled dynamics, and experiments on physical hardware. Results show that the Simplex architecture maintains safety under fault injection and successfully shields a behavioral-cloning-trained HP controller. Crucially, the decision module introduces negligible overhead (0.028 ms average, 4.13 ms worst-case), consuming under 10 % of the control period.
comment: 13 pages, 8 figures, 5 tables
★ Explicit Globally Asymptotically Stable Attitude Control with Reaction Wheel Angular Momentum and Torque Constraints
We present an explicit Lyapunov attitude controller for rigid spacecraft actuated by three orthogonally mounted reaction wheels, parameterized using Modified Rodrigues Parameters (MRPs), which enforces hard bounds on wheel angular momentum and applied torque at every instant without solving any online optimization. A barrier-inspired momentum feedback prevents wheel saturation; a continuous positive scalar uniformly scales the feedback whenever a torque limit would otherwise be breached, preserving Lyapunov convergence. For the regulation problem, global asymptotic stability (GAS) is established via LaSalle's invariance principle, and constraint satisfaction is proved analytically for arbitrary symmetric positive-definite inertia matrices. Replacing the barrier by an error-state counterpart extends the design to tracking, with asymptotic (though no longer global) stability and the same safety guarantees under explicit feasibility conditions on the reference. Simulations confirm convergence and constraint satisfaction, and compare performance against other controllers.
comment: Preprint accepted in CDC 2026 for presentation
★ Quantile-Based Command-Reserve MPC for Current-Command Gimbal Pointing
This paper studies the constrained line-of-sight pointing control problem of a two-axis current-command gimbal subjected to mismatched mechanical disturbances. Friction, torque ripple, current-response lag, model mismatch, and limited drive commands make accurate pointing more challenging yet interesting. A quantile disturbance predictor is proposed to compensate predictable acceleration residuals and describe their remaining uncertainty, with the aid of learning median and outer quantiles from state, command, and reference histories. A tube-inspired model predictive controller is proposed to reserve command headroom according to prediction uncertainty, with the aid of calibrated radii and stage-dependent input margins. The median improves the nominal prediction, while the radius adapts the command reserve without adding optimization variables. A finite-horizon tube inclusion result is established for a specified linear error model and fixed ancillary policy; its conditions are distinguished from the approximations used in hardware. Native command bounds and a final command limiter keep the sent commands within their prescribed range. On a two-axis embedded platform, four controller configurations are compared in seven repeated trials each. The proposed configuration achieves a two-dimensional pointing root-mean-square error of $3.94 \pm 0.10$ mrad with no command-limit samples in the evaluation windows. Its performance supports the combined use of residual prediction and adaptive command margins in the tested operating condition. Solver times and logged loop overruns are reported separately for the nominal 100 Hz implementation.
comment: 10 pages, 4 figures. Supplementary hardware trial data and recomputation scripts included as ancillary files. Submitted to Journal of the Franklin Institute
★ Compositional Approximately Bisimilar Abstractions of Stochastic Interconnected Systems
This paper addresses the challenge of constructing (alternating) bisimilar symbolic models for discrete-time, stochastic, interconnected systems. By leveraging a refined over-approximation of the stochastic reachable sets and the q-th moment input-to-state stability (delta-ISS-Mq) property, we establish an approximate bisimulation relation between stochastic systems and their deterministic (noise-free) counterparts. This approach significantly simplifies the analysis of the approximate (alternating) bisimulation between the global interconnected stochastic system and its symbolic model. We derive formal conditions under which local bisimulation relations are preserved when composing subsystems into a global interconnected system. The engineering relevance of the proposed compositional framework is demonstrated through temperature regulation case studies involving a fixed ten-room building and a circular cascade interconnection scaled from six to fourteen rooms. The results show that, beyond a small number of subsystems, the proposed method achieves substantially faster abstraction computation times than standard monolithic approaches, with the performance gap widening as the number of interconnected subsystems grows and the monolithic approach failing to converge beyond ten rooms.
comment: 31 pages, 7 Figures
★ Physics-Informed and Data-Driven Composite Disturbance Separation and Predictive Control for Airborne Electro-Optical Gimbals with Multiple Constraints
This paper studies the constrained pointing control problem of airborne electro-optical gimbals subjected to composite periodic and aperiodic disturbances. Under carrier vibration, input delay, and actuator constraints, achieving line-of-sight stabilization and tracking is challenging. A disturbance-separated model predictive control (MPC) scheme is proposed to compensate these heterogeneous disturbances according to their physical characteristics. An adaptive periodic observer predicts multi-harmonic vibrations over the MPC horizon using linear-regression frequency estimation. A linear observer estimates the aperiodic disturbance, and a bounded long short-term memory (LSTM) network predicts its change over the input delay. Embedding periodic predictions into the MPC model while applying aperiodic compensation outside the optimizer limits amplification of aperiodic estimation errors and improves pointing accuracy under frequency changes and parameter mismatch. On the stated headroom envelope, input-set tightening provides feasible commands within the prescribed limits, and a regional practical tracking bound is established under explicit input-sequence and predicted-state margin conditions. A UAV-mounted two-axis gimbal platform is assembled for comparative indoor hovering experiments, complemented by simulations under frequency changes and parameter mismatch. In the reported trials, the proposed method achieves yaw and pitch relative-angle root-mean-square errors (RMSEs) of 0.770 degrees and 0.350 degrees, reducing tracking errors by 25.1% and 45.9% compared with linear active disturbance rejection control.
comment: 20 pages, 8 figures. Includes supplementary derivations and proofs. Demonstration video included as an ancillary file
★ Physical Adjoint Learning in Wave Systems with Asymmetric Couplings
Developing self-optimizing physical systems that learn directly from their response to external stimuli offers a promising route toward adaptive and intelligent machines. A key ingredient is the ability to evaluate optimization gradients within the physical hardware, bypassing the need for accurate numerical models. In wave systems, \textit{in-situ} adjoint methods provide this capability by exploiting reciprocity to physically realize the adjoint dynamics. However, this approach generally breaks down in nonreciprocal systems, where the adjoint field cannot be directly accessed through the original device. Here, we develop an \textit{in-situ} adjoint optimization framework that extends physical gradient computation to wave systems with asymmetric couplings through computationally low-cost similarity transformations between the forward and transposed operators. We utilize available degrees of freedom like tailored injected excitations, targeted measurements, and tunable parameters, to enable adjoint field reconstruction through forward experiments on the same physical platform. We demonstrate the versatility of the protocol through broadband transmission and isolation, coherent perfect absorption, programmable wavefront shaping, and spatially selective sensing. Our results provide a route to self-optimization in complex nonreciprocal wave systems through efficient physical gradient-based learning. The number of experiments required by the protocol does not increase with the number of optimization parameters, and the method remains applicable even when parts of the system are completely inaccessible.
comment: Main text: 13 pages, 7 figures; (+) Supplementary Information: 5 pages, 1 figure
★ From AI-assisted to AI-native Industrial Control: Architecture, Agent Design, and Challenges
The integration of AI and industrial control is rapidly transforming industrial automation. However, current approaches remain predominantly AI-assisted, where AI functions are incorporated as auxiliary components without fundamentally altering the underlying architecture of industrial control systems. Consequently, the potential of AI to enable higher levels of autonomy, adaptability, and continuous evolution remains underutilized. This paper proposes an AI-native industrial control paradigm, which reconceptualizes industrial control systems by positioning AI as the intrinsic core rather than an external enhancement. The proposed paradigm is built upon an AI-native infrastructure and a collection of collaborative agentic AIs, each encapsulating a set of advanced skills. Among them, the industrial control agent plays a central role in orchestrating the system lifecycle. Through this paradigm, industrial control shifts from a closed, capability-oriented, human-driven, and passive engineering process to an open, task-oriented, intent-driven, and autonomous solution delivery mechanism. The paper further analyzes the key challenges of the AI-native paradigm from both AI and industrial control perspectives, including issues related to AI model reliability, formal verification, real-time assurance, and heterogeneous resource orchestration. Overall, this work outlines a forward-looking vision for industrial control in the AI era and identifies critical research directions toward autonomous, evolvable, and adaptive control systems in complex and dynamic industrial environments.
★ Compound Environmental Operating Stress and Cyber-Induced Pump Disruption in Water Distribution Networks: Matched Hydraulic Impacts, Service Continuity, and Sustained Recovery
Digitally controlled water distribution systems must maintain service during disruptions, yet restoring water delivery does not necessarily ensure hydraulic recovery. This study evaluates cyber-induced pump outages under hypothetical environmental operating stress using two EPANET benchmark networks (Net1 and Net3) simulated with the Water Network Tool for Resilience. The experimental design comprises 288 simulation records (240 distinct configurations), including 192 verified attack scenarios, with four- and twelve-hour outages, six hydraulic configurations, and four mitigation strategies. All attack scenarios satisfied pump-shutdown verification and pre-attack hydraulic-equivalence criteria. Twelve-hour outages produced an average of 404.96 m$^3$ of unmet demand in Net1, while Net3 maintained full modeled demand delivery despite pressure deviations. Sustained pressure recovery occurred in 103 of 192 scenarios, with 65 exhibiting subsequent relapse, whereas all scenarios satisfied the post-outage water-service recovery criterion. Combined mitigation eliminated unmet demand in evaluated Net1 scenarios, while outage shortening achieved the highest sustained-pressure-recovery count. These findings highlight network-dependent resilience and the distinction between service restoration and hydraulic recovery. Environmental stress scenarios represent sensitivity assumptions rather than empirically calibrated climate impacts.
★ Learning Force-Reactive Humanoid Loco-Manipulation
Many everyday tasks require controlling contact force as well as motion. On a humanoid, this force must often be coordinated across both hands and the whole body, and cannot be specified by kinematic trajectories alone. Therefore, we present FLORE, a framework for learning force-reactive humanoid loco-manipulation. We collect robot-free demonstrations using two force-sensing UMI-style handheld grippers for hand motion and contact wrenches, and wearable trackers for body motion. A force-reactive diffusion policy takes vision, proprioception, and measured wrenches as input, then predicts actions and per-hand force commands. A hybrid compliance whole-body controller with force tracking executes these commands while maintaining balance: an integral of the force error measured by real force/torque (F/T) sensors lets each hand track force accurately along the commanded direction and remain compliant along the others. In simulation, the controller tracks per-hand force commands of varied magnitude and direction, with little loss in whole-body motion accuracy. Across three contact-rich tasks on the robot, the force-reactive policy succeeds in 80% of rollouts, compared with 26.7% for a position-only policy trained on the same demonstrations and executed by the same controller. For more details, please visit our website: https://louis-leee.github.io/FLoRe/.
comment: 9 pages, 6 figures
★ Nonprehensile Robotic Catching Using Virtual Constraints
The problem of nonprehensile robotic catching is addressed using virtual nonholonomic constraints (VNHCs); the motion of the robot end-effector is designed and controlled directly in terms of the states of the object before it is caught on the surface of the end-effector. In contrast to contemporary approaches, our approach avoids the need for object trajectory prediction, and consequent time-varying robot trajectory generation. With a focus on the planar case, we demonstrate that our VNHC approach can be used to catch an object on an end-effector at a desired height, with minimal relative normal velocity at the contact instant. Moreover, the VNHC function depends only on the desired catching height, and the same VNHC can be used for a range of initial conditions. We use model predictive control (MPC) to bring the object and end-effector to rest at a desired location after contact is made. The approach is validated in simulation.
comment: 9 pages, 6 figures; an abridged version of this article has been submitted to the 2027 American Control Conference (ACC)
★ Security Region Compression via Hybrid Vertex Pruning for Enhanced Computational Efficiency
The security region (SR) specifies a joint feasible region in the operational space involving various installations within the power system. It enables dynamic allocation of capacity among these energy units, thereby unlocking additional grid potential for power delivery and mitigating congestion. By constructing a convex hull from a set of vertices to form a polytope, the SR can be effectively represented with an acceptable accuracy loss. However, the SR computation inevitably generates many low-quality vertices, which contribute little to the overall hypervolume and can be discarded. By introducing SR compression, the SR data size can be reduced, which in turn facilitates the memory-efficient computation of SR and time-efficient solution of SR-related optimization problems. Accordingly, this paper proposes a vertex pruning approach that mixes local and backward search policies to achieve effective SR compression. In particular, the backward search strategy allows previously pruned vertices to be recycled when they are re-identified as critical. Subsequently, we incorporate this approach to enhance computational efficiency in the aforementioned applications. Numerical validations are conducted on two realistic medium-voltage Dutch grid cases. The results demonstrate at most 1.65% hypervolume loss with over 96.0\% SR data size reduction after applying the proposed compression approach. Empowered by this compression, the 8-dimensional (8D) SR becomes computationally tractable. Moreover, in relevant grid-aware computation challenges, the solving time can be reduced by at least 52.9% with at most an 1.36% optimality loss. All above validate both the efficacy and values of the proposed SR compression approach.
comment: 10 pages, 14 figures
★ World Model for Energy Efficiency Maximization in Dynamic Metasurface Antennas
Dynamic metasurface antennas (DMAs) have emerged as a promising technology for realizing large-scale antenna arrays in next-generation wireless networks owing to their ability to enable dense and energy-efficient antenna architectures. However, jointly optimizing digital beamforming and DMA element weights under time-varying channel conditions remains challenging due to their coupled effects on the received signal and interference powers. We formulate this problem to maximize time-averaged energy efficiency (EE) under time-varying channel conditions, subject to transmit power constraints, Lorentzian phase limitations, and minimum rate requirements. To enable long-term planning and improve sample efficiency compared with model-free reinforcement learning, we develop a world model framework that learns latent system dynamics from interactions with the environment and jointly optimizes digital beamforming and DMA element weights through imagined trajectories. Furthermore, we incorporate feasibility-preserving action mappings and safety cost signals to ensure constraint satisfaction during policy learning. Simulation results demonstrate that the proposed framework improves the time-averaged EE by approximately 20% compared with the deep Q-network baseline, while achieving higher sample efficiency and more reliable constraint satisfaction.
★ Grid-Forming AC/DC Converters for Meshed DC Networks via Dual-Measurement Frequency Droop
This paper proposes and analyzes a dual-measurement frequency-droop (DMFD) controller for meshed MVDC networks. Using only two local DC measurements, bus-voltage deviation and a filtered DC-port current or power deviation, the controller enables grid-forming operation at both AC and DC ports across the mesh. Although the DMFD regulates neither the DC voltage nor the DC current directly and acts only on the terminal frequency, it is shown that AC-side synchronization closes an effective DC droop at every synchronized terminal, so that the steady operating point of the mesh exists and is unique. Constant frequency deviations at a terminal and single-terminal islanding enter the same DC equations as equivalent current offsets, with the island frequency fixed by the island power balance, allowing node voltages and branch currents to be obtained from one linear system. The steady-state model yields design-use envelopes to map the droop ratios and the net DC-injection levels into bounds on DC-voltage deviation and terminal currents. A small-signal dynamic model is developed and used to generate eigenvalue loci and damping-ratio charts that guide controller-gain selection. Averaged EMT simulations on a four-terminal ring corroborate the analysis: in regimes where PLL-based grid-following control is known to lose damping, the DMFD controller maintains stable operation and follows the predicted steady-state characteristics.
comment: 10 pages, 12 figures, 1 table
★ FedAvgIm: Federated Learning for Multi-Site Impedance Identification of Inverter-Based Resources
With the increasing integration of inverter-based resources (IBRs) into power systems, assessing small-signal interaction stability has become essential. Impedance/admittance models enable frequency-domain characterization of IBR dynamics without requiring access to proprietary internal control details, but measurement-based identification is time-consuming and must be repeated across operating points. This paper proposes FedAvgIm, a privacy-aware federated-learning framework for multi-site dq-admittance identification under severe local data scarcity and raw-data-locality constraints. Each IBR site trains a local feed-forward neural network using a small set of operating-point-frequency samples, while only model parameters are shared with a cloud aggregator that performs sample-size-weighted FedAvg to update a global model and broadcast it back to all sites. To demonstrate effectiveness, EMT measurement-based admittance datasets are constructed for nine grid-following IBRs with different parameters, architectures, measurement-noise levels, and non-IID settings, where each client has only 40-390 training samples. Results show that FedAvgIm achieves performance close to centralized training and consistently outperforms local-only learning and raw-data-locality-preserving transfer-learning baselines. Additional studies show that FedAvgIm maintains stable learning behavior under measurement noise and remains beneficial under moderate non-IID conditions. The estimated dq-admittance models are further applied to generalized Nyquist stability analysis and validated against EMT time-domain simulations under multiple operating points and grid-strength conditions.
comment: Author-accepted manuscript (not the IEEE version of record)
★ Efficient ADMM for Soft-Constrained Quadratic Programs via Generalized Slack Projections
Quadratic programs (QPs) are a core building block in many applications. In practice, constraints are often softened with slack variables to guarantee feasibility. The standard approach tackles this by introducing additional decision variables, growing the resulting optimization problem. In this paper, we propose an Alternating Direction Method of Multipliers (ADMM) scheme for a broad class of convex slack penalty functions, including all $\ell_k$-norms, without increasing the problem size. The only change to the ADMM scheme is a modified slack projection function, while the remaining algorithm stays the same. We demonstrate our approach on numerical examples, showing significant speedups, and show its flexibility on an energy optimization problem.
★ ADACT: PDE-Constrained Thermal Hardware/Software Co-Design Adjoint-Driven Optimal Control via Finite Difference Discretization
By virtue of the sheer compute demands of large-scale artificial intelligence (AI) and high-performance computing (HPC), modern multi-core microprocessors suffer from severe power density limits and thermal throttling, entailing an urgent need for principled methods for predictive, physics-informed dynamical thermal management (DTM). Here, we introduce a hardware/software co-design physics-informed PDE-constrained DTM throttling optimal control algorithm, ADACT (Adjoint-Driven Adaptive Co-Design Thermal Control), that minimizes thermal risk and cooling energy overhead while maximizing core execution throughput. The underlying physical temperature field is governed by a two-dimensional nonhomogeneous parabolic reaction-diffusion equation with spatially localized, time-varying power source terms and variable convective cooling subject to nonhomogeneous Dirichlet boundary conditions. We employ a discretization of the continuous state PDE via a Crank-Nicolson finite difference scheme with a 5-point discrete Laplacian operator on a uniform spatial mesh, which preserves unconditional numerical stability and $O(Δx^2 + Δy^2 + Δt^2)$ second-order temporal and spatial accuracy. To efficiently compute exact gradients, we construct and implement a continuous backward-in-time adjoint state method with L-BFGS as our optimization algorithm of choice. The accuracy and second-order convergence rates of both forward state and backward adjoint solvers are quantitatively verified via the Method of Manufactured Solutions (MMS), and gradient accuracy is validated using directional derivative Taylor tests. Numerical experiments demonstrate that ADACT dynamically throttles software power workloads and modulates hardware cooling efforts to strictly uphold physical thermal thresholds while maintaining operational throughput.
★ Trim: Fee Level Matters More Than Skew in Lightning Routing
Lightning routing nodes lend their channel balance for a posted fee. Operators' tools move that fee with the balance around a hand-set level, and which part of the fee drives revenue is unclear. We present Trim, which models a channel as a two-sided inventory with costly refills. Its optimal fee is a level, set by demand and replenishment, plus a balance skew, the shadow price of liquidity. The level is first order: no fee rule beats the ceiling demand and replenishment set, and on exactly solved channels the best flat fee comes within $1.6\%$ of the optimum at the median, whereas a level off by a factor of two loses tens of percent. The skew is second-order insurance: it adds at most what the flow-balancing flat fee loses to refused forwards. So the level should be chosen: Trim's rule traces the revenue--success frontier, posts one level on every channel, and adds a skew only where refills stay frequent and costly. On a 2026 snapshot with synthetic payments, no tuned tool beats that frontier by over a few percent, and a skew changes gross revenue by a few percent.
★ Accelerated Push-Pull with Local Updates for Decentralized Stochastic Optimization
We consider decentralized stochastic nonconvex optimization over directed networks. Push--Pull provides an efficient decentralized optimization framework for such networks, but in the stochastic setting, gradient variance can slow its convergence and complicate the analysis of linear speedup. Motivated by this observation, we incorporate recursive momentum into stochastic Push--Pull for variance reduction, yielding the accelerated stochastic Push--Pull (ASPP) algorithm. Under mean-squared smoothness, we prove that ASPP improves the convergence rate of stochastic Push--Pull and achieves asymptotic linear speedup with respect to the number of agents. To further reduce communication, we develop LASPP, a local-update variant of ASPP that allows each agent to perform multiple local updates between communication rounds. We show that LASPP achieves an additional asymptotic linear speedup with respect to the number of local steps. With a suitable number of local steps, LASPP reduces communication complexity while maintaining the same stochastic gradient complexity as ASPP. Numerical experiments on real datasets demonstrate the effectiveness of the proposed algorithms.
comment: 38 pages, 4 figures
★ A PDE-Based Model of Runner Flow in Mass Participation Events
This paper presents a macroscopic approach for modeling pedestrian movement. A convection-diffusion partial differential equation (PDE) is described and used to model the flow of runners along a marathon course. A marathon is a good modeling case study for human movement because the location of runners is only known in sparse locations, so an accurate model is needed to know where people are at all times. As well, marathons offer many safety and logistical problems that can be solved by modeling human movement. Data is obtained from the 2023 Columbus Marathon, and model improvements are made to include a temporally varying convection term. Results show an increase in model accuracy which can allow for better resource allocation and safety measures.
comment: 6 pages, 8 figures, 2 tables. Accepted to the 2026 Modeling, Estimation and Control Conference (MECC)
★ Right Model, Right Task: Capability-Constrained Carbon Labeling to Decarbonize Generative AI
The carbon footprint of generative-AI inference depends not only on hardware efficiency and electricity supply, but also on whether the selected model is appropriate for the task. A frontier model may expend unnecessary computation on a routine request, whereas an under-capable model may increase energy use through failure, retries, extended reasoning and escalation. However, a standardized evaluation framework that accounts for task-specific capability requirements and the carbon emissions associated with successful task remains lacking. In this Perspective, we argue for a capability-constrained carbon labeling framework that shifts environmental assessment from carbon intensity per inference towards the expected carbon emissions per successfully completed task at the required capability level. Under this framework, models would first be assessed against task-specific capability requirements, and only models capable of meeting those requirements would be compared on their expected energy use and operational carbon emissions. By linking service quality to carbon cost, this approach could enable more informed matching of the right model to the right task, offering a demand-side pathway to reducing emissions associated with generative-AI token use. Realizing this vision will require standardized capability-based service assessments, comprehensive energy accounting and transparent carbon disclosures.
★ Adaptive Control for General Nonlinear Stochastic Systems
This paper extends the least-squares and certainty-equivalence minimum-variance framework of classical linear self-tuning regulators to a broad class of nonlinearly parameterized systems with compact parameter spaces, including commonly used deep neural network models. First, historical nonlinear least-squares fitting guarantees are converted into online one-step prediction bounds under a general modulus of continuity, with explicit regret rates under parameter Lipschitz continuity and finite-moment noise. Second, approximate NLS estimation combined with certainty-equivalence control achieves closed-loop stability and asymptotically optimal tracking under mild conditions without requiring excitation or parameter identifiability. Explicit closed-loop regret and tracking error rates are established in the parameter-Lipschitz case, and closed-loop examples show that the regret rates cannot, in general, be improved for this regulator. Finally, robustness to small unmodeled dynamics is established by showing that closed-loop stability is preserved and the tracking error bounds comprise a vanishing term and a residual term determined by the model discrepancy.
★ A 2-DOF MultiBody Track Joint for Race Car Optimization
TrackJoint is a Modelica MultiBody component that turns a three-dimensional road surface into a joint with two degrees of freedom. The two joint coordinates are the distance travelled along a reference line and the sideways offset from that line. Together they give the position, orientation and angular velocity of a single frame_b connector. All road geometry reaches the joint as a small set of periodic piecewise-quintic splines, so one component can represent any circuit whose splines have been fitted beforehand. The road constraint is closed with d'Alembert's principle. The two generalized forces that belong to the joint coordinates are set to zero, so no Lagrange multipliers are needed. The joint is connected to a multibody race car in Dymola and the pair is exported as one FMI~3.0 FMU. The same FMU is then used for time-domain simulation and for CasADi/IPOPT minimum lap-time optimization, with no change to the model in between. This works because the joint keeps the arc-length coordinate as a real state and publishes its time derivative, which is the quantity that lets the problem be rewritten with distance We report results for minimum lap time simulations on the Singapore Formula~1 circuit.
comment: American Modelica & FMI Conference 2026
★ Data-Driven Hamiltonian for Neural Optimal Control
Reinforcement learning (RL) has emerged as a prominent data-driven approach to optimal control, but often requires substantial amounts of data and provides limited guarantees when encountering out-of-distribution state-action pairs. In this paper, we propose an alternative approach grounded in the well-established Hamilton-Jacobi framework in optimal control theory. Rather than requiring an explicit analytical model of the system dynamics to evaluate the Hamiltonian, we replace this with a data-driven Hamiltonian (DDH), which we generalize to a broad class of optimal control problems. In contrast to standard RL approaches, the DDH provides a principled mechanism, requiring only minimal prior system information such as a Lipschitz constant, for quantifying and accounting for uncertainty in regions not covered by the data, enabling the derivation of controllers with performance guarantees. Additional prior system knowledge, when available, can be readily incorporated to improve performance. For high-dimensional systems, we further introduce a physics-informed neural approximation of the value function, trained using a novel procedure that generates worst-case fictitious rollouts evaluated through the DDH. On an LQR problem, the DDH reaches near-optimal cost on every dataset we test, including random data on which offline RL methods remain far from optimal, and needs far fewer samples than online RL. On a nonlinear lunar lander benchmark, the DDH achieves an optimality gap to a trajectory-optimization reference more than $3\times$ smaller than CQL and FQL, and matches the performance of SAC with approximately $37\times$ fewer samples. These results establish the DDH as a promising framework for data-driven optimal control, particularly in settings where available data are limited in quantity or quality.
comment: 21 pages, 9 figures
★ Multi-time-scale Control of Power Systems
The power grid has been dominated by large synchronous generators and clearly separated control layers. Today, power-electronics energy conversion interfaces are rapidly being deployed calling for another look at system-level control hierarchies. This article examines how multi-time-scale modeling, feedback control, and feedback optimization can help reinterpret the traditional organically grown hierarchy of primary control, automatic generation control, and economic dispatch, and enable new approaches for control of contemporary power systems. Beginning with an overview of power system models and conventional operational practices, the article develops connections to modern control architectures that can coordinate heterogeneous resources, account for operational constraints, and exploit faster actuation without overlooking first-principles physics.
comment: Submitted for publication
★ Distribution Grid Reconfiguration and Restoration via Linearizable Multi-phase Power Flow Modeling
Accurate and tractable power-flow (PF) modeling is essential for distribution grid topology optimization, where the presence of discrete branch-status decisions therein significantly increase the computational complexity. To tackle this challenge, we develop a topology-adaptive, low-complexity neural piecewise-linear (N-PWL) PF model. We propose a sequential layer design that can directly incorporate branch status variables into the PF mapping from branch flows to nodal injections. As a result, our N-PWL model not only preserves the PF branch-node consistency but also facilitates the representation of any unseen topologies without retraining. A low-complexity model design is further pursued to attain a linear NN parameter scaling by using a fixed-size area partition of the feeder. The resulting scalable N-PWL model enables mixed-integer linear programming (MILP) formulations of feeder topology optimization tasks, including distribution network reconfiguration (DNR) and restoration ordering problems (ROP). Numerical studies on the unbalanced, multi-phase IEEE 123-bus feeder demonstrate our N-PWL model outperforms existing end-to-end baselines in PF prediction and, more importantly, can fill the gap by achieving accurate representation of any unseen topologies. For solving DNR/ROP, our model can significantly improve the AC feasibility over linearized PF models, with a notable reduction of computation time compared with AC-PF based nonlinear formulations, particularly under a large number of topology decisions.
★ Improving Learned AC-OPF Feasibility under Distribution Shift with PF-in-the-Loop Correction
Learning-based AC-OPF enables fast dispatch, yet its feasibility can degrade under distribution shift. We analyze three state-of-the-art methods and identify a shared deployment-time limitation: none use feedback from the power-flow (PF)-realized state to ensure feasibility of the predicted control. Using a simple PF-in-the-loop feasibility restoration filter, we empirically show that such feedback substantially improves hard feasibility across four out-of-distribution scenarios with modest runtime overhead.
♻ ★ GPU-Accelerated Quantum Annealing-Inspired UAV Path Planning for Smart Agriculture
With the rapid advancement of smart agriculture, Unmanned Aerial Vehicles (UAVs) have become indispensable for efficient coverage and path planning in grid-mapped agricultural fields. However, optimizing UAV flight paths in large-scale environments remains a complex Nondeterministic Polynomial-time (NP)-hard problem. Traditional path planning methods are often constrained by local optima, limited scalability, and slow convergence, which significantly restrict their effectiveness in solving large-scale problems. To address these limitations, this paper proposes a novel optimization framework utilizing a Graphics Processing Unit (GPU)-parallelized Ising solver. Our method mimics the operational principles of quantum annealing on GPU hardware, enabling rapid search for the minimum-energy state of Ising models. Unlike physical quantum devices, which are often constrained by the number of qubits, our approach leverages the Fixstars Amplify (FA) platform to perform parallel annealing on highly parallelized GPUs. This enables the simultaneous evaluation of thousands of potential path candidates and vast state transitions. By leveraging large-scale parallel processing, this framework's core strength lies in minimizing computation time even as the problem scale increases. Simulation results demonstrate that our proposed method consistently identifies superior flight paths while maintaining stable computational performance compared with the genetic algorithm and simulated annealing method.
comment: 6 pages, 3 figures, conference
♻ ★ On the invariance of risk-sensitive LQR gain under input randomization
This paper shows that the optimal gain of the risk-sensitive linear quadratic regulator (LQR) problem is invariant under input randomization when the cost penalizes the realized noisy input. This may appear counterintuitive at first glance because certainty equivalence does not hold for risk-sensitive LQR and input randomization inflates the effective process noise. Nonetheless, the gain is preserved because the input noise enters not only the system dynamics but also the cost functional, and its total effect eventually vanishes. Consequently, the optimal gain and the Riccati recursion need not be recomputed, and the increment in the optimal cost can be readily evaluated in closed form. This result facilitates the use of risk-sensitive LQR in applications that employ input randomization for privacy or exploration, such as dynamic watermarking for replay attack detection.
comment: 6 pages, revised extended version submitted to IEEE L-CSS and ACC 2027, including Section III-F
♻ ★ Windex: Index-Driven Contextual Scheduler for Scalable Service Guarantees in NextG Cellular Networks
Next-generation cellular networks must support heterogeneous service objectives, ranging from throughput-oriented eMBB to latency- and regularity-constrained URLLC and XR traffic. Enforcing these objectives at the RAN scheduler is difficult because wireless channels, queues, and application arrivals evolve at the millisecond timescale while spectrum resources are shared across users. We formulate service-aware RAN scheduling as a constrained Markov decision process with per-user throughput and service-regularity constraints. A Lagrangian relaxation gives an exact decomposition of the relaxed problem into per-user control problems with a common resource price. We prove fixed-dual indexability for an illustrative time-since-last-service (TSLS) debt arm with a throughput objective and TSLS constraint, note the analogous scalar throughput-debt result, and assume fixed-dual indexability for the full vector state used in deployment. We design Windex, an index-driven scheduler that learns dual-dependent per-user break-even resource prices using compact neural approximators over queue, channel, TSLS, and violation-deficit state. We further design CWindex, a contextual extension that infers service class from queue-history time series when explicit labels are unavailable. We implement Windex as an application compatible with a realtime 5G radio intelligent controller. Inference takes 10-20 $μ$s per user on a general-purpose CPU, well within a 1 ms control budget. Trace-driven emulation using over-the-air channel measurements and larger simulations show that Windex and CWindex substantially reduce per-user service violations while maintaining throughput efficiency.
comment: 10 pages, 11 figures. To appear in MobiHoc 2026
♻ ★ PREFER: Personalized Review Summarization with Online Preference Learning
Product reviews significantly influence purchasing decisions on e-commerce platforms. However, the sheer volume of reviews can overwhelm users, obscuring information most relevant to their individual needs. Existing e-commerce summarization systems typically produce generic, static summaries that fail to account for the fact that (i) different users prioritize different product characteristics, and (ii) these preferences may evolve with interactions. To address these challenges, we propose an online learning framework that generates personalized summaries for each user by representing user interests in a latent aspect space and selecting preference-aligned review evidence. Our system then iteratively refines its understanding of user preferences by incorporating feedback on the generated summaries directly over time. We evaluate our framework on Amazon Reviews'23 using controlled feedback simulations and robustness checks with held-out-history, independent-embedding, and external LLM-based evaluators. These experiments show improvements in preference-aligned evidence selection under the evaluated feedback settings. A separate human-based study finds that scenario-conditioned summaries are preferred over generic (uniformly-aggregated) alternatives in 69.9% of paired comparisons.
♻ ★ SateBeam: Beam-Aware Radio Map Estimation With Physics-Consistent Parametric Modeling for Unknown Multiple Satellites
Dense low-Earth-orbit (LEO) networks increasingly reuse spectrum across moving satellites and overlapping beams, making a spatial radio map essential for interference diagnosis and coordination. Constructing such a map from sparse ground power measurements is difficult because the active satellite subset and its cardinality are unknown, candidate footprints can be highly coherent, and beam pointing, width, effective power, Doppler, and background must be inferred jointly. Known ephemerides do more than enumerate visible candidates. Propagated positions determine candidate-specific ranges, viewing geometry, and spatial footprints, while radial velocities induce distinct time-frequency signatures. Their joint spatial--Doppler structure rejects candidates whose geometry or time--frequency signature conflicts with the observations before continuous beam inference. SateBeam therefore screens all candidates through Doppler-resolved nonnegative fitting. GPU-accelerated grid correlations initialize continuous off-grid refinement; a Gamma-likelihood Bayesian information criterion (BIC) selects model cardinality, and backward deletion and exchange repair greedy support errors. Across paired real-ephemeris scenarios, SateBeam recovers the exact support in every case and reduces independent-query RMSE from $0.655$ to $0.050$ dB relative to orthogonal matching pursuit, with consistent advantages under model mismatch and larger candidate sets. The code is available at https://github.com/UNIC-Lab/SateBeam.
♻ ★ Repeated Binary Direct Collinear Impacts Under Incremental Contact Laws With Residual Indentation: A Hybrid Systems Formulation
Incremental contact laws specify the normal contact force through a differential relation, possibly carrying an internal state, driven by the indentation and its rate. In some, the force is extinguished at a nonzero indentation, whether by plastic deformation or by an elastic aftereffect, so that a residual deformation remains at separation. Such laws sit uneasily within rigid body dynamics, which admits no deformation. The tension is tolerable when the indentation is small relative to the bodies, so that it may be carried constitutively rather than geometrically. Even then, the contact law alone does not determine the interaction of the bodies. Because force and indentation no longer vanish together, conditions for the commencement and termination of contact must be supplied separately. So must the fate of the deformation and internal state at separation, neither of which the equations of motion contain. This article formulates the repeated direct collinear impact of two convex bodies under external forces as a hybrid dynamical system. The contact interface is modeled as a massless element carrying the contact law and its state, coupled to the bodies through relative velocity and an interaction force dictated by the contact state. Consequently, all switching and resets are confined to the interface, leaving the geometry and the equations of motion of the bodies unaltered. The principal analytical properties of the resulting formulations are established, among them passivity and completeness; the branching of solutions at the onset and termination of contact is also examined. The framework is demonstrated through numerical simulations.
comment: 30 pages, 6 figures; v1-v5: a variety of amendments and corrections; v6: additional analytical properties of the contact laws used in examples
♻ ★ A Novel Path-Tracking Algorithm for Automated Tractor-Trailer Forward and Backward Maneuvers
Fully autonomous tractor-trailer systems are increasingly deployed in logistics, agriculture, and industrial environments, where precise and robust path-tracking capabilities are essential. However, the articulation between the tractor and the trailer introduces additional nonlinearities and significantly complicates lateral and longitudinal control, particularly during reversing maneuvers. This paper introduces a novel path-tracking algorithm specifically designed for articulated vehicles with a single trailer. The proposed method combines a lateral control law applied at the trailer level with a short-horizon predictive adjustment of the tractor steering angle, ensuring stable convergence toward the desired path in both forward and backward motion. The approach is geometry-based and requires no per-vehicle calibration or training. Simulation studies in a high-fidelity physics simulator show steady-state or final lateral errors of $0.02$--$0.05$~m on straight-line, circular, and reverse docking trajectories, which is of the same order of magnitude as, or lower than, the errors reported in the literature. Real-world experiments on a small-scale tractor--trailer robot, with the whole controller running on a low-cost microcontroller, further confirm the feasibility of the approach, with lateral errors below 13~mm during repeated reverse docking maneuvers despite a pose detection latency of about 200~ms.
♻ ★ Rethinking electrified transportation: The case for dynamic wireless charging
Long-term infrastructure investment often requires choosing between an established technology and a capital-intensive alternative that gains value as demand matures. For electric vehicle (EV) charging, this trade-off is between incumbent stationary charging technology and emerging dynamic wireless charging (DWC). DWC could fundamentally alter transportation electrification by enabling high-throughput charging in motion and reducing reliance on large onboard batteries; however, its high upfront cost makes the timing and location of deployment critical. Overcommitting to stationary charging stations (SCS) risks technological lock-in and forgoing the benefits of DWC, whereas premature DWC deployment can strand capital in underutilized corridors. Through a joint investment assessment framework coupling DWC, SCS, and supporting grid infrastructure across nearly 49,000 miles of the US Interstate freight network, we find that higher utilization and longer project lifetimes rapidly shift investment toward DWC, whose share rises sharply from 48% at a project lifetime of 30 years to 100% at 32 years. Contiguous DWC segments can share grid connections, enabling them to provide over six times the rated charging capacity of SCS while requiring 46% less investment in new distribution-grid infrastructure. We also find that a combined DWC and SCS portfolio reaches a maximum lifetime value of USD 60 billion, 63% above the case with only SCS. We further identify specific traffic demand, construction, and grid access conditions that shape where each technology becomes advantageous, yielding actionable thresholds for SCS and DWC deployment that directly inform long-term EV charging infrastructure investment and policy.
♻ ★ Uncertainty-Aware Parameter Estimation for Condition Monitoring of Power Converters
Changes in circuit parameters can indicate component degradation in power converters, but measurement noise and limited parameter observability can make these changes difficult to detect reliably. This paper proposes an uncertainty-aware parameter estimation framework to distinguish genuine parameter changes from variations caused by estimation uncertainty. Using a differentiable switched converter model, the method requires only one or two samples per switching period. Model-derived residual whitening and a Bayesian posterior approximation quantify estimation uncertainty and reveal poorly identifiable parameters. Simulations demonstrate improved estimation accuracy over least-squares and physics-informed neural-network benchmarks, with uncertainty estimates consistent with repeated noise realizations. Experiments on a laboratory buck converter validate the estimated parameters against independently derived references and demonstrate the detection of a controlled output-capacitance reduction. The proposed framework provides a basis for reliable parameter-change detection in power converter condition monitoring.
♻ ★ Minimum Time Consensus of Multi-agent System under Fuel Constraints
This work addresses the problem of finding the minimum time consensus point in the state space for a set of $N$ identical double integrator agents with bounded input and fixed fuel budget constraints. The attainable set of each agent under these constraints is computed, and its boundary is used to characterize the set of initial conditions from which consensus is possible. The minimum time to consensus is the least time at which the attainable sets of all agents intersect, and the corresponding consensus state is their point of intersection. Using convexity of the attainable set and Helly's theorem, this reduces to examining the triplets of agents, for which a closed-form expression for the minimum time and the consensus point is obtained. The triplet computations are independent and are distributed evenly among the $N$ agents. The overall minimum time and consensus point are given by the triplet with the greatest minimum time, and each agent then computes its own control input locally. An example with $12$ agents is used to illustrate the results.
♻ ★ Distributed Optimization with Nesterov Momentum and Finite-Time Quantized Averaging over Directed Networks: Extended Version
In modern large-scale networked systems, rapidly solving optimization problems while utilizing communication resources efficiently is critical for addressing complex tasks. However, distributed optimization over bandwidth-limited networks must cope with two obstacles: limited-bandwidth communication, and slow convergence when the objective is ill-conditioned. In this paper, we consider unconstrained distributed optimization over strongly connected directed networks, where nodes exchange quantized (i.e., compressed) messages. We propose QANM (Quantized Averaged Nesterov Momentum), an algorithm that combines Nesterov momentum with a finite-time quantized averaging protocol. At every iteration, QANM enables all nodes to reach exact agreement, without requiring doubly stochastic weight matrices or undirected communication. For strongly convex and smooth objectives, we show that the objective gap of QANM decays at the accelerated rate $1-κ^{-1/2}$ (where $κ$ is the condition number), up to a residual of order $Δ^2$ determined by the quantization level $Δ$. This improves the condition-number dependence of the transient iteration complexity from $κ$ to $\sqrtκ$ compared with gradient descent that utilizes the same quantized averaging mechanism. Finally, we evaluate QANM on a distributed sensor fusion problem, showing that our algorithm exhibits the fastest initial error reduction compared with other methods from current literature.
♻ ★ Describing Functions and Phase Response Curves of Excitable Systems
The describing function (DF) and phase response curve (PRC) are classical tools for the analysis of feedback oscillations and rhythmic behaviors, widely used across control engineering, biology, and neuroscience. These tools are known to have limitations in networks of relaxation oscillators and excitable systems. For this reason, the paper proposes a novel approach tailored to excitable systems. Our analysis focuses on the discrete-event operator mapping input trains of events to output trains of events. The methodology is illustrated on the excitability model of Hodgkin-Huxley. The proposed framework provides a basis for designing and analyzing central pattern generators in networks of excitable neurons, with direct relevance to neuromorphic control and neurophysiology.
comment: 6 pages, 7 figures, accepted and presented at European Control Conference 2026
♻ ★ Crazyflow: An Accurate, GPU-Accelerated, Differentiable Drone Simulator in JAX
High-quality, large-scale synthetic data from simulations is becoming a cornerstone for pushing robots' capabilities. While aerial robotics simulators have evolved to support specialized needs such as fidelity, differentiability, and swarms independently, a unified platform that synthesizes data across all these domains is missing. In this work, we propose Crazyflow, a simulator designed to push the frontier of aerial robotics algorithm development, supporting model-based and data-driven methods, gradient-based and sampling-based approaches, single- and multi-agent systems, and state- and vision-based policy learning. Crazyflow combines high-fidelity dynamics with JAX for speed and differentiability, scaling to millions of environments or thousands of swarms with thousands of drones on single CPUs and GPUs, and beyond on clusters. Real-world experiments show Crazyflow supports both analytical-gradient-based policy learning, achieving high-accuracy tracking without domain randomization, and sampling-based obstacle avoidance at speeds exceeding half a billion steps per second. Breaking the traditional train-then-deploy paradigm, Crazyflow's performance enables in-flight reinforcement learning; we demonstrate this by throwing a physical drone into the air and training a recovery policy from scratch in 0.37 seconds, catching it before it hits the ground. By pushing accuracy, speed, and differentiability simultaneously, Crazyflow serves as an open-source resource for synthetic data generation, with emerging capabilities for large-scale parallelization for online, in-execution learning and optimization, opening the door to novel algorithm development.
comment: Revision
♻ ★ Low-Altitude Platform Stations for Dense Urban Connectivity
This paper evaluates the downlink performance of 5G non-terrestrial networks (NTNs) realized via low-altitude platform station (LAPS)-mounted base stations, and compares it against conventional 5G terrestrial network (TN) in a realistic dense urban scenario. Using a 3GPP-compliant multi-cell hexagonal layout with 19 sites and 57 sectors, we model LAPS altitudes ranging from 100 to 1000 m and evaluate the average effective signal-to-interference-plus-noise ratio and user throughput via system-level simulations. Results show that LAPS-based NTN deployments can significantly outperform terrestrial 5G in per-user throughput, provided the LAPS altitude is properly chosen to balance increased line-of-sight probability against inter-beam interference.
♻ ★ Programmable Sequential Memory in Coupled Oscillator Networks
The Hopfield network established that static memories can be stored as energy minima of an energy-based model. Real intelligent agents, however, must navigate sequences of memories rather than isolatedcsnapshots. Biological cortex addresses this through a separation of timescales, where fast synaptic dynamics encode individual states while slow neuromodulatory processes govern transitions between them. Inspired by this organization, we propose a sequential memory architecture that is fully continuous and admits exponential storage capacity. The architecture is programmable, since the transition structure is carried by a mechanism that is decoupled from the stored patterns, driven by the input context, and free to be chosen, optimized, or learned from data. Specifically, we construct an autonomous three-timescale dynamical system with three coupled layers. A fast Kuramoto layer stores phase patterns as exponentially stable phase-locked configurations, an intermediate recognition layer sets the dwell times, and a slow attention layer routes the transitions. We analyze the stability, dwell time, and capacity of the system, and we validate the dynamics through numerical simulations of sequential memory retrieval.
♻ ★ Gradient-Free Neural Hamilton-Jacobi Reachability for Scalable Safety-Critical Control
Hamilton-Jacobi (HJ) reachability provides a principled framework for synthesizing safety certificates and robust controllers for safety-critical robotic systems. However, applying reachability analysis to high-dimensional nonlinear systems remains challenging: classical grid-based solvers suffer from the curse of dimensionality, continuous-time neural solvers require accurate spatial value gradients, and reinforcement-learning-based approaches often suffer from weak boundary anchoring and non-stationary adversarial policy optimization. We propose a discrete-time neural reachability framework for control-disturbance-affine systems that learns backward reachable tubes (BRTs) and backward reach-avoid tubes (BRATs) through Bellman-Isaacs value propagation. Our key idea is to combine equation-driven self-supervision with structured policy learning: rather than computing explicit PDE-gradients, we exploit the bang-bang structure of optimal safety interventions to construct approximate teacher actions from gradient-free value probes, converting adversarial actor learning into supervised policy learning. To stabilize long-horizon value propagation, we leverage the learned actor to train the value function backward from the terminal boundary using a windowed temporal curriculum, where each window is used as the boundary condition for the next window. Across benchmark problems up to 80 dimensions, our method learns accurate reachability value functions while improving stability over existing learning-based solvers. We further demonstrate observation-space scalability on F1-tenth racing with over 16,000-dimensional egocentric inputs. The learned safety filter generalizes zero-shot to unseen tracks and transfers to a physical RC car, achieving real-time robust collision avoidance.
♻ ★ A Benchmark Library for Distributed Power System Analysis and Optimization
DPLib is an open-source benchmark library created to support research and development in distributed power system analysis and optimization. Unlike centralized tools such as MATPOWER and PGLib, no general purpose, reproducible data library package currently exists for distributed power system studies. DPLib, available at \href{https://github.com/LSU-RAISE-LAB/DPLib.git}{GitHub}, fills this gap by providing 40 multi-region benchmark test cases ranging from 5 buses to 20758 buses, along with a graph-based partitioning toolkit that converts MATPOWER-compatible systems into distributed regional datasets. The toolkit generates standardized \texttt{.mat}, \texttt{.csv}, and \texttt{.m} files, regional MATPOWER version 2 cases, local and global bus mappings, generator and cost assignments, explicit inter-regional tie-line records, and bus-to-region partition maps. It supports unweighted, electrically weighted, and user-defined partitions, and is compared with METIS, KaFFPa, and an IPA-inspired baseline. DPLib also provides ADMM-based distributed DC and AC OPF solvers for validation. Numerical studies report partitioning sensitivity, centralized run times, distributed OPF iterations, run times, and optimality gaps. These results establish DPLib as a reproducible data layer for distributed power system research.
♻ ★ Ctrl-A: Control-Driven Machine Learning using Adaptive Online Data Augmentation
We introduce ControlAugment (Ctrl-A), an automated data augmentation algorithm for image-vision tasks, which incorporates principles from control theory for online adjustment of augmentation strength distributions during model training. Ctrl-A eliminates the need for initialization of individual augmentation strengths. Instead, augmentation strength distributions are dynamically, and individually, adapted during training based on a control-loop architecture and what we define as relative operation response curves. Using an operation-dependent update procedure provides Ctrl-A with the potential to reduce the use of augmentation operations that negatively impact model performance, alleviating the need to manually engineer augmentation policies for new image classification tasks. Experiments on the CIFAR-10, CIFAR-100, SVHN-core, and Tiny ImageNet benchmark datasets using WideResNet-28-10 and ResNet architectures demonstrate that Ctrl-A provides competitive performance across a range of image classification benchmarks.
comment: 19 pages (13 pages main manuscript), 12 figures (9 in main manuscript). v2 note: Title slightly changed to make it less generic/more descriptive. Experimental section overhauled to demonstrate and exemplify closed-loop control mechanism, experiment with Tiny ImageNet added
♻ ★ Reach-Avoid Differential game with Reachability Analysis for UAVs: A decomposition approach
Reach-avoid (RA) games have significant applications in security and defense, particularly for unmanned aerial vehicles (UAVs). These problems are inherently challenging due to the need to consider obstacles, consider the adversarial nature of opponents, ensure optimality, and account for nonlinear dynamics. Hamilton-Jacobi (HJ) reachability analysis has emerged as a powerful tool for tackling these challenges; however, while it has been applied to games involving two spatial dimensions, directly extending this approach to three spatial dimensions is impossible due to high dimensionality. On the other hand, alternative approaches for solving RA games lack the generality to consider games with three spatial dimensions involving agents with non-trivial system dynamics. In this work, we propose a novel framework for dimensionality reduction by decomposing the problem into a horizontal RA sub-game and a vertical RA sub-game. We then solve each sub-game using HJ reachability analysis and consider second-order dynamics that account for the defender's acceleration. To reconstruct the solution to the original RA game from the sub-games, we introduce a HJ-based tracking control algorithm in each sub-game that not only guarantees capture of the attacker but also tracking of the attacker thereafter. We prove the conditions under which the capture guarantees are maintained. The effectiveness of our approach is demonstrated via numerical simulations, showing that the decomposition maintains optimality and guarantees in the original problem. Our methods are also validated in a Gazebo physics simulator, achieving successful capture of quadrotors in three spatial dimensions space for the first time to the best of our knowledge.
comment: Paper version 1 accepted to the Journal of Guidance, Control, and Dynamics (JGCD). Version 2 strengthens the horizontal invariance and updates figures in the experiment
♻ ★ Formalizing and Estimating Task-Space Complexity for Zero-shot Generalization
Generalization is crucial for modern control, but learning across tasks can be costly. Recent work shows that zero-shot transfer, in which a controller designed for a source task is directly deployed to a target task, can achieve strong performance. We formalize task-space complexity, which characterizes the difficulty of achieving sufficiently good performance across a collection of tasks, by casting source selection as a covering problem. We provide bounds on the complexity and demonstrate its applicability.
Robotics
★ Intrinsic Goals for Autonomous Agents: Hierarchical Exploration Captures Future Volitional Action in Drosophila
Animals have a remarkable capacity for autonomy---driven by intrinsic motivation---without external sensory cues or extrinsic objectives, yet the computational mechanisms behind this ability largely remain a mystery. We combine whole-brain recordings in the fruit fly (Drosophila melanogaster), which reveal neural representations of spontaneous exploratory decisions more than 10 seconds in advance, with a neuromechanical digital twin to test whether optimizing intrinsic goals yields realistically embodied artificial agents that quantitatively match fly behavior and internal representations of future volitional action during exploration. Our hierarchical policy selects from a discrete library of low-level motor primitives, consistent with ethological theories of behavioral syllables. We train agents without external reward using 20 intrinsic motivation algorithms spanning curiosity, entropy, and competency methods. The agents are evaluated in a virtual tethered-ball assay modeled on the fly experiment, comparing spontaneous locomotor statistics and internal representations and decoding future turn direction from kinematics and neural activity on the same timescales observed in flies. DIAYN, a hierarchical competency algorithm learning discrete policies that maximize behavioral entropy and the separability of induced sensory observations, achieves the strongest joint behavioral and neural prospective coding performance relative to flies. Its learned policies closely capture individual variability in fly exploration strategies, suggesting that this diversity serves to maximize state-space coverage at the population level. DIAYN's latent activity also closely captures the sum-difference rule for interhemispheric encoding of kinematics in biological flies. These results establish hierarchical, intrinsically motivated agents as a powerful framework for reverse-engineering animal autonomy.
comment: 27 pages, 5 figures
★ What Does World Modeling Actually Improve in Robot Action Learning?
World Action Models (WAMs) combine future-video prediction with robot action learning, yet improvements in task success alone do not reveal how predictive supervision manifests in execution behavior or whether its benefits persist under task and environmental shifts. We conduct controlled future-video prediction loss ablations within FastWAM and OpenWAM, keeping model architecture, initialization, robot training data, and training steps fixed within each model family. Under clean conditions on seen tasks, predictive supervision increases success rates by 24.38 and 72.91 percentage points, respectively. Trajectory clustering further reveals higher success rates within several coarse trajectory clusters, including cases where cluster frequencies change little, indicating that the gains are not solely reflected in changes in the prevalence of trajectory patterns. However, these benefits do not consistently persist under distribution shifts: improvements on unseen tasks vary across models and tasks, while both models exhibit low success rates under the evaluated environmental randomizations, without a consistent advantage from predictive supervision. We further introduce future-video prediction as an auxiliary training objective for $π_{0.5}$, where a lightweight Transformer predictor used only during training increases success from 82.94% to 89.19%. These findings support future-video prediction as effective auxiliary supervision for action learning, while distinguishing improved execution on familiar tasks from reliable transfer and showing that such gains need not rely on a large pretrained video diffusion transformer (DiT) in the evaluated setting.
★ The German Tank Problem & Navigating by Scent
A robot searching for the source of an odor must eventually answer a deceptively hard question: have I arrived? Near the source, concentration gradients flatten, turbulence makes the signal intermittent, and a small robot often lacks the positional accuracy to map the plume. Here, we describe a lightweight remedy inspired by a World War II statistics conundrum: the German tank problem. To approximate enemy armament production, Allied analysts developed an estimator based on serial numbers from captured tanks. Analogously, we use the same procedure over sampled chemical sensor readings from a robot to create estimates for the peak concentration of an odor plume, and therefore the odor's source. Treating each sensor reading as an observation, our unbiased estimator needs only a running maximum and a sample counter. An exact confidence bound turns both numbers into a principled rule that the robot or UAV can use to determine whether to keep searching for the plume source or land. We derive the estimator and its bound, demonstrate its sample efficiency, and examine its robustness even when its assumptions fail. Finally, we summarize the estimator's role as the termination criterion for olfactory navigation.
comment: 8 pages
★ Dual Gradient Consensus for Adaptive Learning to Transient Out-of-Distribution Disturbances in Quadrotor Control
Quadrotor trajectory tracking under unmodeled disturbances often relies on online residual adaptation, but short out-domain residuals can corrupt the learned disturbance model when constant-step updates treat every residual gradient uniformly. We propose Dual Gradient Consensus (DGC), an adaptive-learning method for mixed in-domain/out-domain disturbances that retains the random Fourier feature model, offline BPTT policy, and least-squares adaptation of a prior domain-adaptive residual policy (DAPL) while modifying the online update step size. DGC compares each residual gradient with long- and short-term gradient memories, classifies the current adaptation regime, and uses a confidence map to suppress unreliable updates while recovering adaptation after persistent domain shifts. We evaluate DGC in numerical quadrotor trajectory tracking under four residuals that mix an in-domain component with short out-domain events. Compared with $L_1$-augmented MPC, RFF-based MPC, constant-step DAPL, and a frozen-parameter baseline, DGC achieves the lowest mean position RMSE in all four cases.
comment: 8 pages, 2 figures, 3 tables
★ Multimodal Socially Aware Framework for Human-Robot Interaction in Virtual Reality
Social robots must coordinate speech and movement while interpreting multimodal human input. We present a multimodal framework that integrates speech, vision, and touch to generate contextually appropriate robot speech and movement. We evaluated the framework in virtual reality with 18 participants across five social interaction scenarios, comparing its performance against a human-teleoperated baseline. Although participants rated the autonomous system above the neutral midpoint for both naturalness and social appropriateness, the teleoperated baseline received significantly higher ratings on both measures. Behavioral coding and qualitative feedback revealed that conversational timing, clarification of ambiguous requests, and spatial coordination were important factors contributing to this performance gap. These findings highlight key challenges in developing autonomous robots capable of natural, multimodal social interaction.
★ How High Can GPT-6 Astra Climb in Contact-Rich Manipulation? PhysLadder: A Hierarchical, Matched Sim-Real Benchmark for Physical Interaction
We introduce PhysLadder, a hierarchical Sim-Real benchmark for evaluating physical interaction capabilities in contact-rich manipulation. Its 16 household tasks span three cumulative levels: contact selection and geometric alignment (L1), force regulation within sustained contact (L2), and coordinated contact reconfiguration and force regulation (L3). Each task is instantiated in physically and visually matched simulated and real-world environments, with shared task definitions and success criteria. We evaluate GPT-6 Astra zero-shot with three trials per task in each environment. Real-world success rates decline from 80.0% at L1 to 47.6% at L2 and 0.0% at L3, with corresponding simulation rates of 66.7%, 47.6%, and 0.0%. Ten of the 16 tasks have identical success counts across environments; the remaining six differ by one success out of three trials. Across both environments, recurring difficulties in spatial reasoning, contact selection, force regulation, and coordination often prevent local physical progress from accumulating into task completion. These findings characterize Astra's observed capability boundary across increasing interaction demands and, for this model, support matched simulation as a repeatable, scalable testbed for studying the sustained, coordinated physical interactions that limit reliable manipulation.
comment: 31 pages, 11 figures
★ TRIAGE: Efficient Vision-Language-Action Inference Needs Both Reuse and Pruning
Vision-language-action (VLA) models are increasingly used for robot manipulation because they map camera images and a language instruction directly to actions. Robots that cannot be tethered to a workstation, such as mobile manipulators and humanoids, must run them on onboard embedded boards, where the prefill latency of the language backbone grows with the hundreds of visual tokens processed at every control step. To meet real-time limits, training-free accelerators recompute only a subset of the visual tokens and skip the rest, either by pruning them or by reusing their cached key-value (KV) states. The two options save nearly the same computation, yet existing methods apply only one of them to every skipped token. Through a token-level diagnostic, we show that which option keeps the action closer to full recomputation depends on the token's drift, the change in its shallow-layer value since its cache was computed. On both backbones tested, reusing tokens below a drift threshold was no worse than pruning them, whereas beyond it, reuse yields up to 4.4x the action error of pruning; the same diagnostic calibrates this threshold per backbone. Building on this finding, we propose TRIAGE, which deems a cache valid when its drift is below this threshold and it is not too old. TRIAGE recomputes a fixed budget of tokens, prioritizing important tokens with invalid caches, reuses every other token with a valid cache, and prunes the rest; prune-only and reuse-only methods are the two extremes of this partition. Because the budget fixes the number of recomputed tokens, it directly sets the prefill latency. On LIBERO and SimplerEnv, TRIAGE stays within 3 pp of full recomputation in average success rate while cutting prefill latency on a Jetson AGX Orin by up to 48%, and with only 7% of visual tokens recomputed, it beats the strongest baseline by 24.4 pp on LIBERO-Object.
comment: 8 pages, 6 figures
★ IntNet: A Communication-Driven Multi-Agent Reinforcement Learning Framework for Cooperative Autonomous Driving
Achieving safety in autonomous driving through Multi-Agent Reinforcement Learning (MARL) is a critical yet challenging task due to non-stationarity, partial observability, and the need for effective coordination among agents. Although earlier cooperative MARL methods have aimed to improve coordination by sharing encoded state observations, we find these strategies are insufficient in safety-critical scenarios. To address this gap, we present IntNet, a novel MARL framework that enhances safety by incorporating the transmission of vehicle intent and adaptive communication scheduling into a unified end-to-end learning paradigm, jointly optimising all components for safe coordination. Central to our approach is an observation prediction module that enables agents to forecast subsequent policy outputs, which they can then share across a vehicle-to-vehicle network. Our model-free architecture employs a Graph Attention Network to process incoming messages and a scheduler component to dynamically optimise the communication graph for bandwidth efficiency. We compare IntNet against state-of-the-art communicative MARL methods in complex urban environments with both autonomous and human-operated vehicles, achieving improved coordination, lower collision rates, and reducing communication efforts by up to 60\%. Through extensive experiments, we assess the framework's learning efficiency, scalability, and the balance between information sharing and bandwidth usage for collision-free trajectories.
★ Trajectory Generation and Control for Quadrotors
In this project, I explored controller design and trajectory generation for a quadrotor. Differential flatness was used for trajectory generation based on one of the references. A minimum snap trajectory is generated from the given algorithm based on waypoint navigation, and a simple proportional plus derivative (PD) controller is used to track the trajectories. Simulation results are presented which generate smooth trajectories for position and velocity of the quadrotor.
comment: 8 pages, 6 figures
★ Nonlinear Control of Aerial Manipulator
This paper presents two controllers for a quadrotor equipped with a two-DOF robot manipulator that physically interacts with the environment. Cartesian impedance and adaptive sliding mode controllers are implemented and compared for a simple hovering task in the presence of some bounded disturbance. A full dynamic model is developed, and the two above controllers are implemented with the same initial conditions and desired position.
comment: 5 pages, 9 figures, 2 tables
★ Scaling Dexterous Manipulation via Action Representation Pretraining
Large-scale vision, language, and video pretraining has substantially improved the perception and generalization capabilities of robotic manipulation models, but action generation for complex dexterous manipulation remains constrained by the scarcity of robot demonstrations. Limited robot demonstrations must be used to learn both action representations and task-conditioned action distributions, making it difficult to adequately learn action representations and, in turn, limiting the ability of downstream generative models to generate complex dexterous actions. To address this limitation, we propose ARPG (Action Representation Pretraining and Generation), which explicitly separates action representation learning from action generation: we first pretrain an action representation encoder on large-scale human manipulation data. We then freeze the encoder and train a compact generative backbone (130M parameters) to learn task-conditioned action distributions in the encoder's representation space. Experimental results show that ARPG achieves state-of-the-art performance on the DexJoCo benchmark and improves the average success rate on bimanual dexterous manipulation from 10.1% for $π_{0.5}$ to 44.7%. Scaling experiments further show that broader coverage of human manipulation behaviors effectively improves downstream performance. These results indicate that action representation pretraining provides an independent scaling dimension for improving complex dexterous manipulation capabilities.
comment: 23 pages. Project page: https://arpg-project.github.io/
★ AuRA: From Atoms to Bits through Autonomous Robot Interaction
Simulation enables robot-learning experiments at scale, but acquiring physics-informed digital twins often requires human manipulation and manual annotation of recorded interactions. To reduce this reliance on human effort, we present AuRA, a system that automates robot interaction and data preparation for digital-twin reconstruction. AuRA combines execution-aware grasp selection, gripper-motion measurements derived from robot states, and registered pre-interaction geometry and appearance priors. These choices enable autonomous collection, avoid reliance on visual gripper tracking, and recover object information obscured by the robot. Across 13 deformable objects, AuRA reduces active human labor and wall-clock collection time per successfully reconstructed asset by 88.5% and 55.4%, respectively. Relative to the human-collected PhysTwin baseline, AuRA improves seven of nine metrics for future prediction under known gripper motion, including a 28.1% reduction in one-way $L_1$ Chamfer distance. It also achieves similar mean silhouette IoU and Chamfer distance under the evaluated unseen interactions. These results highlight the potential of autonomous robot interaction to support more efficient digital-twin acquisition with comparable reconstruction quality.
comment: Project page: https://aura-robot.github.io/
★ On Distributed Sensing and Adversarial Localization of Narrowband Acoustic Sources for Offshore Data Center Monitoring
This report investigates an acoustic surveillance framework for localizing and tracking narrowband adversarial acoustic sources that threaten offshore data centers. Unlike conventional sound source localization (SSL) methods that rely on broadband signals and compact receiver arrays, the considered scenario involves narrowband emissions and sparse sensors. In such scenarios, the observed time-difference-of-arrival (TDOA) measurements are aliased and produce an ambiguous solution space, rendering conventional localization methods ineffective. To address this challenge, we integrate a mobile receiver that temporarily forms anchor pairs with selected fixed receivers, constraining the feasible solution space. The proposed two-stage pipeline first estimates the initial source position using anchor constraints, then initializes a particle filter that jointly tracks source position and velocity using one live anchor measurement and quality-adaptive fixed-pair measurements. We evaluate the proposed method through Monte Carlo acoustic simulations, ROS-Gazebo robotic demonstrations, and open-water field experiments. Our framework achieves a localization success rate of 91% in simulation, while 9 of 10 field trials show tracking error below 2m under receiver-position uncertainty and noisy sensing conditions. These findings demonstrate the feasibility of passive acoustic surveillance for protecting large-scale infrastructure against acoustic threats.
comment: Technical report, 19 pages, 11 figures
★ ThinkLocoManip: A Whole-Body Control Agentic Harness for Legged Loco-Manipulation
In this paper, we propose ThinkLocoManip, an agentic harness for legged loco-manipulation designed to tackle complex, long-horizon tasks in large-scale environments. Existing approaches typically follow either direct action generation or a plan-then-act paradigm, both of which can struggle with the generalization and error-recovery capabilities required for extended task execution. In contrast, ThinkLocoManip adopts an interleaved reasoning-and-acting paradigm, allowing the agent to continuously reason about its current state, execute actions, observe their outcomes, and adapt its strategy accordingly. Building on this paradigm, our agentic harness orchestrates a suite of specialized tools for perception, navigation, manipulation, and locomotion, enabling flexible composition of heterogeneous capabilities. Unlike existing robotic agentic harnesses, ThinkLocoManip explicitly incorporates locomotion as a tool domain for agent reasoning and execution. We further design a set of skills that explicitly account for both tool usage within each functional domain and coordination across domains, supporting effective execution of complex behaviors over long horizons. To tackle task failure during long-horizon interactions, we develop keyframe thinking mechanism to preserve critical execution context and support efficient error recovery. Extensive real-world experiments demonstrate that ThinkLocoManip substantially outperforms representative methods based on both direct action generation and plan-then-act paradigms across challenging long-horizon legged loco-manipulation tasks. Notably, ThinkLocoManip achieves a success rate of 77%, more than twice that of the strongest plan-then-act baseline (37.1%). Our experiments further validate the effectiveness of interleaved reasoning and acting, tool orchestration, and coordinated skill execution.
★ EAST: Embodied AI Safety Testing via Systematic Fault Injection in Digital Twins
Task success is not a sufficient measure of safety for embodied artificial intelligence (AI), because robots may complete objectives while violating safety requirements. Existing evaluations often emphasize performance and robustness, but lack a systematic route from documented safety hazards to executable, verifiable tests. We present EAST (Embodied AI Safety Testing), a digital-twin framework that converts literature-grounded safety evidence into fault-injection tests across perception, instructions, environments, actions, and timing. EAST combines an expert-reviewed safety-fault taxonomy and eight executable operators with applicability and validity constraints. Its three-stage workflow generates test specifications, verifies fault realization during simulated execution, and independently assesses safety challenges, violations, task outcomes, and behavioral signatures using simulator-state evidence. The framework preserves execution traces and replayable records, supporting attribution and reproducibility across experiments. We evaluate EAST on six RoboTwin manipulation tasks and two policy architectures, ACT and RDT, against alternative testing strategies under matched budgets. Across five comparative campaigns, EAST targets all 74 admissible task-policy-operator combinations and obtains valid evidence for 63, while achieving safety-challenge exposure rates close to Random-Legal and Rule-Based. Moreover, 82 valid EAST executions complete their tasks despite safety violations, revealing risks concealed by success-oriented evaluation. These findings support systematic, validity-aware, reproducible safety testing beyond task-completion metrics.
★ RynnVLA-Latent: Scaling Robotic Manipulation Foundation Models with 3D-Grounded Latent Actions
Scaling vision-language-action (VLA) models is constrained by scarce action-labeled demonstrations and incompatible control spaces across robot embodiments. We present RynnVLA-Latent, a framework that turns heterogeneous interaction videos into scalable supervision for robot policy pretraining. At its core, RynnLAM learns continuous, 3D-grounded latent actions from visual transitions without requiring native robot controls. By combining residual future-feature prediction, camera-compensated 3D-flow supervision, and camera-motion decoupling, RynnLAM encourages representations that capture control-relevant motion rather than appearance changes alone. We apply RynnLAM to 30.1K hours of human, robot, and simulation videos, producing 745.52M latent-action descriptors. RynnVLA-Latent then learns to predict latent-action chunks from visual observations and language through flow matching, before adapting its pretrained backbone and action expert to embodiment-specific robot actions. The resulting 4B policy achieves 98.8% success on LIBERO, 84.7% on LIBERO-Plus without benchmark-specific adaptation, 51.9% average success across five VLABench tracks, and 88.6% average success on the RoboTwin 2.0 bimanual benchmark. It also achieves the highest observed success rates among the compared methods on six real-world manipulation tasks across dexterous-hand and mobile-gripper platforms. Together with representation ablations and data-scaling results, these findings demonstrate the potential of latent-action supervision for scaling robot policy learning beyond native action-labeled data.
comment: Project page: https://alibaba-damo-academy.github.io/RynnVLA-Latent.github.io/, GitHub: https://github.com/alibaba-damo-academy/RynnVLA-Latent
★ ScheNav: Schematic-Guided Object Goal Navigation
Object goal navigation requires an embodied agent to locate and reach target objects in complex environments. Humans often rely on low-cost yet informative schematic maps, such as rough sketches or directional annotations. We develop a workflow for schematic-guided embodied navigation with two components: a training-data generation pipeline and ScheNav, a closed-loop navigation framework. Our lightweight data-generation pipeline turns 2D floorplans into navigation training environments, yielding Furnished-RPLAN, a dataset of over 33k furnished layouts with paired navigation episodes and noise-controllable schematic maps. ScheNav turns rough schematic guidance into exploration and stopping decisions by using SchemaField, a learned spatial module that fuses the guidance with accumulated observations and predicts spatial value fields. The framework updates these fields as exploration proceeds, with trajectory-based training and a cross-frame consistency objective supporting coherent guidance across changing viewpoints. ScheNav learns to use schematic guidance entirely from simulated floorplans and transfers this ability to unseen embodied environments. Experiments show improved navigation success and path efficiency across schematic maps, goal ellipses, and pointed directions, together with tolerance to the tested schematic perturbations and omissions. Together, these results demonstrate that rough schematic guidance offers a practical way to improve embodied navigation.
★ AdaPolicy: Action-Driven Anticipatory Robot Policies via Next-Chunk Prediction
Endowing robot policies with anticipatory capabilities has proven effective in improving control performance. However, existing methods predominantly rely on explicit supervision derived from future observations, in either pixel or latent space, at the cost of additional architectural and computational complexity. Departing from explicit future supervision, we ask whether such foresight can be elicited directly from the action signal. To this end, we propose \textbf{AdaPolicy}, an \textbf{A}ction-\textbf{d}riven \textbf{a}nticipatory policy framework that shapes future-aware representations via cascaded next-chunk prediction. Specifically, an Online Action Head jointly produces the current action chunk and an evolved latent state. Building on this state, a cascaded Unrolled Action Head forecasts the next chunk, and the resulting loss flows back through the evolved state to instill implicit foresight into the online policy. Importantly, this training-only unroll requires no future-observation encoding, keeping AdaPolicy lightweight to train with zero inference-time overhead. Experiments on MetaWorld, RoboTwin 2.0, LIBERO, and real-world deployments demonstrate consistent gains across diverse settings, including up to $14$ percentage points on millimeter-precision tasks such as earphone assembly and shoelace threading. More details and visualizations are available at \href{https://teee111.github.io/AdaPolicy/}{https://teee111.github.io/AdaPolicy/}.
★ Robust Single-Parameter Point Cloud Registration using Truncated Lp Functions
We propose a novel method for robust point cloud registration with only one tunable parameter. Our method, dubbed TLP, uses truncated $L_p$ functions with different values of $p$ to assign priority scores to correspondences, select the hypothesis, and refine the final model. By leveraging the robustness of truncated functions, TLP effectively handles the high outlier rates commonly encountered in real-world 3D feature matching. We also exploit GPU parallelization to evaluate hypotheses efficiently, enabling registration from thousands of correspondences in under a second on average. With only one tunable parameter, our method offers greater practicality than multi-parameter methods that require dataset-specific tuning of several parameters. Evaluations on public benchmark datasets demonstrate that TLP achieves higher accuracy than state-of-the-art single-parameter methods. Furthermore, it outperforms two widely used multi-parameter methods on three of the four evaluated datasets while maintaining competitive runtime.
★ HuGe: Scaling Humanoid Generalist Vision-Language-Action Models with Egocentric Experience
Extending Vision-Language-Action models (VLAs) to humanoid robots requires coordinated whole-body control and diverse demonstrations that are costly to collect in real-world scenarios. Although egocentric human demonstrations provide a complementary source of task experience, differences in visual embodiment and missing robot proprioception complicate transfer to humanoid policies. To address these challenges, we introduce HuGe, a family of Humanoid Generalist VLAs pre-trained on over 2,000 hours of egocentric human and robot data. Specifically, HuGe employs three existing representative VLA architectures to predict latent motion tokens for a pretrained whole-body controller, together with hand-control signals. Based on this, we introduce a three-stage training pipeline that combines egocentric pre-training, humanoid adaptation, and downstream fine-tuning. To facilitate transfer across embodiments, we construct pseudo-proprioceptive states through simulation playback during pre-training and modify arm appearance in egocentric observations through re-rendering during downstream fine-tuning. With this pipeline, HuGe achieves the state-of-the-art on simulation benchmark HumanoidArena. Furthermore, real-world experiments on the Unitree G1 indicate that augmenting robot demonstrations with visually adapted egocentric demonstrations significantly increases success on a pick-and-drop task.
★ Execution Logs Sway VLM Judges of Robot Progress SP
Vision-language models now grade robot videos, and text placed beside the frames can change their verdicts. On clips where the robot is losing progress, we add one log line saying the robot is making progress. Across Gemini, Qwen and GPT judges, this flips correct verdicts to progress 10 to 35 percentage points more often than a neutral line does. A one-line warning that the log may be wrong removes most of this effect. Selected fake "task complete" claims flip half of Qwen's correct verdicts, against a tenth for unrelated log lines. They have no detectable effect on Gemini.
comment: SPAIS Workshop @ CoRL 2026
★ PS-DLO: Property- and Spatially Aware Deformable Linear Object Manipulation Policy
Deformable linear objects (DLO) pose a coupled perception and control challenge: their slender geometry makes precise localization difficult, while their deformations depend on object-specific physical properties. To address this challenge, we propose PS-DLO, a property- and spatially aware DLO manipulation policy. First, contrastive image-text learning maps the initial observation to categorical descriptions of DLO properties, providing explicit physical and structural priors for action generation. Second, SAM3 extracts mask-guided features from future observations, which are distilled into the policy's action expert to encode task-relevant DLO regions and their subsequent configurations. We evaluate PS-DLO on nine real-world tasks across single-arm and bimanual platforms, training a shared multi-task policy for each embodiment. PS-DLO achieves an average success rate of 68.9%, exceeding the strongest baseline by 20.0 percentage points. In generalization experiments, PS-DLO achieves higher success rates than the baseline in six of seven visual-shift conditions on butterfly-knot untying. These results demonstrate that combining object-property priors with future spatial supervision improves multi-task DLO manipulation and robustness to visual variation.
★ Semantic-Coupled Perception Framework for Autonomous Laparoscope Control
In minimally invasive surgery (MIS), maintaining an appropriate field of view (FoV) is essential for surgical safety and efficiency. However, conventional autonomous laparoscope control methods often treat visual tracking and surgical phase recognition independently, limiting robustness under occlusion, entity overlap, and camera motion. This paper proposes a closed-loop autonomous laparoscope control framework that couples real-time surgical phase recognition with camera-motion-compensated visual perception. By exploiting phase-dependent relationships among surgical entities, the framework improves target state estimation under challenging visual conditions, supporting task-aware autonomous FoV control. Experiments on benchmark datasets evaluate the proposed framework in surgical phase recognition and target tracking, including scenarios involving occlusion and entity overlap. Phantom experiments further demonstrate the feasibility of autonomous FoV control, showing improved target visibility and task execution performance compared with fixed-FoV control. These results highlight the potential of incorporating task-level semantics into visual state estimation for autonomous laparoscope guidance.
comment: 8 pages, 5 figures. Submitted to IEEE Robotics and Automation Letters (RA-L) for possible publication
★ HumanoidNDM: Closing the Training-Deployment Gap of Motion Tracking via Humanoid Neural Dynamics Model
Humanoid robots have recently achieved remarkable success in motion tracking. However, during real-world deployment, robots inevitably encounter dynamics gaps arising from either extra physical disturbances or inaccurate simulation modeling, resulting in a significant training-deployment gap. Therefore, effectively leveraging real-world data to fine-tune policies represents the crucial last mile for robust deployment. To bridge this gap, we propose HumanoidNDM, an online fine-tuning framework for humanoid motion tracking driven by a real-world-trained neural dynamics model. First, in the data collection stage, we introduce a low-frequency reference perturbation method with an adaptive intensity mechanism, balancing deployment feasibility and exploration efficiency. Second, for the model training stage, we train a robust history-based, proprioceptive neural dynamics model in Euclidean space by incorporating differentiable forward kinematics, eliminating the reliance on costly motion capture systems. Finally, in the policy fine-tuning stage, we directly fine-tune the policy via backpropagated model gradients, achieving high data efficiency. Through physical validation on the Unitree G1 robot, we demonstrate that HumanoidNDM enables rapid real-world adaptation to various perturbations, representing the first online, MoCap-free fine-tuning framework for humanoid tracking policies.
comment: Accepted to CoRL 2026
★ RoboDuo: Attempt, Check and Atomic Recovery through Code for Vision-Language-Action Models
Vision-Language-Action (VLA) models map a language instruction and the current observation directly to actions, without checking whether the instruction has been completed. For long-horizon tasks, a single failed step therefore goes unnoticed and leaves the whole task unfinished. Such failed episodes, however, often end with the task partially complete. Moreover, the unfinished steps are reusable atomic skills that recur across demonstrations. A failed attempt can thus be recovered by composing with atomic skills. We propose RoboDuo, a closed-loop dual-system framework that uses code as the interface between a coding agent and two VLAs. The agent writes a program that lets a full-task VLA attempt the whole task and checks whether it is complete; on failure, an atomic VLA finishes the remaining steps. Since the agent emits no actions and only decides which VLA for execution, RoboDuo wraps an existing full-task VLA without updating its weights. To train the atomic VLA and the coding agent with the completion-checking ability, we build LIBERO-Atomic, LIBERO-Code, and LIBERO-Judge from a single set of LIBERO demonstrations. RoboDuo consistently improves X-VLA and $π_{0.5}$ on LIBERO and LIBERO-PRO, with the largest gains on long-horizon tasks, and recovers failed rollouts of a frozen full-task VLA on a real robot.
★ X-World: Transferring Video Priors to Variable-View Robot Control
Video diffusion models provide a source of interaction representations for robot control, but transferring these representations requires reconciling variable camera inputs with heterogeneous action interfaces and closed-loop execution. We present X-World, a framework that transfers a video prior through a fixed-length token sequence to a shared action expert. A residual cross-view attention branch extends a pretrained video backbone to one, two, or three views while retaining its original self-attention path. A layer-aligned Q-Former aggregates backbone features, and dataset-specific modules map between native state and action interfaces and the shared action expert. Training separates video adaptation from action-only pretraining by freezing the video backbone in the latter stage, followed by joint target-domain post-training. The policy extracts video features with a single backbone pass and uses asynchronous action chunking for execution. On LIBERO, X-World reaches 98.5\% mean success; its LIBERO-trained checkpoint obtains 76.5\% on LIBERO-Plus without further adaptation. The seven-shift breakdown reveals gains in robot, language, lighting, and layout perturbations, alongside weaknesses under camera, background, and noise shifts relative to OpenVLA-OFT. On six dual-arm real-world tasks, it achieves 81.7\% mean success over 20 trials per task. Removing video adaptation or cross-view fusion produces the largest observed success drops among the tested removals.
★ Doppler-aided Position-Velocity Tracking of Hidden Objects using mmWave Radar
Safe indoor operation requires autonomous tracking of objects that are not directly visible to the robot. Optical non-line-of-sight (NLoS) methods depend on ambient lighting and scene geometry or on weak multi-bounce returns that need long acquisitions. Millimeter-wave (mmWave) radar offers an attractive alternative: walls act as reflectors for mmWave signals, and frequency-modulated continuous-wave (FMCW) radars enable fast, practical measurement of range, angle, and velocity. However, the low resolution of these radars leads to spatially ambiguous observations. The Doppler information, although particularly sensitive to target motion, remains underexplored in indoor NLoS localization: existing methods emphasize spatial representations and either collapse or discard the Doppler dimension before learning. To leverage the complete Range-Azimuth-Doppler (RAD) information, we acquire the first indoor NLoS radar dataset that pairs sequential RAD measurements with synchronized position and velocity ground truth under controlled variations in motion: 576 sequences totaling 720,856 frames across 12 trajectories of a mobile robot moving along varying speeds, starting positions, and material configurations. We then formulate NLoS tracking as sequential position-velocity estimation rather than frame-wise localization: a shared 3-D encoder processes the full RAD representation of consecutive frames, separate recurrent branches estimate position and velocity, and post-inference refinement combines the two into a motion-consistent trajectory. Our method achieves 8.95cm position and 1.75cm/s velocity errors on held-out trajectories and generalizes to unseen scenarios involving a moving human target. Together, our dataset and tracker establish a benchmark for tracking hidden moving targets with a low-cost radar.
comment: 8 pages, 8 figures
★ DSGN: A Dual-Stream Gated Network for Scalable Simultaneous Surveillance and Containment
The simultaneous surveillance and containment (SSC) problem arises in multi-robot applications such as perimeter security and environmental monitoring, where robots must survey a region of interest and coordinate to enclose targets once they are detected. Surveillance favors spatial dispersion for coverage, whereas containment requires target-centric cohesion for enclosure. This conflict makes heuristic switching unstable, and the instability becomes more pronounced as larger robotic networks introduce more inter-agent and agent-target relations. End-to-end DRL reduces reliance on hand-crafted switching, but conventional single-stream policies often mix surveillance and containment cues in one shared feature space that could entangle dispersion-oriented and target-centric signals and weaken task transitions. In this paper, we propose the Dual-Stream Gated Network (DSGN), an architecture designed to address scale-amplified task-transition difficulties by separating surveillance and containment into specialized representation streams and coordinating them through a context-aware gating mechanism. A probabilistic prior further regularizes the gating process, reducing ambiguous intermediate decisions and supporting clearer behavioral transitions under large-scale interaction complexity. Evaluations show that DSGN maintains strong task efficacy and scalability under constant-density cross-scale settings. Additional ablation and phase-specific evaluations show that the proposed dynamic gating mechanism and probabilistic prior support effective task arbitration and reliable mode switching in both pure surveillance and task-switching scenarios. The source code is available at https://github.com/zuozysama/DSGN.
comment: 10 pages, 6 figures, 8 tables. Full version of the paper accepted for publication in IEEE Robotics and Automation Letters (RA-L), October 2026
★ Learning Interaction-Aware Path Corrections from Unlabeled Trajectories: A LiDAR-Based Sim-to-Real GNN Framework for Construction Robots
Construction robots operating near workers require navigation that can anticipate human motion while remaining robust to dynamic and unstructured site conditions. However, existing social navigation methods depend on hand-crafted interaction rules, labeled demonstrations, or robot-specific control policies, limiting their scalability and transferability in construction environments. We propose a real-time, LiDAR-only framework that learns interaction-aware path corrections from unlabeled demonstrations. Constant-velocity extrapolation generates residual learning targets, while a GNN models relationships among the robot, pedestrians, and goal context. During deployment, an extended Kalman filter tracks LiDAR detections to construct dynamic graphs, and the GNN offsets planner-generated waypoints through a planner-agnostic interface. Within the bricklaying trial, the GNN reduced path irregularity by 39.2\% and increased minimum TTC. Across Dynablox sequences, it increased minimum TTC by 38.3\% while producing shorter, smoother paths than the baselines. Ablation studies supported a 1.0-s prediction interval and confirmed the benefits of target context, reflection augmentation, and feature normalization.
comment: 12 pages, 8 figures, submitted to Automation in Construction
★ Lightweight Change Detection for Adaptive Collective Perception in Dynamic Environments
Swarm robotics enables scalable inspection through collective perception, where robots infer environmental properties from distributed observations. A common benchmark considers robots exploring black-white tiled environments and determining the majority tile type. While most work studies static environments, dynamic environments remain underexplored. We study collective perception in dynamic environments where the tile proportion and the spatial correlation of surface patterns may change over time. To enable implementation on resource-constrained robots, we integrate a lightweight change detection mechanism into a Bayesian collective decision-making framework that resets robot beliefs when distribution shifts are detected. The proposed detector requires constant memory and constant per-sample computation, unlike two recent competitive approaches that store the full sample stream, making it suitable for memory-constrained robots. We further introduce a method that detects spatially correlated sampling and adapts sampling intervals online to mitigate correlation effects. Our simulation results in Webots show competitive change detection delays, more consistent precision across communication settings, and improved decision accuracy under changing spatial correlations.
★ Extended LASMP: Language Aided Subset Sampling Based Motion Planner for Drone Navigation
This paper proposes a command-aligned sampling distribution that incorporates navigation commands directly into the exploration process to improve the efficiency of sampling-based motion planning for quadcopters in three-dimensional environments. Inspired by the Language-Aided Subset Sampling Motion Planner, this paper extends command-aware subset sampling to aerial systems. Subset sampling is generalized to command-aligned regions, enabling structured 3D exploration while preserving probabilistic completeness. Natural language instructions are grounded into yaw-conditioned and altitude-aware geometric constraints using a prompt-based large language model for entity extraction and instruction validation. Extensive simulations in complex environments demonstrate that the proposed method generates collision-free, consistent trajectories while significantly reducing sampling effort compared to uniform RRT-based planners.
★ Behavior Spatial Benchmark: Evaluating Spatial Understanding for Robotics
Robot tasks require grounding spatial language in physical scenes, but task success alone cannot establish whether spatial instructions are followed. We present BEHAVIOR-Spatial, a benchmark built on BEHAVIOR-1K that measures spatial question answering, point-based target grounding, and robot action to examine how VLM spatial understanding relates to VLA behavior. It comprises 3,256 spatial questions across 49 activities, 1,240 grounding queries across 14 manipulation tasks, and action evaluations covering those 14 tasks and 10 navigation tasks. Our manipulation protocol evaluates held-out configurations and instruction following when the requested target changes within a fixed training layout. The baseline achieves 41.6% on held-out configurations but only 16.1% under changed instructions. VLM adaptation and robot-data pretraining provide limited improvements, whereas oracle visual target cues raise these rates to 60.3% and 38.0%, respectively. Perception evaluations reveal policy-dependent correlations between VLM question-answering and grounding accuracy and VLA manipulation performance. Together, these findings motivate evaluating spatial instruction following alongside task completion and examining when spatial perception performance translates into robot behavior. Project website: https://arielchen07.github.io/behavior-spatial/.
★ T-ConSG: A LiDAR-Based Real-Time Temporal Construction Scene Graph for Dynamic and Evolving Environments
Dynamic and continuously evolving environments present significant challenges for robotic mapping and scene understanding, particularly on construction sites where moving agents, temporary obstacles, and persistent structural changes coexist. While dynamic and long-term SLAM methods can detect motion and environmental changes, their outputs are predominantly metric representations that provide limited structure for higher-level spatial and temporal reasoning. This paper presents a LiDAR-based real-time temporal construction scene graph (T-ConSG) that jointly represents free-space regions, static objects, dynamic objects, and environmental events with spatial-relational and event-participation edges. The proposed framework incrementally constructs the graph from LiDAR-derived geometric evidence, maintaining persistent free-space and static-object information while tracking dynamic objects. Changes in accumulated spatial evidence are further represented as explicit installation and removal events, allowing the graph to capture both transient dynamics and persistent environmental evolution. We demonstrate the proposed representation through three complementary application studies: dynamics-aware navigation in a bricklaying testbed, construction progress monitoring on the ConSLAM dataset, and online change detection in a simulated construction environment. The results demonstrate that the proposed T-ConSG provides a unified representation for organizing spatial and temporal information and supports robotic reasoning across environments containing both short-term dynamics and persistent changes.
comment: 8 pages, 7 figures, submitted to IEEE Robotics and Automation Letters (RA-L)
★ Cut a Wall, See a Wall: Identity Substitution in Occluded Point-Cloud Perception for Construction Robots
Construction robots must label building elements they can only partly see. We show that when occlusion at test time removes whole regions of a scene, rather than parts of an object as in training, point-cloud segmentation networks do not simply miss the hidden elements: they relabel them, and the errors collapse onto a single sink class. With fixed synthetic layouts the column absorbs up to 70% of all test points; with randomized layouts the sink moves to the stair, predicted at 4.5-5.4x its true share across three training seeds. Baseline mIoU falls to 0.15-0.27 (fixed) and 0.36-0.72 (randomized), and up to 269 whole objects vanish in a single test set. Appending 14 hand-computed geometric features per point (GPE) restores mIoU to 0.54-0.92 and removes 64-100% of whole-object losses in every run, for PointNet++ and a PTv3-style transformer in both layout regimes; on real S3DIS scans it removes every whole-wall loss (10 to 0). In an as-built column check computed from the predictions, GPE raises the share of correctly checked views from 17-58% to 71-92%. Feature permutation on fixed, randomized and real scenes explains the pattern: each class relies on whichever cue is stable in its training data, position for height-banded slabs and beams, intrinsic shape for walls and stairs once their position varies. Occlusion augmentation is a strong alternative on mIoU, but in every setting it loses more whole objects than GPE (e.g., 25-35 vs. 0).
comment: 6 pages, 4 figures, 5 tables
★ Enabling Motion-Induced WiFi-CSI Sensing in Multi-Robot Systems: A Simulation Framework
WiFi Channel State Information (CSI) enables fine-grained sensing of spatial relationships between wireless devices. The WiFi Sensor for Robotics (WSR) toolbox demonstrated that WiFi-CSI measurements collected during robot motion, combined with odometry, can be used to synthesize virtual antenna arrays, enabling motion-induced bearing estimation in multi-robot experiments. However, deploying the WSR approach has remained challenging due to its limited integration with standard robotics tools and reliance on deprecated WiFi-CSI acquisition hardware. This work presents an open-source simulation framework that integrates the WSR sensing pipeline within the robotic simulator Gazebo. The framework generates synthetic WiFi-CSI measurements with realistic physical signal properties and leverages realistic robot motion required for motion-induced sensing and downstream localization. It further supports processing both simulated and real WiFi-CSI data for direct comparison and validation. Our aim is to make WSR-style sensing accessible as a modular sensing modality for robotic research. We validate our framework by comparing simulated and real-world experiments using TurtleBot3 robots, and demonstrate its use for bearing estimation and robot localization. This work lowers the barrier to entry for WiFi-CSI-based sensing in multi-robot systems and provides a foundation for integrating WiFi-CSI sensing with multi-modal perception for localization and coordination. All code and experimental details are publicly available at https://github.com/baharhaghighat/DARS_2026_Dijkstra_et_al
★ ElevateHead: Decide Once, Then Commit: When the Action Head Matters for Multimodal Imitation
Robots that learn from people often see several valid ways to do the same task: pass an obstacle on the left or on the right, open the microwave or move the kettle first. A policy that re-decides at every step can waver between these ways mid-task. We ask when the action head (the part of a policy that turns the state into actions) matters, and how it should make this choice. On a controlled task, heads that sample one way succeed far more often than a head that averages the ways when the demonstrations contain several ways and the state gives no hint which to take (+0.72 success with two ways); this trend was not reproduced on the other tasks we ran. We then present ElevateHead, a head that decides once at a learned decision point, commits to that choice for the episode, and exposes the choice as an input a user can set. We designed it by diagnosis: each version was frozen and tested on tasks it had not seen, and the next version changed only what a measured failure pointed to; predictions were registered before each run. The first four versions each fell below BeT on at least one held-out task. The fifth completed all four subtasks of the held-out Franka Kitchen task in 45.2% of episodes against 29.3% for BeT, the strongest baseline across our tasks (+0.160, 95% CI [+0.129, +0.188]); when a mode was forced, the first completed subtask matched it in 95% of successful episodes. Diagnosis also located where commitment breaks: varied start positions (repaired on the controlled task: commitment 0.22 to 0.81, success 0.74 to 0.91) and varied object placement (partly repaired: in an exploratory variant, commitment 0.03 to 0.59 on aligning with no detectable change in success). On held-out block stacking, version 4 was comparable to BeT and did not exceed it. A real-robot test with predictions fixed in advance is planned for a follow-up version of this paper.
comment: 8 pages, 7 figures, 7 tables, including appendix
★ I Won't Know Until I Try: Interactive Information Gathering for Long-Horizon Manipulation
Long-horizon manipulation often depends on object properties that cannot be reliably inferred from appearance alone. Physical interaction can reveal these properties, but it can also change the world in ways that invalidate subsequent actions. We present a task and motion planning (TAMP) framework that integrates task-directed interactive information-gathering with interleaved planning and execution. Building on PDDLStream, our framework plans optimistically with these interactive actions, verifies their outcomes during execution, and adapts the remaining plan by retaining still-applicable actions and replanning only the connections needed to continue execution. We evaluate the framework in simulation and on a real robot across tasks that require determining whether an object is movable, a pen has ink, a drawer can be opened, or a container is empty. These experiments demonstrate how interaction-driven state updates and recovery replanning support continued execution under uncertain outcomes and external disturbances.
comment: 8 pages. Under review
♻ ★ Shooting for Contact: Contact-Implicit Multiple Shooting for Dynamic Motion Retargeting
Motion retargeting approaches often prioritize kinematic similarity over whole-body dynamics, contact consistency, and actuation limits, yielding references that are difficult for reinforcement learning (RL) policies to reproduce, particularly for contact-rich behaviors. We present a contact-implicit, direct simulation-based multiple shooting (DSMS) framework that transforms kinematically feasible references into dynamically feasible whole-body trajectories. By embedding a differentiable simulator within a nonlinear program, DSMS resolves contact, friction, impacts, self-collision, and joint limits internally while enforcing tracking, actuation, and task constraints without prescribing a contact schedule or introducing explicit contact constraints. Compared with existing retargeting methods, DSMS accelerates motion-imitation RL training and yields policies with high success rates and low tracking error. We further demonstrate zero-shot sim-to-real transfer on the Unitree G1 through command-conditioned walking, jogging, and contact-rich crawling, and a highly dynamic 180-degree jump-turn.
comment: Project website with additional material: https://shooting-for-contact.github.io/
♻ ★ CRAFT-E: An Interpretable and Editable Neuro-Symbolic Framework for Affordance-Grounded Active Object Retrieval WACV 2027
Assistive robots operating in unstructured environments must reason not only about what objects are, but how they can be used, including non-canonical and improvised uses. This requires grounding language-based action queries to objects that afford the requested function, are visually present, and can be physically retrieved. Existing approaches often rely on black-box models, fixed affordance labels, or direct object predictions, limiting transparency and controllability. We introduce CRAFT-E, a modular neuro-symbolic framework that composes an LLM-derived verb-property-object knowledge graph with CLIP-based visual alignment and energy-based grasp reasoning. The system exposes interpretable grounding paths, incorporates grasp feasibility into affordance inference, and supports targeted graph edits without retraining. We also construct a benchmark dataset with unified annotations for verb-object compatibility, segmentation, and grasp candidates, and deploy the full pipeline on a physical robot. Across static scenes, real-world trials involving 20 verbs and 39 objects, ImageNet-based functional retrieval, and COCO-Tasks transfer, CRAFT-E provides a competitive training-free prior for affordance-grounded retrieval while exposing component-level diagnostics and graph-level interventions. Under staged and full-pipeline perception, CRAFT-E matches strong flat LLM priors in several settings while adding failure attribution and editable reasoning paths. By coupling symbolic reasoning with embodied perception, CRAFT-E offers an inspectable and customizable framework for affordance-grounded object retrieval.
comment: WACV 2027. Project page: https://zzc0053.github.io/CRAFT-E-page/
♻ ★ CLARO: Clutter Aware Robot Orchestrator for Open-Vocabulary Grasping using VLM Agents WACV 2027
Robots that follow natural-language instructions unlock a wide range of autonomous applications in households, warehouses, and labs. Real-world environments are diverse and often heavily cluttered, requiring robots to plan over multiple steps, act, and replan as they work toward a goal. We present CLARO (CLutter-Aware Robot Orchestrator), a VLM agent that orchestrates a set of perception and manipulation skills: open-vocabulary object detection, segmentation, 6-DoF grasping, and non-prehensile pushing to locate and retrieve a language-specified target in clutter. CLARO maintains a memory of its past actions to support long-horizon execution, and emits manipulation targets (grasp poses/3D waypoints) allowing transfer across robot arms without task-specific training. We evaluate CLARO against a range of methods on a dynamic simulation environment and standard image-based benchmarks, demonstrating strong generalization across object classes, lighting, and clutter levels. CLARO attains an average success rate of 89.1%, a relative improvement of over 16% on the strongest competing method in high cluttered scenes. We further analyze performance along grounding and grasping axes to characterize where each method succeeds and fails. Finally, we deploy CLARO on a robot arm across diverse cluttered scenes, achieving a success rate of 88.8%, and demonstrating strong sim2real generalizability.
comment: Accepted to WACV 2027
♻ ★ Designing Versatile Samples for Learned Trajectory Scoring WACV
Many current end-to-end driving policies emit a pool of candidate trajectories and select one, which makes selection a separable component: a scorer can be retrained while the planner, its backbone, and its trajectory generator all stay frozen. However, many strong planners concentrate their proposals around safe mode, providing limited supervision near decision boundaries. In this work, we design a training dataset that provides more informative supervision for the scorer. In particular, we construct two generators that perturb the logged human trajectory along the two axes a vehicle can be displaced: laterally toward the drivable boundary and longitudinally toward a leading vehicle. The designed dataset produces more informative positive and negative samples than the base planner's proposal pool. We attach a transformer-based scorer to two frozen generative planners, DiffusionDrive and MeanFuser, and train it on the NAVSIM navtrain dataset. The results of the experiments show that we achieve 90.1 EPDMS on DiffusionDrive and 90.4 EPDMS on MeanFuser when using ResNet-34, with 0.4 and 0.3 EPDMS respectively, from the designed training dataset. The code is publicly available at https://github.com/PurdueDigitalTwin/Versatile-Scoring.
comment: Accepted to IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2027
♻ ★ IndustryNav: Evaluating Spatial Reasoning and Safety of Embodied Agents in Dynamic Industrial Navigation
Visual large language models (VLLMs) are increasingly used as embodied agents, yet industrial warehouses demand more than reaching a goal: agents must navigate safely around moving workers and vehicles. To evaluate embodied agents' goal reaching and safety, we introduce IndustryNav, a simulation environment with 24 warehouse scenes for closed-loop navigation to specified destinations. Alongside task success and efficiency, we introduce safety-aware metrics that capture collisions and near-collision warnings along each trajectory. Experiments with eleven leading open- and closed-source VLLMs and three navigation baselines show that agents can reach their goals yet collide with moving obstacles, exposing a gap between success and safe navigation. This gap matters in safety-critical settings, where a single collision can end the task. IndustryNav also supports policy learning through automated oracle trajectories and simulator interaction, with held-out scenes for generalization tests. These experiments show that policies can succeed on new routes in familiar warehouses yet fail to transfer to held-out layouts, making cross-scene generalization an open challenge.
♻ ★ MPPI-based Informative Trajectory Planning for Search and Capture of Drifting Targets with ASVs
Autonomous surface vehicles (ASVs) offer an efficient solution for environmental cleanup as well as search and rescue operations in open waters. Targets in these settings drift continuously, so efficient search must balance exploration of unobserved regions with tracking of known targets. However, most target tracking and pursuit scenarios consider simple guidance behaviours and short-term predictions for decision-making. In this letter, we address the problem of search and capture of multiple drifting targets, such as litter, in marine environments, using an adaptive informative planning approach. A key aspect of our strategy is a spatiotemporal informative planning method based on model predictive path integral (MPPI) control, a sampling-based model predictive control method. Our planner balances search and tracking objectives with a cost design that couples spatiotemporal target drift predictions and exploration objectives. The planner directly generates kinematic-level commands by optimising continuous trajectories over long horizons. In the interception stage, we use pure pursuit guidance for physical capture of the targets. Simulations show that our planner captures up to 18% more targets than the strongest baseline, proving its efficiency for environmental cleanup. Finally, we validate our approach in field trials with an ASV.
comment: 8 pages; Accepted for publication in IEEE RA-L
♻ ★ Staged Multi-Agent Training for Human-Exoskeleton Co-Adaptation: Simulation-Based Learning and Real-World Metabolic and Biomechanical Validation IROS 2026
Learning-based controllers can enable exoskeleton assistance to be trained entirely in physics-based simulation, but achieving effective assistance on real users remains challenging because the human and device continuously adapt to one another. We investigate a Staged Multi-Agent Training (SMAT) approach that jointly trains a musculoskeletal human actor and a bilateral hip-exoskeleton actor (control policy) to learn coordinated assistance strategies while accounting for human-device co-adaptation. The resulting policy was transferred to a physical hip exoskeleton and evaluated in eight healthy adults across users and locomotion conditions without subject-specific retraining. Indirect calorimetry was used to compare no-exoskeleton, passive, and active conditions. Active assistance reduced net metabolic rate by 19.7% relative to the passive device (p < 0.001). The policy produced predominantly positive hip mechanical power across subjects, with a positive-power ratio of 0.984, and maintained effective assistance across walking speeds and terrains beyond the training conditions. These results demonstrate that simulation-based multi-agent co-adaptation can produce a single transferable exoskeleton policy that elicits meaningful physiological benefits in real users without subject-specific retraining, providing a framework for reducing the experimental burden of human-in-the-loop controller development.
comment: 15 pages, 9 figures, including supplementary material. Extended version of a paper to appear at IROS 2026 (arXiv:2603.07618)
♻ ★ Robot Learning with Vision-Language-Action Models for Agricultural Manipulation
Robotic manipulation in agricultural environments remains challenging due to variations in plant geometry, occlusion, and camera viewpoints. Pretrained vision-language-action (VLA) and generalist robot policies offer a promising approach to addressing these challenges by learning manipulation behaviors from multimodal observations and language instructions. However, adapting these policies to agricultural tasks requires domain-specific demonstrations, which are costly and time-consuming to collect on physical robots. In this work, we develop a simulation-to-real robot-learning framework for agricultural reaching and grasping that integrates task-specific demonstration collection and pretrained policy adaptation. A MuJoCo-based agricultural environment is developed with diverse tomato plant structures to support demonstration collection and controlled evaluation. Expert demonstrations are collected through joystick teleoperation, with a low-level controller used to execute robot actions. Two pretrained generalist robot policies, Octo and $π_{0.5}$, are adapted using simulated demonstrations, followed by further fine-tuning on a physical Franka Research 3 manipulator using real-robot demonstrations. The adapted policies are evaluated under in-distribution, viewpoint/spatial-shift, and out-of-distribution conditions in simulation, followed by physical-robot experiments. The influence of proprioceptive information on policy performance is also investigated. Results show that $π_{0.5}$ achieves higher task-performance scores than Octo and exhibits less degradation under distribution shifts. These findings demonstrate the potential of integrating pretrained generalist policies with task-specific demonstration learning for agricultural manipulation.
♻ ★ High-Speed, Scalable Sensor Readout for Dexterous Robotic Hands via Shift-Register Multiplexing
Dexterous robotic hands require high-speed multimodal sensing across many degrees of freedom, yet existing readout architectures often impose trade-offs between sensor count, wiring complexity, and sampling bandwidth. This paper presents a scalable analog sensor readout architecture based on a serial-in parallel-out (SIPO) shift-register principle. The proposed architecture supports versatile integration of heterogeneous analog-output sensors, scalable expansion using only three signal lines between sensor modules, and fast, configurable sampling. We validate the approach on a tendon-driven robotic hand integrating 16 joint sensor modules and one four-channel tactile sensor module, enabling acquisition of 20 sensor channels at a full-scan rate of 1 kHz, with stable operation up to 1.5 kHz. Joint sensor characterization showed a maximum slope absolute percentage error (APE) of 0.446% and sub-degree estimation error, indicating that the proposed readout system preserves sensing performance. For tactile sensing, LSTM-based models enabled real-time force estimation and contact-location classification at 1 kHz. System-level experiments showed that the joint sensors provided more accurate feedback than motor-based estimation during interaction. Closed-loop experiments further demonstrated improved joint trajectory tracking using joint-sensor feedback and force-controlled object pinching using tactile feedback. The proposed architecture offers a practical path toward sensorized robotic hands for dexterous manipulation.
comment: 8 pages, 8 figures. Updated to the accepted manuscript for IEEE Robotics and Automation Letters. Project website and videos: https://srl-ethz.github.io/scalable-sensor-readout/
♻ ★ Imagine the Future, Internalize the Gist: Efficient VLA Reasoning via Internalized Spatiotemporal Imagination
Vision-language-action (VLA) models increasingly incorporate intermediate reasoning to improve robotic manipulation, yet existing approaches primarily reason about observed states without explicitly anticipating future scene evolution. Extending such reasoning to explicit future rollouts at every inference step, however, introduces substantial computational overhead. We propose IG-VLA, a VLA reasoning framework that enables models to imagine the future and internalize the gist. Our Latent Spatiotemporal Reasoning learns to imagine task-relevant future scene evolution directly in visual representation space, guiding action prediction without costly pixel-level video generation. To further reduce inference overhead, we introduce Scene Gist Memory, which internalizes reasoning-derived scene-behavior associations into a compact Scene Gist Token, preserving the benefits of future reasoning while bypassing explicit future imagination at inference. Extensive experiments on LIBERO, LIBERO-Plus, and VLABench demonstrate the effectiveness and efficiency of IG-VLA. On the LIBERO-Plus Language suite, both the reasoning and gist policies outperform the strongest baseline by nearly 6% in success rate. The gist policy also achieves up to 6.38x speedup over baselines, reducing inference latency from 1081ms to 169.5ms per action chunk on a single NVIDIA A6000 GPU. These results demonstrate that future spatiotemporal reasoning can be effectively internalized for efficient VLA deployment.
♻ ★ GeoHAT: Geometry-Adaptive Hybrid Action Transformer for Mobile Manipulation
Whole-body mobile manipulation requires coordinating mobile base and manipulator under shifting viewpoints, posing challenges in geometric perception and action generation. Current policies either rely on 2D features or sparse 3D representations that lack dense spatial structure, and typically encode arm and base within one action vector that ignores their distinct control demands. Moreover, existing dense fusion strategies risk corrupting pretrained representations under noisy depth while incurring heavy computational overhead. We present GeoHAT, an end-to-end diffusion-based framework built on a simple principle: geometry should be injected only where reliable and attended to only where needed. GeoHAT employs a lightweight Fourier spatial encoder that maps dense per-pixel 3D coordinates into geometric tokens without an additional 3D vision backbone. These tokens are then selectively injected into vision foundation model features through per-token gated fusion modulated by depth validity, preserving the semantic prior while enriching spatial understanding. For action generation, a Hybrid Whole-Body Action Decoder decomposes arm and base into distinct subspaces and lets each action modality attend to its task-relevant visual context through sparse cross-attention, while causal temporal modeling captures intra-timestep coordination and inter-timestep dependencies. Experiments on the ManiSkill-HAB simulation benchmark demonstrate that GeoHAT achieves a 79.2% mean success rate, surpassing the strongest baseline by 23.6 pp. Furthermore, real-world experiments on diverse tasks also confirm consistent improvements over all baselines.
♻ ★ Toward Physical Recursive Self-Improvement: A Call for Body-Control Co-Design for Humanoid Robot NeurIPS 2026
Humanoid robots, as general-purpose physical agents, must integrate both intelligent control and adaptive morphology to operate effectively in diverse real-world environments. While recent research primarily optimizes control policies for fixed bodies, we advocate body-control co-design as a path toward physical Recursive Self-Improvement (RSI): using experience from a humanoid's current body and policy to guide successive improvements to both. Inspired by biological evolution, this approach enables robots to recursively adapt both their form and behavior to optimize performance within task-specific and resource-constrained contexts. Despite its promise, co-design in humanoid robotics remains a relatively underexplored domain, raising fundamental questions about its feasibility and necessity in achieving true embodied intelligence. To address these challenges, we discuss potential co-design methodologies grounded in strategic exploration, Sim2Real transfer, and meta-policy learning. We further argue for the essential role of co-design by analyzing it from methodological, application-driven, and community-oriented perspectives. Striving to guide and inspire future studies, we present open research questions, spanning from short-term innovations to long-term goals. Ultimately, co-design offers a path toward physical RSI in humanoids that improves their bodies and policies while learning to make this improvement process more effective.
comment: NeurIPS 2026 Position Paper Track
♻ ★ Guardian: Detecting Robotic Planning and Execution Errors with Vision-Language Models WACV2027
Robust robotic manipulation requires reliable failure detection and recovery. Although recent Vision-Language Models (VLMs) show promise in robot failure detection, their generalization is severely limited by the scarcity and narrow coverage of failure data. To address this bottleneck, we propose an automatic framework for generating diverse robotic planning and execution failures across both simulated and real-world environments. Our approach perturbs successful manipulation trajectories to synthesize failures that reflect realistic failure distributions, and leverages VLMs to produce structured step-by-step reasoning traces. This yields GuardianFail-36k, a large-scale failure reasoning dataset built upon the RLBench simulator and the BridgeDataV2 real-robot dataset. Using GuardianFail-36k, we train Guardian, a multi-view reasoning VLM for unified planning and execution verification. Guardian achieves state-of-the-art performance on three unseen real-world benchmarks: RoboFail, RoboVQA, and our newly introduced UR5-Fail. When integrated with a state-of-the-art LLM-based manipulation policy, it consistently boosts task success rates in both simulation and real-world deployment. These results demonstrate that scaling high-quality failure reasoning data is critical for improving generalization in robotic failure detection. Code, Data, and Models are available at https://www.di.ens.fr/willow/research/guardian/.
comment: Code, Data, and Models available at https://www.di.ens.fr/willow/research/guardian/. The paper contains 8 pages, 7 figures, 7 tables Accepted at WACV2027
♻ ★ Collaborative Continuum Robots: A Survey from an Automation-System Perspective
Continuum robots (CRs), owing to their compact structure, inherent compliance, and flexible deformation, have been widely applied in various fields. By coordinating multiple CRs to form collaborative continuum robots (CCRs), task adaptability, flexibility and load capacity can be further improved, thus offering significant advantages. In recent years, interest in this emerging field has grown steadily within the continuum-robotics community. In this survey, a closed-loop task-execution architecture for CCRs is constructed from an automation-system perspective, clarifying the system-level logic that transforms target-task specifications into multi-CR collaborative execution. Based on this architecture, CCRs are classified into three representative collaboration modes, namely separated collaboration, assistance collaboration, and parallel collaboration. Research progress is then systematically summarized and analyzed in terms of key technologies, including multi-CR task allocation, structural design, modeling, motion planning methods, control strategies, and sensing mechanisms. Finally, research opportunities in key technologies and system integration challenges for real-world deployment are discussed, providing guidance for research and application of CCRs.
♻ ★ LAW: Unifying Language, Action, and World Modeling for Agentic Physical Intelligence
World-action models jointly predict actions and future world states, but long-horizon manipulation further requires language reasoning to plan subtask objectives, assess execution progress, and guide action selection. We present LAW, a closed-loop VLM--WAM framework that unifies language reasoning, action generation, and world modeling through semantic subtask progress. Within the LAW agent, a hybrid-expert vision-language model plans the remaining subtasks, estimates the values of and fuses actions proposed by a world-action model, and asynchronously assesses execution status to support continued execution and subtask transitions. Within the world-action model, Mixture-of-Transformer Experts integrates modality-specific and shared computation for joint video--action modeling. Dual-stream hierarchical visual memory maintains temporal context for both models, while subtask-local value learning aligns action evaluation with semantic objectives and preserves supervision from successful prefixes of failed trajectories. LAW achieves 94.21% average success across 50 RoboTwin tasks. Replacing the WAM backbone with OpenWAM-$alpha$ further raises the average success rate to 95.56%, establishing a new open-source state of the art on RoboTwin. Despite having only 2B parameters, our VLM also outperforms variants using Embodied-R1.5 and GPT-5.5 High. On three long-horizon real-robot tasks, LAW achieves 65% average success, compared with 28.33% for $pi_{0.5}$ and 25% for Fast-WAM.
♻ ★ X-WBC: A Cross-Embodiment Foundation Model for Humanoid Whole-Body Control
Scaling humanoid whole-body control toward general-purpose deployment requires large human motion corpora and training experience shared across robot bodies. Existing methods usually train one policy per robot, leaving motion experience isolated across embodiments. We introduce X-WBC, a cross-embodiment foundation framework that separates relatively shared human motion semantics from embodiment-specific physical execution. Human-centered command tokens align full human motion, robot reference motion, and sparse VR observations. A causal Transformer learns reusable temporal structure from mixed multi-robot rollouts, while lightweight robot-specific modules map the shared representation to each robot's proprioception and action space. Across nine simulated embodiments, external motions, and four real robots, experiments show that joint training improves tracking, the aligned representation supports consistent control across command sources, and the learned policy remains competitive beyond the training corpus. These results support heterogeneous humanoids as joint data sources and establish cross-embodiment joint training as a practical route toward whole-body control foundation models.
comment: Accepted to CoRL 2026
♻ ★ Embodied Playground: A Long-Horizon Agentic Robotics Benchmark for Physical, Strategic, and Spatial Reasoning
Foundation models offer a promising basis for robotic agents, but long-horizon tasks require them to sustain coherent reasoning and reliable physical execution across interdependent stages. We introduce Embodied Playground, a benchmark for evaluating how foundation-model agents coordinate physical, strategic, and spatial reasoning throughout such tasks. Agents must track task progress, satisfy intermediate constraints, and revise their plans using execution feedback as earlier actions shape subsequent decisions. The playground comprises 410 task environments spanning basic manipulation, sequential puzzles, and multi-stage household activities, with variations in planning depth, geometric constraints, and physical execution demands. We curate EP-100, a representative selection of 100 tasks across five suites and 16 task types, together with the EP-Easy and EP-Hard difficulty splits and the compact EP-20 selection. A shared protocol specifies observation and tool access, resource budgets, repeated trials, and independent checks of goal completion, task-rule compliance, and physical validity. Embodied Playground provides a unified framework for studying whether foundation-model agents can sustain effective reasoning and action over extended task horizons. https://winniechen2002.github.io/Agent-bench-website/
comment: 38 pages, 2 figures
♻ ★ SentryVLA: Uncertainty-Guided Action Selection and State Monitoring for Vision-Language-Action Robotic Control
Vision-language-action (VLA) models have emerged as generalist robotic controllers that map visual observations and language instructions to continuous actions. However, pretrained VLAs typically provide limited information about the reliability of their generated actions, making deployment challenging when execution failures must be anticipated. We address this problem by introducing SentryVLA, a lightweight runtime wrapper for frozen VLA policies. At the action level, SentryVLA uses conformalized quantile regression to estimate a calibrated upper bound on the deviation between sampled VLA actions and expert-like behavior under the calibration distribution. The predicted deviation is then used to rank multiple stochastic candidates for action selection. At the execution-monitoring level, SentryVLA applies a Mahalanobis-distance-based anomaly score to the contextualized latent execution state extracted before final action prediction, providing a complementary signal for detecting abnormal execution states during closed-loop control. The framework does not require retraining or architectural modification of the underlying VLA policy. Experiments in simulation and on a physical robot show that SentryVLA improves action selection and provides useful runtime warning signals for abnormal execution states. These findings demonstrate the practical value of post-hoc reliability scoring for improving the runtime behavior of pretrained VLA policies. Project webpage: https://robotic-vision-lab.github.io/sentryvla/.
comment: To be published in the 2026 Conference on Robot Learning (CoRL)
♻ ★ LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies
Vision-Language-Action models (VLAs) leverage large-scale vision-language pretraining for semantic robot control, but often lack explicit foresight into how robot actions change the scene. World-Action Models (WAMs) address this limitation by conditioning policies on predicted futures, yet existing approaches typically rely on computationally expensive video generation with substantial pixel-level redundancy. We present LaWAM, a Latent World Action Model that exposes predictive dynamics to robot policies through compact latent visual subgoals instead of reconstructed future video. At the core of LaWAM is a latent-action-conditioned Latent World Model (LaWM). We obtain LaWM by training a latent action model in the latent space of a pretrained vision foundation model and repurposing its forward decoder to predict future observation features for scene evolution. LaWAM then conditions action generation on these predicted latent visual subgoals to enable dynamics-aware robot control. LaWAM achieves state-of-the-art or competitive success rates (SRs) across LIBERO (98.6% SR), RoboTwin (91.22% SR), and real-world manipulation tasks while retaining low-latency inference. LaWAM runs in 187 ms per action-chunk prediction and achieves up to 24x lower wall-clock latency than pixel-space WAMs.
comment: Accepted at CoRL 2026. Project website: https://rlinf.github.io/LaWAM/
♻ ★ H2RBench: A Real-to-Sim Benchmark for Evaluating Human-to-Robot Transfer
Learning robot manipulation policies from human video demonstrations constitutes a promising avenue for scalable robot learning. However, comparing different human-to-robot (H2R) transfer methods remains challenging, as existing approaches are evaluated under different settings, including differing task suites, scene layouts, object instances, and amounts of robot supervision. To address this challenge, we present H2RBench, a Real2Sim benchmark for evaluating H2R transfer methods. H2RBench provides a standardized protocol built on real human video demonstrations and simulated robot demonstrations, and includes four manipulation tasks spanning diverse interaction requirements. We evaluate multiple representative H2R transfer methods, each adopting a different strategy for bridging the embodiment gap. Using H2RBench, we systematically characterize how each method scales with the amount of human demonstrations, revealing that methods differ substantially in their ability to leverage additional human data. We further show that simulation performance is broadly predictive of real-world robot performance, with an overall Pearson correlation of r = 0.89, Spearman correlation of \r{ho} = 0.85 and Mean Maximum Rank Violation (MMRV) of 0.06 across method-task configurations. These results establish H2RBench as a practical and scalable benchmark for comparative H2R evaluation prior to real-world deployment.
comment: 10th Conference on Robot Learning (CoRL 2026), Austin, TX, USA
♻ ★ VLN on the Fly: An Onboard Vision-Language Navigation Stack for Aerial Robots
Running vision-language navigation fully onboard an aerial robot is hard, since grounding, planning, and control must share limited compute and a single-stage error is difficult to isolate in flight. End-to-end aerial policies fuse these stages into one network, giving up the observability and safety checks a modular stack keeps available. We propose VLN on the Fly, an onboard stack that keeps grounding, planning, and control as separate, inspectable stages. A quantized VLM grounds an instruction to a coarse image cell, depth lifts it to a 3D goal, a fast B-spline planner returns a feasible trajectory, and a pretrained reinforcement learning policy tracks it to motor commands across quadrotors. Across 15 onboard flights over three everyday referents in a controlled indoor volume, the stack reaches the target in 13 of 15 trials with 5.72 cm mean goal error and 39.3% average GPU utilization. In 6 additional cluttered-environment trials, the stack tracks collision-free trajectories under onboard perception gating.
comment: 7 pages, 5 figures. Project page: https://vln-on-the-fly.github.io/
♻ ★ DiVeR: Decision-Critical Verifier Learning for VLA Test-Time Scaling
Scaling robot data and model capacity has improved Vision-Language-Action (VLA) policies, but further progress is constrained by the high cost of robotic data. Verifier-guided test-time scaling offers an efficient alternative by sampling multiple action candidates and selecting the one most likely to lead to task success at inference time. Existing classification-based verifiers learn from trajectory-level outcomes but treat all visited states equally, even though their value for candidate discrimination can vary across a trajectory. At many states, plausible actions are similar and provide limited discrimination signal, while only a sparse set of decision-critical states admits meaningfully different actions that can substantially affect downstream outcomes. To address this, we propose DiVeR, which estimates decision criticality from the dispersion of sampled action representations. DiVeR then uses this signal to reweight verifier learning toward states where action selection is most consequential, without requiring step-level annotations or additional environment interaction. Across LIBERO, RoboCasa, and real-world experiments on a Franka Research 3 robot, DiVeR consistently improves task success through more effective verifier-guided action selection, while adding negligible verifier inference overhead.
Multiagent Systems
★ AgentProofBench: Benchmarking Reusable Verifiers for Web Agents
Reusable Web-agent verification requires a rule to follow changing requirements and support subsequent execution. We introduce AgentProofBench, which connects paired tests of a frozen acceptance rule to execution from identical saved openings. The resource spans 97 tasks, 79 families, and four Web services. Experiments with DeepSeek Flash/Pro establish two linked measurements. Across 48 families, complete-panel and decision-replay tests retain 82/96 Flash and 79/96 Pro source programs; paired reuse-rebuild BA intervals are [-5.47, 1.57] and [-10.16, 0.00] percentage points. Across 22 execution families, deployment-state panel correctness separates program assignments whose two saved openings are both delivered correctly: 32/33 assignments with error-free panels, versus 1/11 with panel errors. All reference-defined program-gate regressions and withheld deliveries occur in the latter group. Neutral continuation attains the highest correct delivery under the tested generic feedback, providing a concrete baseline for measuring the value added by a gate. Released instructions, states, programs, reference rules, and trajectories connect each test failure to its execution consequences. AgentProofBench makes reusable verification assessable at both the requirement and policy levels.
★ Online Fair Division with Frequency Predictions
We study online fair division of indivisible goods among $n$ agents with frequency predictions: before any good arrives, the algorithm learns the multiset of each agent's values for the goods, but neither the arrival order nor which values of different agents belong to the same good. Neoh et al. (ICML 2026) showed that such predictions allow every share-based guarantee that is achievable offline, and left open whether a positive approximation of envy-freeness up to one good (EF1) is possible. We show that deterministic online algorithms can guarantee $1/2$-EF1 for three agents and $(3-\sqrt5)/2$-EF1 for four agents, whereas for five or more agents, no online algorithm can guarantee any positive approximation of EF1, even with $n+1$ goods and identical predictions. Together with the known result for two agents, a positive approximation of EF1 is therefore possible if and only if $n\leq4$. We also rule out any positive approximation of EFX for three or more agents, even when the valuations take only two distinct positive values. Next, we use the meta-algorithm of Neoh et al. (ICML 2026) to compute an epistemic EFX (EEFX) allocation in polynomial time for any $n$, and hence a $1/(n-1)$-EF$(n-2)$ allocation for $n\geq3$, whereas algorithms that fix a picking sequence in advance can guarantee a positive approximation of EF$k$ if and only if $n\leq k+2$. Finally, for noisy predictions, we give algorithms that return EEFX allocations when the predictions are exact, and guarantee $(1-2\varepsilon+O(\varepsilon^2))$-EEFX under underestimates and, for $n\geq3$, $(1-2(n-1)\varepsilon+O(\varepsilon^2))$-EEFX under overestimates; the linear terms are the best possible, and larger overestimates rule out any positive approximation for such algorithms.
★ Agentic AI Orchestration Framework for Spatial Intelligence: Supply Chain Deployment SP
Industrial spatial decisions are assembled across systems that share neither a reference frame nor an owner, and analysts spend substantial effort reconciling those systems rather than reasoning about space. This paper presents ANASSA, a reference architecture integrating structured reasoning, multi-agent workflow orchestration, authoritative spatial grounding, and feedback-driven execution, and reports its instantiation in an operating supply-chain deployment. The architecture places typed data contracts at agent boundaries, validates spatial claims against a certified authoritative foundation, and enforces two blocking human-in-the-loop approvals inside a six-step cognitive loop. The deployment covers multi-hazard risk assessment on an ocean-and-inland freight lane, including cold-chain excursion detection over container telemetry. Across a three-month proof of concept (POC) with three implementation iterations, three deployment outcomes are reported: 25% improvement in data quality, 50% reduction in spatial workflow runtime, and 60% lower decision latency relative to the preceding manual workflow. Absolute deployment-scale quantities and baseline counts are confidential; reported percentages are observed relative changes. These are uncontrolled deployment observations, not controlled experiments. We therefore specify instrumentation and ablation protocols for mechanism-level evaluation.
comment: 10 pages, 6 figures. Accepted for publication in the proceedings of GeoIndustry 2026 (5th ACM SIGSPATIAL International Workshop on Spatial Big Data and AI for Industrial Applications). In-person presentation scheduled for November 3, 2026, Riverside, California. Proceedings forthcoming in the ACM Digital Library
★ FlockWorld: Toward Population-Scale Multi-Agent Video World Models through Artificial Life Simulation NeurIPS 2026
Most action-conditioned video world models remain limited to a single player, while existing multiplayer models support only small, fixed groups of agents. We introduce FlockWorld, an artificial-life environment built on Reynolds's Boids algorithm for studying population-scale multi-agent video world models. Specifically, FlockWorld renders synchronized egocentric crops, assigns distinct colors to preserve camera-agent identity across views, exposes rule-derived accelerations as actions, and lets the number of camera agents be configured independently of the size of the surrounding population. We further introduce FlockDiT, an action-conditioned latent diffusion transformer trained with flow matching on FlockWorld data to jointly generate synchronized egocentric videos for multiple agents. Our results suggest that joint generation benefits agent detection at density-matched settings and that diffusion forcing improves rollout fidelity. Together, FlockWorld and FlockDiT provide a controlled testbed for multi-agent video generation.
comment: Accepted to NeurIPS 2026, Creative AI Track
★ DynaFlow: Learning Free-Form Multi-Agent Orchestration
Multi-agent systems often rely on predefined collaboration structures, limiting their ability to adapt to diverse task requirements. We introduce DynaFlow, a framework that integrates free-form executable workflow synthesis into the action space of the main agent. The agent decides when collaboration is needed and generates task-specific programs that coordinate sub-agents through sequential or parallel execution, structured information exchange, and verification. Execution results and traces inform subsequent actions, enabling the agent to adapt collaboration as reasoning progresses. We train only the main agent through supervised fine-tuning on distilled trajectories, followed by refinement on examples filtered for unsupported premises, biased verification, and information-flow defects. All sub-agents remain fixed, separating orchestration learning from executor capability improvement. Across six benchmarks spanning mathematical reasoning, scientific question answering, and multi-hop retrieval, DynaFlow achieves 88.08% overall accuracy in Forced mode, outperforming the strongest evaluated baseline by 10.90 percentage points with the same Qwen3.5-27B backbone. Workflow analysis finds more explicit use, verification, and refinement of upstream outputs after training, highlighting cross-stage information flow as a key dimension of multi-agent orchestration.
★ EvolveMAS: A Unified Self-Evolution Framework for Decentralized Multi-Agent Systems
Decentralized multi-agent systems (MAS) enable large language model agents to interact without centralized control. Agents need to adapt to other agents at test time and reuse accumulated experience to refine their interaction strategies for future interactions across cooperative, mixed-motive, and adversarial settings. However, existing approaches often develop adaptation mechanisms for particular interaction settings. This raises a central question: how can decentralized MAS support test-time adaptation and reliable experience reuse across interaction settings? We propose EvolveMAS, a unified self-evolution framework that supports experience-driven adaptation within and across episodes. EvolveMAS pairs each main agent with a dedicated MemoryAgent for test-time memory governance, which maintains isolated Structured Memory outside the main agent's reasoning context. EvolveMAS organizes adaptation into two complementary processes: Within-Episode Online Adaptation and Cross-Episode Knowledge Evolution. During Within-Episode Online Adaptation, MemoryAgent updates episodic, semantic, and working memory from interaction evidence to inform subsequent actions. During Cross-Episode Knowledge Evolution, the main agent uses memory and episode outcomes to distill reusable interaction knowledge comprising partner profiles and interaction strategies for subsequent episodes. Across five benchmarks and six evaluation configurations spanning the three interaction settings, EvolveMAS achieves the highest normalized average score of 77.2 among the compared methods. Ablations show that test-time memory governance and cross-episode knowledge reuse both improve performance. Interaction traces further illustrate how updated memory and distilled knowledge guide subsequent interactions. These results support experience-driven adaptation in decentralized MAS without updating model parameters.
comment: 27 pages, 9 figures
★ 6G Reasoning Agent Societies: Composition, Scaling, and Consistency-Time Laws
Emerging 6G control planes would compose Large Language Model (LLM) agents that exchange interpreted conclusions, so hallucinations can propagate even over error-free transport. Recent architectures audit each message locally against the sender's observation mask, but isolated audits miss collective behavior. Modeling the agent society as a cellular sheaf, we establish three laws valid on arbitrary topologies. First, the \emph{composition law}: network-wide divergence exceeds the sum of per-agent audit scores by exactly their hidden total correlation. This blind spot masks up to 23\% of joint divergence and, because the declared signal cancels in the cognitive signal-to-noise ratio (SNR), overstates it by $1.14$\,dB purely through topology; the resulting trust inflation preserves valid consensus states but slows convergence. Second, the \emph{scaling law}: reconciliation decouples across metric counters, so the worst-connected dimension sets the pace. Consistency time grows for a society size $n$ as $Ω(n^{2/d})$ on polynomial-growth topologies of degree $d$, and paths breach a $1$\,s near-real-time budget at $n^\ast=13$ agents, whereas expanders ($Θ(1)$) and radius-3 local audits ($O(1)$, $\approx300$\,ms) stay bounded. Third, the \emph{consistency-time law}: the timing error from omitted correlations is one-sided and capped by the worst-measured link. These laws, validated on simulated topologies and deployed LLM societies, give provable worst-case latency ceilings for near-real-time operation.
★ Weighted Fairness in Multi-User Earth Observation Satellite Scheduling
As constellations of satellite grow in size, and become more costly, they are increasingly funded by multiple stakeholders who expect access to the constellation's resources that is commensurate with their investment. Such cooperation raises significant challenges for constellation operation, in particular regarding the fairness of the computed schedules among stakeholders. While several existing works deal with fairness in the Earth Observation Satellite Scheduling Problem (EOSSP), most of them assume that users (e.g. space agencies, research labs, countries) have the same weights, i.e. contributed equally to the project and therefore have equal access to the constellation. However, in practice, users' contributions often differ significantly. Hence, in this paper, we study a weighted variant of the EOSSP where users are assigned weights reflecting their respective contributions to the constellation. We propose several solution concepts with varying focus on efficiency and fairness, namely weighted utilitarianism, weighted egalitarianism, weighted leximin, and weighted ordered weighed averaging (WOWA). We formulate the related optimization problems as Mixed-Integer Linear Programs (MILPs), and compare them against two natural heuristics: weighted greedy and weighted round robin. All methods are evaluated on scenarios drawn from the literature, under strict runtime constraints, in order to assess the performance of each approach with respect to both efficiency and fairness metrics.
★ BE-CTDE: Blockchain-Enabled Multi-Agent Reinforcement Learning with Anomaly Detection and Token Economics
Multi-agent reinforcement learning supports cooperative decision-making in Internet of Things systems, but strategic deviations can disrupt coordination and undermine confidence in the training process. This paper proposes Blockchain-Enabled Centralized Training and Decentralized Execution (BE-CTDE), which uses blockchain as an active training-management layer. A behavioral verifier compares executed trajectories with shared-policy recommendations; its flags drive a token mechanism combining slashing, burning, rewards, and compensation. Retained balances determine each agent's contribution to the global policy update, connecting auditable behavior to learning influence. We derive a one-round incentive condition that accounts for action-dependent compensation and the payoff from deviation, and a finite expected exclusion-time bound under explicit conditional detection and accounting assumptions, showing perfect detection is unnecessary. An exact finite-population capacity bound characterizes the limits of the population-relative detector. In multi-agent path finding experiments, BE-CTDE outperforms a state-of-the-art learning-based path planner with substantially higher success rates under attack, even when adversaries form the majority. It also outperforms partially defended baselines in all configurations, with collision rates up to 35% lower than under detection without economic consequences and up to 43% lower than under conventional stake slashing. Together, these results characterize how behavioral verification and token feedback can support accountable cooperative learning.
comment: 12 pages, 4 figures, 6 tables. Submitted to the IEEE Internet of Things Journal
★ EMMA: Structured Episodic Memory for Multi-Agent Workflow Synthesis
Deploying LLM agents in specialized domains often require fine-grained task-specific procedures and knowledge a general model lacks, yet parameter training is costly and non-interpretable, and hand-engineered agent systems shift the cost to expert labor. Episodic memory updates at context level, but single agent adaptation struggles to alleviate cross-task interference, especially with complex tasks such as multi-application navigation or data pool analysis. Although multi-agent evolution alleviates this problem, existing methods lack clear definitions on update rules and rely on the backbone model to decide where and how much the system changes. This ill-scoped evolution makes learning follow the backbone's structural preferences rather than the observed failures: one backbone accumulates skills while another spawns agents. We present EMMA (structured Episodic Memory for Multi-Agent workflow synthesis), which treats the configuration of a multi-agent system (controller routing memory, specialists inheriting a generalist prompt, and per-step procedural skills) as episodic memory. EMMA makes the ownership and admissibility of updates explicit: a reflector assigns each diagnosed failure to the scope owning its correction and proposes one edit within it, and backbone-independent guards keep every accepted edit scoped, incremental and bounded. Across KramaBench, AppWorld and BBEH with five backbones and four baselines (ACE, GEPA, EvoMAS and ADAS), EMMA attains the best overall performance on every benchmark, up to 32.5% over the strongest baseline, maintains consistent improvements across backbones, and uses 17-38% fewer training executions than the most efficient baseline.
★ A Hyper-Logical View on Multi-Agent Path Planning
Multi-Agent Path Planning (MAPP) requires collision-free paths for multiple agents. MAPP has numerous real-life applications, including the navigation of automated vehicles, and has therefore many variants considering different constraints, requirements, and objectives. For each such variant, a new algorithmic treatment is often developed. This paper takes a complementary, specification-centered view: we show that many MAPP variants can be modeled in HyperLTL, a temporal logic that reasons about multiple paths and their evolution through time, and in its omega-regular extension HyperQPTL. The contribution is not a claim that current hyperproperty model checkers replace specialized MAPP solvers. Rather, the hyper-logical view gives a unified language for comparing MAPP variants, adding cross-agent constraints, expressing optimality, and connecting MAPP to verification, explanation, and causality techniques for hyperproperties.
★ AgentBaton: Efficient Model Collaboration in Agentic Tasks
Solving long-horizon agentic tasks requires language agents to interact with their environments over multiple steps, and reliable performance often comes from the expensive capable models available through cloud providers. Meanwhile, alternative efficient models with inferior performance but a much lower cost are emerging, including lower-tier cloud models and smaller open-weights models deployable on local hardware. This raises a natural question that remains less explored than traditional query-level routing: can capable and efficient models collaborate within a single long-horizon agentic task to substantially reduce cost while preserving task accuracy? In this paper, we systematically investigate key design choices for such collaboration through model handoffs, where one model takes over from another, focusing on handoff direction and context transfer. We find that efficient-to-capable handoffs generally underperform capable-only execution with limited or even no cost reduction, whereas capable-to-efficient handoffs at suitable stages of task progress largely preserve or even improve accuracy while substantially reducing cost. These findings suggest that a one-time, simple, well-timed handoff strategy is good enough for cost-effective agent collaboration, motivating AgentBaton, a lightweight detector that learns when to hand off from recurring patterns in task progress. Across 11 model pairs and two representative benchmarks, AgentBaton reduces API costs by 65.9% on average while largely preserving task accuracy, with accuracy gains of up to 8.8 percentage points over capable-only execution. Experiments across various reasoning-effort settings further show that AgentBaton extends the empirical accuracy-cost Pareto frontier.
★ The Coordination Tax and the Heterogeneity Dividend: Break-Even Conditions for Multi-Agent LLM Pipelines
Multi-agent LLM pipelines built on interoperability protocols such as the Open Floor Protocol (OFP) or A2A are usually justified on capability grounds, with their runtime cost treated as an overhead of unknown size. Comparing such a pipeline with one model call is underspecified, because a pipeline can give each role a different model. We give a closed-form model that sets the coordination tax (decomposition overhead plus per-handoff protocol cost) against the heterogeneity dividend from routing work to a cheaper model, with a break-even threshold and a feasibility bound, $α< 1/r$, and measure every term, with OFP as the protocol under test, on 135 HumanEval and ClassEval tasks in 2430 runs of six architectures with Claude Opus 5.5 and Haiku 5.5. The protocol adds 7.2 ms per handoff, 0.11% of pipeline wall time. Decomposition does not pay on these tasks: a planner-coder-reviewer pipeline costs 3.22x as much as a single strong call, takes 3.24x as long, and solves no more tasks. Moving planner and reviewer to the cheap model cuts the pipeline's cost to 0.39x but changes pass@1 by -6.7 points, leaving it dearer, slower and less accurate than the single call, and the break-even forecast misses by -19% because the cheap roles inflate token volume. The saving lies in choosing a model per task instead: an oracle choice between the two single calls solves 94.3% of runs at half the strong call's cost per solved task.
comment: 16 pages, 3 figures, 4 tables
♻ ★ Multi-Agent Debate for Belief Optimization in Defeasible Domains
Multi-agent debate has emerged as a promising approach for improving LLM reasoning on ground-truth tasks, yet current methodologies face certain structural limitations: debate tends to induce a martingale over belief trajectories, majority voting accounts for most observed gains, and LLMs exhibit confidence escalation rather than calibration across rounds. We argue that the genuine value of debate, and dialectic systems as a whole, lies not in ground-truth tasks but in defeasible domains, where every position can in principle be defeated by better reasoning. We present the Council of Hierarchical Agentic Language (CHAL), a multi-agent dialectic framework that treats defeasible argumentation as an engine for belief optimization. Each agent maintains a CHAL Belief Schema (CBS), a graph-structured belief representation with a Bayesian-inspired architecture, that facilitates belief revision through a gradient-informed dynamic mechanism by leveraging the strength of the belief's thesis as a differentiable objective. Meta-cognitive value systems spanning epistemology, logic, and ethics are elevated to configurable hyperparameters governing agent reasoning and adjudication outcomes. We provide a series of ablation experiments that demonstrate systematic and interpretable effects: the adjudicator's value system determines the debate's overall trajectories in latent belief space, council diversity refines beliefs for all participants, and the framework generalizes across broad fields. CHAL is, to our knowledge, the first framework to treat multi-agent debate as structured belief optimization over defeasible domains. Further, the auditable belief artifacts it produces establish the foundation for dedicated evaluation suites for defeasible argumentation, with broader implications for building AI systems whose reasoning and value commitments are transparent, aligned, and subject to human oversight.
♻ ★ Safe Equilibrium
The standard game-theoretic solution concept, Nash equilibrium, assumes that all players behave rationally. If we follow a Nash equilibrium and opponents are irrational (or follow strategies from a different Nash equilibrium), then we may obtain an extremely low payoff. On the other hand, a maximin strategy assumes that all opposing agents are playing to minimize our payoff (even if it is not in their best interest), and ensures the maximal possible worst-case payoff, but results in exceedingly conservative play. We propose a new solution concept called safe equilibrium that models opponents as behaving rationally with a specified probability and behaving potentially arbitrarily with the remaining probability. We prove that a safe equilibrium exists in all strategic-form games (for all possible values of the rationality parameters), and prove that its computation is PPAD-hard. We present exact algorithms for computing a safe equilibrium in both 2 and $n$-player games, as well as scalable approximation algorithms.
♻ ★ Observable Perfect Equilibrium
While Nash equilibrium has emerged as the central game-theoretic solution concept, many important games contain several Nash equilibria and we must determine how to select between them in order to create real strategic agents. Several Nash equilibrium refinement concepts have been proposed and studied for sequential imperfect-information games, the most prominent being trembling-hand perfect equilibrium, quasi-perfect equilibrium, proper equilibrium, and recently one-sided quasi-perfect equilibrium. These concepts are robust to certain arbitrarily small mistakes, and are guaranteed to always exist; however, we argue that none of these is the correct concept for developing strong agents in sequential games of imperfect information. We define a new equilibrium refinement concept for extensive-form games called observable perfect equilibrium in which the solution is robust over trembles in publicly-observable action probabilities (not necessarily over all action probabilities that may not be observable by opposing players). Observable perfect equilibrium correctly captures the assumption that the opponent is playing as rationally as possible given mistakes that have been observed (while previous solution concepts do not). We prove that observable perfect equilibrium is always guaranteed to exist, and demonstrate that it selects a unique equilibrium in a simplified no-limit poker game where the prior extensive-form refinements retain a continuum of equilibria. We expect observable perfect equilibrium to be a useful equilibrium refinement concept for modeling many important imperfect-information games of interest in artificial intelligence.
♻ ★ Trading Agents or Market Crashers? Dissecting Robustness and Security Failures in Academic Financial LLM Trading Schemes
Large language model (LLM) agents already trade real capital, and builders copy a few published designs that share models and news feeds, so one shock reaches many agents at once. We present FARSIGHT (Financial Agent Robustness and Security Investigation and Global Holistic Taxonomy), a systematization that rates trading agents on robustness to market stress and on security against source manipulation, agent compromise, and agents acting as attackers, and checks the ratings against released code. Across 15 academic schemes, no paper describes a halt that skips LLM inference or combines source provenance with cross-source corroboration, and the 7 code releases enforce no stop-loss or position limit outside the LLM. Three LLMs sell whole positions on a fake headline and on real hoaxes, including the fake press release behind the 2000 Emulex crash. In a simulated market driven by these decisions, one fake headline crashes the market in all runs with 400 agents, and 400 agents that sell when prices fall nearly double a 2010-sized crash. The pause that U.S. exchanges apply to large stocks starts in no headline run, whereas automatic halts and hard loss limits outside the model remove both effects. Restated as a domain-independent threat model, FARSIGHT maps the same gaps onto agents in healthcare, law, software, and crypto, where compromised agents spread harm as worms and as cascades. We derive 14 open problems and urge deployers to corroborate sources, keep interrupts that bypass the model, and enforce loss limits outside it.
comment: 57 pages, 19 figures, 21 tables, 221 references. v3: adds a flash-crash experiment (trading decisions of three LLMs on a fake headline and four real market hoaxes, a simulated market with up to 800 agents, the effect of trading halts, hard loss limits, source corroboration, and the exchange Limit Up-Limit Down pause, and robustness checks) and drops the SoK label from the title
♻ ★ CAFE: Causally-Guided Automated Feature Engineering with Multi-Agent Reinforcement Learning NeurIPS 2026
Automated feature engineering (AFE) enables AI systems to autonomously construct high-utility representations from raw tabular data. However, existing AFE methods rely on statistical heuristics, yielding brittle features that fail under distribution shift. We introduce CAFE, a framework that reformulates AFE as a causally-guided sequential decision process, bridging causal discovery with reinforcement learning-driven feature construction. Phase I learns a sparse directed acyclic graph over features and the target to obtain soft causal priors, grouping features as direct, indirect, or other based on their causal influence with respect to the target. Phase II uses a cascading multi-agent deep Q-learning architecture to select causal groups and transformation operators, with hierarchical reward shaping and causal group-level exploration strategies that favor causally plausible transformations while controlling feature complexity. Across 15 public benchmarks (classification with macro-F1; regression with inverse relative absolute error), CAFE achieves statistically significant gains over the strongest AFE baseline (Wilcoxon p<0.001, large effect), reduces episodes-to-convergence, and delivers competitive time-to-target. Under both controlled covariate shifts and naturally occurring distribution shifts (Folktables), CAFE reduces out-of-distribution performance drop by up to 4x relative to a non-causal multi-agent baseline, and produces more compact feature sets with more stable post-hoc attributions. These findings underscore that causal structure, used as a soft inductive prior rather than a rigid constraint, can substantially improve the robustness and efficiency of automated feature engineering.
comment: 10 Pages, References, and Appendix. Accepted at NeurIPS 2026
♻ ★ When Does Execution Provenance Help Agent Memory Retrieval?
A language agent's execution history can exceed its context window, requiring its memory system to retrieve complete supporting evidence under a hard token budget. Evidence may span multiple execution events, yet conventional retrievers use fixed token windows and fixed-k metrics that reward individual fragments without showing whether the complete evidence set fits in context. Smaller windows reduce irrelevant text but scatter evidence across candidates, while flat-versus-graph comparisons can conflate candidate design with graph propagation. To address these limitations, we formulate agent-memory retrieval as budgeted evidence completion and score exact gold spans in shared source coordinates. We first construct source-aligned provenance units from tool arguments and outputs. We then apply a zero-initialized residual R-GCN to refine frozen dense-retrieval scores over typed provenance edges. We evaluate 2,000 span-grounded memory queries over 1,207 held-out execution-grounded ISETrace trajectories. With matched Dense-FT scoring, provenance units improve Full Support@2048 by 19.07 points over flat 512-token windows and remain 11.96 points above a per-metric oracle over four flat chunk sizes; the pattern also holds with cross-encoder scoring. Holding the candidates and seed scores fixed, graph propagation adds 4.55 points in Full Support@2048 (95% CI [2.98, 6.18]). This gain is concentrated when gold evidence spans multiple events; entity co-occurrence expansion produces no comparable benefit, and relation and topology controls confirm dependence on typed transformations and observed graph structure. Overall, source-aligned candidates address the dominant granularity trade-off, while graph-conditioned propagation adds a smaller, targeted benefit for distributed evidence.
comment: 14 Pages, 4 Figures, 8 Tables
♻ ★ Calibration Is Not Verification: Answer-Level Conformal Guarantees for Mixture-of-Agents
Web answer synthesis with Mixture-of-Agents (MoA) requires reliable factual support for every atomic claim, yet agreement among agents is not verification. Correlated proposers can repeat a shared misconception, and a correct specialist fact receives little support when the other proposers stay silent. We introduce C-MoA, which decomposes MoA outputs into atomic claims, scores inter-agent semantic support, and calibrates claim retention at the level of whole answers. Under exchangeability and a policy frozen before calibration, it provides distribution-free marginal control of the probability that an answer retains any claim unsupported by the reference labels. Because calibration only selects a cutoff on a fixed ranking and cannot repair shared errors, we also study CONTRA-MoA, an optional extension that adds blinded near-miss tournaments, leave-one-agent-out stability, and availability-aware fusion, and we derive a competence--discrimination law for its verifier. Across 300 splits of the FactScore biography benchmark, the mean rate of fully supported answers (Fact) exceeds all six nominal targets, but only $1.7\%$--$9.8\%$ of supported claims are retained and $30\%$--$75\%$ of answers are empty. At $α=0.10$, mean agreement retains $3.1\%$ of supported claims, about $3.4$ times the $0.9\%$ retained by the best single-proposer score. On the medical Golden Dataset, it retains $37.3\%$ at a mean Fact of $0.898$, slightly below the $0.90$ target, and on NQ mean Fact reaches $0.904$. On a separate balanced FactScore cohort, memory-only counterfactual scoring is close to chance (AUC $0.522$ on 182 claims), and raw maximum fusion lowers the agreement AUC from $0.687$ to $0.652$. These results separate the statistical calibration of factuality from informative claim verification and clarify the role of each in multi-agent Web synthesis.
comment: 11 pages, 3 figures; revised manuscript with updated experiments and analysis
♻ ★ Consistent Opponent Modeling in Imperfect-Information Games
The goal of agents in multi-agent environments is to maximize total reward against the opposing agents that are encountered. Following a game-theoretic solution concept, such as Nash equilibrium, may obtain a strong performance in some settings; however, such approaches fail to capitalize on historical and observed data from repeated interactions against our opponents. Opponent modeling algorithms integrate machine learning techniques to exploit suboptimal opponents utilizing available data; however, the effectiveness of such approaches in imperfect-information games to date is quite limited. We show that existing opponent modeling approaches fail to satisfy a simple desirable property even against static opponents drawn from a known prior distribution; namely, they do not guarantee that the model approaches the opponent's true strategy even in the limit as the number of iterations approaches infinity. We develop a new algorithm that achieves this property given sufficiently informative observations and runs efficiently by solving a convex minimization problem based on the sequence-form game representation.
Systems and Control (EESS)
★ Uniform observability criteria for linear time-varying systems
Verifying uniform observability of a linear time-varying system generally requires its state-transition matrix. This paper develops necessary and sufficient conditions that can be checked directly from the coefficient matrices for two classes of systems. For constant dynamics with real spectrum and a measurable, essentially bounded output matrix, uniform observability is equivalent to persistent excitation of the output matrix on each eigenspace. An equivalent condition requires persistent excitation of the Kalman observability stack. The second result concerns cascades with a constant observable measured subsystem coupled to a hidden state. Under uniform regularity assumptions, linear combinations of output derivatives eliminate the measured state and yield an equivalent observability test for the hidden dynamics. When those dynamics have real spectrum, the test reduces to excitation on each eigenspace intersected with the kernel of the direct measurements. An application to arbitrary-order integrator chains with measurements of ranges to fixed anchors gives an exact excitation condition for the associated lifted LTV system. A second application treats direct hidden measurements and compares the condition with existing sufficient tests.
comment: 17 pages including a technical-report cover, 2 figures
★ Dual Gradient Consensus for Adaptive Learning to Transient Out-of-Distribution Disturbances in Quadrotor Control
Quadrotor trajectory tracking under unmodeled disturbances often relies on online residual adaptation, but short out-domain residuals can corrupt the learned disturbance model when constant-step updates treat every residual gradient uniformly. We propose Dual Gradient Consensus (DGC), an adaptive-learning method for mixed in-domain/out-domain disturbances that retains the random Fourier feature model, offline BPTT policy, and least-squares adaptation of a prior domain-adaptive residual policy (DAPL) while modifying the online update step size. DGC compares each residual gradient with long- and short-term gradient memories, classifies the current adaptation regime, and uses a confidence map to suppress unreliable updates while recovering adaptation after persistent domain shifts. We evaluate DGC in numerical quadrotor trajectory tracking under four residuals that mix an in-domain component with short out-domain events. Compared with $L_1$-augmented MPC, RFF-based MPC, constant-step DAPL, and a frozen-parameter baseline, DGC achieves the lowest mean position RMSE in all four cases.
comment: 8 pages, 2 figures, 3 tables
★ Symbolic Direct Construction of Augmented Systems for Levenberg-Marquardt Power Flow
Levenberg-Marquardt power flow (LMPF) addresses ill-conditioned equations and provides least-squares stationary points for infeasible operating requests. Conventional implementations form damped normal equations, where J^T J introduces fill, squares the spectral condition number of the undamped part, and adds sparse-product cost. We propose an augmented LMPF formulation with a symbolic-permutation-based Jacobian operator that fills the augmented blocks directly. We prove that the matrix is symmetric quasi-definite for positive damping, enabling fixed-order, pivot-free QDLDL factorization. This property also motivates the solver choice: KLU value-only refactorization can reuse an unsuitable pivot order, forcing fresh full factorizations on difficult LMPF trajectories. We further introduce one-probe adaptive trust-region initialization for the radius and damping. Direct assembly is 7.5-11.9x faster than the fastest coordinate-list path and 18.0-58.8x faster than cached normal-equation sparse-product formation; QDLDL reduces numerical refactorization and triangular-solve time by up to 3.40x on identical matrices. Adaptive initialization solves flat-started RTE cases in 19-25 factorizations, 7.5-11.6x faster than reproduced three-step LM. On case2383wp, our 84-iteration run performs 84 factorizations and 168 resolves in 237 ms, versus published times of 200.780-441.72 s with fewer outer iterations, corresponding to speedups of 847-1860x. These results require complete factorization and resolve accounting and reveal initialization and legacy-parameter effects. DC-initialized NR exhibits fast quadratic convergence throughout every tested solvable loading range, motivating clearer, reproducible definitions of ill-conditioned power-flow benchmarks.
comment: 8pages, 1 figure
★ Reinsertion-Aware Parameter Estimation of the WECC Composite Load Model via Closed-Loop Equivalent-Network Co-Estimation
Measurement-based parameter estimation of the WECC composite load model (CMLD) commonly relies on voltage replay, where measured voltage is imposed on an isolated CMLD and its parameters are optimized to reproduce measured active and reactive power. However, this open-loop formulation neglects the two-way dynamic interaction between the load and grid, potentially causing large errors when the estimated model is reinserted into dynamic grid simulations. To address this limitation, we propose a closed-loop parameter estimation framework that alternately estimates the CMLD parameters and a five-parameter equivalent network representing the external grid. This framework restores load-grid interaction during estimation and enables voltage, active power, and reactive power to jointly constrain the parameter estimation. Compared with four baseline methods, the proposed approach achieves the highest reinsertion accuracy in both the CMLD benchmark and transmission and distribution (T&D) co-simulation environments. In the T&D case, it reduces active-power, reactive-power, and voltage errors by 73%, 64%, and 62%, respectively. The identified model also maintains improved reinsertion accuracy under disturbances not used for estimation, demonstrating transferability beyond the identification event.
comment: 10 pages, 10 figures, 7 tables. Submitted to IEEE Transactions on Power Systems
★ Trajectory Generation and Control for Quadrotors
In this project, I explored controller design and trajectory generation for a quadrotor. Differential flatness was used for trajectory generation based on one of the references. A minimum snap trajectory is generated from the given algorithm based on waypoint navigation, and a simple proportional plus derivative (PD) controller is used to track the trajectories. Simulation results are presented which generate smooth trajectories for position and velocity of the quadrotor.
comment: 8 pages, 6 figures
★ Nonlinear Control of Aerial Manipulator
This paper presents two controllers for a quadrotor equipped with a two-DOF robot manipulator that physically interacts with the environment. Cartesian impedance and adaptive sliding mode controllers are implemented and compared for a simple hovering task in the presence of some bounded disturbance. A full dynamic model is developed, and the two above controllers are implemented with the same initial conditions and desired position.
comment: 5 pages, 9 figures, 2 tables
★ Optimal Steady-State Regulation of Nonlinear Systems: A Feedback Optimization Approach
This paper studies the optimal steady-state regulation problem for a class of nonlinear systems. The regulated output is required to track a trajectory that minimizes a steady-state cost depending on both the output and the exo-system. To characterize the desired steady-state behavior, we introduce a generalized regulator equation and develop a two-layer framework. The optimization layer generates the optimal reference online through a steady-state generator combined with gradient-feedback optimization dynamics, while the control layer employs backstepping to stabilize the plant and achieve the desired optimal steady-state regulation. Under suitable immersion and minimum-phase assumptions, global asymptotic stability is established.
★ Recursive Neighbor Pruning: Decentralized Community Detection with Certified Behavior from Saturated Opinion Dynamics
Detecting communities, i.e., groups of densely connected nodes, usually requires a central entity that knows the whole network. We study Recursive Neighbor Pruning (RNP), a fully decentralized alternative inspired by social learning. Each agent holds a private opinion but only shows its neighbors a coarse public action, namely its opinion amplified and clipped to a fixed range. Agents repeatedly move their opinion toward the actions of their neighbors and, once opinions settle, drop the links with neighbors whose action has the opposite sign. Repeating this procedure on the remaining network splits it into communities, without any agent knowing the network or the number of communities. We use tools from nonlinear control to characterize this behavior. We show that splitting requires the amplification to exceed a threshold set by how strongly the network is bottlenecked. We also show that a group in which every member has enough of its links inside the group can no longer be split once its members agree, and that a single run separates such groups exactly when started from suitable opinions. Finally, we bound the number of runs and messages. On two standard families of synthetic networks, RNP is as accurate as the best centralized method as long as each node has fewer than about forty percent of its links outside its community, and it fails close to the value predicted by our analysis. It is more accurate than two decentralized competitors below this level, but exchanges many more messages.
★ Necessary and Sufficient Conditions for Adaptive Stabilizability of Continuous-Time Markov Jump Linear Stochastic Systems
The purpose of this paper is to provide a mathematically rigorous solution to a well-known problem in stochastic adaptive control, i.e., to find the weakest possible structural condition for global stabilizability of continuous-time Markov jump linear stochastic systems with known candidate modes but with an unobservable Markov parameter process. Under some regularity assumptions, it is shown that the necessary and sufficient condition for global adaptive stabilizability hinges on the solvability of the same coupled Riccati equations as in the non-adaptive case. This result highlights a fundamental difference between the continuous-time case considered here and the discrete-time case studied earlier by Xue and Guo (2001), in which the coupled Riccati equations for adaptive and non-adaptive stabilizability are different.
comment: 19 pages
★ Evaluating Wi-Fi HaLow for Next-Generation Capsule Endoscopy
Capsule endoscopy is limited by its wireless link: insufficient throughput restricts image resolution and frame rate, whereas high-rate radios typically use higher carrier frequencies that experience greater attenuation through tissue. This work evaluates Wi-Fi HaLow (IEEE 802.11ah), a sub-GHz Wi-Fi standard, as an image link for capsule endoscopy. Across all tested combinations of modulation and coding scheme and bandwidth, the saturation TCP throughput reached 15.55 Mbps, and the fastest configuration also achieved the lowest energy per bit. Through power optimization strategies, the combined power consumption of the HaLow module and microcontroller at 1 frame per second (FPS) was reduced from 101.70 to 14.00 mW, which is comparable to that of Bluetooth Low Energy systems. In a simulated in vivo setting, the default rate control selected the highest tested MCS and bandwidth, indicating sufficient link quality; this configuration also had the lowest measured power in both the 1 FPS mode (13.59 mW) and the 30 FPS mode (174.21 mW). The capsule-to-receiver latency was 20.68 ms for a 24 KB frame, while the receiver-to-capsule latency was 4.77 ms for a command; this low-latency bidirectional communication supports real-time detection and manipulation for next-generation capsule endoscopy.
★ Semantic-Coupled Perception Framework for Autonomous Laparoscope Control
In minimally invasive surgery (MIS), maintaining an appropriate field of view (FoV) is essential for surgical safety and efficiency. However, conventional autonomous laparoscope control methods often treat visual tracking and surgical phase recognition independently, limiting robustness under occlusion, entity overlap, and camera motion. This paper proposes a closed-loop autonomous laparoscope control framework that couples real-time surgical phase recognition with camera-motion-compensated visual perception. By exploiting phase-dependent relationships among surgical entities, the framework improves target state estimation under challenging visual conditions, supporting task-aware autonomous FoV control. Experiments on benchmark datasets evaluate the proposed framework in surgical phase recognition and target tracking, including scenarios involving occlusion and entity overlap. Phantom experiments further demonstrate the feasibility of autonomous FoV control, showing improved target visibility and task execution performance compared with fixed-FoV control. These results highlight the potential of incorporating task-level semantics into visual state estimation for autonomous laparoscope guidance.
comment: 8 pages, 5 figures. Submitted to IEEE Robotics and Automation Letters (RA-L) for possible publication
★ Do Bio-Inspired Algorithms Save Energy in Data Centres? Separating Server Packing from Migration Search in Virtual Machine Consolidation
Energy-efficient virtual machine (VM) consolidation matters as data centre electricity demand rises. Bio-inspired metaheuristics, including ant colony optimisation (ACO), particle swarm optimisation (PSO), and genetic algorithms, have reported substantial energy savings. However, these improvements are commonly evaluated against conventional allocation policies that differ in both initial VM placement and subsequent migration optimisation, making it difficult to determine the contribution of metaheuristic search. This study reproduces an existing consolidation implementation in CloudSim Plus and introduces a matched control that retains the same initial packing and empty-host deactivation mechanisms but excludes migration search. ACO, PSO, and a modified genetic algorithm (MGA) are evaluated across two fleet configurations and four workload regimes, with ten stochastic replications per condition. The matched control achieves energy savings of 0.8-81.4% relative to the spreading baseline, the amount available falling sharply as load rises, without migrations or incomplete workload execution. Across 24 comparisons, metaheuristic search increases energy consumption in 11 cases, produces identical outcomes in seven, and reduces consumption in six. Of these, ten increases and five reductions remain statistically significant after Holm correction. Five of the six apparent reductions coincide with unfinished tasks and disappear after normalisation by delivered computational work. Only ACO under light heterogeneous load retains a 3.22-percentage-point improvement (95% CI 2.96-3.47). These findings demonstrate that initial packing and host deactivation account for most of the measured energy savings in the examined implementations, highlighting the importance of matched control conditions and workload-completion measures for reliable evaluation of VM consolidation algorithms.
comment: 38 pages, 8 figures, 6 tables. Data, source and analysis scripts: doi:10.5281/zenodo.23632346. Submitted to Simulation Modelling Practice and Theory
★ Optimal Linear Filtering of Markov Jump Systems
This paper deals with the optimal state estimation problem for discrete-time Markov jump linear systems in the sense of linear minimum mean-square error where the noises corrupting the system are mode-dependent and need not be Gaussian. An orthogonal projection extension-based equation (OPEBE) is established, which can decompose a random vector into several parts where the random vector contains conditional expectation of another random vector. Using the OPEBE and some results obtained in this paper, an optimal linear filter for the system under consideration is proposed. The computational and storage load of the proposed filter does not increase with time. Also, the proposed filter is finite-dimensionally computable and has a recursive structure. Computer simulations are provided to verify the effectiveness and performance of the proposed optimal linear filter.
★ Water as a Battery: Using Hydropower Flexibility to Support the Grid
This study presents a two-layer optimization framework to determine the optimal sizing of PV, wind, and battery storage co-located with existing hydroelectric plants to support rapidly increasing data center loads while minimizing cost, grid reliance, and transmission upgrades. The outer layer uses Particle Swarm Optimization (PSO) to select capacity combinations, while the inner layer performs rolling-horizon dispatch simulations to model hourly energy balance among PV, wind, hydro, and BESS, considering operational constraints. Two case studies were analyzed. At The Dalles Dam (Oregon), favorable wind resources and land availability enabled deployment of 111.2 MW PV and 117.9 MW wind, with no BESS required. Hydro flexibility allowed the system to meet the load with minimal grid support. In contrast, Conowingo Dam (Maryland) faced land constraints and limited renewables to 130 MW PV, leading to higher grid dependence. Co-locating hydro with renewables significantly enhances reliability and cost effectiveness by leveraging hydro's dispatchability to balance daily and seasonal variable generation from PV + Wind, reducing the need for storage or costly grid imports. This analysis provides a framework to meet the 99.9% availability targets of this new load type on the grid with sustainable energy resources.
★ Bi-Objective Receding-Horizon Scheduling of Nurses and Hospital Service Robots
This paper studies the joint scheduling of nurses and hospital service robots as a bi-objective mixed-integer linear programming problem. The formulation represents heterogeneous nursing skills, alternative skill-coverage modes, nurse availability and temporal rules, workload balance, robot task capability, charging and battery dynamics, and human supervision. Robot workload relief is credited only when an eligible service is actually executed, and explicit supervision-load variables deduct the corresponding nurse capacity from clinical coverage. The receding-horizon formulation also includes a cross-window night--morning constraint and a terminal battery reserve. The computational study is presented as numerical verification on synthetic hospital-like instances rather than clinical validation. Under the proposed formulation, the eight-nurse reference case gives $Z_1=0.78$ and $Z_2=3.164$. A common-budget comparison on B20 seed~1 yields total shortage $53.70$ for the integrated model, compared with $58.03$ for sequential nurse--robot scheduling and $59.46$ for the nurse-only baseline. Additional experiments examine multiple epsilon budgets on a larger B30 instance, sensitivity to robot-relief/supervision parameters and objective weights, and one-day versus two-day rolling horizons against a static full-horizon reference. The battery and routing representations remain aggregate planning approximations, and the scalability evidence is interpreted accordingly. No patient records or real hospital staff data are used, so the results establish model behavior and comparative numerical evidence rather than practical clinical effectiveness.
comment: Ghorbani-Moghadam, K., and Khalili-Amirabadi, R. (2026). Bi-objective receding-horizon scheduling of nurses and hospital service robots. Journal of Intelligent Monitoring and Control Systems, 1, 56-65
★ Unifying Performance and Equity via Percentile Optimization and Cell-Free Massive MIMO
Improving the throughput of the weakest users is a central objective of wireless resource allocation, and the objective around which the user-centric cell-free massive multiple-input multiple-output (UC-mMIMO) architecture is built. This paper brings the two together by formulating and solving percentile programs for UC-mMIMO networks. We maximize the sum-least-qth-percentile (SLqP) rate via downlink power control, a utility that targets the user equipments (UEs) in the lowest q percentile of the rate distribution and subsumes max-min-rate and sum-rate maximization as special cases. The problem is non-convex, non-smooth and NP-hard. The cyclic maximization approach of one of the previous percentile optimization series does not carry over: the coherent joint transmission technique introduces cross terms in the powers that destroy the block-concavity on which it relies. We develop a change of variables that restores block-concavity, yielding an algorithm with closed-form auxiliary updates and a concave power update, and we give a self-contained proof of non-decreasing convergence to a directional stationary point. Simulations establish two facts: first, percentile optimization retains its value in an architecture with no cell edges; relative to the weighted minimum mean-square error benchmark, it raises the average rate of the lowest-10th-percentile UEs by 56--72% and Jain's fairness index from below 0.92 to above 0.99. Second, compared directly with massive MIMO and small cells under the same SLqP utility, UC-mMIMO delivers the highest lower-percentile rates at every access point density tested, by up to 62% and 2.7$\times$ respectively, at the same fairness, and its advantage widens with densification.
♻ ★ Shooting for Contact: Contact-Implicit Multiple Shooting for Dynamic Motion Retargeting
Motion retargeting approaches often prioritize kinematic similarity over whole-body dynamics, contact consistency, and actuation limits, yielding references that are difficult for reinforcement learning (RL) policies to reproduce, particularly for contact-rich behaviors. We present a contact-implicit, direct simulation-based multiple shooting (DSMS) framework that transforms kinematically feasible references into dynamically feasible whole-body trajectories. By embedding a differentiable simulator within a nonlinear program, DSMS resolves contact, friction, impacts, self-collision, and joint limits internally while enforcing tracking, actuation, and task constraints without prescribing a contact schedule or introducing explicit contact constraints. Compared with existing retargeting methods, DSMS accelerates motion-imitation RL training and yields policies with high success rates and low tracking error. We further demonstrate zero-shot sim-to-real transfer on the Unitree G1 through command-conditioned walking, jogging, and contact-rich crawling, and a highly dynamic 180-degree jump-turn.
comment: Project website with additional material: https://shooting-for-contact.github.io/
♻ ★ Fair Dynamic Operating Envelopes using Distributed Multi-Period Optimal Power Flow and Jain Index for Active Distribution Networks
Dynamic operating envelopes (DOEs) are increasingly used to publish time-varying export limits that keep distribution networks within operational limits. Purely technical DOE allocation, however, can systematically privilege electrically favorable prosumers, while embedding fairness directly into a single-period optimal power flow (OPF) objective mixes network feasibility, equity and efficiency in a way that obscures the cost of fairness. This paper proposes a two-stage, multi-period framework that addresses both of these. Initially, a technical distributed OPF computes network-feasible export envelopes. The subsequent stage then applies a dynamic aggregate export budget and redistributes capacity through cumulative proportional fairness, limiting the additional curtailment by an admissible efficiency budget. The resulting fair DOEs are treated as first-stage decisions, while battery storage provides scenario-dependent recourse under demand and renewable uncertainty. The operational problem is solved by a calibrated regional alternating direction method of multipliers (ADMM) on a lossless LinDistFlow model and independently validated using AC power flow. We observe a trade-off between increases in curtailment \emph{versus} improvements on fairness, as expected, reaching Jain fairness indices close to unity. Results show that considering both storage (which alleviates curtailment impact) and multi-period fairness (which increases curtailment) is an interesting approach for modern DOE design, which in turn requires a multi-period, co-designed approach.
comment: 10 pages, submitted to CLAGTEE 2026 Conference
♻ ★ Multi-Agentic AI for Fairness-Aware and Accelerated Multi-modal Large Model Inference in Real-world Mobile Edge Networks
Generative AI (GenAI) has transformed applications in natural language processing and content creation, yet centralized inference remains hindered by high latency, limited customizability, and privacy concerns. Deploying large models (LMs) in mobile edge networks emerges as a promising solution. However, it also poses new challenges, including heterogeneous multi-modal LMs with diverse resource demands and inference speeds, varied prompt/output modalities that complicate orchestration, and resource-limited infrastructure ill-suited for concurrent LM execution. In response, we propose a Multi-Agentic AI framework for latency- and fairness-aware multi-modal LM inference in mobile edge networks. Our solution includes a long-term Global Planning Agent, a short-term Prompt Scheduling Agent, and multiple on-node LM Deployment Agents, all powered by foundation language models. These agents cooperatively optimize prompt routing and LM deployment through natural language reasoning over runtime telemetry and historical experience. To evaluate its performance, we further develop a city-wide testbed that supports network monitoring, containerized LM deployment, intra-server resource management, and inter-server communications. Experiments demonstrate that our solution reduces average end-to-end latency by 42.9% compared with a state-of-the-art online-learning baseline and by 80.6% compared with the full-activation baseline, while maintaining a mean service-type Jain's index of 0.925. Moreover, our solution adapts quickly without fine-tuning, offering a generalizable solution for optimizing GenAI services in edge environments.
♻ ★ VLN on the Fly: An Onboard Vision-Language Navigation Stack for Aerial Robots
Running vision-language navigation fully onboard an aerial robot is hard, since grounding, planning, and control must share limited compute and a single-stage error is difficult to isolate in flight. End-to-end aerial policies fuse these stages into one network, giving up the observability and safety checks a modular stack keeps available. We propose VLN on the Fly, an onboard stack that keeps grounding, planning, and control as separate, inspectable stages. A quantized VLM grounds an instruction to a coarse image cell, depth lifts it to a 3D goal, a fast B-spline planner returns a feasible trajectory, and a pretrained reinforcement learning policy tracks it to motor commands across quadrotors. Across 15 onboard flights over three everyday referents in a controlled indoor volume, the stack reaches the target in 13 of 15 trials with 5.72 cm mean goal error and 39.3% average GPU utilization. In 6 additional cluttered-environment trials, the stack tracks collision-free trajectories under onboard perception gating.
comment: 7 pages, 5 figures. Project page: https://vln-on-the-fly.github.io/
Robotics
★ Learning Functional Specialisation for Bimanual Robotic Control
Bimanual robotic control requires simultaneous management of distinct functional demands under motor noise, energetic costs, and communication delays. We investigate how learning responsibility should be organised across modular controllers and the two arms in a simulated musculoskeletal system with two communicating recurrent controllers. Using a 2 x 2 design, we compare differential versus shared controller objectives and fixed versus balanced hand-task assignments. Task-aligned functional specialisation was strongest when differential objectives were combined with a fixed assignment aligned with contralateral control routing. This condition achieved the highest canonical-task performance and control efficiency, reaching 81.33% bimanual success. Its advantage over shared objectives increased with motor noise but narrowed under stronger energy penalties. Stronger specialisation also reduced hand-role flexibility: reversing the hand roles decreased bimanual success by 80.46 percentage points under Differential-Fixed training, compared with 1.51 percentage points under Shared-Balanced training. These results show that controller objectives and hand-task experience jointly determine functional organisation. Stronger role-specific specialisation favours performance and efficiency when roles remain stable, whereas more distributed training favours interchangeability when roles change.
comment: 24 pages, 6 figures, 10 tables
★ Real-Time Human Trajectory Prediction from RGB-D for Mobile Robots Using Bi-Mamba
Predicting the short-term future trajectory of a person from RGB-D cameras is a key capability for mobile robots navigating in dynamic and human populated environment. The key challenge to deploying learned predictors on real robots is the train-deploy pipeline gap. Training data are often prepared with an idealized, smoothed ground truth, whereas at inference, the model is fed noisy estimates from pose detectors and depth sensors. The model is therefore evaluated on an input distribution it never encountered during training. Most existing models rely on recurrent neural networks (RNNs) or transformers, which either struggle to capture short-term temporal dependencies or suffer from computational cost. We present TrajMamba, a lightweight, bidirectional state space model for short-term human trajectory prediction using an RGB-D camera. The model uses a single shared processing path for both training and deployment, aligning human pose extraction, depth back projection, and smoothing processes. It encodes a short window of persons's hip center velocities with stacked bidirectional Mamba blocks and decodes future velocities with a GRU head. On the ETH/UCY benchmark, TrajMamba reaches an average ADE of 0.29 m FDE of 0.51 m. With 262K parameters, it runs in real time on modest hardware and is deployed on a TIAGo robot with an onboard NVIDIA Jetson Orin.
comment: Accepted at the 14th International Conference on Control, Mechatronics and Automation (ICCMA 2026), Bari, Italy, November 3-5, 2026. 7 pages, 6 figures
★ Sybil-Resilient Multi-Drone Exploration of Indoor and Confined Spaces
When a team of drones explores a building together, each drone tells the others which frontier it is going to and shares the map it has built. This avoids duplicated effort, but it also means that the team trusts every identity from which it receives messages. In a Sybil attack, one compromised drone creates fake identities, and these fake teammates claim frontiers and send fabricated scans, so the real drones leave parts of the building unexplored and end up with walls in their maps that do not exist. We study five such attackers, including one that seals off the openings between floors, and five levels of defence that need no security infrastructure, only the drones' own sensors and maps: an audit of received map data against what a drone sees itself, a check that a body is present where an identity claims to be, trust only for identities whose body has been seen, a quarantine for identities that no drone has observed, and a rule that binds each observed body to one identity. Tests in simulation cover 70 maps of seven kinds, from offices and tunnels to three-storey buildings and caves. Without a defence, one compromised drone leaves on average up to 25% of a map unexplored (45% in offices), and the floor-sealing attacker cuts the explored part of a three-storey building to 82.3%, or 34.6% on the worst map. The quarantine brings exploration back to the level of an unattacked team in every setting, and in most settings it also brings the false data back to that level. The exceptions are two colluding attackers, fakes placed next to real drones, three of the ten caves and a team of 16 drones, and binding bodies to identities fixes the last three. Over 2,350 missions, an honest drone was revoked by a teammate only twice.
comment: 8 pages, 5 figures, and 8 tables. Code, worlds and mission results: https://github.com/IDZTPHKKKS/Sybil4Drones
★ RoboLongR: Causally Consistent Rewards Teach Robots to Progressively Master Long-Horizon Manipulation
Long-horizon manipulation requires actions whose validity depends on earlier interactions, yet visual progress can conceal missing task prerequisites. We propose RoboLongR, an offline-to-online framework that learns history-dependent rewards to guide policy adaptation to new tasks. Its central objective is causal consistency: aligning rewarded progress with prerequisites supported by execution history. Stage-shuffle supervision encourages execution-order sensitivity, recurrent memory preserves evidence of earlier interactions, and two-round training refines trajectory segments using predicted progress. The resulting rewards guide stage-conditioned policy refinement from behavior cloning to online reinforcement learning. Sparse terminal feedback connects policy experience to reward learning by triggering correction for repeated high-scoring failures. Pretrained on demonstrations from 88 RLBench tasks and Open-X data, RoboLongR achieves 31.8% average success on 18 long-horizon tasks excluded from reward pretraining, exceeding the strongest evaluated baseline by 9.2 percentage points. Across four real-robot tasks, it achieves 57.5% success, a 32.5-percentage-point improvement over the comparison method. Reward-level controls probe sensitivity to synthetic order perturbations, while ablations examine the contributions of individual components to policy improvement.
comment: 11 pages including appendices, 8 figures, 6 tables. Project website: https://chenzixuan99.github.io/RoboLongR-public/
★ Securing Edge & Physical AI: Post-Quantum IoT Blockchain for Mission-Critical Cyber-Physical Agents
Distributed Cyber-Physical Systems (CPS) and Edge AI networks increasingly depend on public-key authentication and key-establishment mechanisms that are vulnerable to cryptographically relevant Quantum Computers (QC). Shor's algorithm threatens widely deployed asymmetric primitives, while Grover's algorithm reduces the effective security margins of symmetric and hash-based mechanisms. Although Post-Quantum Cryptography (PQC) provides a migration path, its larger public keys, ciphertexts, and signatures introduce additional computation, memory, communication, storage, and packet-fragmentation costs that can constrain dynamic multi-agent systems. In this article, we introduce rotational secrecy, an architectural objective that limits temporal key exposure through authenticated transitions between cryptographic epochs. We present Private Authentication for Rotational Secrecy (PARS), a reference mechanism that coordinates policy-triggered key establishment, rotation, and reconciliation among authorized Physical AI agents. For deployments that need distributed agreement on security state, we further develop the PARS-Enabled Edge Security Ledger (PARS-ESL), an optional permissioned architecture maintained by authorized validators or capable edge gateways. PARS-ESL does not store high-volume operational transactions. Instead, it records authenticated state for key epochs, group membership, revocation, algorithm-suite policies, challenge validation, and recovery checkpoints. As a result, resource-constrained agents can take part as lightweight clients, and low-latency operational traffic stays outside the ledger consensus path, protected by the currently active symmetric epoch keys. Our results show that ledger-assisted post-quantum key-epoch management is feasible for selected Edge AI and CPS security-control workloads under the stated trust, compromise, connectivity, and resource assumptions.
comment: 47 pages, 39 figures, 10 tables, 5 algorithms, to be published in IEEE IoTJ, Science Directy (Elseiver IoTJ, Embodied Intelligence in Physical Systems)
★ AdaDexTrack: Dynamic Modulation for Adaptive and Generalizable Dexterous Manipulation Tracking CVPR 2026
Language is a natural way to command robots, but converting a single instruction into a long-horizon, contact-rich hand-object interaction remains challenging: synthesized references are noisy, human-to-robot retargeting introduces embodiment bias, and fixed-reference tracking lets small errors snowball. We address this with AdaDexTrack, a modulator-in-the-loop framework for language-conditioned manipulation tracking. A distilled generalist tracker serves as the skill carrier, while a tightly aligned modulator performs three feedback corrections: reference modulation (continual adjustment of what to track), object-latent modulation (online adaptation of the object representation to recruit suitable skills), and positional-target modulation (small state-dependent refinements for execution). The tracker is learned via large-scale specialist to generalist distillation on a corpus of language-conditioned hand-object trajectories; the modulator is trained with RL under the same task objective, ensuring tight coupling. Across large-scale evaluations, AdaDexTrack consistently outperforms prior SOTA on unseen-trajectory and unseen-object sets in both average tracking error and success rate, demonstrating robustness and generalization. We further show zero-shot sim-to-real transfer on real hardware, where adding the modulator yields substantial gains over a tracker-only variant. AdaDexTrack reframes language-conditioned dexterous manipulation as modulated tracking, replacing the open-loop, fixed-reference tracking with in-loop modulation that adjusts the reference, object latent, and positional target, yielding drift-resistant execution from noisy text references.
comment: CVPR 2026. 18 pages, 14 figures, including supplementary material. Project page: https://janebek.github.io/AdaDexTrack/
★ RoboWorM: Evaluating Movement Realism, Consistency and Acceleration Affects in Embodied Video World Models
Video world models (VWMs) are rapidly emerging as scalable neural simulators for robotic manipulation. However, most benchmarks only evaluate them as open-world video generators, using Vision-Language Models (VLMs) that lack strict 3D spatial and kinematic reasoning. To bridge this gap, we introduce RoboWorM, a comprehensive, deployment-aware benchmark that leverages known hardware priors such as robot kinematic trees and fixed camera configurations, to systematically examine structural and spatiotemporal fidelity. RoboWorM curates 1,605 episodes across single-arm, bimanual, and humanoid morphologies, spanning both Markovian and long-horizon non-Markovian tasks under single- and multi-view setups. To explicitly measure physical consistency, we propose two deterministic, VLM-free metrics: KIVA for intra-view kinematic stability, and RIGIS for multi-view geometric consistency. By exploiting intrinsic robotic invariants, KIVA improves hallucination detection alignment with human annotations by 25.8% relative to the strongest frontier VLMs, while RIGIS achieves a 130% higher Spearman correlation with human preferences. Furthermore, since real-world deployment requires low inference latency, we utilize RoboWorM to conduct the first large-scale audit of the quality-efficiency trade-off. Evaluating 9 state-of-the-art acceleration techniques across 3 leading world models (over 29,000 generated rollouts), we demonstrate how these acceleration methods can retain superficial visual quality while silently compromising deployment-critical physical constraints. Using our metrics, we achieve 2.3$\times$ maximum inference speed-up with minimal quality loss.
comment: First version, 52 pages
★ One Scan is Enough: Demonstration-Free Adaptation for Language-Guided Navigation NeurIPS 2026
Deploying a language-guided navigation policy in a new building usually means watching it fail. Public-benchmark VLAs degrade scene by scene as layout, appearance, and language drift, and the standard fix of teleoperating fresh trajectories on site does not scale. We replace human demonstration with a single brief iPhone scan. Scan2Instruction compiles that scan into a complete training environment, automatically synthesizing executable navigation episodes with grounded instructions. PRPT-KS then adapts the policy through a closed-form Pareto utility read directly off the reconstructed scene, with no learned critic and no human labels. Across five real environments, the two designs together drive our RealNav to an 85% real-robot success rate, compared with 49% for a RealNav baseline post-trained on human demonstrations, and still achieve 82% in unscanned regions of the same buildings, compared with 41% for the same baseline. Our work turns demonstration-free scene adaptation for language-guided navigation from an aspiration into a working reality.
comment: NeurIPS 2026
★ Segmentation of Moving Objects During Camera Motion Using the OWL Function
We present a principled, analytical framework for segmenting multiple independently moving rigid objects during single camera motion, built directly upon the OWL perception function. OWL encodes two fundamental visual motion cues, perceived looming $L$ and perceived rotation $ω$, into a single closed-form representation of relative 3D structure. A key consequence of this structure is that the unit direction of relative translation $\hat{t}$ is analytically recoverable per image point from $L$ and $ω$ alone. All points belonging to a non-rotating rigid body share the same $\hat{t}$, whereas points on different rigid bodies generally yield distinct values. This furnishes a natural segmentation criterion: clustering per-point $\hat{t}$ estimates partitions the visible scene into independently moving rigid bodies. Building on the 2D complex form of OWL, we derive the framework in 3D using quaternions, for a single moving camera. The approach is parallel, camera rotation-invariant, and requires no training data, no depth information, and no semantic priors.
★ RoboReason: Boosting Robotics-Relevant VLM Reasoning Through Scalable Data Generation EMNLP 2026
Vision-Language Models (VLMs) have demonstrated impressive performance when integrated into robot manipulation pipelines, but real-world evaluation of VLM capabilities on downstream robotics tasks is both difficult and expensive to scale. We propose that spatial and temporal ordering tasks using images from stereo and wrist cameras attached to real-world robots are particularly well-suited for evaluating visual reasoning capabilities, as they assess skills directly applicable to robots. We evaluate VLM reasoning abilities on three questions: ordering shuffled frames conditioned on a language description of the task, ordering objects depthwise based on distance from the camera, and ordering objects from left-to-right in pixel coordinates. The complexity of these tasks can also be modulated by changing the size of the set to sort. Our experiments show that several state-of-the-art vision-language models struggle on ordering tasks. To fix this, we propose a scalable autonomous training data generation pipeline, and show that both supervised finetuning and reinforcement-learning on the generated data can significantly improve VLM reasoning performance both on spatiotemporal ordering tasks as well as downstream robotics and visual reasoning tasks.
comment: EMNLP 2026 Main
★ C-DFF: Concept-Level Diagnostics for Mechanism Parity in Simulation-Based Validation of Autonomous Driving Perception SC
Simulation is widely used in autonomous-driving validation, but matching task outputs between real and synthetic images does not guarantee that a system relies on the same visual evidence. This creates a validation gap for transfer, especially in safety-critical perception functions. We present Concept-Decisive Feature Fidelity (C-DFF), a concept-level mechanism-parity diagnostic for paired real-synthetic evaluation. Starting from decisive evidence extracted from a frozen system-under-test (SUT), C-DFF projects that evidence into a shared semantic concept space using off-the-shelf segmentation and compares real and synthetic concept-importance vectors. The result is not only a scalar mechanism-parity score, but also a concept-shift diagnostic that identifies which scene concepts become more or less decisive in simulation. We evaluate C-DFF on KITTI $\leftrightarrow$ Virtual KITTI 2 with 2126 matched pairs across three frozen SUTs (steering regression, YOLOP drivable-area, YOLOP lane-line). Across these tasks, C-DFF is SUT-discriminative and yields interpretable concept-level summaries that support validation triage and simulator debugging. When reported alongside task-specific output-value context metrics, C-DFF shows that output agreement does not by itself establish mechanism parity. These results position C-DFF as a practical semantic diagnostic layer for validation and verification workflows in simulation-based AV testing.
comment: 7 pages, 4 figures, 5 tables. Accepted and presented at the 2026 IEEE 29th International Conference on Intelligent Transportation Systems (ITSC), Naples, Italy, September 15-18, 2026
★ POD-RoPE: Rotary Position Encoding over Physical Observation Distributions for Embodied AI
Embodied perception requires positional representations that capture both physical geometry and the spatial and temporal extent of observations. We introduce POD-RoPE (Physical Observation Distribution RoPE), which represents token observation footprints as distributions over camera centers, viewing directions, and sampling times. Under independent sampling, expected rotations encode relative relationships between distributions; when each distribution collapses to a single address, the construction recovers pointwise RoPE. The same rotation statistics support Q/K matching and coherence-dependent V/O transformations, allowing observation geometry to guide both information selection and content aggregation. Per-token pre-aggregation avoids increasing sequence length or processing pairs of observation nodes. We evaluate POD-RoPE across four robot policy architectures, simulation and real-world manipulation, and complementary reconstruction, depth-estimation, and novel-view-synthesis tasks. Integrating POD-RoPE into Fast-WAM improves mean success from 91.83% to 92.88% across all 50 RoboTwin 2.0 tasks, and from 32.5% to 52.5% on two real-robot tasks on Sharpa North. Controlled reconstruction ablations support the contributions of distribution encoding and V/O transformations. In H20 model-block benchmarks, POD-RoPE achieves median forward-plus-backward speedups of $1.91\times$ over P-RoPE and $5.34\times$ over RayRoPE, with a median latency overhead of 17.4% over Standard RoPE. These results support physical observation distributions as an effective and computationally efficient basis for geometry-aware attention.
comment: 21 pages, 5 figures
★ Floor Friction Assessment from Robot Tire-Surface Interaction
Floor traction influences wheel-slip behavior, dead-reckoning accuracy, and the navigation performance of wheeled mobile robots that rely on wheel odometry. As these robots become increasingly common in GNSS-denied indoor environments, including automated manufacturing and retail facilities, their onboard measurements could also be used to characterize floor conditions and improve localization. In this study, a skid-steer robot was driven over five floor surfaces with distinct traction characteristics using commanded velocity profiles designed to induce wheel slip through repeated acceleration and direction reversal. Differences between the body velocity estimated from inertial sensing and the velocity obtained from wheel odometry were evaluated using the integrated relative-velocity error, phase-derived wheel-to-body response delay, and velocity-reversal response delay. Welch power spectral density analysis was additionally applied to characterize the frequency distribution of the velocity mismatch during positive and negative acceleration. Band-integrated spectral power, spectral centroid, and spectral decay rate were extracted from each recording and evaluated using linear discriminant analysis and multiclass classification. The response-delay measurements produced consistent surface-dependent trends, with the longest delays observed on soap and the shortest delays observed on carpet. The spectral features also provided clear separation among the tested surfaces, achieving a balanced classification accuracy of 97.1% under recording-level cross-validation. These results demonstrate that both the magnitude and frequency domain characteristics of wheel-to-body velocity mismatch contain useful information about tire-surface interactions. Such information could support surface recognition and the development of slip-aware wheel-odometry weighting within robot state-estimation frameworks.
comment: 11 pages and 14 figures; Extended abstract submitted to IEEE/ION PLANS 2027, under review
★ RoboJev: An Open Vision-Language Judge for Robot Success Detection
Many robot manipulation videos do not indicate whether the task was successfully completed. RoboJev-27B is an open-weight vision-language model that takes a task instruction, 10 frames of a manipulation attempt, and (if available) final frames from auxiliary cameras and predicts whether the robot accomplished the task. RoboJev-27B is a variant of JEV-27B-VL, trained with a rank-32 LoRA adapter on the language model only (keeping the vision encoder frozen) using 52,598 episode records. It also uses a source-specific KL penalty that applies only to research data, anchoring predictions closer to the original model on those sources. It uses a calibrated two-token readout that outputs success/failure probabilities in a single forward pass. On 268 held-out episodes of the same robot model (SO-101), RoboJev-27B attains 95.2% balanced accuracy and 0.993 AUROC. It also generalises to episodes collected by unseen labs/robots (AUROC = 0.890 over 459 episodes) as well as to clearly labelled community-collected episodes (accuracy = 93.3% over 135 episodes), where the base model attains an AUROC = 0.900 on the unseen-lab/robot subset with the same inputs. It takes 0.26 seconds per episode (median, serial) to infer on a single H100, which is about \$0.19 per 1,000 success predictions at \$2.69 per GPU-hour (not including media processing and serving costs). RoboJev-27B is released together with its inference code, along with a transparent evaluation: 20,628 predictions organized into 29 model/protocol entries are made available to facilitate browsing and to enable reproducible benchmarking of success detection. We also compare against strong API-based frontier VLMs, which take in fewer input frames (4 images from the primary camera only); note that this comparison is not strictly like-for-like because RoboJev-27B is given more frames at inference.
comment: 15 pages, 5 figures
★ Quantum Closed Loop Inverse Kinematics Through Hilbert-Space Encoding of Robot Kinematics
The integration of quantum computing into robotics requires mathematical representations that preserve the geometric structure and configuration dependence of robot motion. This paper introduces a structure-preserving quantum representation of rigid-body transformations and serial-manipulator kinematics, establishing a correspondence between classical kinematic models and parameterized quantum circuits. Building on the Bloch-sphere representation, spatial directions are encoded into single-qubit states, while quantum rotations reproduce the ordered composition of reference-frame transformations. Translational contributions are combined through quantum-state aggregation, with their metric scales retained classically, enabling the reconstruction of end-effector position and orientation from Pauli expectation values. The formulation preserves the original joint variables and accommodates different kinematic conventions and joint types. Beyond forward kinematics, we establish that the manipulator geometric Jacobian can be reconstructed from quantum-observable derivatives using the parameter-shift rule, with analytical equivalence to its classical counterpart under ideal readout. To demonstrate the applicability of this representation to robot-level algorithms, we introduce Quantum Closed Loop Inverse Kinematics, a hybrid closed-loop method that exploits quantum-derived pose and Jacobian information. The proposed framework provides a geometrically interpretable foundation for developing and investigating quantum and hybrid computational methods for robotic kinematics and motion.
★ Open-World Law-to-Action: Vision-Language Priors and Touch for Parts with Hidden Physics
A robot in a home it has never seen must open doors, drawers, lids and taps whose physics a picture does not show. Instructions such as "open the drawer halfway, gently" depend on that physics: how far a part travels, how it resists, and what locks it. We present Open-World Law-to-Action (L2A). A vision-language model (VLM) proposes, for each movable part, its motions, the terms of its law of motion, and what may lock it. Each proposal becomes a prior with a floor, so that touch can always overturn it. The robot identifies each part's law online by Bayesian linear regression with closed-form evidence, for parts that move in one way or in two, and keeps a hypothesis that an unseen part or motion locks the target. We prove that touch alone cannot rule this hypothesis out, derive when another look is worth its cost, and show that a constant load and friction cannot be told apart while a part moves one way. All experiments are in simulation, developed on some scenes and run once on held-out ones. In new kitchens the robot completes 80% of tasks from the VLM's proposals, against 98% when every part is known. Proposed physics raises success on objects that need new terms from 353 to 378-383 of 400 episodes, and a second look raises success in mixed scenes from 284 to 329 of 366 tasks. A mobile robot that takes nothing from the simulator completes 70 of 100 requests in new scenes, against 62 when the simulator says which part each box is (after one change made between two held-out tests).
★ EXE: Egocentric Cross-Embodiment Pretraining via Factorized Latent Actions for VLA Models
Vision-Language-Action (VLA) models are bottlenecked by the scarcity of robot data. Egocentric human manipulation videos offer an abundant, diverse, and semantically-rich alternative that inherently captures manipulation intent. However, the embodiment gap makes it difficult to extract action-relevant representations from these videos. From the perspective of action execution, human hands and robots share analogous manipulation primitives---reaching, grasping, placing---yet exhibit distinct kinematics tied to their embodiments. This might motivate our key approach: egocentric video pretraining can yield shared action representations capturing transferable manipulation skills, while human-specific motion patterns should be disentangled to avoid interfering with robot policy learning. To this end, we propose EXE, an Egocentric Cross(X)-Embodiment pretraining framework that learns a factorized latent action representation from egocentric human videos, separating shared manipulation priors from embodiment-specific signals. By leveraging the shared action space, the robot policy acquires generalizable manipulation semantics from diverse human demonstrations without being biased by human-specific kinematics. This shared knowledge transfers across embodiments with different kinematic structures, enabling the same pretraining to benefit both parallel grippers and dexterous hands. EXE yields consistent gains over baseline without pretraining on RoboTwin 2.0, RoboCasa GR1, and real-world manipulation tasks.
★ Energy-Based Fault Detection and Isolation Framework for Flat Port-Hamiltonian Mechanical Systems
Fault diagnosis during regulated motion requires informative signals even when feedback attenuates tracking deviations. This paper develops an energy-based diagnostic framework that constructs mechanical port-power residuals without a dedicated dynamic observer. Spectral Hamiltonian Algebraic Power Evaluation (SHAPE) combines port-Hamiltonian variables and differential flatness to generate nominal power references from planned motion for fully actuated mechanical systems with collocated actuation. Subtracting these references from commanded effort multiplied by measured flow yields residuals whose final evaluation requires neither recursive observer updates nor measured-state dynamics calculations. Continuous wavelet transform (CWT) scalograms and a shared-encoder convolutional neural network are used to distinguish nominal operation, actuator loss of effectiveness, anomalous friction, and vibration from complete trajectories. Evaluation uses a simulated six degree of freedom Kinova manipulator under model predictive control and fixed plant-model mismatch, with single faults at a specified joint. Across five training seeds and 3000 held-out test runs, SHAPE-CWT achieves a mean accuracy of 84.69 percent, versus 81.67 percent for torque residuals and 97.17 percent for the generalized momentum observer under common CWT processing. The respective sample standard deviations are 0.70, 0.70, and 0.37 percentage points. These results demonstrate useful fault-type discrimination from algebraic power residuals, while severity-excitation profiles and offline velocity perturbations reveal limits to their diagnostic capability.
★ HARP: Handoff-Aware Recovery Planning for Active Target Acquisition in Multi-UAV Handoff
Continuous aerial tracking often requires multiple drones to maintain target coverage over long periods. During handoff, the receiving UAV may lack prior target observations and must actively acquire a usable view without disrupting the sender's ongoing coverage. This work introduces HARP, a handoff-aware recovery planner that registers bounded sender predictions in the receiver's local map, distinguishes field-of-view failure from physical occlusion, and generates failure-specific yaw, translation, and parallax viewpoints. Candidate motions are checked for receiver clearance, sender-fiducial observability, protected sender--target visibility, temporal validity, and deadline compliance. When a complete trajectory cannot be admitted, HARP may execute a sampled admissible nonzero prefix and replan from updated evidence. In simulation, HARP acquired the target in all 49 standard trials and in 15 of 16 stress trials. Field trials validated HARP's recovery geometry under field-of-view limitation and environmental occlusions.
★ HA-DiPO: Diffusion Policy Optimization for Heterogeneous Cooperative Transport
Cooperative transport with heterogeneous legged robots requires coordinating different motion capabilities while supporting an unfixed shared payload. We present HA-DiPO (Heterogeneous-Agent Diffusion Policy Optimization), a hierarchical multi-agent reinforcement learning framework under centralized training with decentralized execution (CTDE). The framework decouples team-level coordination from embodiment-specific motion control: at the high level, a centralized critic uses team-value estimates to jointly train independent coordination policies, aligning robot-specific action responses with a shared transport objective; at the low level, embodiment-specific robust motion policies track the coordination commands. The high-level diffusion policies offer more flexible action distributions than single-Gaussian parameterizations, allowing potentially multimodal coordination responses to be represented. The coordination policies learn from scratch through reward-driven simulator interaction without demonstrations. At execution, each robot generates coordination commands using only local robot and payload observations, without partner-state inputs or inter-robot messages. Simulation comparisons show that HA-DiPO achieves higher final mean returns and velocity-tracking success rates, as well as lower velocity errors, than the evaluated baselines. Real-world experiments further demonstrate quadrupedal-wheeled-legged and human-robot cooperative transport.
comment: 9 pages, 6 figures. Submitted to IEEE Robotics and Automation Letters (RA-L). Project website: https://ha-dipo.github.io/
★ Replicate What Varies: Resource Sharing for Domain-Randomized Embodied Simulation
Simulation provides a low-cost sandbox where embodied agents can attempt tasks, observe outcomes, and learn from experience. Running many diverse simulation environments concurrently can consume substantial CPU and GPU memory because independently constructed scenes duplicate resources that remain unchanged across instances. We present Patchwork, a resource-sharing system for domain-randomized simulation, built around the representation $E=T+Δ$: a shared template contains common resources, and a patch records the mutations needed to construct a target environment. Patchwork separates randomization sampling from replay and selects reusable templates together with patches that preserve each target environment's intended configuration. Its layered runtime shares compiled model structure and immutable assets while keeping mutable parameters and simulation state private to each instance. We implement Patchwork on robosuite and MuJoCo and evaluate it on LIBERO-plus and RoboCasa365 workloads. At 32 replicas, shared-template execution reduces CPU RSS by 88-92% relative to process-per-environment execution. Controlled experiments show how sharing shifts memory growth toward template diversity and unique asset content rather than instance count, within the supported sharing boundaries.
★ Memory-Aware Keyframe Management for Dense Maps on Memory-Constrained Robots ICRA 2027
Running a language model and a dense 3D map on one device means the two share one memory, and exceeding its limit is an out-of-memory failure. Map systems that keep a bounded working set bound it by a count of keyframes, but the memory one keyframe occupies varies widely with what it carries, so a count does not control what the working set occupies. The unit of admission is as poorly matched, since a radius around the pose, a fixed cell, or the graph neighbors of the last matched keyframe all move parts of the map that are not needed. We therefore propose two decisions. First, the memory the resident map uses is bounded directly, and the cost of every admission is known before any data is read, from per-keyframe resident sizes computed offline and carried with the map. Second, keyframes are admitted and released at a semantic spatial unit, the rooms and corridors of the building, so that the robot holds the zone it occupies together with the zone it is about to enter while the zones it has passed are released. Every admission measures the resident total, makes room, and only then reads, so the budget is never overshot. On two NVIDIA Jetson boards the budget holds in every run and the model gives up at most 0.8 percent of its throughput, whereas without it the operating system kills the model or the map. Under that budget, and over the environments evaluated, admission by semantic zone localizes 6-9 points more of the query frames than a geometric partition of matched zone count and about twice as much as the retrieval of RTAB-Map, at a lower ATE. Bounding memory lets the map and the model coexist, and moving the map in semantic zones makes the bounded map usable.
comment: 6 pages, 4 figures, 6 tables. Submitted to ICRA 2027
★ WorldFailBench: Do World Models Understand Policy Failures?
Robot policy evaluation informs deployment decisions and policy improvement, but physical testing and simulator preparation require substantial effort. World models offer a complementary route through closed-loop interaction in generated environments. Recent benchmarks have demonstrated their potential as policy evaluators through outcome-based evaluation (i.e., assessing task success or failure). However, such evaluation does not establish whether world models faithfully simulate how policies fail. Misrepresenting failures can lead to incorrect diagnoses of policy weaknesses and make policies appear ready for deployment even when small scene changes cause them to fail. We therefore ask: do world models faithfully simulate how and at which stages policies fail, and under what conditions they transition from success to failure? To answer this question, we introduce WorldFailBench with three evaluation tasks. Failure reason tests whether models reproduce the observable errors policies make. Failure stage assesses whether failures concentrate at the same stages of full-task execution. Failure boundary examines whether models reproduce the conditions under which policies transition from success to failure under controlled scene changes in simulation. The benchmark covers manipulation tasks in the LIBERO and RoboTwin simulation environments and real-world manipulation tasks performed by the G1 humanoid robot. Experiments across seven world models reveal substantial gaps in their understanding of policy failures, providing guidance for developing more reliable world models for policy evaluation.
★ MOON: Learning When to Forget and How to Move On in Vision-and-Language Navigation
Recent vision-and-language navigation (VLN) policies act on their visual history, and an incorrect action can take them off the instructed route. Paired interventions on four VLN policies show that clearing the history doubles pooled success at the onset of a deviation but lowers it on route, while later in a deviation, removing only the off-route frames helps much less. After a deviation, the retained history keeps the policy from turning where a turn is needed, which we call behavioral inertia. We propose MOON, a framework that equips a frozen VLN policy with two lightweight modules, adding less than 1% of its parameters. A deviation judge decides when to forget, raising an alarm that clears the history when it predicts that the agent is off route. A recovery policy, trained with GRPO from alarm states, learns how to move on, turning more of the successes found only by sampling into a single greedy rollout. Experiments on R2R-CE and RxR-CE show that MOON reaches the highest R2R-CE success rate among the compared methods and raises the success rate of its frozen base by 5.2 and 6.0 points, and an ablation with a random trigger indicates that this gain depends on when the history is cleared. Without retraining, MOON also improves zero-shot object-goal navigation on HM3D-OVON. Together, these results suggest that deviation recovery depends on knowing when to forget, not only how to act. Code is available at https://github.com/AutoLab-SAI-SJTU/MOON.
★ MimicAll: Structured Embodiment Conditioning for Cross-Humanoid Whole-Body Motion Tracking
Humanoid motion trackers reproduce diverse behaviors, but sharing a policy across bodies requires accounting for differences in structure, morphology and dynamics. We present MimicAll, a reinforcement-learning framework for shared whole-body motion tracking across heterogeneous humanoids. A structured embodiment descriptor is encoded into global and joint-aligned conditions, which are combined with observation history and body-matched motion references to predict joint-target offsets. Physics-consistent embodiment randomization, body-calibrated retargeting and heterogeneous simulation support training across 4,096 variants from 16 humanoid families. We introduce a benchmark of 464 bodies across five distribution strata to evaluate motion generalization, within-family embodiment shifts and unseen-family transfer. Controlled ablations show that embodiment information improves motion completion with simple concatenation, while self-attention yields the lowest global position errors. With equal training transitions, shared training achieves 99.4% 30-second success on randomized G1 bodies in 100STYLE, versus 42.9% for a single-body specialist. With Flat-MLP encoding and concatenation, continued training on a motion subset excluding obstacle categories achieves 98.3% and 93.5% 30-second success on 100STYLE for sampled training bodies and out-of-distribution variants within seen families, respectively. On an eight-motion LAFAN1 subset excluding ground and get-up sequences across 464 bodies, mismatched body conditions reduce self-attention's 30-second success from 61.7% to 47.0%, demonstrating a closed-loop benefit from embodiment information.
★ ReSight: Task-Driven Self-Improving Memory for Embodied Agents
Robots can use historical observations to answer spatiotemporal queries in familiar environments, but the presence of relevant evidence does not imply that existing tools support the operations a task requires. We introduce ReSight, a task-driven self-improving memory system for embodied agents that expands the capabilities for processing and accessing historical observations as task demands arise. ReSight retains keyframes and their spatiotemporal associations, separating observation evidence from extensible derived fields so that stored observations can be reprocessed for new needs. An execution agent and a coding agent share this memory through different interfaces: the former calls registered tools within a bounded online budget, while the latter uses a programmable workspace and a larger exploration budget to inspect evidence, write programs, and acquire models as needed. To realize capability expansion in ReSight, we develop Explore-Distill-Mount (EDM), which identifies capability gaps from execution and exploration records without task reference answers and converts exploration-derived solutions into reusable memory builders, query tools, and skills, extending memory representations and access capabilities as needed. Experiments on three public benchmarks demonstrate effective base memory access and capability expansion. With the execution model and online budget held fixed within each benchmark, EDM reduces mean position error on OC-NaVQA from 33.10 m to 24.72 m and improves high-level success on FindingDory from 38.3% to 63.8%. With the tools and skill frozen, success still improves by 28.3 percentage points on FindingDory episodes excluded from exploration and distillation. Experiments on real robot patrol memories further demonstrate reuse of these capabilities for subsequent queries and new memories, with fewer online tool calls on average.
★ Learning Dexterous Manipulation: Hands, Simulators, Training, and Evaluation
A dexterous hand can change an object's pose without setting it down. This survey covers 113 learning-based methods that attempt it, the hands they run on, the simulators they train in, and how they report results. The field has largely converged on one recipe: reinforcement learning on privileged state in a GPU simulator, distilled into a vision-only student for transfer to hardware. Almost half the corpus trains that way, nearly all of it in one simulator, whose own successors are barely used. A second and faster-growing branch learns from teleoperated demonstration instead, through vision-language-action models, diffusion policies, and systems whose contribution is the collection rig itself. Published work has tripled since 2022. Grasping remains the largest task family and bimanual coordination the fastest-rising, and almost every experiment runs on one of four hands. What the survey adds comes from reading released code and vendor specifications beside the papers. Eight hands that can be bought or built from published designs appear in no method paper here. Where the depth a hand sinks into the object it holds is reported at all, it is not measured in the physics that produced the motion. Section VII proposes a protocol for the unreported quantities; the body gives the counts behind every claim above.
comment: 36 pages, 14 figures, 7 tables. Survey of 113 learning-based dexterous manipulation methods, with a comparison of each paper's stated reward against its released code
★ MDPO: Multi-constraint Dexterous Policy Optimization with Reinforcement Learning
Dexterous manipulation requires high-dimensional control policies to accomplish complex contact-rich tasks while simultaneously satisfying multiple safety requirements, such as joint limits, unsafe finger configurations, arm collisions, and unstable object interactions. Constrained reinforcement learning (RL) provides a principled framework for incorporating these requirements into dexterous manipulation policy learning. However, existing constrained RL methods for dexterous manipulation primarily focus on cumulative cost constraints and struggle to simultaneously handle multiple heterogeneous constraints during dexterous policy learning. In this work, we propose Multi-constraint Dexterous Policy Optimization (MDPO) for safe dexterous manipulation. We first formulate dexterous manipulation as a multi-constraint policy optimization problem involving both cumulative cost constraints and state-wise cost constraints. Based on this formulation, we integrate multi-constraint projection into the dexterous manipulation policy optimization process. Specifically, we employ constraint-specific policy projections to map the intermediate policy that maximizes task performance onto the feasible regions defined by the cumulative cost constraint and the state-wise cost constraint. Extensive experiments on ReDMan, including safe joint control, finger safety, and grasping tasks, demonstrate that MDPO achieves superior constraint satisfaction while maintaining competitive task performance compared with the baselines. Our code is available at this link.
★ A Differentiable Chassis Safety Layer for Intelligent Vehicle Motion Control
Learning-based vehicle motion control requires coordination between diverse task objectives and state-dependent chassis safety constraints. Conventional cascaded architectures may compromise task performance when safety filters modify nominal commands independently of policy learning. This article proposes a differentiable chassis safety layer that modularly integrates task-oriented control with explicit physical constraints. Actuator limits, individual-tire force constraints, and vehicle handling envelopes are incorporated into sequential quadratic programs. Each projected command propagates the predicted vehicle dynamics and updates subsequent constraints, enabling temporal gradient coupling through implicit differentiation. A two-stage learning strategy combines expert imitation with risk-guided supervision and conditional intervention guidance to coordinate nominal control decisions with chassis safety constraints. Closed-loop evaluations under unseen scenarios, emergency maneuvers, and ablation studies demonstrate that the proposed method improves safety-performance coordination and exploits vehicle dynamic potential while maintaining handling stability.
★ SCET-CBF: Speed-Calibrated Soft Guidance for One-Step Dynamic Navigation
Control barrier functions (CBFs) support collision avoidance, yet dense dynamic encounters can leave a vehicle with no feasible command. Geometric refinements and soft constraints help preserve motion choices. Using these choices also requires accounting for the vehicle's changing turning response. For a steering-limited bicycle, braking can lower closing speed while slowing the lateral motion needed before an obstacle arrives. We propose SCET-CBF, a speed-calibrated soft guidance law for one-step dynamic navigation. A short lateral escape maneuver gives an allowable approach speed. Its speed derivative allows a quadratic program to weigh both effects of braking when choosing acceleration and steering. Soft penalties accommodate competing obstacle preferences within the input set permitted by segment and braking constraints. We derive the guidance and establish protection over each accepted motion segment, together with conditional braking invariance for a single obstacle. On 2500 paired tasks with a 60 s deadline, SCET achieves 72.8% completion, compared with 49.8% for CBF/velocity-obstacle control and 69.8% for an enhanced 1 s CBF-MPC. At 100 obstacles, SCET and the enhanced MPC complete 52.4% and 33.6% of tasks. Paired studies identify the contributions of speed feedback and early activation. Further evaluations examine the operating range under lateral acceleration limits and bounded motion and execution errors.
comment: 9 pages, 6 figures
★ MOTIO: Agentic Skill Engineering for Reality-Grounded Humanoid Loco-Manipulation
We present MOTIO, an agentic skill-engineering system that turns ordinary human-action videos, reality-grounded 3D assets, motion references, and reusable robot skills into executable humanoid loco-manipulation programs. Through in-context task and contact reasoning, an agent constructs staged robot-native interactions and improves them using physics execution, multi-frame feedback, and human validation---without task-specific teleoperation or robot demonstration collection. Agentic 3D supplies metric product and environment assets, while existing whole-body, inverse-kinematics, and dexterous-hand interfaces realize the programs. Across 33 timestamped first approvals, the median recorded agent-turn time is 97.0 minutes and request-to-approval duration is 2.80 hours. Simulation-side layout and object variation prepare a long-horizon bottle-disposal skill for a continuous 101.75-second physical G1 execution in a scanned office; physical refinement reduces screw-tightening thread residual by 76.0%. Agent-generated skills additionally supervise BC/DAgger distillation into one shared reference policy, completing 18 simulated tasks under the same checkpoint without inference-time expert intervention. Project page: https://iiiiqiiii.github.io/motio/
comment: 24 pages, including 11 pages of supplementary material
♻ ★ PRISM: PRior-guided Imagination Sampling in world Models
Latent world models learn representations that enable them to predict how actions change the environment - a capability we call physical intuition. We find that these same representations also support action intuition: a lightweight head can learn a goal-conditioned action prior directly from the frozen encoder's features. We introduce PRISM to investigate how action intuition can be extracted from frozen world-model representations and used to guide goal-conditioned planning. The prior's mean and predicted uncertainty guide the search for useful actions, while the world model evaluates their consequences against the goal. Experiments on four simulated control tasks and two real-robot platforms show that a lightweight action head improves planning performance, and that the same fusion interface accepts a diffusion prior over the same frozen features without modification. These results demonstrate the potential of reusing world-model representations to both predict action consequences and guide action selection.
♻ ★ StepTrigger: Contact-State-Triggered Backdoor Attacks on VLM-Powered Legged Robots
Large language models and vision-language models are increasingly used as high-level planners in robotic systems, using task goals and sensor summaries to select navigation or manipulation actions. This creates a new backdoor surface: a compromised planner can behave normally in most runs, yet change its target selection when a hidden trigger is present. Prior attacks on LLM-powered or embodied agents mainly rely on triggers that appear in language, camera-visible objects, scene semantics, or specific sequences of past actions. This paper presents StepTrigger, a contact-state-triggered backdoor attack for VLM-powered legged robots. The trigger is not a prompt token or a visible marker. It is produced by pressure and foot-ground contact patterns that arise when a Unitree Go1 quadruped walks across a dense terrain patch. Unlike conventional visual or textual triggers, contact signals are inherently noisy and may also arise during benign locomotion. To avoid treating every pressure anomaly as a trigger, StepTrigger learns a selective backdoor policy from multimodal robot state, using incidental pressure events as benign examples and dense-patch contacts as poisoned examples. In a stratified offline evaluation, the trained planner achieved 98.75% clean behavior preservation, 92.50% false-trigger rejection, 76.25% true-trigger activation, and 89.17% overall parsed behavior accuracy. These results reveal a backdoor surface in proprioceptive and contact channels that is not captured by defenses focused only on language, vision, or action history.
♻ ★ BEACON: Cross-Domain Co-Training of Generative Robot Policies via Best-Effort Adaptation
We introduce BEACON--Best-Effort Adaptation for Cross-Domain Co-Training--a theory-guided framework for training generative robot policies with abundant source demonstrations and limited target demonstrations. BEACON casts cross-domain co-training as a discrepancy-aware importance-reweighting problem, jointly learning a diffusion-based visuomotor policy and per-sample source weights through an objective inspired by the Best-Effort Adaptation (BEA) bound. To make this approach practical for high-dimensional sequence policies, we develop scalable instance-level discrepancy proxies, stochastic alternating updates for policy and weights, and a multi-source extension that balances heterogeneous source domains. Across sim-to-sim, sim-to-real, and multi-source manipulation settings, BEACON improves robustness and data efficiency over target-only, fixed-ratio co-training, and feature-alignment baselines. Without an explicit alignment objective, BEACON also brings source and target representations closer together as an implicit result of discrepancy-aware co-training.
♻ ★ Making Space for Time: The Special Galilean Group and Its Application to Some Robotics Problems IROS
The special Galilean group, usually denoted SGal(3), is a 10-dimensional Lie group whose important subgroups include the special orthogonal group, the special Euclidean group, and the group of extended poses. We briefly describe SGal(3) and its Lie algebra and show how the group structure supports a unified representation of uncertainty in space and time. Our aim is to highlight the potential usefulness of this group for several robotics problems.
comment: In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Workshop From Geometry to General Autonomy of Robotic Systems, Abu Dhabi, United Arab Emirates, October 15, 2024. 3 pages, 1 figure
♻ ★ WorldOracle: Benchmarking Action-Conditioned World Models against Real-World Executions
We introduce WorldOracle, a benchmark specifically designed to evaluate action-conditioned world models (ACWMs) against paired real-world executions. Given the same initial observation and low-level action sequence, WorldOracle asks whether a predicted rollout reproduces what happens in the real world, including physical consequences and task-state transitions. We evaluate models along four complementary dimensions: physical state correctness, interaction spatial alignment, key interaction area fidelity, and multi-view action coherence. To support this evaluation, we curate real-world training and evaluation suites that cover different interaction dynamics and embodiments. The evaluation suite contains long, multi-segment episodes with both successful and failed executions, enabling evaluation across varied action outcomes and behavior distributions. Experiments on seven representative ACWMs reveal distinct model strengths and failure modes. WorldOracle provides a systematic foundation for measuring and advancing action-conditioned world modeling for manipulation. We release the datasets, evaluation code, and leaderboard: https://worldoracle-benchmark.github.io/.
♻ ★ UniManip: Open-World Robotic Manipulation via Agentic Operational Graphs
Achieving general-purpose robotic manipulation requires robots to bridge high-level semantic intent with low-level physical interaction in unstructured environments. However, zero-shot generalization remains challenging: end-to-end Vision-Language-Action (VLA) models often lack the precision required for long-horizon tasks, while traditional hierarchical planners suffer from semantic rigidity under open-world variations. We present UniManip, a unified framework centered on an Agentic Operational Graph (AOG), an explicit cognitive and operational backbone for task orchestration, state tracking, execution verification, and closed-loop failure recovery. High-level semantic intents are grounded into actionable primitives via a physically grounded scene graph that provides geometric affordances, contact-aware interaction parameters, and collision-free trajectory synthesis. By coupling a high-level Agentic Layer for task orchestration with a low-level Scene Layer for dynamic state representation, UniManip continuously aligns abstract planning with geometric constraints, enabling robust execution. Unlike static pipelines, UniManip operates as a dynamic agentic loop: it instantiates object-centric scene graphs from unstructured perception, parameterizes these representations into collision-free trajectories via a safety-aware local planner, and exploits structured memory to autonomously diagnose and recover from execution failures. UniManip achieves state-of-the-art performance on LIBERO-Pro, while extensive real-world experiments demonstrate robust zero-shot generalization across diverse tasks, embodiments, and dexterous hands, outperforming leading VLA models and hierarchical baselines by 22.5% and 25.0% in task success rate, respectively. Open-source project page: https://henryhcliu.github.io/unimanip.
comment: 18 pages, 15 figures, 9 tables, project page: https://henryhcliu.github.io/unimanip
♻ ★ GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation
World-action models (WAM) predict future states to guide robot actions, enabling learning from both action-free video and action-labeled interaction. Most inherit pretrained video generators, leaving WAM pretraining and scaling underexplored. We introduce Genie Envisioner Act 2.0 (GE-Act 2.0), a world-action model whose trainable generative and action components are all initialized from scratch on manipulation data. It combines a control-oriented autoencoder (CoAE), a single-step visual planner (SVP), and an inverse dynamics model (IDM). CoAE retains action- and instruction-relevant information under aggressive compression, while SVP produces a complete future state in one differentiable pass, so visual planning and inverse dynamics can be pretrained separately on complementary data. The components are then jointly trained with knowledge-aligned selective optimization (KASO), which reduces mismatched supervision by selecting only predicted futures judged behaviorally compatible with the recorded action. We evaluate pretrained checkpoints directly, without per-task fine-tuning, on 100 tasks across 20 manipulation skill groups with held-out scenes, backgrounds, lighting, and object instances. Scaling co-training data from 300 to 30,000 hours raises success from 17.1% to 44.1% on G1-OP and from 13.4% to 31.1% on G2-90D; despite comprising less than 2% of the co-training data, G2-90D improves by 17.7 points, suggesting cross-embodiment transfer. Gains span 19/20 and 18/20 skill groups, and skill-specific coverage strongly correlates with zero-shot out-of-distribution (OOD) success (Pearson r=0.80; Spearman rho=0.85). Under the same protocol, the model grounds object, color, shape, and position references in at least 90% of trials and follows explicit instructions even when they conflict with an already-committed behavior or a conventional scene association.
comment: Technical report by the AgiBot Research Team. Project page: https://ge-act-v2.github.io/
♻ ★ Haptic Sorter: A Unified Planning Framework for Online Shape Estimation and Real-Time Pose Inference
Robotics manipulation usually assumes that the shape and pose of the object are known to the robot prior to motion planning. However, precise geometric information is not always available in practice, and pose inference suffers from sensor uncertainties and view occlusion. In this work, we propose a unified model-based geometric framework integrating robotic haptic perception, modeling, and manipulation planning. Our novelties involve: \textit{i)} Introducing Bayesian Optimization (BO) to guide the haptic exploration for object shape inference, where superellipses are used to approximate geometric boundary; \textit{ii)} Adaptive formulation of manipulation potential encoding object geometry for quasi-static robot-object interaction; \textit{iii)} Proposing an online Ordinary Differential Equation (ODE) for real-time pose inference based on model prediction and tactile feedback. We deploy our system on a 2D robotic sorting task, and vary object geometries to validate the robustness and generalizability of our framework in both simulation and a real-world multi-arm setup.
♻ ★ Motion Retargeting with Spatiotemporal Preferences for Humanoid Loco-Manipulation
Human motion retargeting bridges the embodiment gap between humans and humanoid robots by converting human demonstrations into reference trajectories for whole-body control and loco-manipulation. For object-interaction tasks, successful execution depends on accurately preserving body--object relations at critical stages. However, uniform per-frame iteration budgets do not account for these localized accuracy requirements, limiting the computational efficiency of retargeting. To address this limitation, we present Spatiotemporal Retargeting (STR), which uses keyframes and key body parts to allocate optimization effort. Specifically, a vision-language model selects relevant motion signals and body parts, and keyframes are identified from changes in the selected signals. Based on these selections, STR assigns additional solver iterations to keyframes and higher interaction weights to the selected parts. To further improve contact fidelity, we introduce Mechanics-Aware Contact Anchoring (MCA) to refine object-surface contact positions using an approximate object-wrench model while retaining the selected surface directions. Building on the resulting references, we develop Spatiotemporal Reinforcement Learning (STRL), which combines adaptive key-stage sampling with a contact-position reward based on target proximity for downstream control. Simulation experiments on a Unitree G1 and real-world experiments on a Unitree R1 demonstrate that STR improves geometric contact coverage and continuity at lower computational cost, while the resulting references improve downstream task completion across diverse object-interaction motions.
♻ ★ Learning to Absorb Corrections: Critic-Weighted Residual Flow for Vision-Based Insertion
Vision-based insertion requires correcting alignment errors while retaining useful behavior learned from demonstrations. We present ACT--Flow, an online learning framework that couples an adaptive Action Chunking Transformer (ACT) with a conditional residual flow policy. Following anchor-guided warmup, ACT absorbs successful executed actions through behavioral cloning, while residual targets are recomputed relative to the updated prior. A chunk-level critic learns from successful and failed interactions and evaluates final composed actions. Its estimates weight generated residual candidates for flow matching, complemented by supervision from successful real executions. Candidate scoring is confined to training; deployment uses RGB observations and samples one residual per action chunk. We evaluate the framework on power-plug, USB, and key insertion, together with cable routing and needle threading. Results show a five-task mean success rate of 66% at 20,000 post-initialization interaction steps, exceeding the strongest baseline by 18 percentage points at this checkpoint. Evaluations at matched checkpoints with the residual disabled show ACT's mean success increasing from 16% to 76%. Component ablations further support the complementary roles of behavior absorption, value weighting, and successful-action supervision.
comment: 9 pages, 7 figures
♻ ★ Geometry-Aware Epipolar Correspondence Graphs for Monocular Visual Odometry
We demonstrate Graph Representation Learning to address challenges that arise in visual odometry. In this context, the change in pose of a camera is typically estimated using matched correspondences in frames that satisfy the epipolar constraint; finding this unique set under different situations is not a simple task. Employing an existing detector-free model to find matching correspondences in a pair of images, a correspondence graph is constructed whose topology is guided by the epipolar constraint. Starting with a graph where every node is connected to $k$ closest neighbours (by distance), this graph is pruned based on an initial estimate of the Essential Matrix; this feature is in stark contrast with traditional graph formulations where graphs from local spatial neighborhoods defined by heuristics are constructed. The graph learning module is then trained to generate an aggregated representative vector that summarises the geometric context of the graph. This vector forms an input to a Multi-layer Perceptron (MLP), whose outputs are standard rotation and translation elements that define the change in pose. To investigate the effect of graph learning on this representation, four variations of Graph Neural Networks (GNNs): GCN, GAT, GIN, and Graph-based Cross Attention are evaluated under a unified learning framework. The application to monocular visual odometry is validated using outdoor benchmark datasets such as KiTTi, ETH3D, King's College, and Tartan Air. Experimental results demonstrate consistent improvements in trajectory and relative pose estimation across diverse metrics when compared with classical consensus-based estimators and learning-guided correspondence methods.
comment: 22 pages, 6 figures, 13 Tables, Submitted to Pattern Recognition Journal
♻ ★ RoboEvolve: Co-Evolving Planner-Simulator for Robotic Manipulation with Limited Data
The scalability of robotic manipulation is fundamentally bottlenecked by the scarcity of task-aligned physical interaction data. While vision-language models (VLMs) and video generation models (VGMs) hold promise for autonomous data synthesis, they suffer from semantic-spatial misalignment and physical hallucinations, respectively. To bridge this gap, we introduce RoboEvolve, a novel framework that couples a VLM planner and a VGM simulator into a mutually reinforcing co-evolutionary loop. Bootstrapped for self-evolution with unlabeled seed images, RoboEvolve leverages a cognitive-inspired dual-phase mechanism: (i) daytime exploration fosters physically grounded behavioral discovery through a semantic-controlled multi-granular reward, and (ii) nighttime consolidation mines "near-miss" failures to stabilize policy optimization. Guided by an autonomous progressive curriculum, the system naturally scales from simple atomic actions to complex tasks. Extensive experiments demonstrate that RoboEvolve (I) achieves superior effectiveness, elevating base planners by 30 absolute points and amplifying simulator success by 48% on average; (II) exhibits extreme data efficiency, surpassing fully supervised baselines with merely 500 unlabeled seeds--a 50X reduction in initial RL data; (III) demonstrates robust continual learning without catastrophic forgetting; and (IV) extends to low-level control, improving cold-started LingBot-VA from 78.7% to 98.7% average task success on LIBERO using environment feedback. Together, these results demonstrate the applicability of our co-evolutionary framework to both internally supervised generative learning and environment-grounded action learning.
comment: v2: Extended RoboEvolve to the world-action model setting, with new low-level control experiments on LIBERO
♻ ★ A Semantic Autonomy Framework for VLM-Integrated Indoor Mobile Robots: Hybrid Deterministic Reasoning and Cross-Robot Adaptive Memory
Semantic indoor navigation requires robots to connect human instructions with spatial destinations, execute missions and retain useful operational experience. This paper presents the Semantic Autonomy Stack (SAS), a six-layer integration framework, and develops its semantic-reasoning, mission-interface and operational-memory layers over a previously validated ROS 2/Nav2 foundation. A deterministic-first cascade combines the route graph, semantic objects, robot state and stored instruction-destination associations, escalating unresolved or guarded cases to a pretrained Vision-Language Model (VLM). Recurrent model-mediated associations can pass a frequency-and-consistency gate and be compiled for deterministic M3 reuse; this gate measures sampled-output stability rather than semantic truth. The frozen v4.8-review implementation was evaluated on two physical robot configurations in one indoor environment, with onboard navigation and perception and shared-workstation reasoning and memory. Of 216 decision cycles outside the start gate, 112 resolved deterministically at sub-millisecond resolver-compute latency. A sealed 28-instruction challenge yielded 17 acceptable outcomes; five fast-path decisions included one incorrect selection. One promoted association was reused on the second platform through deliberate shared-digest installation. A controlled 594-observation offline comparison showed greater agreement with the stored associations and lower decision latency from M3 reuse in five familiar cases. The M3-disabled cascade reduced VLM calls but matched direct-VLM decision quality in standardized context. Large latency savings were concentrated in paths avoiding VLM inference. Together, the results establish a working integration of selective hybrid reasoning and reusable operational memory, while identifying implementation and evaluation boundaries for further development within SAS.
comment: Published version (open access, CC BY 4.0). 32 pages, 6 figures, 10 tables, 2 algorithms. This version replaces the submitted manuscript (v1); the analysis, results and claims were revised during peer review, and the published article supersedes v1
Multiagent Systems
★ Build, Don't Play: Self-Improving LLM Research Agents Develop a Hearthstone Battlegrounds AI That Rivals Top Players
Can LLM agents carry a hard real-world problem from raw data to a system that holds its own against people? We study this in Hearthstone Battlegrounds, an eight-player auto-battler with stochastic shops and battles, hidden information and nearly a thousand interacting cards. We present AutoTavern, a self-improving research system: specialised LLM research agents share a typed research memory, every claim passes verification gates, and live games close the loop, so the agents improve the simulator, opponents and policy and, from the same feedback, their own research process; a human sets goals and rules on ambiguous mechanics. In 20 days AutoTavern built TavernGym, the first faithful, full-content reproduction of the live game, and delivered 23 versions of TavernAgent, trained with PPO against opponents calibrated to real games. TavernAgent plays complete games on the live ladder by itself: its rating rose from 6,000 to about 15,000 and, on the public leaderboard's measure, its last 20 records average a placement of 1.60 with 80% first places, better than the server's ten highest-rated players on every statistic (2.34, 59%). TavernBench, 960 tasks in 40 scenarios, shows that the system the agents built is the strongest player: TavernAgent answers 91% of the play tasks against at most 63% for frontier LLMs, and decides in 0.03 s where they need up to 26 s; within the second a live turn allows, it keeps 90% and no LLM more than 26%. LLM agents build far better than they play. We will open-source TavernBench and TavernGym at https://github.com/MCG-NJU/AutoTavern.
comment: 20 pages, 8 figures, 15 tables. Code and benchmark: https://github.com/MCG-NJU/AutoTavern
★ Secure and Functional Code Generation with Self-Evaluation
Large Language Models (LLMs) can generate functionally correct code that remains insecure. Security-focused fine-tuning can be expensive and is not always available, motivating inference-time approaches. Security-oriented prompting and retrieval can reduce vulnerabilities, but may also remove required behavior or apply defenses incompatible with the task. The challenge is to identify relevant security knowledge and use it to improve security without sacrificing functionality. We present SecGent, an inference-time framework that retrieves weakness context from the Common Weakness Enumeration (CWE) database and applies it to secure code generation. By connecting task specifications to relevant textual CWE records, SecGent provides task-specific security guidance without relying on retrieved vulnerable or patched code examples. Complementary functional and security tests check the resulting implementation, and their execution feedback guides repairs toward satisfying both objectives. We evaluate SecGent with four open-source LLMs and Claude Sonnet 5.5 across five programming languages on the CWEval benchmark. It achieves the highest joint functional--security success against four inference-time baselines for all five models. Across the open-weight models, mean functional-security@1 (FS@1) improves by 15.5% over RAG, with gains in both functionality and security, and by 19.8% over the strongest prompt optimizer compiled for each model. On Sonnet, FS@1 reaches 92.1%, outperforming the strongest baseline AutoGen by 4.8%. In a separate comparison, SecGent outperforms Claude Code with security-focused prompting by over 6% in FS@1. Feedback-guided repair further improves FS@1 by up to 6.7% over generation without feedback, primarily by recovering functionality while checking security.
comment: 18 pages, 7 figures
★ Agent-to-Agent Synthetic Social Engineering on the Web
An AI agent can prepare an accurate report and still request an action its user has not authorized. Cooperation across Web services therefore requires separate judgments about identity, evidentiary support, and permission. We define synthetic social engineering (SSE) as deliberately deceptive or misleading influence intended to exploit AI recipients' responses to credibility, authority, or obligation cues and redirect behavior away from authorized objectives. We focus on separately tasked sender agents that produce or adapt misleading communication. Drawing on cybersecurity, social engineering, and persuasion research, we develop an operational framework connecting actors, deceptive representations, candidate influence routes, evidence or permission boundaries, responses, and consequences. The framework distinguishes attempts from successful redirection, without assuming human psychological processes. We implement it in the Agent Service Exchange and Evidence Commons (ASE-EC), a reproducible testbed with independently specified records and authorization scopes. Across 1,260 research episodes using Qwen3-4B in two fictional briefing tasks, matched comparisons test authority and commitment appeals, while defense comparisons assess protection alongside authorized task completion. Invoking a limited prior agreement increased unauthorized requests from 43.3% to 96.7%, although recipients retained supported decisions and permissions remained unchanged. These controlled findings identify action selection as a distinct security concern, even when evidence handling succeeds, and they provide a reusable basis for evaluating safeguards that preserve useful cooperation across Web services.
★ PASS Is a Claim, Not an Observation: Mechanical Re-verification of 10 Self-Asserted Fix Claims in a Live Multi-Agent Audit Layer
When agents claim to have fixed their own defects, how do we know the fix is real? We mechanically re-verified 10 self-asserted code-level fix claims from a privileged coordination agent (azone_god) operating in AITRAP, a live multi-agent knowledge-tree platform. These 10 claims were extracted from a corpus of 179 coordination broadcasts (28 containing structured fix assertions, 31 unique fix-ID/message pairs, 18 distinct fix identifiers). Fix claims use mixed English/Chinese formats (PASS, checkmarks, and "verification passed" declarations). Each code-level claim was checked against three criteria: (1) file modification timestamp (mtime) consistency -- 70% pass; (2) presence of the claimed defensive pattern at the specified identifier and call site -- 60% pass in the claimed file (80% anywhere in the codebase); (3) deterministic rule-based re-verification from the broadcast announcement alone -- 80% pass. The critical finding: among claims that provide sufficient information for re-verification (C3-pass), 50% fail the identifier+call-site check (C2-fail) -- the fix either does not exist at the claimed location or does not exist at all. This demonstrates that self-asserted fix claims cannot be trusted without independent mechanical verification. Our central claim: a PASS declaration is a claim requiring independent verification, not an observation of fix completion.
comment: 8 pages, 4 tables. v1.1 revised Oct 24, 2026
★ Network Topology Does Not Matter: The Breakdown of Classical Consensus in LLM Populations
As multi-agent language model networks become widely deployed, predicting their collective behavior is a critical challenge. Classical distributed consensus theory relies on network topology to predict convergence, structurally assuming that agent updates are convex combinations, population means are conserved, and weights are content-independent. In this work, we demonstrate that these foundational assumptions fail for large language models (LLMs). By deterministically extracting continuous credence from token-level probabilities across eight instruction-tuned models and 73,080 controlled exchanges, we directly measure the empirical LLM update operator. We find that LLMs systematically violate convexity by updating outside the hull of observed opinions, exhibiting active repulsion (negative weights) that fundamentally breaks row-stochastic modeling and standard bounded-confidence repairs. Scaling these empirically fitted operators to simulations of 10,000 agents reveals that algebraic connectivity explains almost none of the final outcome variance. Instead, population dynamics are governed entirely by the model's internal update bias and message framing. Consequently, populations of several models, including those with up to 7B parameters, reliably compound small local drifts to reach confident agreement on objectively false claims regardless of their network structure. Our results prove that classical graph-theoretic metrics are inadequate for LLM populations, highlighting a profound deployment risk where multi-agent deliberation predictably saturates in collective delusion.
comment: 19 pages, 3 figures, 3 tables
♻ ★ Asymmetric-Information Resource Allocation Games: An LP Approach to Purposeful Deception
In this work, we introduce the Deceptive Resource Allocation Game (DRAG), which studies purposeful deception within a Bayesian game framework. In DRAG, a Defender allocates resources across the true asset and several decoys to influence an Attacker's beliefs and actions, with the goal of diverting the Attacker away from the true asset. We seek to characterize purposeful deception, whereby the Defender deceives only when doing so improves its performance. To this end, we solve for the Perfect Bayesian Nash Equilibrium (PBNE) of the corresponding game. We show that, despite the coupled belief-policy interdependence, the problem admits an efficient, non-iterative linear programming formulation. Numerical results demonstrate that the resulting policies naturally balance effective allocation and belief manipulation, giving rise to purposeful and emergent deceptive behaviors.
♻ ★ A Theory-Guided LLM Pedagogical Agent Supporting Open-Ended STEM+C Learning
Open-ended learning environments (OELEs) can support STEM+C (science, technology, engineering, mathematics, and computing) inquiry through activities such as computational modeling, which requires students to translate disciplinary knowledge across domains and coordinate scientific concepts with computational representations. However, this is often challenging for students, many of whom require support during problem solving. In this exploratory study, we present CoSMoAgent (Computational Science Modeling Agent), a GPT-5-based multi-agent knowledgeable peer for high-school kinematics modeling grounded in the Evidence-Decision-Feedback (EDF) framework. CoSMoAgent goes beyond dialogue: it infers students' strategies, learner states, and knowledge gaps from environment logs and evolving model states. These inferences guide dialogue policies that prioritize probing and hints over direct answers. We deployed CoSMoAgent with 11 dyads across three modeling tasks (33 dyad-sessions), using a mixed-methods design to examine learning outcomes, processes, and adaptive scaffolding. As task mastery increased, students verbalized their understanding more successfully and sought help less often, and the agent's policies shifted from probing toward suggesting actions and extending reasoning. Two contrasting case-study dyads followed markedly different problem-solving paths, yet achieved similar learning gains despite substantial differences in task mastery and CoSMoAgent interaction. Students valued the agent's questions but were often frustrated by its withholding of answers. The study contributes a design pattern and classroom evidence for LLM scaffolding grounded in environmental activity rather than chat alone within an open-ended modeling environment.
comment: Submitted to Educational Technology Research and Development (ETR&D). Currently under review
♻ ★ Scam Town: Can Scam Education Spread Like a Vaccine?
Most older adults who hear a warning about scams hear it from someone they know: a relative, a neighbour, or a friend who was targeted. Yet research on scam prevention almost always studies one person at a time. We ask whether scam awareness can spread through a community and protect it the way a vaccine does, since a warning protects whoever hears it and can be passed on. To find out, we built Scam Town, a simulated community in which a large language model plays each resident. Scam centres phone the residents, and after a call a resident can warn the people they know. Word of mouth helped, but it did not take off on its own: each warning led to fewer than one more, so a community cannot count on warnings "going viral". Across 80 simulated towns, everyday talk cut the number of people scammed by roughly a fifth. The largest gains came when an official warning, from a bank, a senior centre, or an agency, reached most of a town and residents then talked about it: in simulation (a combination we did not test live) the two together cut scams by about half. Protection was greatest when warnings reached the people scammers actually call, who here were older residents. Because our residents are language-model stand-ins rather than real people, we treat these findings as predictions to test with people, not as proof. They point to a simple recipe for scam education: give people a clear story they can retell, design campaigns to be passed along, and reach first those most likely to be called.
comment: 28 pages, 12 figures, 11 tables
♻ ★ Computing Evolutionarily Stable Strategies in Multiplayer Games
We present an algorithm for computing evolutionarily stable strategies in symmetric normal-form games with three or more players.
♻ ★ Computing Evolutionarily Stable Strategies in Imperfect-Information Games
We present an algorithm for computing evolutionarily stable strategies (ESSs) in symmetric two-player perfect-recall extensive-form games of imperfect information. The algorithm operates directly in sequence form, avoiding explicit enumeration of normal-form supports. In a biologically motivated cancer signaling game, the algorithm recovers all seven symmetric Nash equilibria and identifies all three ESSs, each with a clear biological interpretation. Experiments on random imperfect-information games evaluate scalability as the numbers of private signals and available actions increase.
♻ ★ The Hive Mind is a Single Reinforcement Learning Agent
Decision-making is an essential attribute of any intelligent agent or group. Natural systems are known to converge to effective strategies through at least two distinct mechanisms: collective decision-making via imitation of others, and trial-and-error by a single agent. This paper establishes an equivalence between these two paradigms. We show that the emergent distributed cognition (sometimes referred to as the hive mind) arising from individuals following simple, local imitation-based rules is that of a single online reinforcement learning (RL) agent aggregating action-value samples from many parallel instances of a multi-armed bandit. More specifically, we show that, in the blindly imitative weighted voter model of honey bees' waggle dance, the update rule through which this macro-agent learns is an RL algorithm that we coin Maynard-Cross Learning. Our analysis implies that a group of individuals following simple imitative strategies can be equivalent to a more complex learning entity, substantiating the idea that group-level intelligence may explain how seemingly irrational individual behaviors are selected in nature. Beyond biology, the framework offers new tools for analyzing economic and social systems where individuals imitate successful strategies, effectively participating in a collective learning process. Our findings may further inform the design of scalable RL-inspired collective systems in artificial domains.
♻ ★ Artificial Leviathan: Exploring Social Evolution of LLM Agents Through the Lens of Hobbesian Social Contract Theory
The emergence of Large Language Models (LLMs) and advancements in Artificial Intelligence (AI) offer an opportunity for computational social science research at scale. Building upon prior explorations of LLM agent design, our work introduces a simulated agent society where complex social relationships dynamically form and evolve over time. Agents are imbued with psychological drives and placed in a sandbox survival environment. We conduct an evaluation of the agent society through the lens of Thomas Hobbes's seminal Social Contract Theory (SCT). We analyze whether, as the theory postulates, agents seek to escape a brutish "state of nature" by surrendering rights to an absolute sovereign in exchange for order and security. Our experiments unveil an alignment: Initially, agents engage in unrestrained conflict, mirroring Hobbes's depiction of the state of nature. However, as the simulation progresses, social contracts emerge, leading to the authorization of an absolute sovereign and the establishment of a peaceful commonwealth founded on mutual cooperation. This congruence between our LLM agent society's evolutionary trajectory and Hobbes's theoretical account indicates LLMs' capability to model intricate social dynamics and potentially replicate forces that shape human societies. By enabling such insights into group behavior and emergent societal phenomena, LLM-driven multi-agent simulations, while unable to simulate all the nuances of human behavior, may hold potential for advancing our understanding of social structures, group dynamics, and complex human systems.
♻ ★ MedCache: Efficient and Temporally Valid Memory for Longitudinal Clinical Agents
Longitudinal clinical agents must maintain an evolving patient state from evidence distributed across visits, time points, and specialties. However, how agent memory should be designed for this setting remains unclear. We introduce a benchmark of multi-visit, multi-specialty patient records that evaluates long-context evidence retrieval, cross-time evidence aggregation, and cross-specialty clinical reasoning. Using this benchmark, we systematically study four memory design choices: curation, organization, retrieval, and memory-augmented reasoning. We find that temporal validity is more important than simply retaining more history; specialty-factorized memory reduces context but can hide shared evidence; and multiple agents help when specialists must reason together, not merely when evidence comes from multiple memories. Guided by these findings, we propose \textit{MedCache}, a hybrid framework that constructs temporally valid patient memory, organizes evidence into overlapping specialty views, routes each query to relevant memories, and adaptively invokes one or multiple specialists. Experiments show that MedCache improves reasoning accuracy and memory efficiency over strong single-agent and multi-agent baselines, while generalizing across model backbones and external datasets.
♻ ★ Agentic Societies Need a Social Harness
An agentic society is a collection of AI agents that coordinate autonomously across trust boundaries on behalf of different principals. We show experimentally that in such societies even honest, competent agents often fail to reach satisfactory outcomes when using current harnesses and messaging primitives; and faulty or malicious agents can completely stall progress, influence outcomes, and pursue harmful goals by exploiting communication vulnerabilities. We argue that agentic societies need a \emph{social harness}. Unlike a \emph{personal harness}, which manages private context and communication with the principal, a social harness manages interactions with other agents. We propose a layered architecture for social harnesses which (i) prevents classes of communication failures outright, (ii) enables agents to detect invalid messages, and (iii) supports post-facto investigation and consequences.
♻ ★ Leadership as Coordination Control: Behavioral Signatures and the Recovery-Advantage Boundary in Multi-Agent LLM Teams
Team science holds that leadership is contingent: it helps only under specific conditions. We ask the same question of multi-agent LLM teams: under what measurable conditions does process-level coordination control add value, and do those conditions match what team science predicts? This takes measurement, not just accuracy: we use behavioral signatures (majority lock-in, exploration, recovery from an incorrect round-0 consensus) and per-action ablations, clean because each controller is an explicit action set. We operationalize three classical leadership styles (transactional, transformational, situational) as controllers over a shared action vocabulary (explore, revise, accept, synthesize). A matched controller with the same actions but an arbitrary rule recovers no better than majority voting. Across four task regimes and three open-weight model families, no controller dominates by accuracy. Transactional control matches a shared round-0 vote on all 12 combinations to within 1.3pp, and accuracy gains appear only on the single llama-4-scout social combination, where the round-0 majority is unreliable. Against the stronger flat baseline, only situational still gains (+8pp). A recovery-advantage account, tested with four boundary probes, says a controller beats plain interaction only where the round-0 majority is unreliable, the task is recoverable, and undirected interaction does not already repair it. Because these conditions map onto contingency theory (leadership substitutes, path-goal redundancy, the readiness gap), a largely null accuracy result is what the theory predicts, not a failure of the controllers. We advance this readiness-gap boundary as an explanatory hypothesis rather than an established general principle. Its null side is corroborated across every other combination, including under held-out prediction, while its positive side is observed in a single combination.
comment: Published in Transactions on Machine Learning Research (TMLR), 2026. 37 pages. Reviews: https://openreview.net/forum?id=gp77UrXyjT. Code: https://github.com/haewoon/leadership-coordination-control. Video: https://youtu.be/_8R5cbJEHw8
Systems and Control (EESS)
★ Efficiency--Complexity Tradeoff and Constrained Percolation
We study the efficiency--complexity tradeoff for resource-limited agents through directed first-passage percolation. Nonnegative i.i.d. edge weights represent physical energy expenditure, while the logarithmic size of a constrained path family measures representational complexity. For translation-invariant families, a path-joining property yields deterministic large-distance costs in rational directions and joint convexity in constraint parameter and direction, under suitable moment and feasibility assumptions. Our main result concerns the limiting cost parameterized by path-family entropy. Using techniques from analytic combinatorics in several variables (ACSV), we establish its almost-sure existence and joint continuity in entropy and direction for nested families satisfying additional generating-function conditions, on the corresponding entropy range. Path-count asymptotics supply the change of parameter linking this cost to the constraint-parameterized cost. Two planar examples constrain the number of turns or the number of East-runs of length at least two.
comment: 27 pages, 2 figures
★ Data-Driven LQR with Uncertainty Refinement and Domain Randomization---Part~II: Fisher-Majorized Certified Policy Iteration
Building on the uncertainty description developed in Part~I, this paper introduces a Fisher-majorized domain-randomized generalized policy iteration framework for data-driven linear quadratic regulator (LQR) synthesis. At each iteration, partial Bellman evaluation over a fixed batch of uncertainty realizations produces a structured Sylvester policy-improvement step. A residual-weighted Fisher curvature controls the policy-induced variation in the stationary covariance and converts the Sylvester model into a certified upper bound on the cost change, thereby guaranteeing descent for every accepted step. A check-and-correct mechanism then certifies stability over the entire uncertainty support, rather than only over the sampled realizations. Consequently, every iterate is robustly stabilizing and monotonically decreases the fixed-batch objective. The Fisher truncation and stability correction become inactive after finitely many times. The objective gap then contracts linearly to a floor determined by the dispersion of the realization-wise Riccati targets and by measurement noise. In the absence of measurement noise, this floor vanishes when the realization-wise targets coincide.
★ Data-Driven LQR with Uncertainty Refinement and Domain Randomization---Part~I: Certified Uncertainty Refinement
Learning controllers from trajectories corrupted by process and measurement noise is difficult when the system matrices and the noise covariances are unknown: noisy regressors bias least squares, and model uncertainty cannot be quantified without the noise covariance, which can only be estimated from residuals of the uncertain model. This paper, Part~I of a two-part study, resolves both obstacles with a finite-sample framework that certifies, and iteratively refines, uncertainty sets for the model, the noise in the collected data, and the noise covariance. A closed-loop parametrization based on delayed instrumental variables (IVs) removes the bias and assigns to every candidate model exactly one noise sequence that reproduces the data. Learning this noise breaks the circular dependence: the noise sequences give the covariance estimate directly, models whose noise violates the prior or the statistical certificates are discarded, and the surviving models yield a tighter covariance certificate. Several delayed instruments further tighten the model set and give a certified test of data consistency. We prove finite-sample containment of the true model, noise, and covariance, recursive nesting, and convergence at the IV rate. Part~II uses these certificates for domain-randomized linear quadratic regulator synthesis.
★ Hyperscale Data Centers as Wide-Area Damping Actuators for Low-Inertia Power Systems
Low-inertia power systems need fast damping actuators as synchronous generation is displaced by inverter-based resources. This paper evaluates hyperscale data-center loads controlled through Dynamic Voltage and Frequency Scaling (DVFS) as wide-area damping actuators. A DAE-consistent bus-input matrix is validated against the ANDES reduced state matrix and a finite-difference input check. On the Kundur two-area benchmark, Bus 10 ranks highest among non-generator buses for the 0.647 Hz inter-area mode. A classical washout-plus-lead-lag controller raises damping from 3.43% to approximately 6%. Modern 10-50 ms battery and DVFS actuators require comparable peak modulation for the same small-signal damping target. Compared with a closest same-system data-center-frequency-droop baseline, the wide-area controller offers a clearer advantage: at comparable peak MW, droop reaches 4.76% damping, while reaching 6% requires approximately twice the peak modulation. In the Pade-delay model, damping crosses the open-loop value at 443 ms for the 10 ms DVFS actuator versus 374 ms for the 50 ms BESS case. Reduced-order saturation stress tests show that a 2 MW envelope is non-binding for modal, fault-cleared-equivalent, and line-outage-equivalent disturbances, but can bind under generator-trip-equivalent stress. IEEE 39-bus tests confirm the DAE-based ranking procedure and show that high-residue buses may require more phase compensation than a single lead-lag stage can supply.
comment: 24 pages, 7 figures. Author manuscript of the published journal article
★ Energy-Based Fault Detection and Isolation Framework for Flat Port-Hamiltonian Mechanical Systems
Fault diagnosis during regulated motion requires informative signals even when feedback attenuates tracking deviations. This paper develops an energy-based diagnostic framework that constructs mechanical port-power residuals without a dedicated dynamic observer. Spectral Hamiltonian Algebraic Power Evaluation (SHAPE) combines port-Hamiltonian variables and differential flatness to generate nominal power references from planned motion for fully actuated mechanical systems with collocated actuation. Subtracting these references from commanded effort multiplied by measured flow yields residuals whose final evaluation requires neither recursive observer updates nor measured-state dynamics calculations. Continuous wavelet transform (CWT) scalograms and a shared-encoder convolutional neural network are used to distinguish nominal operation, actuator loss of effectiveness, anomalous friction, and vibration from complete trajectories. Evaluation uses a simulated six degree of freedom Kinova manipulator under model predictive control and fixed plant-model mismatch, with single faults at a specified joint. Across five training seeds and 3000 held-out test runs, SHAPE-CWT achieves a mean accuracy of 84.69 percent, versus 81.67 percent for torque residuals and 97.17 percent for the generalized momentum observer under common CWT processing. The respective sample standard deviations are 0.70, 0.70, and 0.37 percentage points. These results demonstrate useful fault-type discrimination from algebraic power residuals, while severity-excitation profiles and offline velocity perturbations reveal limits to their diagnostic capability.
★ Physics-Informed Graph-Based Safe RL with Constraint-Specific Critics for Community Battery Coordination in Distribution Network
High penetrations of distributed energy resources (DERs) and fast-varying electric vehicle (EV) charging intensify voltage and line-limit violations in distribution networks (DNs), making safe, low-cost operation increasingly challenging. This paper proposes a physics-informed safe deep reinforcement learning (DRL) framework for coordinating community battery energy storage systems to mitigate constraint violations while minimizing operating cost. To represent nonlocal spatiotemporal interactions, we develop a dual-level physics-informed graph representation in which line impedance and capacity shape spatial information propagation, while power-balance, voltage-profile, and flow-reconstruction losses regularize the latent state toward control-relevant physical consistency. Building on this representation, we develop Lagrangian Dedicated-Critic PPO (LDCPPO), which implements constraint-specific value decomposition: independent reward and cost critics preserve heterogeneous voltage- and line-loading-related learning signals and combine their advantages only during the Lagrangian policy update. Case studies on DNs with high rooftop photovoltaic penetration and EV charging demonstrate that the proposed method reduces the key voltage-violation degree by up to 80.5% compared with the tested baselines, while also achieving the lowest voltage-violation count and line-overload degree with competitive operating cost.
★ Nodal Inertia Pricing for RoCoF Security with Integration of Synchronous and Virtual Resources
Frequency regulation is becoming a more significant challenge in the undergoing low-carbon transition of modern power systems, which is heavily rely on system inertia provision. In the context of market-based energy allocation, reasonable valuation of the system inertia is crucial to its procurement, namely a worthy price should be determined for inertia providers. However, current research on the value of inertia remains insufficient, particularly during the initial phase immediately following a contingency, a critical period when the system primarily relies on the inherent inertia from conventional units and pre-engaged virtual inertia resources. Furthermore, it is found in existing studies that the Center of Inertia (COI) models exhibit significant deviations from actual physical conditions. For market-oriented reforms that explore individual differences, COI models also fail to capture the heterogeneous contributions of inertia at different nodes to system security. Given these circumstances, this paper proposes a nodal inertia pricing mechanism via a day-ahead unit commitment and inertia co-optimization model with nodal resolution. Following the proposed pricing scheme, the day- or hour-ahead inertia provision is monetized and promoted, providing regional guidance to units and devices with virtual inertia provision, which is further verified by simulation results of the South East Australia Power System.
★ Hybrid-time PINNs: Discontinuity-Aware Physics-Informed Neural Networks for Learning Hybrid Dynamical Systems
Hybrid dynamical systems flow continuously but jump at discrete events. Physics-informed neural networks (PINNs) fit data while enforcing known dynamics, but a single network is smooth in time and cannot model these jumps. We show that the more closely a PINN follows a jump, the larger its physics loss must be. We propose hybrid-time PINNs (HT-PINNs), which assign one network to each flow interval of the hybrid time domain and train the networks jointly with the jump times and the unknown parameters. The number of jumps and the sequence of modes are found by enumeration. HT-PINNs recover the number of jumps and the restitution coefficient of a bouncing ball. In a MuJoCo simulation of robotic disassembly, where a robot pulls a press-fitted pin out of a housing through a spring, HT-PINNs find the number of jumps and the sequence of modes, and estimate the static and kinetic friction coefficients with small errors of 0.1% and 1.7%.
★ A Personalized and Adaptive Intelligent Tutoring System for Psychomotor Learning
Intelligent tutoring systems (ITSs) have advanced learner motivation and personalization in cognitive domains, but extending these gains to psychomotor tasks remains challenging. Existing psychomotor ITSs rely on learner models built before instruction or updated once per module, leaving open how to use real-time assessments of learner cognitive state for finer-grained personalization. Psychomotor learning involves multiple tutoring objectives, including calibrating self-confidence to performance, calibrating workload per the Yerkes-Dodson Law, and driving skill progression toward higher learning stages. How to balance these objectives for learners, and adjust that balance over time, remains open. This study proposes and evaluates an online personalization algorithm that continuously reprioritizes tutoring objectives to match learners' evolving needs. Rather than weighting objectives equally, the actor-critic-inspired algorithm dynamically updates reward weights using recent measures of self-confidence, workload, and performance. These weights inform a multi-input, multi-output cognitive control framework that synthesizes learner-specific tutoring policies via model-based planning. The approach was evaluated in a human-subject study (N = 40) with a 2D quadrotor landing task. Clustering of the learned reward weights revealed distinct prioritization patterns across learners, indicating the algorithm captures heterogeneity in learner needs. Learners whose weights shifted more substantially tended to reach higher final performance, and those whose weights consistently emphasized workload calibration achieved the strongest outcomes. These findings advance online learner modeling and control-based personalization in psychomotor ITSs, showing how adaptive tutoring can better support learning by aligning instructional decisions with learners' evolving cognitive and performance states.
comment: 13 pages, 6 figures
★ POMDPs with Lipschitz-Continuous Belief-Dependent Rewards Have Lipschitz-Continuous Value Functions
We establish the vector (or elementwise) Lipschitz continuity of value functions for partially observed Markov decision processes (POMDPs) with reward functions that are Lipschitz-continuous functions of the posterior probability distribution of the state (i.e., the belief). Our results significantly tighten existing Lipschitz results for this class of POMDPs on infinite horizons, with previous results only yielding finite constants for discount factors less than 0.5, while our results yield finite Lipschitz constants for all discount factors less than 1. The significance of our Lipschitz constants is illustrated through their use in heuristic search value iteration (HSVI) to construct (finite) piecewise-linear value function approximations.
comment: 7 pages, 3 figures, accepted for presentation at 2026 IEEE Conference on Decision and Control (CDC)
★ Performance Gaps Between Open-Loop, Certainty-Equivalent, and Optimal Control Under Action-Dependent Observations
In dynamical systems, feedback can improve performance by allowing decisions to depend on past observations, rather than being designed a priori. Under full observation, the current state is sufficient for decision-making; under partial observation, the controller must instead estimate the state from measurements. When actions also affect observations, control can have a dual role: actions influence both the state and the information available for future decisions. This interaction can invalidate certainty-equivalent (CE) control, a widely used strategy that applies state feedback policies as if the estimated state were the true state. We ask when this suboptimality leads to a quantitative performance gap that grows with the horizon. We study this question in a linear-quadratic system with bilinear observations by comparing the optimal policy, certainty-equivalent control, and the optimal open-loop (OL) policy. We identify an instance-dependent information-strength parameter $q$ and establish distinct performance regimes as $q$ varies. When $q$ is sufficiently small, we show that no policy can improve over OL control by more than $O(1)$ cumulative cost. For large values of $q$, we find that CE achieves an $Ω(T)$ advantage over OL control. Our main finding is that for intermediate values of $q$, we can construct a policy that occasionally probes for information and achieves an $Ω(T)$ advantage over OL control while CE only achieves an $O(1)$ advantage, establishing an $Ω(T)$ gap between CE and optimal control. Our analysis relies on a regret decomposition that captures the tradeoff between the immediate cost of gathering information and its future benefit.
★ From Caveman to COBA: Coordinated Optimization-Based Attacks on Grid-Edge Smart Inverters in DER-Rich Power Systems
Aggregation platforms and smart-inverter control channels can allow a compromised coordination layer to alter active- and reactive-power commands across a geographically distributed DER fleet. This paper introduces the Coordinated Optimization-Based Attack (COBA), which uses network information and location-specific DER P/Q setpoints to maximize thermal loading on a selected transmission corridor. COBA is compared with low-information CAV baselines limited to a single uniform fleet command: all-off, all-on, maximum reactive absorption, or maximum reactive injection. The Hawaii Synthetic Grid-37 Buses case is evaluated with rooftop-photovoltaic-like capability aggregated at 20 selected 69-kV buses and verified using MATPOWER power flow. Under the case's single-reference-bus balancing convention, COBA overloads B86 (WAIALUA69-WAIANAE69) near 30% penetration and B51 (COCONUT69-PALM69) near 55%, while same-target uniform loadings remain lower and uniform commands primarily stress other natural weak lines. The results show that topology, target selection, and command capability determine DER-induced thermal risk beyond aggregate penetration alone.
comment: 5 pages, 4 figures
★ PMU Foundation Model Bridge: A Tool for Connecting Synchrophasor Streams to Time Series Foundation Models
This paper presents PMU Foundation Model Bridge, an open source tool that connects online phasor measurement unit (PMU) or phasor data concentrator (PDC) streams to publicly available time series foundation models (TSFM). Through a desktop interface or command line interface, users can select measurement channels, define preprocessing, choose TSFMs, and configure outputs. In the initial release, 4 publicly available TSFMs were supported, including Chronos-2, TimesFM 2.5, Timestral, and TimesFM-3. The tool supports both file and synchrophasor streaming outputs, therefore enabling both offline use and integration into C37.118-based online workflow applications. In addition to supporting raw synchrophasor data, the framework provides a preprocessing layer for deriving customized additional quantities for forecasting. The tool is validated using a virtual PMU and real-time laboratory measurements from a frequency disturbance recorder (FDR). The tool aims to support community research on combining synchrophasor technology with TSFMs.
♻ ★ Asymmetric-Information Resource Allocation Games: An LP Approach to Purposeful Deception
In this work, we introduce the Deceptive Resource Allocation Game (DRAG), which studies purposeful deception within a Bayesian game framework. In DRAG, a Defender allocates resources across the true asset and several decoys to influence an Attacker's beliefs and actions, with the goal of diverting the Attacker away from the true asset. We seek to characterize purposeful deception, whereby the Defender deceives only when doing so improves its performance. To this end, we solve for the Perfect Bayesian Nash Equilibrium (PBNE) of the corresponding game. We show that, despite the coupled belief-policy interdependence, the problem admits an efficient, non-iterative linear programming formulation. Numerical results demonstrate that the resulting policies naturally balance effective allocation and belief manipulation, giving rise to purposeful and emergent deceptive behaviors.
♻ ★ Data-based Low-conservative Nonlinear Safe Control Learning
This paper develops a data-driven safe control framework for nonlinear discrete-time systems with parametric uncertainty and additive disturbances. The proposed approach constructs a data-consistent closed-loop representation that enables controller synthesis and safety certification directly from data. Unlike existing methods that treat unmodeled nonlinearities as global worst-case uncertainties using Lipschitz bounds, the proposed approach embeds nonlinear terms directly into the invariance conditions via a geometry-aware difference-of-convex formulation. This enables facet- and direction-specific convexification, avoiding both nonlinearity cancellation and the excessive conservatism induced by uniform global bounds. We further propose a vertex-dependent controller construction that enforces convexity and contractivity conditions locally on the active facets associated with each vertex, thereby enlarging the class of certifiable invariant sets. For systems subject to additive disturbances, disturbance effects are embedded directly into the verification conditions through optimized, geometry-dependent bounds, rather than via uniform margin inflation, yielding less conservative robust safety guarantees. As a result, the proposed methods can certify substantially larger safe sets, naturally accommodate joint state and input constraints, and provide data-driven safety guarantees. The simulation results show a significant improvement in both nonlinearity tolerance and the size of the certified safe set.
♻ ★ Stability and stabilization of semilinear single-track vehicle models with distributed tire friction dynamics via singular perturbation analysis
This paper investigates the stability and stabilization of semilinear single-track vehicle models with distributed tire friction dynamics, modeled as interconnections of ordinary differential equations (ODEs) and hyperbolic partial differential equations (PDEs). Motivated by the long-standing practice of neglecting transient tire dynamics in vehicle modeling and control, a rigorous justification is provided for such simplifications using singular perturbation theory. A perturbation parameter, defined as the ratio between a characteristic rolling contact length and the vehicle's longitudinal speed, is introduced to formalize the time-scale separation between rigid-body motion and tire dynamics. For sufficiently small values of this parameter, it is demonstrated that standard finite-dimensional techniques can be applied to analyze the local stability of equilibria and to design stabilizing controllers. Whilst the proposed controllers build on classical approaches, the novelty of this work lies in establishing the first singular perturbation framework for ODE-PDE vehicle models with distributed tire dynamics, providing a theoretical justification for their quasi-static reduction and for the use of finite-dimensional tools for analysis and control design.
comment: 15 pages, 10 figures. Accepted at Automatica
♻ ★ Convex Inner Approximations for Safe AC Dispatch With Device Dynamics
Safe AC dispatch requires both feasible operating points and enough ramping capability to reach the next target. This paper constructs convex inner approximations that connect nonlinear network feasibility, constrained device response, and dispatch-interval tracking. A fixed-point certificate enforces AC operating limits under bounded load uncertainty. Componentwise response envelopes convert heterogeneous device dynamics into affine conditions on measured states and power references. A command governor then combines these conditions with available ramping capability and a hard terminal-error bound. It selects an admissible target through a minimum-deviation objective and holds the corresponding reference while reserved fast regulation remains active. The resulting quadratic program optimizes controllable references and auxiliary variables. Tests on the IEEE 39- and 118-bus systems include 96 single-target executions and 64 transitions in continuous dispatch sequences at five- and fifteen-minute intervals. All 160 transitions satisfy the AC constraints, independent balancing-generator capability checks, and the prescribed regulation-adjusted terminal accuracy. Complete admission-cycle medians are 3.45 and 8.79 ms. A connected synthetic system with 3,200 buses and 1,000 generating units extends the computational assessment to 999 independent references, with a median cycle time of \LargeMedian{} s. The construction provides a tractable link between available ramping capability and safe, timely execution of AC dispatch.
Multiagent Systems
★ Estimation of Propagated Risk in LLM-based Multi-Agent Systems
Large Language Model (LLM) based multi-agent systems introduce a distinctive reliability challenge in which actions taken by upstream agents can affect the correctness of downstream agents. While prior work has investigated error propagation and developed methods for estimating uncertainty inherited from upstream agents, existing approaches do not explicitly model how realized sender-receiver interactions contribute to downstream error risk under different sender correctness states. In this work, we study agent-level error-risk estimation from multi-agent execution traces. We propose Interaction-Effect Propagation (IE-Prop), an interaction-conditioned model that learns separate effects for correct and incorrect upstream agents. For each sender-receiver interaction, the model combines these conditional effects using the sender's estimated error risk, and recursively propagates the resulting risk estimates through the execution graph. We evaluate our approach across three multi-agent architectures and three benchmarks. Across all evaluated architectures, our approach raises the final-node AUROC over the established baseline by approximately 0.05 points on average. Furthermore, tracking risk across multiple agent steps contributes a distinct +0.03 AUROC gain in sequential systems. These results show that realized inter-agent interactions provide predictive information beyond receiver-local evidence, while the value of multi-step risk tracking depends on the system's information-flow structure.
★ Coral Discovery: The Division of Cognitive Labor in Autonomous Research
A research community is only as strong as the way it divides its cognitive labour. The philosopher Philip Kitcher argued that researchers should not all pursue the most promising approach. Credit rules, not a central plan, keep some of them on other lines. We bring this question to autonomous research, where Large Language Model agents improve programs against a deterministic grader. Agents that share only a score follow the leader and converge onto one approach. Coral Discovery gives the community a direction board. A verifier labels every attempt with a direction, the mechanism earning the score, and a method, one change within it. Per direction, the board tracks how much each eval improves the score, adds an exploration bonus for little-tried directions, prices the option of opening a new direction, and suggests how many agents each direction deserves. Credit is earned against a direction's own best, so a lagging direction keeps its agents. Agents finish their lines, then treat the board as advice. We test on nine tasks with the same model and budget, without grader access except for Kernel. Coral Discovery beats the strongest open autonomous multi-agent system on 6 tasks, ties on 1, and loses on 2. Against published numbers, it is best or tied on 4. It improves the reported Kernel result from 1103 to 1095 cycles and passes 1103 at eval 275, under half of the 596 used before. It sets a new Polyominoes state of the art, 91.78 against 87.0. Direction labels show that the leading scores rest on directions a score-only community never entered. Code will be released shortly.
★ Neuromorphic Decision Dynamics for Deadlock-free Safe Navigation in Congested Environments
Safe navigation in congested environments requires both collision avoidance and coordinated passage. We introduce neuromorphic decision dynamics (NDD), a decentralized, event-based controller, that provides navigation in congested spaces with guarantees on safety, liveness, robustness, and scalability. NDD makes safe-and-live deconfliction a controllable, locally tunable emergent property of local agent interactions, without requiring communication, preassigned priorities, models of neighbors' objectives or future actions, or estimates of group size or global network properties. By design, NDD couples the dynamics of internal decision and attention states with changing physical states and locally observed relative motion. Each agent continuously modulates its speed along its prescribed path, responding as soon and as much as conflict resolution requires. Agents self-organize into disjoint, locally engaged groups that resolve conflicts concurrently. When any group's conflict becomes safety-critical, decision-action feedback destabilizes collective indecision, and coordinated conflict-resolution emerges that stabilizes a safe traversal order within the group. Order persists as conflicts clear, but groups may split or merge as agents autonomously adapt to new conflicts and those with resolved conflicts resume their nominal motion. We derive explicit conditions on sensing range, feedback gains, response time scales, and actuation limits that guarantee collision avoidance and traversal-order formation in finite time with no deadlock or livelock for finite, heterogeneous agent populations of arbitrary, changing size. Computational cost scales with the number of local interactions sensed by each agent. Large-scale simulations, comparisons with existing approaches, and hardware experiments demonstrate NDD in complex congested scenarios and validate its scalability.
★ Do "True" and "False" Add Up to One? Complement Consistency in LLM-Agent Opinion Simulations
In LLM-agent opinion simulations, each agent reports the probability that a claim is true after seeing its own previous probability and the probabilities reported by its neighbors, and that number becomes its state in the next round. If the number describes the agent's position rather than the sentence it was shown, then rewriting the claim as its complement and replacing every probability p with 1 - p should return the complementary answer, so the two reports should sum to one. We test this with three small open models: Qwen3-4B, Llama-3.2-3B and Gemma-3-4B. On 2,304 agent states taken from a running network simulation, Qwen's paired reports sum to 1.18 on average, and on 48 real fact-checking claims from AVeriTeC they again exceed one. A controlled benchmark of 24 worlds, each with exactly two possible outcomes, shows the same failure in all three models. The discrepancy is two to ten times larger than the change produced by a same-meaning paraphrase, and it persists under repeated sampling and across peer group sizes and report strengths. When the reports are fed back into two populations that start from complementary states, the populations do not remain mirror images: their divergence is positive after five rounds for every model. Averaging each answer with the complement of its paired answer restores consistency by construction and lowers Brier loss averaged over the two wordings, but at the final round it beats the better of the two original wordings for only one of the three models. A schedule-by-targeting experiment on the same model gives a small, stable interaction because such sweeps keep the claim wording fixed. Consistency under rewording therefore needs its own test before simulated probabilities are read as beliefs
★ J-Comm: Jacobian-Guided Latent Communication between LLM Agents
Latent communication requires both an effective continuous reasoning process and an interface through which an external model can leverage its intermediate computation. We present J-Comm, a framework comprising two core components. First, a training-free Geometric Agreement for Latent Reasoning (GEAR) reweights token candidates over existing model embeddings, which monotonically optimizes geometric consensus among candidate tokens in embedding space at each step. Second, we introduce an H+J (Hidden-and-Jacobian) communication interface that gives the Receiver two complementary views of the Sender's reasoning. The H channel supplies continuous inputs derived from feedback distributions, while the J channel conveys intermediate-state cues in vocabulary-indexed Jacobian-lens coordinates. Intuitively, H provides inputs for the Receiver to reason from, while J offers additional guidance within its computation. The Receiver reconstructs these cues using its own dictionary and injects them into intermediate layers during prefill, enabling communication without sharing key-value caches or requiring identical model widths. In Qwen3-8B to Qwen3-4B communication, J-Comm achieves the highest mean score across seven benchmarks, including 94.80% on MATH500, 54.55% on GPQA-Diamond, 73.33% on AIME2024+2025, and 62.93% success on ALFWorld. Compared with TextMAS, it improves performance by 1.29-8.59 percentage points and reduces generated text tokens by 19.2-74.8%.
★ Agent-Based Transport Model Calibration via Online Stochastic Batch Optimization
Calibrating agent-based models (ABMs) is computationally demanding, as traditional black-box optimization typically requires hundreds or thousands of independent simulation runs. To reduce this burden, we introduce an online calibration framework that estimates parameters within a single simulation run. The approach combines black-box optimization with stochastic batch optimization, a method widely applied in machine learning but not yet explored in ABM calibration. Using MATSim as a case study, we treat each simulation iteration as a stochastic batch, enabling incremental parameter updates as agents make decisions. This online calibration paradigm allows model parameters and agent behaviors to co-evolve, achieving convergence without restarting the simulation. We demonstrate the effectiveness of our method by calibrating utility function parameters in MATSim for a large-scale transportation scenario of Switzerland. Results show that mode shares align closely with observed data within a typical simulation timeframe, confirming the feasibility and efficiency of the approach. By embedding calibration directly into the simulation loop, our method opens new possibilities for real-time, scalable, and efficient calibration of complex ABMs.
comment: Presented at the World Conference on Transport Research (WCTR 2026), Toulouse, France
★ HGWM: Hierarchical Graph-Guided World Model for Zero-Shot Object Navigation via Graph Matching
Zero-shot object navigation requires an embodied agent to choose where to search while retaining evidence from previous observations. HGWM couples an LLM-generated room-anchor prior with persistent spatial-semantic memory. A coarse stage assigns visual-semantic scores to candidate headings on a 0-10 scale; a fine stage measures symbolic compatibility between observed objects, room context, and the goal prior. Phase-dependent fusion combines these scores with an LLM direction recommendation before local action selection. On HM3D v0.1, the reported HGWM configuration achieves 62.6% success rate (SR) and 31.5% success weighted by path length (SPL), compared with 58.1% and 31.2% for WMNav. On MP3D, the corresponding values are 46.8% and 17.9%, compared with 45.4% and 17.2%. The results indicate higher success point estimates and modest path-efficiency gains. On HM3D v0.2, the full model obtains 74.4% SR and 31.9% SPL. Additional component ablations reduce SR by 0.9-2.3 percentage points and SPL by 0.6-1.5 points. These are author-reported aggregate measurements. The accompanying implementation repairs pass offline tests; episode-level rerun records are not available. Evaluation is confined to simulation.
★ PULSES: multi-agent AI for autonomous excited-state model building and dynamics
Simulating molecular excited-state processes requires resolving electronic and nuclear dynamics, but the cost of direct quantum-chemical calculations limits accessible system sizes and timescales. Machine-learning models can extend these simulations, yet they often need to be tailored to specific molecular systems and scientific objectives. Building and deploying such models requires coordinated decisions across conformational sampling, quantum-chemical labelling, model training, validation and dynamics. Here we introduce Paired agents for Unified Learning and Simulation of Excited States (PULSES), a multi-agent system that automates this workflow within expert-approved boundaries. A human expert and planner define the scientific objective, methods and decision boundaries, while an orchestrator coordinates domain agents and artifact-specific scientific assurance during execution. We evaluate PULSES on 15 computational chemistry tasks and demonstrate end-to-end applications to perylene exciton Hamiltonians, solvent-dependent biphenyl relaxation, and excited-state proton transfer in 10-hydroxybenzo[h]quinoline. These results demonstrate traceable, expert-bounded construction and deployment of specialized excited-state models, while retaining scientific validation as a requirement rather than assuming that successful execution establishes correctness.
★ Empirically Calibrated Differential Games for Simulating Driver Interactions
Game-theoretic models can resolve vehicle interactions without hand-written priority rules, but whether they describe human drivers once calibrated has rarely been tested. This study formulates a general-sum differential game of vehicle interactions at urban intersections, calibrates it to observed driving, and evaluates it on data not used for calibration. Each vehicle minimizes a cost that combines speed tracking, adherence to a map-derived lane route, control effort, gap keeping based on the intelligent driver model, a predictive conflict term, and a contact barrier, while perceiving the other vehicles with a delay. The game is solved with a feedback iterative linear-quadratic algorithm, and ten behavioral parameters per vehicle are calibrated by Bayesian optimization on trajectories recorded at four intersections in Washington, D.C. The calibrated game reproduces the trajectories of 819 vehicles with a mean position error of 1.00 m. When only each vehicle's route is known, as in a simulator, it is the most accurate of the compared models on 96 held-out vehicles (7.42 m), and ten-fold cross-validation shows that it is significantly more accurate than calibrated SUMO (by 3.61 m), an inverse reinforcement learning model (by 0.93 m), and independent and cooperative variants of the same model (by 0.33 and 0.32 m). The solutions, computed in 2.1 s for a typical interaction, are verified approximate equilibria. Finally, the conflict and steering weights, which the data do not identify, can be set from safety and comfort targets; together with a stiffer contact barrier, this removes the collisions that otherwise occur in 43% of tight closed-loop conflicts.
comment: This manuscript is intended for submission to IEEE Transactions on Intelligent Transportation Systems (IEEE T-ITS)
★ Markets of Minds: A 51-Billion-State Economy of Intelligence
When intelligence is shared, reasoning becomes a constrained allocation problem. In a population of agents with limited access to expert models, an intervention can benefit its recipient yet still be a poor use of shared capacity if another feasible intervention has greater expected value. Assistance must therefore be evaluated through both its marginal benefit and the opportunity cost induced by budgets, access, scarcity, and service capacity. To this end, we introduce Markets of Minds, a framework for allocating expert reasoning across agents before expert outputs are observed. For each feasible agent-service pair, the framework estimates the expected gain from assistance relative to the agent's default continuation, explicitly separating repair benefit from regression risk, and jointly clears assistance contracts under shared resource constraints. We then study this problem in a dependency-aware agent economy with 51.2 billion addressable persona-market states spanning capabilities, cognitive budgets, provider access, scarcity, and network roles. These dependencies couple allocation feasibility across agents rather than the outcome of a fixed contract. Empirically, in a matched replay of 240 cases under a common 30% assistance quota, the frozen allocator reaches 85.42% task-level persona adherence versus 73.86% for equal-capacity random allocation, a gain of 11.56 percentage points. The results show that assistance value is heterogeneous and can be negative, and that scarcity changes which interventions are worth purchasing. Together, these findings motivate a population-level view of intelligence where the value of assistance depends not only on what it can achieve, but also on which alternative uses of scarce cognitive capacity it displaces.
♻ ★ A-MapReduce: Executing Wide Search via Agentic MapReduce
Contemporary large language model (LLM)-based multi-agent systems exhibit systematic advantages in deep research tasks, which emphasize iterative, vertically structured information seeking. However, when confronted with wide search tasks characterized by large-scale, breadth-oriented retrieval, existing agentic frameworks, primarily designed around sequential, vertically structured reasoning, remain stuck in expansive search objectives and inefficient long-horizon execution. To bridge this gap, we propose A-MapReduce, a MapReduce paradigm-inspired multi-agent execution framework that recasts wide search as a horizontally structured retrieval problem. Concretely, A-MapReduce implements parallel processing of massive retrieval targets through task-adaptive decomposition and structured result aggregation. Meanwhile, it leverages experiential memory to drive the continual evolution of query-conditioned task allocation and recomposition, enabling progressive improvement in large-scale wide-search regimes. Extensive experiments on five agentic benchmarks demonstrate that A-MapReduce is (i) high-performing, achieving state-of-the-art performance on WideSearch and DeepWideSearch, and delivering 5.11% - 17.50% average Item F1 improvements compared with strong baselines with OpenAI o3 or Gemini 2.5 Pro backbones; (ii) cost-effective and efficient, delivering superior cost-performance trade-offs and reducing running time by 45.8\% compared to representative multi-agent baselines. The code is available at https://github.com/mingju-c/AMapReduce.
comment: 27 pages
♻ ★ Interchangeable in Role, Not in History: Replacing Members of Formed LLM Teams
Multiagent LLM systems often treat a role as a slot: when a member is replaced, any agent competent at the role is expected to fill it. We test this view on teams whose members keep short written notebooks. A one-seat swap replays a formed, frozen team with its own member, with a same-role newcomer carrying another team's notebook for that role, or with a newcomer carrying none. In cooperative cooking, wildfire response and medical question answering, the experienced newcomer, averaged over seats, outscores the one without a notebook in every setting, so the usual comparison would approve it. Against the original team, the same replacement falls short in every setting, and the team spends more messages per unit of progress. On the hardest cooking level, the shortfall appears only after the team has worked together, persists across model families and decoding temperatures, and narrows by two thirds with added planning-only members. Because the team's history is written down, we can take it apart. In the default configuration on that level, notebooks from the same donor teams leave a newcomer that leads, holding the recipe, worse off than no notebook and one that follows better off. The notebook lines about the partner reproduce this whole difference between the seats. Whether an agent can replace a teammate depends on the seat and the team it enters.
♻ ★ The fixed-point bundle method over product-of-simplex domains arising from game equilibria
Developing general-purpose algorithms with global asymptotic convergence guarantees for Nash equilibria of general-sum games remains a longstanding challenge. This paper extends the fixed-point bundle framework for finite-dimensional variational inequalities (VIs) from simplex domains to product-of-simplex domains, enabling its direct application to Nash equilibrium computation. The resulting fixed-point bundle exhibits a composite fiber bundle structure. The key innovation is to construct an equivalent VI on a simplex and establish the equivalence between the two fixed-point bundle frameworks via a fiber bundle isomorphism. Exploiting this geometric equivalence, we show that the path-following algorithm for VIs on product-of-simplex domains inherits the global linear convergence guarantees of the simplex-domain framework. In numerical experiments, the algorithm attained a precision of $10^{-5}$ on all 5600 randomly generated game instances, with dimension ranging from 2-player 128-action to 128-player 2-action. Code and data are available at https://github.com/shb20tsinghua/FiberBundle_VI.
comment: 26 pages, 1 table, code and data are available at https://github.com/shb20tsinghua/FiberBundle_VI
♻ ★ Temporal Fair Division in Multi-Agent Systems: From Precise Alternation Metrics to Scalable Coordination Proxies
Many multi-agent games and simulated environments involve several independent, often learning, agents repeatedly competing for a shared resource. Conventional fairness measures assess whether resources are shared equally overall, but cannot distinguish orderly turn-taking from irregular access patterns that produce long, unpredictable waiting times despite similar cumulative outcomes. We introduce Rotational Periodicity, a family of computationally efficient metrics combining two complementary measurements, the regularity of waiting times between successful accesses and the balance of access frequencies across agents. We evaluate them against a more detailed family of alternation metrics in a repeated threshold-congestion game, the Honey-Jar Game, in which two to ten reinforcement-learning agents compete for exclusive access to a shared high-value resource. Our experiments show that independently trained Q-learning agents often coordinate substantially worse than random-policy agents, even though conventional fairness metrics consistently report favourable outcomes. Every member of the family closely reproduces the rankings of the more computationally expensive alternation metrics, while computing six to twenty times faster as the number of agents grows. A direct timing study, which separates the two quantities that drive that cost, measures the advantage growing in proportion to the number of agents out to two thousand agents. These findings show that evaluating multi-agent learning systems requires temporally aware measures of coordination, not only aggregate outcomes, and that efficient proxies make this practical for larger multi-a
comment: 18 pages, 6 figures, 11 tables. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible
♻ ★ System Design Framework for Distributed Space Antennas Using Electromagnetic Formation Flight
Distributed space antennas using electromagnetic formation flight (EMFF) are a promising architecture for large-aperture, long-life space communication systems. Their system design, however, is governed by coupled constraints on antenna performance, satellite mass, power generation, coil geometry, and formation-keeping power. This paper proposes a system-level design framework for EMFF-based distributed space antennas. It links phased-array requirements with satellite and formation-system sizing for a static grid under a fixed total system mass. Before system-level optimization, distributed-control simulations are used to derive a control index that quantifies the formation-maintenance requirement. This index is integrated into an optimization problem with a user-defined objective and sizing, power, coil, and sidelobe-envelope constraints. Parametric case studies examine the margin magnetic moment, prescribed transmit power, and large inter-satellite spacing. Results show that increasing system mass improves footprint reduction or effective isotropic radiated power only while satellite-level design headroom remains. In direct-to-device cases with 0.15-m spacing, generated power and coil-geometry constraints dominate the feasible aperture. In the 0.30-m large-spacing case, the required coil burden can exceed satellite-level mass, size, and power capacities, making the design infeasible despite favorable communication performance. Higher-fidelity simulations of representative designs evaluate formation maintenance under nonlinear $J_2$ dynamics and antenna gain with mutual coupling, further supporting the framework's usefulness for preliminary design. The proposed framework enables the design and evaluation of feasible static grid-based EMFF distributed antennas under coupled antenna, satellite, and control constraints.
comment: Submitted to IEEE Access and currently under review
♻ ★ BeforeIT.jl: High-Performance Agent-Based Macroeconomics Made Easy
BeforeIT is an open-source software for building and simulating state-of-the-art macroeconomic agent-based models (macro ABMs) based on the recently introduced macro ABM developed in [1] and here referred to as the base model. Written in Julia, it combines extraordinary computational efficiency with user-friendliness and extensibility. We present the main structure of the software, demonstrate its ease of use with illustrative examples, and benchmark its performance. Our benchmarks demonstrate that the base model built with BeforeIT is orders of magnitude faster than its original Matlab implementation, while achieving performance comparable to a highly optimised HPC implementation. Furthermore, we show how BeforeIT supports an easy construction of model variants such as the 'CANVAS' model of [2], how these variants can be evaluated at scale via an automated multicountry forecasting pipeline, and how the entire simulation can be automatically differentiated. BeforeIT is designed to facilitate reproducibility, extensibility, and experimentation. As the first open-source, industry-grade software to build macro ABMs, BeforeIT can significantly foster collaboration and innovation in the field of agent-based macroeconomic modelling. The package, along with its documentation, is freely available at https://github.com/bancaditalia/BeforeIT.jl under the Apache 2.0 license.
comment: 16 pages, 11 figures
♻ ★ S$^2$-MAS: A Society-Scale Multi-Agent Simulation Framework with 500 Million Household Persona Agents
Simulating economic responses at society scale requires agents whose decisions reflect household circumstances. We introduce S$^2$-MAS, a society-scale multi-agent simulation framework built on 500 million synthetic household personas grounded in real-survey data. Its Genese engine separates decisions from state updates, while reusable financial scenarios and application ledgers track consumption, cash, debt and payments. Trained with SFT + RLCD on 2.93 million examples, sharp-421m-mlx achieves 74.55% accuracy and 38.37% question-macro F1 on one-shot consumer financial-choice tasks derived from held-out survey responses. Its accuracy exceeds GLM-5.3-Flash, Jev 1.13, DeepSeek-V4.1-Flash, GPT-6-Luna and GPT-6-Astra in the common benchmark (full test for the first four; matched 500-group sample for Astra). We evaluate eight financial task families and simulate 12 million distinct household agents in paired purchase-funding conditions. Jointly raising purchase goals and approved credit limits increases consumption, with responses differing across age and income groups. Education, employment and household-size differences persist after age-income standardization and mainly reflect transitions from deferral to purchase. Asset and debt groups differ in cash drawdown, repayment and closing debt. S$^2$-MAS links heterogeneous household decisions to evolving consumption, cash and repayment paths, supporting economic experiments across household groups and population scales.
comment: 50 pages, 9 main figures and 7 supplementary figures. Project resources: https://github.com/KataDavidXD/s2-mas
♻ ★ OACap: Caption-Guided Active Observation for Detailed Video Captioning
Unlike question answering, where a query specifies a perceptual target, open-ended detailed captioning must determine what evidence to acquire and how to integrate it into a coherent global description. We propose Observation-Adaptive Video Captioner (OACap), a caption-centered agent that turns unsupported or missing information into targeted observation requests, uses spatiotemporal tools to acquire evidence, and iteratively verifies and revises the caption. OACap captions consistently improve performance across general video-understanding benchmarks using a fixed text-only reader. To better diagnose the information they preserve, we introduce Factual-Semantic Captioning Bench (FSCap-Bench). Its Core-Support Factuality Score (CSFS) combines claim precision with importance-aware fact coverage and responds as expected to four controlled perturbation types on 93.0-98.3% of valid caption pairs. Its Cross-Modal Semantic Round-Trip Score (CSRS) measures semantic preservation through a video-caption-video-caption round trip by matching atomic facts from source and recovered captions. Together, they connect downstream textual-memory utility with factual and semantic preservation.
comment: 14 pages, 4 figures, 4 tables. Technical report
♻ ★ NeuroMAS: Multi-Agent Systems as Neural Networks with Joint Reinforcement Learning
Multi-agent language systems are often built as hand-designed workflows, where agents are assigned semantic roles and communication protocols are specified in advance. We propose NeuroMAS, a method that first treats a multi-agent language system as a trainable and scalable neural-network-like architecture with LLM agents as nodes and intermediate textual signals as edges. In NeuroMAS, agent nodes are role-free but structure-aware: the topology only determines how information can flow in general, while reinforcement learning training determines how nodes communicate, specialize, and coordinate. This formulation shifts multi-agent design from workflow engineering toward architecture design, where depth, width, connectivity, and growth protocol become scalable sources of capability. Further, we provide a theoretical perspective showing why such modular textual computation is more parameter-efficient when tasks admit hierarchical decompositions. Experiments show that NeuroMAS improves significantly over both inference-time and trained multi-agent baselines. We further find that organizational scaling is path-dependent: larger systems can be challenging to train from scratch, but become feasible when grown progressively from smaller trained systems. These results suggest that learned neural multi-agent systems are a promising scaling axis for LLMs.
Systems and Control (EESS)
★ A SysML v2 Domain Library for ROS2 Robot Architectures
Systems engineers who model a ROS2 robot in SysML v2 must first model ROS2 itself. Its nodes, communication endpoints, message types, lifecycle states, quality-of-service policies, and coordinate frames have no ready definitions in the language, so each project reconstructs them. We present ros2-sysmlv2, a native SysML v2 domain library that supplies these definitions for reuse. A robot model imports the library and specializes its definitions to declare its own nodes, ports, connections, frames, and parameters. The library maps ROS2 concepts to SysML v2 definitions and usages and organizes them in Primitives, Infrastructure, and Application layers. We check its declarations for parity with the ROS2 Jazzy sources they model. Because every ROS2 element of a model specializes a library definition, a tool can select those elements by type. We state what this selection is and use it for a round trip. A ROS2 package is generated from the selection, and a monitor generated with the package compares the running graph with the same selection and reports every departure against the model. The generation-gap workflow derives topic endpoints and parameter declarations while preserving developer-owned node algorithms. In a case study on a generated ground robot, the generated conformance monitor reported each of the three departures that we introduced in the implementation. The library is distributed as a versioned package that any SysML v2 model can import.
★ Contract-Based Agentic Systems Engineering: Delegation as Quotient for Assured System Design
A component an agent builds from a prompt can satisfy it and still break the system it joins, because the prompt omits what its neighbours assume of it. The artifact to hand over is computed, not written. A SysML v2 requirement with assumed and required constraints is an assume/guarantee contract, so when a system requirement is decomposed over fixed neighbours, the open component's requirement is the contract quotient, the weakest sound one, and again a SysML v2 requirement. Its required constraints carry the neighbours' assumptions, which a requirement for the mission alone drops and unit tests with mocked neighbours cannot reach. Signal Temporal Logic is closed under the quotient. We build the requirements, with derivation, allocation, satisfaction and verification, as a SysML~v2 model and compute the quotient from it. A case study delegates the local controller of a simulated warehouse robot, whose route crosses a worker's path and ends at a dock, to one language model under the quotient requirement, a mission-only requirement and a natural-language task, and scores the deliveries on live episodes against the system requirement from ground truth.
★ Active Reachability Learning for Humanoid Recoverability Monitoring
Deploying humanoid task policies requires reliable runtime monitors that anticipate unrecoverable falls in time to trigger a fallback. We formulate this task as learning the stochastic, finite-horizon recoverability envelope of a given fallback policy. Synthesizing this monitor is challenging: existing methods assume training data naturally covers the recoverability envelope, yet acquiring such data remains a major bottleneck for contact-rich systems. Consequently, they recover only a fraction of the true envelope, leaving them prone to optimistic hallucinations for out-of-distribution (OOD) inputs. We resolve this via a unified framework that iterates between value learning and active exploration. Data exploration is reframed as a reach-avoid problem, where an uncertainty-guided explorer systematically uncovers unseen states while avoiding confidently unrecoverable regions. This loop is further supported by a combined Temporal Difference and Monte Carlo objective for accurate in-distribution value prediction and reliable uncertainty estimation. By explicitly treating the remaining high-uncertainty regions as unrecoverable, the monitor mitigates optimistic OOD hallucinations. Evaluated on stochastic benchmarks and a 29-DoF Unitree G1 humanoid, our approach drastically improves monitor coverage and accuracy and successfully transfers zero-shot to real-world hardware.
★ Concurrent Space-Based Observer Placement and Scheduling for Cislunar Space Domain Awareness
As interest in cislunar space grows, so does the need for space domain awareness (SDA) capabilities to safely support missions. To enhance cislunar SDA capabilities, the use of space-based observers has been proposed. Prior research has explored optimizing observer placement but has not considered scheduling limitations. We propose a mixed-integer linear programming formulation that concurrently optimizes the placement and scheduling of space-based observers for cislunar SDA. Our work considers three aspects of observer scheduling: sensor tasking, communication, and charging. We utilize a scenario-based approach that generalizes observer placement for unknown targets while applying scheduling constraints to each scenario. However, this problem quickly becomes difficult for commercial solvers to handle as the number of scenarios grows. As an alternative, a column generation approach is employed to introduce well-performing schedules into the problem for each orbital slot. A comparative analysis is performed across multiple instances to evaluate the performance of column generation against that of a commercial solver. A case study is then conducted in which the performance of the observers' placement from the concurrent formulation is tested against a set of validation target sets and compared with the performance of two baselines in which placement and scheduling are performed sequentially.
comment: 41 pages
★ On the Discernibility of Topology Changes in Networked LTI Systems
This article investigates the discernibility of topology changes in networked linear time-invariant systems, with particular focus on edge disconnection. We introduce a new notion of discernibility motivated by a limitation of the definition commonly adopted in the literature. In consensus networks, topology changes are inherently indiscernible for all initial conditions belonging to the synchronization subspace. To overcome this issue, we first introduce the notion of dynamical discernibility of a topology change for elementary consensus networks. This notion explicitly disregards the intrinsic indiscernibility associated with states in the consensus subspace. We characterize this property for directed communication graphs and then specialize the resulting conditions to topology changes arising from either the disconnection of an edge or a variation in its weight. We then extend the analysis to networks of homogeneous agents by introducing discernibility modulo synchronization, which factors out the unavoidable indiscernibility associated with synchronized trajectories. We derive a general characterization showing how discernibility depends jointly on the topology change and on the dynamics of the individual agents. Finally, we specialize the analysis to undirected communication graphs and show that, under mild additional assumptions, discernibility modulo synchronization is equivalent to the dynamical discernibility of the corresponding topology change. This establishes a direct connection between the two notions and extends existing results on the discernibility of topology variations in dynamical networks. The resulting characterizations are then further specialized to the case of edge disconnection.
comment: Submitted as an invited paper to the Bicentennial Special Issue of the Journal of the Franklin Institute. Under review
★ How AI Load Swings Reduce Datacenter Power Conversion Efficiency
AI training and inference workloads make GPU power swing repeatedly between compute and communication phases. As a result, the cascaded converters in a datacenter rack keep moving between operating points with different efficiencies. This paper uses an energy based, cycle averaged method to quantify that effect along a 240 V AC to 400 V DC to 51 V DC rack power path, using measured efficiency curves of commercially available GaN based PFC and LLC converters. For a 1 Hz, 50% duty-cycle load swinging between 5 kW and 510 W, conversion efficiency falls to 95.97%, compared with 96.82% at the equivalent constant average load, a loss of 0.88%. When 51/48V to 1V point of load conversion to the chip rail is included, it is shown that the loss grows to 1.66% (89.45% vs. 90.96%). These results show that efficiency rated at average load overstates real world performance, and that the penalty grows with each load dependent conversion stage between the grid and the GPU.
★ Terra-Fi: Decoupling Sensing and Wireless Depths for Sustainable Deep Underground Monitoring
Wireless underground sensor networks (WUSNs) enable in-situ monitoring of subsurface processes and buried assets to deliver substantial social and economic benefits across diverse verticals. However, meter-scale underground monitoring remains challenging since conventional WUSNs place the entire underground device at the target sensing depth, where severe soil-induced attenuation significantly degrades communication reliability and makes wireless energy transfer (WET) inefficient and even impractical. To address this limitation, we propose Terra-Fi, a novel shallow-hub and deep-sensing architecture that decouples the sensing depth from the wireless communication and energy-replenishment depth. In Terra-Fi, a shallow wireless hub is deployed at a depth favorable for communication, WET, and maintenance, while application-specific sensing probes are extended to meter-scale target layers through power-and-signal feeders. On this basis, we first examine the depth limitations of underground communication and WET and compare representative monitoring architectures. Then, we present its system architecture, operation flow, and potential benefits. Through a precision agriculture case study, our numerical results reveal that Terra-Fi establishes more robust wireless connectivity and a longer operational lifetime than conventional fully buried WUSN architectures, particularly at greater sensing depths and higher volumetric water content. Moreover, its performance gain can be enhanced through time allocation and transmission-configuration optimization. Finally, we pinpoint practical challenges and future research directions for deploying Terra-Fi into WUSN systems.
comment: 8 pages, 4 figures, 2 tables, submitted to IEEE IoTM
★ Modeling Liquid-Cooled Data Centers in EnergyPlus
As AI workloads accelerate data center transition toward high-density racks with direct-to-chip (DTC) liquid cooling, existing simulation tools remain limited in evaluating whole-building and multi-year efficiency. To bridge this gap, we develop a native EnergyPlus object, ElectricEquipment:ITE:LiquidCooled, that models DTC liquid cooling for chilled-water and warm-water architectures as plant-loop components. It couples facility-water loops, rack-side secondary loops, coolant distribution units, and chip-junction temperatures within a single solver. Validation against an independent Modelica implementation across 21 operating conditions, including two full-year runs compared at one-minute resolution, yields loop-temperature RMSE at or below 0.021 K, well within the 0.1 K acceptance threshold. Our case study demonstrates that warm-water liquid cooling reduces annual PUE from 1.43 to 1.25, eliminates on-site water use, and cuts total facility energy by 37.9% relative to an air-cooled baseline. By embedding liquid-cooling logic into EnergyPlus, our work provides whole-building, long-term energy modeling for high-density AI infrastructure within an open-source framework.
comment: 14 pages, 7 figures, 6 tables. Code: https://github.com/Urban-Informatics-Lab/energyplus-liquid-cooling
★ Classical, Machine Learning, and Hybrid Solvers for Partial Differential Equations: An Evidence-Weighted Review and Decision Framework
Machine learning for partial differential equations raises a practical question: when are classical, learned, or hybrid solvers justified? This survey compares the three families in a decision framework organized by data availability, query volume, and certification needs. An amortization law, tested on parametric Darcy flow, shows that surrogates trained on solver-generated data break even only after many queries, beyond thirty thousand for errors below one percent. An evidence ladder, a benchmark reanalysis, and three case studies show that claims of speedup and generalization carry the weakest evidence. Solver choice should follow regime, query volume, and evidence quality, not method novelty.
★ Polyhedral RIS Architecture with Quantum-Assisted Facet Selection for Uniform Coverage
Reconfigurable intelligent surfaces (RISs) direct coverage toward moving user equipment (UE), but practical deployments involve discrete phase control, uncertain facet positions, and UE mobility, which require the problem to be re-solved continually. We consider these challenges for the chamfered hexagonal polyhedral RIS, which provides nearly uniform coverage. With both incidence and departure angles, we bound in closed form the gain loss of its best facet relative to the best possible reflecting facet, include the known b-bit quantization loss, and extend the bound to joint random mounting errors of all facets. For a precoded multi-user downlink, each facet-selection subproblem of the max-min rate design is shown to be an exact quadratic unconstrained binary optimization (QUBO) problem, suited to repeated time-constrained solving. The deployment of the base station in relation to the RIS allows the polyhedral surface to bring down the percentage of dead zones in vertical planar panels from 77% to 0.3% and increase the minimum rate by 34% relative to the STAR panel. The simulated annealing algorithm finds the optimum solution in 95% of the cases. The quantum approximation algorithm fails to do so.
comment: This work has been submitted to the IEEE WCL for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible
★ Trust-Region Benders for Scalable Multiperiod Expansion Planning for Transmission and Storage
Transmission expansion planning (TEP) seeks to identify cost-effective investment strategies that enable power systems to reliably meet future demand, while accounting for the location, timing, and operational impacts of infrastructure upgrades. As demand grows and renewable penetration increases, there is growing interest in jointly modeling energy storage investments to enhance operational flexibility. This work presents a multiperiod formulation that co-optimizes transmission reconductoring and energy storage installation across multiple planning periods. To solve the resulting large-scale problem efficiently, a Benders decomposition with per-period trust-region stabilization and a relaxation-based warm-start procedure is proposed. The approach is demonstrated on a 2,000-bus Texas system, producing an investment plan spanning 2030-2045 with optimality gap below 1.0%, and is compared against a rolling-horizon approach to highlight the benefits of multiperiod coordination.
comment: Presented at PowerUp 2026, September 9-11, 2026, Boulder, Colorado
★ Density control of large-scale multi-agent systems via tuning of agent interactions
Collective behavior in biological systems emerges through adaptive interactions among units, enabling large populations to coordinate without centralized control. This mechanism raises the question of whether collective behavior in engineered multi-agent systems can likewise be regulated by modifying the interaction law between agents, rather than by directly controlling individual units. To answer this question, we consider large-scale multi-agent systems evolving on a ring and treat the interaction kernel as a time-varying control variable whose evolution regulates the collective dynamics. Using a mean-field formulation, we develop a framework for the analysis and design of kernel-based controllers. We derive explicit conditions for controllability in application-relevant scenarios, and synthesize a state-feedback controller to regulate the kernel dynamics, validated on the microscopic, discrete dynamics. Numerical simulations demonstrate the approach's effectiveness and robustness to unmodeled perturbations. This work constitutes a first step in the design of bio-inspired control strategies for large-scale multi-agent systems by modulation of interactions between agents.
★ Cross-Scenario Controller Parameter Transformation from Real-World Driving Data through Flow Matching
Advanced Driver Assistance Systems have become increasingly widespread, aiming to support drivers, reduce workload, and improve traffic safety. However, many systems are rarely used because their behavior feels unnatural and does not match individual driving behavior. The growing availability of large-scale, real-world human driving data enables more human-like and driver-individual assistance functions. This paper formulates the adaptation of Adaptive Cruise Control parameters as a driver-individual scenario-to-scenario transport problem, transforming controller parameters from one driving context to another. Using flow matching, we train a generative model to predict the target-scenario behavior of previously unseen drivers from only a few demonstrations in a source scenario. We compare the approach with a linear parameter-shift baseline and evaluate flow-matching variants incorporating driver-consistent coupling, driver embeddings, and classifier-free guidance based on driving profiles extracted from trajectory data using a variational autoencoder.
★ Foundation Models Meet Power Grid Topology for Cascade Prediction
Accurate early-stage prediction of cascading failure severity is essential for assessing the vulnerability of power transmission networks. However, existing data-driven approaches often rely on network-specific representations, limiting their generalizability across power system topologies. This work investigates the complementary capabilities of Large Language Models (LLMs) and Graph Neural Networks (GNNs) for cascading failure severity prediction. First, standalone LLM predictors are developed using a LoRA-fine-tuned LLaMA-3-8B-Instruct model with varying degrees of topological information. A hybrid Graph-LLM fusion framework is then proposed, integrating a Topology-Aware Graph Attention Transformer (TAGAT) with the LLM through confidence-gated fusion. TAGAT captures structural dependencies among transmission lines, while the LLM extracts representations from early-stage power-flow observations and aggregated cascade descriptors. The frameworks are evaluated on 30,000 simulated cascading failure scenarios for each of the IEEE 118-bus and IEEE 300-bus systems. Using only the first four cascade steps, the hybrid framework achieves weighted F1 scores of 0.94 and 0.92, respectively. In cross-system transfer from IEEE 118-bus to IEEE 300-bus, it achieves 0.72 zero-shot accuracy, improving to 0.83 with limited target-system calibration, outperforming standalone models and conventional baselines. These results show the potential of combining graph-based structural learning with foundation models to improve cascading failure severity prediction and cross-system transferability.
comment: 10 pages
★ Robust Offset-Free Lane-Keeping Control Framework for Vision-Only Automated Vehicles
Achieving accurate, real-time lane-keeping control during high-speed cornering on large-curvature roads while ensuring robust vehicle stability is a critical challenge in autonomous driving. To address this challenge, this study developed a robust offset-free model predictive control (OF-MPC) framework for vision-based autonomous vehicle lane-keeping. The regulation problem in this framework is solved using several algorithms. Numerical simulation results indicated that the interpolation-tube-based constrained iterative linear quadratic regulator (itube-CILQR) algorithm outperformed solvers based on the standard Interior Point Optimizer (IPOPT) and Operator Splitting Quadratic Program (OSQP), achieving a mean computation time of less than 5 ms. The itube-CILQR algorithm was then used for vehicle control in vision-based lane-keeping experiments in a simulation environment to quantify the perceptual uncertainty of a visual perception model (multitask UNet). This quantification was performed using the OF-MPC framework for traffic scenes that were not encountered during model training. Overall, the developed OF-MPC framework and the proposed method for characterizing the perceptual uncertainty of a visual perception system in unseen environments can contribute to the future development of advanced autonomous driving systems.
comment: 13 pages, 10 figures
★ Iterative Behavioral Identification for Covert Misappropriation Attacks
We present a data-driven, three-phase framework for covert misappropriation attacks on closed-loop networked control systems. A man-in-the-middle (MITM) adversary, with access only to intercepted input-output data, incrementally identifies the plant's behavioral subspace while maintaining stealth under an infinity-norm intrusion-detection system (IDS). Phase 1 collects an initial rank-deficient Hankel matrix using a low-amplitude probing signal that satisfies an analytical stealth bound. Phase 2 adapts online rank-increasing experiment design to the closed-loop setting, using causal predictor-based masking to limit IDS residuals while the Hankel rank grows from 1 to the full behavioral rank n + mL in exactly n + mL - 1 steps. We show that the identified subspaces are strictly nested and that the projection error of any desired trajectory decreases monotonically, strictly so almost surely at each rank increase. Phase 3 employs the final Hankel matrix in a DeePC-style optimization to synthesize a covert attack that tracks an arbitrary reference while respecting the IDS constraint. Simulations on a DC motor with PI control demonstrate full-rank identification and stealthy tracking. Stealth is guaranteed analytically in Phases 1 and 3 and is numerically supported in Phase 2. The method is model-free and assumes noise-free LTI dynamics.
comment: Presented at the 17th APCA International Conference on Automatic Control and Soft Computing (CONTROLO 2026), Coimbra, Portugal, September 9-11, 2026
★ Enabling 3GPP-Driven Services Over Optical Transport Network
We demonstrate a multi-layer slicing architecture enabling deployment of 3GPP network slices as self-contained, connected functions atop optical transport slices. The system provides Optical Spectrum as a Service (OSaaS) by integrating disaggregated optical control, multi-tenant orchestration, and telemetry-based optimization via standard interfaces.
comment: 4 pages, 3 figures. ECOC 2025 demonstration paper
★ State Estimation as In-context Tabular Prediction
This letter poses state estimation (SE) as an in-context tabular prediction (ICTP) problem. The key idea is a tabular arrangement in which historical measurement vectors and corresponding state estimates serve as the context, and a new measurement vector forms the query for which we predict a state vector. We propose to use tabular foundation models (TFMs), pretrained on synthetic data, whose weights are never updated, so power system data is needed only as context. With frozen TFM weights we estimate the state of 14-, 30-, 33-, 70-, and 118-bus transmission and distribution networks, and do so where weighted least squares (WLS) fails or degrades, under low observability and with bad data. ICTP errors are 11 to 57$\times$ below the per-bus average of the context at full metering, and 1.3 to 4.0$\times$ below it with only 5% of measurements available. With measurements at six buses biased by 8$σ$, the ICTP angle error rises 1.8 to 5.0$\times$ over its bias-free value against 2.5 to 12.7$\times$ for WLS with bad-data detection. Without retraining, ICTP has lower voltage error than DSS$^2$ on both of its feeders, and lower than a retrained TAPINE on two of three reconfiguration sets.
comment: 3 pages, 5 figures
★ Solid State Transformer - Next Generation MV Interface for Native DC loads
About PowerNexa PowerNexa is the project name for a start- up initiative driven by Marco Liserre and Steffan Hansen with the ambition to develop the next generation power grid interface. Battery integration and DC loads Ongoing electrification is fundamentally changing how electrical energy is produced and consumed. DC loads are becoming an integral part of the infrastructure while battery integration is increasingly needed to manage peak demand and address grid capacity constraints. The next generation grid interface PowerNexa aims to combine a fully controllable Medium Voltage grid interface with native battery integration and multiple DC voltage levels, based on an architecture designed for high efficiency, modularity and redundancy.
comment: 8 pages, 5 figures
★ Do LLM-derived text features improve electricity price forecasts beyond document activity? Evidence from Australia's National Electricity Market
Large language models (LLMs) can turn market notices and news into features for electricity price forecasting. Such features also record when documents arrive and whether their extraction failed, so a gain measured against a model without text can reflect document activity as well as content. We add the counts of the same documents and of their failed extractions to a no-text LightGBM model and measure each LLM representation against this activity reference, which needs no additional LLM requests. With gpt-6-luna as the extractor, we forecast Australian electricity prices from operator notices (5-30 minutes ahead) and WattClarity news (day-ahead) after the extractor's documented knowledge cutoff, averaging forecasts over three training seeds. For news, the counts alone lower mean absolute error (MAE) by 3.0%, most of the 3.6% reduction that the better LLM representation achieves relative to the no-text model; its increment beyond the counts is not significant. For notices, two of the three LLM representations lower MAE beyond the counts, by 0.14-0.21% and 0.45-0.52%, whether the counts are cumulative or cover recent windows, whereas the third shows no significant reduction. Time-shifted placebos give mixed evidence on whether these gains require text aligned with the target. Benchmarks using the operator's pre-dispatch forecasts (notices) or price history alone (news) are more accurate than all these text models. Gains against the no-text model and the activity reference differ in sign at individual seeds and, for notices with cumulative counts, in seed-averaged forecasts. Text gains should be reported against both references.
comment: 27 pages, 4 figures, 12 tables. Code and data: https://github.com/ningfanxing-sys/nem-llm-text-activity
★ A Robust Control Architecture for Multirate Systems and their Interconnections
We introduce a framework for analyzing and designing linear multirate control systems and their interconnections through the lens of robust control. The proposed approach models multirate systems as impulsive systems with multiple independent jump maps and corresponding jump-time sequences. We tackle analysis and design problems with stability, dissipativity, and robustness objectives, relying on a multi-clock-dependent Lyapunov function parameterization. Further, we develop a compositional approach for analyzing interconnections of multirate systems based on individual dissipativity properties. The theoretical conditions can be verified via linear matrix inequalities and they require weak assumptions on the sampling rates, allowing for asynchronous and time-varying sampling rates with or without separated time scales. The practicality of the framework is demonstrated with numerical examples.
★ Regret-Based Max-Pressure Traffic Signal Control under Uncertain Traffic States
Max-pressure (MP) traffic signal control selects phases using pressure scores computed from estimated traffic states, but these states are uncertain and their consequences depend on whether they alter phase ranking and how much pressure is lost. This paper develops a decision-oriented Regret-Based Max-Pressure framework that represents uncertainty through phase-selection regret, defined as the pressure loss of a candidate phase relative to the pressure-maximizing phase under each plausible latent state. We propose Stochastic Regret MP, which minimizes expected regret under a calibrated residual distribution, and Distributionally Robust (DR) Regret MP, which minimizes worst-case expected regret over a Wasserstein ambiguity set. Robustifying regret rather than absolute pressure makes the formulation invariant to state-dependent pressure components shared by all candidate phases, thereby focusing robustness on relative phase quality. We further derive an explicit queue-linear pressure-loss bound and establish strong queue stability when the net-service reserve exceeds the resulting loss coefficient. Numerical experiments show that both methods reduce normalized true-state pressure regret by about 70% relative to Point-estimate MP, while DR Regret MP yields lower network queues than direct robustification of absolute pressure. In microscopic SUMO experiments on a 28-intersection real-world network, Stochastic Regret MP and DR Regret MP reduce mean delay by 18.9% and 14.9%, respectively, and retain benefits under detector undercount and failure. The results show that representing uncertainty through its decision consequences can improve reliable traffic control under imperfect state information.
comment: 29 pages, 7 figures
★ Decision Timescales in Max-Pressure Traffic Signal Control: Theory, Methodology, and Validation
Max-pressure (MP) control adapts signal phases to prevailing traffic states, but its decision timescale is typically fixed a priori. This paper treats the decision timescale as an explicit dimension of MP control. We first characterize the trade-off between responsiveness and switching efficiency by quantifying how an equivalent switching loss reduces service capacity over a finite decision interval. This yields a load-dependent critical timescale determined by switching loss and normalized traffic load, together with a performance-oriented operating range above the stability boundary. Building on this result, we propose Variable-Timescale Max-Pressure (VT-MP), in which each intersection selects its decision interval according to local operating conditions, allowing the control timescale to vary across time and space. We establish strong stability of the resulting asynchronous heterogeneous controller under uniformly admissible local loads and bounded decision intervals, and introduce a downstream receiving-supply correction for finite-storage implementation. Three experiments provide mechanism validation and network-level evaluation. A single-intersection experiment confirms that the preferred decision interval increases systematically with traffic load and follows the derived characteristic timescales. In a store-and-forward network, heterogeneous VT-MP reduces total queueing time by 31.0% relative to the best fixed-interval benchmark. In a microscopic 5x5 SUMO network, it reduces mean trip delay by 24.3%, from 213.3 s to 161.5 s, and peak network accumulation by 12.3%. These findings show that the MP decision timescale should be treated as a traffic-state-dependent control variable rather than a fixed implementation parameter.
comment: 31 pages, 11 figures
★ Heterogeneous Space Data Centers: A Key Paradigm for Future Non-Terrestrial and In-Orbit Services
Lower space-launch cost and advances in AI are driving space-based data centers (SDCs) that provide AI-centric computing and communication services to in-orbit and non-terrestrial assets. However, architectural principles, service models, and energy provisioning for such SDCs, particularly in low-Earth orbit (LEO), remain largely unexplored. We introduce heterogeneous SDCs composed of large and small nodes, differentiated by computing capability, size, and power budget, to support applications spanning ground, aerial, and in-orbit users. We define use cases, node sizing and placement strategies, and interconnection topologies integrating space-to-space and space-to-ground links. Focusing on dawn-dusk LEO configurations, we analyze solar power availability, visibility windows, and latency-aware coverage. Using baseline workloads for typical scenarios, we evaluate latency and service reliability. We quantify energy requirements of different SDC roles and deployment scales, highlighting trade-offs among node size, orbit selection, and service portfolio, and provide design guidance and future research directions for AI infrastructure in space.
★ Quadrotor Guidance for Tasks along Elliptical Boundary
This paper presents a guidance method that enables a quadrotor uncrewed aerial vehicle to perform inspection tasks along an elliptical boundary enclosing a structure of interest. The proposed guidance approach leverages a bifurcation theory-based framework that supports multiple inspection tasks, including convergence to a desired viewpoint, switching between viewpoints along the boundary, constant-speed elliptical circumnavigation, and circumnavigation with sectorial dwell-time constraint. Stability analysis of the proposed guidance dynamics establishes convergence of the generated reference trajectories to the desired behavior. The control parameters in the proposed method further allow the settling time of the reference trajectory to be specified for each task. The resulting reference velocity expressions are closed-form and algebraically simple, facilitating practical implementation. A qualitative comparison with existing guidance methods highlights the flexibility of the proposed approach in achieving different inspection tasks through a unified framework, while relying on simpler guidance computations. The proposed guidance method is validated through numerical simulations using a six-degree-of-freedom quadrotor model with model uncertainties, and a cascaded sliding-mode controller that tracks the generated reference trajectories under bounded disturbances.
★ Model-Free Output-Admittance Reshaping for Stability Enhancement of Grid-Connected Inverters
Grid-connected inverters can be unstable when their current-control loops interact with an uncertain external network. Returning the existing controller to improve damping also alters current tracking because tracking dynamics and output admittance are coupled. This paper adds a local model-free control (MFC) loop around the existing current controller to reduce output admittance while closely preserving nominal current-tracking dynamics. The added loop uses local inverter parameters without an external grid or neighbouring converter model. A Norton formulation separates reference tracking from the current response to terminal-voltage disturbances and establishes a tracking-preservation condition for admittance reshaping. Reducing the disturbance response makes the inverter behave more like an ideal controlled current source. Averaged-model simulations show reductions exceeding 65% in the magnitudes of all four dq output-admittance elements over 10-200 Hz, with closely preserved current-tracking dynamics. Generalised Nyquist analysis predicts small-signal stability for all sampled grid-impedance cases with MFC, compared with 90.0% without MFC.
comment: Accepted at the 2026 IEEE PES Australasian Universities Power Engineering Conference (AUPEC)
★ Stability Enhancement of Cascaded DC Systems via Source Output-Impedance Reshaping Using Model-Free Control SC
Cascaded DC systems may become unstable through source-load impedance interactions even when their constituent converters are locally stable. This paper proposes a source-side model-free control (MFC) method that reuses bus-voltage feedback to reshape the source output impedance while preserving nominal reference-to-output dynamics. The controller uses only nominal source-loop information and requires neither a load-converter model nor load-side measurements. A current-controlled buck source supplying a regulated buck-resistor load that exhibits constant-power load (CPL) behaviour at low frequencies is used as the case study. Impedance analysis predicts that the proposed MFC increases the maximum stable load power by approximately 50%, thereby improving the stability margin of the cascaded system. Switched-model simulations confirm preservation of the nominal reference-to-output response and show that the MFC-controlled system remains stable at load levels for which the baseline system becomes unstable.
comment: Accepted at the 2026 CPSS & IEEE International Symposium on Energy Storage and Conversion (ISESC)
★ A Dynamic Inventory Rationing Model for Last-Mile Delivery of Medication in Liberia's Community Health Program
We study the dynamic allocation of scarce medications in Liberia's community health program. A central warehouse supplies a health facility and the Community Health Assistants (CHAs) it supervises. When a CHA runs out of medication, patients are referred to the facility, incurring travel costs if they complete the referral and remain untreated otherwise. Allocating inventory to CHAs allows more patients to receive treatment without traveling, while retaining inventory centrally preserves flexibility to respond to uncertain demand. The allocation problem must balance these considerations while accounting for fairness across locations. We formulate this problem as a finite-horizon stochastic control problem whose objective combines the costs of referrals and untreated patients with a demand-weighted fairness utility. Because standard dynamic programming methods are computationally intractable in this setting, we develop a continuous-time approximation that involves controlling the drift rate of a reflected Brownian motion. We then adapt the deep splitting method, using deep neural networks to approximate the value function and its gradient and compute a dynamic allocation policy. We calibrate the model using data on malaria cases among children in the Fangonda region of Lofa County, Liberia, and evaluate the proposed policy in the original discrete-time model. In the baseline setting, our policy reduces operating costs by about 6% relative to the strongest benchmark, the fairness-only policy, while retaining approximately 98% of its fairness utility. Additional experiments show that higher referral costs shift inventory toward CHAs and reduce local unmet demand, while greater demand variability increases our policy's performance over the fairness-only benchmark. Our model and computational approach provide a framework for studying similar last-mile allocation problems.
comment: 56 pages, 8 figures, 8 tables, including appendices
★ On Exact Critical-Region Boundaries in Continuous-Time Multiparametric Optimal Control Problems
Continuous-time multiparametric model predictive control solves the optimal control problem once, offline, for every initial state, and divides the initial-state space into critical regions, each with its own control law and sequence of active constraints. A boundary between two regions is either a hyperplane, given by one linear equation, or curved. The curved boundaries have not been computed to date; a classifier has been trained instead to distinguish the regions, so the partition has been known only approximately. In this paper we show that a curved boundary can be computed exactly, without solving any optimal control problem. The method rests on one property. Once the sequence of active constraints is fixed, together with the instants inside it at which a constraint becomes active or inactive (the switching times) and the instant at which the change of the neighboring region first appears (the event time), every condition on a boundary is a linear equation in the initial state, so the computation of a boundary, curved or not, is exact. We fix one of these instants, a switching time or the event time, and solve the linear equations that correspond to it for the corresponding initial state on the boundary; stepping the fixed instant along the horizon yields the whole boundary. A hyperplane is the case in which no instant is left to fix. We illustrate the method on the Newell-Lee evaporator, which has seven critical regions and eight boundaries, four hyperplanes and four curved, computing one boundary of each type in full with the adjacent regions at its ends.
★ Scalable Reservoir Computing for Disturbance Localization and Identification in Power Grids
In this paper, we present a novel method for the localization and temporal waveform reconstruction of disturbances in complex power grids using observations of system state variables. Although various methods exist for detecting these events, accurately identifying and responding to them remains a significant challenge due to the grid's size, speed, complexity, and inherently dynamic behavior. We formulate a method for disturbance localization and reconstruction based on reservoir computing, a recurrent neural network architecture. We show that our method is able to recover the time dependence of previously unobserved disturbances, is robust to missing information, and can operate online. The approach requires only observed system data and knowledge of the network topology, without requiring a physical model of the system dynamics, and can recover simultaneous disturbances occurring at multiple locations. We implement a scalable parallelized reservoir architecture and demonstrate our algorithm on test IEEE cases and a reduced WECC system.
comment: 11 pages, 9 figures, submitted to IEEE: transactions on power systems
★ Strengthening AC optimal power flow relaxation through offline bound tightening and linear approximations
The alternating-current optimal power flow (ACOPF) problem is fundamental to power system operation and planning, but its nonconvexity makes globally optimal solutions of large-scale instances computationally challenging. Convex relaxations provide tractable lower bounds for assessing solution quality and supporting global optimization, although their effectiveness depends strongly on relaxation strength. This paper proposes an offline convexification and bound tightening framework for constructing strong linear programming (LP) relaxations for families of related ACOPF instances. Rather than repeating the tightening procedure for each instance, we compute variable bounds and second-order cone outer-approximation (SOC-OA) cuts offline using a demand-slack model and reuse them for subsequent instances with the same topology and admissible demand variations. Computational results on PGLib instances show that the fully offline approach achieves an average gap of $4.32\%$, compared with $3.80\%$ for the fully online approach, while achieving an average speedup of approximately $272\times$. The results demonstrate that offline convexification and bound tightening can recover most of the relaxation improvement of online tightening while significantly reducing the computational burden for repeated ACOPF evaluations.
comment: 15 pages, 4 figures
♻ ★ Symmetry-breaking in a discrete-choice LQG mean field game
Mean-field games provide a continuum framework for modeling the dynamics of large, interacting populations of non-cooperative agents. This paper studies symmetry-breaking in a finite-horizon, two-choice min-LQG mean-field game in which identical agents with linear stochastic dynamics choose one of two equally desirable terminal destinations, while trading off control effort against social pressure to conform. The model has an odd symmetry between the two destinations and therefore always admits a symmetric, dynamic Nash equilibrium in which the population splits evenly between them, producing a deadlock collective state at final time. Numerical studies have suggested that, as the penalty for social nonconformity increases, this symmetric equilibrium loses stability as a fixed point of an associated scalar, self-consistency map, and asymmetric consensus Nash equilibria emerge, where most agents select the same destination. By analyzing the linearized forward-backward PDE system through the scalar map representation, we provide a proof of this loss of stability of the symmetric equilibrium. Together with the odd symmetry of the map, this implies the existence of symmetry-broken mean-field game equilibria corresponding to consensus on either destination.
comment: 26 pages, 4 figures. Originally submitted for review 5/18/26. v2 fixes issue with proof of lemma 4.3 related to terminal differentiability of the linearized scalar map, and other minor issues/typos
♻ ★ Why the Swing Parameters Are Sloppy
Evangelou, Stanković, Kevrekidis and Transtrum compared analytic and data-driven assessments of parameter identifiability for the infinite-bus synchronous generator. Fisher information with the manifold boundary approximation method (MBAM), and output diffusion maps with geometric harmonics, report the same six identifiable parameters on the late window $t \in [3, 5]\text{ s}$, and both place the inertia $H$ and the damping $D$ among the unidentifiable ones. We explain this result from the geometry of the dynamics rather than from that of the parameter-to-data map. The rotor swing is a fast fibre that has collapsed onto a two-dimensional normally attracting invariant manifold (NAIM) of the transient fluxes before the window opens. The zeroth-order NAIM is exactly independent of $H$, $D$, $T_{d0}''$ and $T_{q0}''$, four of the five MBAM limits. We prove, under stated smoothness and rank conditions, that each of the four limits is a boundary of the model manifold at a distance equal, to first order in the parameter, to its current profiled speed. The same analysis predicts how identifiability changes with the experiment: neither a larger initial speed deviation nor a load change restores $H$ or $D$ in the late window, whereas the early window $[0, 2]\text{ s}$ makes $H$ the best-determined parameter. Evaluated in a physical metric fixed by synchrophasor accuracy limits, the late window identifies five parameters rather than six and the early window nine. The spiked-covariance structure of the linearised trajectory map explains why the principal components of whitened, sampled trajectories reproduce the prior-scaled Fisher spectrum, and regression on those components attains the Fisher-predicted recovery error in 113 of 114 parameter--scenario pairs.
comment: The draft is withdrawn so as to improve before resubmission. Regret the inconvenience
♻ ★ Privacy-Preserving Fully Distributed Gaussian Process Regression
Although distributed Gaussian process regression (GPR) enables multiple agents to jointly learn a model of the target function, its collaborative nature poses a risk of private data leakage. To address this, we propose a privacy-preserving fully distributed GPR protocol based on secure multi-party computation, which hides each agent's individual contribution from semi-honest coalitions of bounded size, beyond what is implied by the aggregated value. Building upon a secure distributed average consensus algorithm, it guarantees that each agent's local model practically converges to the same global model obtained by the standard distributed GPR. Formal privacy guarantees are established within the simulation based security paradigm. The protocol is further extended to privacy-preserving optimization of kernel hyperparameters, which is critical yet often overlooked in the literature. Experimental results demonstrate the effectiveness and practical applicability of the proposed method.
comment: 12 pages, 3 figures, 1 table, accepted to IEEE Transactions on Control of Network Systems
♻ ★ Safe Leader-Follower Density Control via Higher-Order Mean-Field Control Barrier Functions
Mean-field control barrier functions (MF-CBFs) enforce safety of large-scale multi-agent systems at the level of densities, but existing formulations require the control to act directly on the population to be kept safe. We consider populations that are not directly actuated and are influenced only through other, controlled populations, possibly in the presence of diffusion. We show that if the constrained population lies at directed graph distance d from the actuated one, then, under a nondegeneracy condition, the safety functional has relative degree d+1 and the control enters its (d+1)-th time derivative affinely. Higher-order safety constraints can therefore still be enforced through a convex quadratic program. We apply the framework to leader-follower density control among dangerous regions, where the follower safety functional has relative degree two and the leader safety functional has relative degree one. We characterize exactly when the two constraints are jointly infeasible and bound the tracking errors induced by the safety correction. Simulations in cluttered environments show that the proposed filter preserves follower safety, whereas enforcing leader safety alone does not.
♻ ★ Controller-Agnostic Runtime Safety Certificates for Tool-Using AI Agents via Data-Driven Robust Reachability
Software agents built on large language models act by issuing tool calls, some of which cannot be undone. To control theory the model is an opaque feedback controller, replaced every few months, whose closed loop cannot be analyzed directly. We propose a runtime filter, or shield, between any such controller and its environment that certifies a bound on the probability of reaching an unsafe condition during a bounded task. Given a predicate abstraction and candidate successor sets, everything else comes from logs: the loop becomes a finite controlled Markov chain, the estimated kernel is surrounded by a distribution-free confidence region, and robust reach-avoid value iteration runs over it. The certificate is probably approximately correct and uniform over controllers respecting the filter, so one model may replace another without recomputation. In all tested datasets the true risk stayed within certificate and budget; in a file-system sandbox the filter stays sound against a hazard the logs barely covered, where a plug-in baseline certifies safety and fails; and it holds seven controllers, three of them open-weights models, to zero unsafe outcomes in 1050 episodes.
♻ ★ The BAR-SOT Method: Long-term Average Cost Control as Stochastic Optimal Self-Transport
We reformulate average-cost (ergodic) control of a Markov jump process as a finite-horizon stochastic optimal transport (SOT) problem that jointly optimizes over the controlled evolution and the marginal law from which it starts and returns to (i.e., a self-transport). The constraint relating the marginal flow to the controlled generator is the basic adjoint relationship (BAR), so we call the resulting problem BAR-SOT. For any time horizon T>0, its optimal value is a constant scaling of the long-run average-cost rate. We give three equivalent formulations (through controlled processes, a Fokker--Planck constraint, and relaxed marginal measures) and show that the optimal dual is a stationary potential plus a term linear in time, with slope equal to the rate. A relative-entropy penalty on the control yields a cost-tilted Schrodinger bridge problem, computable by a Sinkhorn-type iteration when every transition rate is controlled, whose value converges to the unregularized optimum as the penalty vanishes. We develop the theory for finite Markov decision processes and then for general controlled Markov jump processes. A neural parametrization of the dual, trained as a physics-informed neural network (PINN), that encodes Harrison's equivalent-workload formulation (see Harrison 2000) matches the strong reinforcement-learning baseline of Dai and Gluzman (2022); numerically conditioning the fit of a sub-dominant transverse correction then improves on both that baseline and the best priority heuristic. On the input-queued switch, a graph-attention parametrization of the dual improves on the strongest matching heuristic we are aware of, with a parameter count that does not grow with the switch size.
Robotics
★ Sensorimotor Alignment: A Platonic Proxy for Embodied Policy Success
Robotic foundation models are typically evaluated through costly closed-loop rollouts, which motivates cheap offline proxies for model capability. The Platonic Representation Hypothesis (PRH) suggests that as representations improve, different modalities such as vision and language converge toward a shared statistical model of reality. We ask whether motion trajectories form another such modality: as embodied models become more capable, should their sensory representations converge toward the structure of physical motion? To examine this, we propose Sensorimotor Alignment ($S^2$), a training-free offline metric that measures how well a model's sensory representation aligns with the geometry of physical motion. We find that $S^2$ correlates positively with downstream robot task success across embodied model families, making it a low-cost offline proxy for prioritizing models before closed-loop evaluation. $S^2$ can also identify promising vision-language model (VLM) backbones even before costly fine-tuning into vision-language-action models (VLAs). Beyond this correlation, $S^2$ offers representation-level explanations for phenomena observed in VLAs and world action models (WAMs). With $S^2$, we extend PRH to embodied models and make sensorimotor alignment empirically measurable, offering insight into why an embodied model succeeds beyond its closed-loop success rate.
comment: 28 pages. Project page and code available at https://yunfeixie233.github.io/s2/
★ OpenTac: An Open Ecosystem for Customizable, Scalable, and Transferable 3D Tactile Sensing
We present OpenTac, a novel sensor and a full-stack ecosystem for tactile skins that are highly customizable, scalable, and readily deployed in downstream robot control and learning applications. Our core contribution is twofold. First, we develop a scalable resistive tactile skin with a structured deformable overlay that converts tangential loading into spatially varying pressure. Differential responses across neighboring "taxels" support decoding of local shear vectors, and the common-mode response encodes normal force. Second, we provide a highly customizable sensor design tool that is able to automatically generate fabrication files for these tactile skins from 2D outlines and developable 3D surfaces, including from abstract descriptions such as text (e.g., "design a sensor for a size 9 shoe insole"). We develop an ecosystem around the OpenTac sensor, including the CAD design tool, a sensor simulator, cross-sensor transfer, and fast/adaptive data acquisition. We demonstrate the broad applicability of the OpenTac-CAD sensor design tool by designing and fabricating more than 30 sensors across a variety of robot embodiments. We characterize the three-dimensional tactile fields produced by OpenTac with a commercial, high-precision force/torque sensor. We demonstrate the broad applicability of 3D tactile sensing across contact-rich feedback control and manipulation policy learning on four real-world robot embodiments. Additionally, adaptive acquisition enables kilohertz-rate partial readouts, exceeding full-array raster scanning on the same sensor by 19.2X at 16 sampled taxels per update for a 1024-taxel skin. We provide all hardware, software, and result videos are at opentac.github.io
comment: 14 pages, 14 figures. Project website, open-source hardware and software, and demonstration videos: https://opentac.github.io
★ Extending Harmonic Function Methods via Root Locus for Coordinated Path Generation
We consider the offline generation of coordinated paths for N interchangeable robots and N goals in planar environments with polygonal obstacles. After a conformal map to a circular domain, starts and goals become the poles and zeros of a rational function R, and the paths are the branches of its Generalized Root Locus (GRL), i.e. the streamlines of the logarithmic potential -log R. We show that, for every phase outside a finite set of critical phases and a set of boundary phases, the induced gradient motion drives each robot to a distinct goal along pairwise disjoint paths that never reach the obstacles; the robot-goal assignment is produced by the construction, and with the gain used as a clock all robots move as the closed-loop roots of the GRL and arrive together at a prescribed time. Obstacles become streamlines through successive Milne-Thomson reflections, and for a finite number of reflections a finite test certifies safety for the potential that is actually computed. Reciprocal root-locus planning is recovered as a special case. The method is illustrated on two polygonal workspaces, 100 random instances compared with independent harmonic planning and artificial potentials, and a scene with ten robots and ten obstacles.
comment: 16 pages, 6 figures. Extended version of a paper submitted to Automatica. Corresponding code: https://github.com/mfrascarelli/grl-certified-planning
★ DiffDreamer: Coupling World Models with Differentiable Simulation for Agile Quadrotor Control
Tasks such as quadrotor dynamic-gate racing require agents to anticipate external motion while maintaining precise high-frequency control. World-model-based reinforcement learning provides a natural solution by learning latent dynamics for future imagination, yet it often entangles environmental evolution and the robot's own dynamics in a single learned rollout, where model bias can affect policy optimization efficiency. We propose DiffDreamer, a differentiable reinforcement learning framework that tightly couples a learned world model with differentiable simulation for actor optimization. In one optimization step, the learned world model propagates latent states that capture external dynamics and history-dependent features, while the differentiable simulator propagates the robot's explicit physical state. We further decompose the actor objective into a critic-driven pathway for non-differentiable task rewards and a simulator-driven pathway for differentiable motion rewards. Experiments on aggressive tracking and dynamic-gate racing show that DiffDreamer improves training convergence over the model-free, model-based, and differentiable-RL baselines, and final performance over these baselines and classical planning and control methods. Additional experiments assess reward decomposition and the odometry-conditioned relative-state representation through ablations. We also provide proof-of-concept real-world flight experiments for validation.
★ Progress Probes $P^2$: Lightweight Runtime Monitoring for Vision-Language-Action Models
Vision-Language-Action (VLA) models can perform diverse manipulation tasks directly from visual observations and natural language instructions, but they remain prone to mistakes and unpredictable failures. Deploying these policies without continuous human supervision will require robust runtime monitors that detect when the policy has gone off-course. Existing monitors use external foundation models, such as vision-language models (VLMs) to supervise the policy, introducing significant complexity and computational cost for monitoring at runtime. In this work, we investigate an alternate approach: \textit{can a VLA's own internal activations provide the information needed to monitor its execution?} Our method trains linear probes on policy activations from demonstrations to estimate progress, then flags executions that deviate from expected progress. In experiments with the open-source VLA $π_{0.5}$ and across simulated and real-world manipulation tasks, this approach is competitive with or outperforms VLM-based monitors in task failure detection, as well as methods which require failure data for training. These findings suggest that simple probes of VLA internals can be used to develop reliable runtime monitors, opening a path toward building policies that can act reliably and monitor their execution without external foundation models.
★ Humanoid Mobility via Attention-based Contacts
Humanoid mobility over complex terrain requires accurate foot placement and coordinated whole-body motion capable of robust adaptation. Reinforcement learning (RL) policies trained without motion references generalize well across terrains, yet eliciting effective whole-body coordination through reward design and exploration alone remains challenging. In contrast, reference-based methods yield natural motions but struggle to adapt beyond the predefined trajectories in their training data. We introduce a learning framework that bridges these two approaches by using motion data strictly as a training prior. An explicit, self-supervised, attention-based contact planner predicts future body-environment contacts, while a contact-conditioned tracker translates these plans into coordinated whole-body motion. Together, they establish contact as an interface between sensory observations and control actions. This abstraction allows the policy to internalize the contact-aware structure of locomotion rather than directly replicating kinematics. Consequently, a single policy can exploit data-driven motion priors for natural whole-body coordination while retaining the flexibility to discover environment-dependent behaviors through reinforcement learning. Building on the learned contact planner and tracker, a subsequent fine-tuning stage bootstraps their acquired capabilities to explore environment variations beyond those covered by the original motion data. Operating entirely from onboard observations without any reference inputs at deployment, this framework enables versatile humanoid mobility in challenging environments.
comment: under review
★ Echo: Coupled Action and Task Status Signal Denoising for Vision Language Action Models
Recognizing task completion and failure enables robots to stop upon success and initiate recovery or replanning after failure. Existing methods augment Vision Language Action (VLA) policies with task status signal prediction to support these execution decisions. However, learning and integrating these signals can require fine-grained supervision, external monitoring models, or complex architectural designs. Moreover, the potential of task status signals to directly guide action generation remains underexplored. We present Echo, a framework that combines task status signal learning without frame-level progress annotations with a compact design for status prediction and action generation within a single VLA. Specifically, Echo represents task status with stop and failure signals, added as two dimensions of each action chunk and denoised together with robot actions. Their combination distinguishes ongoing execution, successful completion, and failure. To learn these signals without fine-grained progress annotations, Echo constructs successful, failed, and recovery trajectories with complementary supervision properties. To enable status signal-guided action generation, Echo controls interactions between the signals and actions and denoises the signals faster than the actions. These allow later action denoising steps to condition on cleaner status estimates while preserving accurate signal prediction. Extensive real-world manipulation experiments show that Echo accurately predicts task status and consistently improves policy performance.
comment: Project page: https://lyfeng001.github.io/Echo/
★ Geometry-Dependent Properties of Physically Crosslinked PPy-PVA Hydrogels Enable Tunable Iontophoretic Drug Delivery
Objective: Iontophoretic drug delivery has primarily relied on current modulation to regulate drug transport. Here, we investigate hydrogel geometry, microstructure, and local electric fields as alternative mechanisms for controlling transdermal delivery. Methods: Bovine serum albumin (BSA) was encapsulated in polypyrrole-polyvinyl alcohol (PPy-PVA) hydrogels. Five geometries, including Hilbert curve-based designs, were fabricated to vary pore architecture and local electric field distributions. We optimized PPy content, drug loading, and current density. Electric fields were evaluated using finite element analysis, and microstructures were characterized by scanning electron microscopy. Drug transport was assessed using Franz diffusion cells with artificial membranes. Results: Hydrogel geometries exhibited distinct microstructures and electric field distributions. Iontophoresis substantially enhanced permeability compared with passive diffusion, while individual geometries produced different cumulative release, flux, and permeability profiles. The average iontophoretic permeability was (3.80 +/- 1.93) x 10^-5, exceeding previously reported values using artificial membranes. Conclusion: Geometry-driven fabrication of drug-loaded hydrogels enables tunable transport characteristics under both iontophoretic and passive conditions. Significance: Geometry-mediated control offers an alternative approach to regulating iontophoretic drug delivery without requiring individual current-density modulation.
★ Decentralized Bi-Level Optimization for Cooperative Orbital Transportation of a Rigid Payload
The rapid growth of the space economy, alongside increasing orbital congestion and debris, is driving demand for reliable on-orbit transportation of large and non-cooperative objects. Performing such transportation with a swarm of small spacecraft distributes risk and removes the single point of failure inherent to monolithic architectures, but requires solving a non-convex trajectory optimization coupling every agent's cone-constrained thrust to the payload's rotational dynamics. Existing approaches span decentralized control laws with asymptotic guarantees but limited constraint enforcement, and centralized and partially centralized planners that reintroduce a single point of failure. In this work, we present a decentralized bi-level framework driven by exact gradients: the problem is split into an inner convex thrust allocation and an outer search over payload torque trajectories. This decomposition yields fuel-efficient solutions quickly, without requiring full NLP convergence, and at short timescales its performance scales with the number of agents. Theoretical convergence guarantees are established, and the method is validated through extensive Monte Carlo simulation, including closed-loop recovery from in-flight faults. These properties make the framework well suited to spacecraft swarm applications.
comment: 10 pages, 6 figures, 4 tables
★ Dispersive Tree Agreements for Multi-Robot Coordination
Multi-robot coordination of nonlinear systems is naturally formulated as joint-space optimal control subject to constraints over groups of robots. Existing methods that approach joint optimality remain largely centralized, offline, and confined to collision-free motion planning, while decentralized, real-time counterparts target safe navigation without pursuing the joint optimum. This paper develops a decentralized, real-time planner that approaches the joint optimum as an anytime process. We extend a recent single-robot framework that searches dispersive trees of forward-propagated commands for differentially flat systems to coordinate a multi-robot team near-optimally. We first show that the inter-robot coupling can be effectively replaced by some local constraints, under which locally grown dispersive trees contain a near-optimal agreement. It is this structure that the robots pursue by decentralized column generation, negotiating rents on the shared space that guide the search of each robot's tree. The negotiation settles on an agreement whose near-optimality is certified at quiescence. Our protocol, WarpWeaver, implements this negotiation as an asynchronous real-time loop over a receding horizon, and demonstrates effective closure to quiesced agreements even when real-time deadlines cut the negotiation short. In simulation, WarpWeaver reduces total travel time by up to 22% against centralized offline planners on the Dynobench suite and by up to 37% against a decentralized real-time baseline, and nearly doubles the success rate of a task-specific planner on graph connectivity. Hardware experiments on six robots planning onboard demonstrate real-world implementability.
comment: 19 pages, 5 figures, 7 tables. Submitted to IEEE Transactions on Robotics
★ SCOPE: Scene-graph Construction Optimized by Plan Error-correction for Robot Long-term Planning Datasets IROS 2026
Task and Motion Planning (TAMP) stands as a fundamental challenge in robotics, essential for enabling autonomous agents to execute complex, long-horizon tasks in diverse environments. While Large Language Models (LLMs) introduce a new paradigm for TAMP by leveraging extensive common sense, their inherent reasoning limitations hinder direct application, and fine-tuning strategies to enhance these capabilities are currently bottlenecked by the scarcity of high-quality datasets. Existing automated generation methods often suffer from homogeneity, limiting model generalization, and unsolvability, where hallucinated scene graphs create physical mismatches with task goals. To address these issues, we propose Scene-graph Construction Optimized by Plan Error-correction (SCOPE), an automated framework designed to generate diverse and executable TAMP datasets. SCOPE incorporates multi-level diversity metrics to maximize the information entropy of scenarios. Crucially, it introduces a Plan Feedback Error-correction pipeline that utilizes planner execution failures as feedback signals to retroactively rectify logical inconsistencies in scene graphs, ensuring physical feasibility. Experimental results demonstrate that SCOPE exhibits greater robustness to erroneous scene graphs and generates higher-quality data. When utilized for LLM fine-tuning, SCOPE-generated data yield a downstream TAMP model that achieves an absolute 12.33% gain in success rate over its counterpart fine-tuned with data generated by DELTA, a state-of-the-art PDDL-based planner.
comment: 8 pages. Accepted at IROS 2026. Project page: https://scope-page.github.io/
★ Point-Level Refinable Semantic Mapping for Spatial Understanding
Open-vocabulary 3D semantic maps have emerged as a key spatial memory representation for long-horizon robot autonomy, enabling robots to reason about objects, their spatial relationships, and past observations across time. However, existing object-centric semantic maps are primarily designed for static environments and rely on an append-only point integration strategy, where newly observed points are accumulated but previously integrated points are rarely reevaluated. Consequently, perception errors accumulate over time, and outdated points persist after objects move or disappear. Although recent dynamic mapping methods update object-level representations in changing scenes, they do not explicitly revise noisy or obsolete points within each object. We propose an online refinable semantic mapping framework that continuously refines object representations through probabilistic point-level confidence updating. Each object point maintains a confidence score that is updated online using multi-view temporal evidence: consistent observations reinforce reliable points, while inconsistent segmentation masks, depth measurements, and visibility cues suppress spurious and outdated points. Combined with object tracking and re-identification to preserve instance identities under motion and relocation, our framework enables object-centric semantic maps to be continuously refined rather than monotonically accumulated. Experiments demonstrate improved mapping accuracy over the previous method by 3.46% on Replica and 2.55% on ScanNet, show robustness in dynamic environments and under perception noise, and provide an effective spatial memory for object navigation and embodied question-answering with success rates of 82.4% and 76.7%.
★ Skeleton-Grounded Latent Action Model for Robot World Models
Robot world models simulate how environments evolve in response to robot actions. Latent action models enable learning dynamics from videos by encoding visual transitions as proxy actions. However, reconstruction-driven latent actions mix robot motion with object responses and other scene changes, complicating their correspondence with robot controls. We propose SkeLAM, a skeleton-guided latent action model for controllable robot world modeling. SkeLAM decomposes latent actions into a kinematic component grounded by robot skeleton supervision and a residual component retaining complementary visual dynamics. Cross-modal swap reconstruction encourages RGB-derived and skeleton-derived kinematic representations to be interchangeable during video reconstruction. The world model incorporates both components through separate conditioning pathways, while branch dropout reduces its dependence on future-derived residual information. A lightweight adapter maps robot actions to the kinematic space, and subsequent world-model fine-tuning adapts generation to these predicted conditions. At inference, the model generates future observations from the current observation and robot actions. On DROID, SkeLAM reduces end-effector ADE by 19.5% and LPIPS by 11.1% compared with DreamDojo. Motion probing supports kinematic specialization, while qualitative results demonstrate zero-shot human-to-robot motion transfer in generated rollouts. Policy evaluation shows agreement between predicted success rates and those measured in real robot executions.
comment: Project Page: https://aopolin-lv.github.io/SkeLAM/
★ VecTile: A Low-Cost Flexible Tactile Array for 3-DoF Force Sensing ICRA
Tactile sensing provides robots with contact information difficult to obtain from vision alone. Shear force is particularly important for understanding tangential contact, guiding corrective motions, and maintaining grasp stability, yet flexible and low-cost tactile arrays often measure only normal force. We introduce VecTile, a low-cost, 6 mm thin, flexible tactile array capable of three-axis force sensing. VecTile combines a piezoresistive layer with passive silicone nubs that convert normal and shear forces into distinguishable pressure patterns. The sensor can be fabricated in-house without specialized equipment. The complete system costs less than 35 USD for a 4x4 array. We validate the sensing mechanism through finite element analysis and hardware experiments. In controlled single-taxel tests over 1-5 N normal force and up to 1.5 N shear force, VecTile achieves root-mean-square errors of 0.39 N and 0.16 N for normal and shear forces. A VecTile calibrated while flat retains accuracy when curved, while a five-minute manual calibration is sufficient for practical use. Two applications demonstrate its utility. First, shear readings are sensitive enough to stop a sharp scalpel from puncturing a balloon after contact. Second, the sensor substantially improves Diffusion Policy-guided power-plug insertion. All designs and code will be released upon paper acceptance. Project website: https://jiannnh.github.io/VecTile/.
comment: 8 pages, 8 figures. Submitted to IEEE International Conference on Robotics and Automation (ICRA) 2027
★ A Sparse Topometric Graph for Scalable Cross-Session Indoor Navigation
A conventional embodied agent in an indoor environment builds a map by accumulating noisy real-world measurements and results from heavy inference to guide navigation. While the detailed measurements may differ due to scene changes or sensor settings, the unique topological structure of navigable passages or the connectivity between rooms may provide training-free guidance that complements local observations. We propose a sparse topometric graph as an efficient structural representation for robust indoor navigation, alleviating the need for fine-grained matching, which is computationally expensive, memory-intensive, and prone to real-world variations. The map consists of nodes that encode local 3D geometry and distance-weighted edges that capture traversable connectivity. A learned graph-matching network finds consistent subgraph structures across sessions by considering both node-level geometric descriptors and neighborhood connectivity. The holistic contextual matching is robust to local ambiguities caused by weak distinctiveness or repetitive structure, and establishes stable correspondences despite local scene changes, different sensor modalities, or limited spatial overlap. We analyze the practicality of the graph through cross-session point-goal navigation, where a goal specified in a prior reference map must be reached from an unknown initial pose in a later session. Experiments show that our method achieves the highest success rates (77.1% on Gibson and 40.0% under 3D-FRONT scene changes), while reducing memory by more than two orders of magnitude compared to point-level representations. The proposed approach further supports cross-session alignment and goal-directed navigation with strong scalability and robust real-world transfer.
★ PhysTrace: Interpretable Temporal Evaluation of Physical Consistency in Video Generative Models
World action models increasingly rely on pretrained video models assumed to capture physical dynamics for prediction, planning, and action generation. Existing evaluations assess plausibility, probe representations, or measure predictive surprise, but rarely identify which history-conditioned physical expectations conflict with observations. We introduce PhysTrace, a training-free framework aligning MLLM-generated, history-conditioned expectations with observed video windows to temporally localize physical violations and provide interpretable descriptions without task-specific probes or classifiers. We also introduce IntPhys2-Timing, adding violation intervals to IntPhys2. PhysTrace achieves a Hit Rate of up to 70.7% across physical principles on IntPhys2-Timing and outperforms the compared methods in overall possible-impossible discrimination accuracy on IntPhys2. Applied to reconstructed videos along the trajectories of pretrained diffusion/flow-based video models, PhysTrace reveals attribute- and noise-dependent patterns in physical violation localization, enabling interpretable physical consistency analysis across timesteps.
comment: 17 pages, 7 figures (including supplementary material). Code: https://github.com/PhysTrace/PhysTrace
★ FastHarness: Experience-Driven Review Scheduling for Vision-Language-Action Policies
Agentic robot systems combine language-model reasoning with robot policies to monitor execution and recover from failures. However, reviewing every action chunk incurs substantial overhead, even in familiar situations. We introduce FastHarness, an experience-driven execution harness that adaptively schedules GPT-6 Astra supervision of a π0.5 policy. A fast path uses stage memory and reviewed outcomes to reduce review frequency as experience accumulates, while a complementary recovery path focuses supervision on correcting errors. Once recovery is verified, routine execution resumes, with failure experience retained to guide future interventions. The policy remains frozen and generates fresh actions from current observations. Across four simulated manipulation tasks, reviewer token use decreases by an average of 57.2% per task from the first evaluated episode to the mean of the final five episodes. Mean episode time decreases from 85.5 to 52.0 minutes over the same interval. On the same tasks, mean success increases from 80% for GPT-as-Policy (π0.5 + GPT-6 Astra) to 97.19% with FastHarness. These results highlight adaptive supervision as a practical route to more efficient agentic robotics without additional model training.
comment: Github:https://github.com/FastHarness/fast-harness
★ FocusVLA: Future-Intent Guided Dynamic View Routing for Vision-Language-Action Models
Vision-language-action (VLA) models have become an important paradigm for robotic manipulation by unifying visual perception, language understanding, and action generation. Since reliable manipulation requires sufficient observation of objects, end-effectors, and their spatial relationships, multi-view inputs are widely used to provide complementary spatial information. However, our experiments show that existing VLA models do not fully exploit such multi-view complementarity. The policy tends to persistently rely on the wrist view, lacking the ability to dynamically adjust global-local view dependency across manipulation stages, which limits stage adaptability and robustness. To address this issue, we propose FocusVLA, a stage-aware multi-view routing framework. FocusVLA predicts future interaction intent from historical actions and uses it to guide dynamic view routing during action generation. By adaptively coordinating global and local observations across different task stages, FocusVLA improves multi-view utilization. On RoboTwin 2.0, FocusVLA improves the average success rate over the strongest baseline $π_{0.5}$ by 10.10 pp. In real-world evaluation, FocusVLA improves the success rate from 71.7\% to 85.0\%, achieving a 13.3\% gain over $π_{0.5}$. Moreover, the intent prediction and dynamic routing modules of FocusVLA can be integrated into existing VLA frameworks without modifying their core architectures.
comment: Accepted at CoRL 2026
★ CoHelix: Event-Based Multi-Propeller Rotation-Rate Tracking with Shared Projective Geometry
Estimating propeller rates from a moving event camera requires separating blade rotation from the changing image geometry. Background activity mixes with blade events, while repeated blade appearances admit half- and double-rate ambiguities. We propose CoHelix, a recursive filter that estimates one shared homography and separate rotation states for four rigidly mounted, coplanar propellers. Each admitted event updates its assigned rotation state, and its accepted geometry measurements correct the shared homography used to interpret later events from every propeller. We report lock, the fraction of evaluated time for which an estimate remains within 2% of synchronized ground truth, because mean error alone does not show how long a tracker follows the true rate. With fixed-rate initialization, leave-one-recording-out evaluation on moving-camera data gives CoHelix 62.9% lock, compared with 39.8% for HelixTrack and 22.2% for EventPro. With other settings fixed, allowing four geometry contributors instead of one raises lock from 3.4% to 62.9%. On a private target-follower dataset with a different target quadcopter, CoHelix retains the measurement settings selected on moving-camera data and uses a fixed initial rate. It reaches 95.9% lock, compared with 56.2% for HelixTrack and 61.1% for EventPro.
★ Probabilistic Energy Modeling Unifies Versatile Swarm Control
Robotic swarms must coordinate across diverse missions, yet existing controllers often require redesign, retuning, or retraining when objectives change. Here we show that probabilistic energy modeling provides a unifying representation for versatile swarm control, in which diverse missions emerge from compositions of shared interaction principles rather than task-specific control formulations. By exploiting the correspondence between Gibbs random field energy minimization and structured reward maximization, we encode fundamental swarm behaviors as reusable energy terms within a unified probabilistic objective. Decentralized policies trained within this representation are composed into a single controller through lightweight policy gating, without retraining. An interpretable action-attention mechanism captures non-uniform local interactions, while dimensionless normalization supports transfer across swarm scales and platforms. Demonstrated on aerial swarms, the unified policy generalizes zero-shot across six mission classes, 10-100 agents, two hardware platforms, and indoor cluttered and GPS-denied woodland environments. Large-scale simulations achieve 0.96-1.00 task success rates with 1-10 ms per-agent inference latency, while physical deployments confirm zero-shot transfer to real platforms and unstructured environments. These results establish probabilistic energy modeling as a scalable and transferable framework for decentralized swarm control.
comment: 25
★ GelSlim 5.0: Unifying Vision and Touch with a Single Fingertip Camera
Vision and touch provide complementary cues for object pose and local contact geometry, but external cameras are often occluded at the contact site. We present GelSlim 5.0, a fingertip sensor that acquires visual and tactile observations with one fixed camera and a continuous UV-responsive contact layer. By switching illumination, the camera sequentially acquires through-membrane visual images and contact-sensitive tactile images from a shared viewpoint, avoiding inter-camera extrinsic calibration. To combine these observations, we introduce VisTacFusion, a multimodal transformer with separate prediction paths for planar object pose and contact geometry. Asymmetric bottleneck fusion incorporates visual context while preserving tactile spatial features for depth and surface-normal estimation. Modality dropout enables inference with paired inputs or either modality alone. On real validation data from 12 known objects, paired inputs reduce depth MSE, angular error, and translation $L_1$ by 61.2%, 72.3%, and 61.3%, respectively, relative to tactile-only inference with the same model. Relative to visual-only inference, the corresponding reductions are 93.7%, 6.1%, and 8.6%.
★ Whole-Body Motion Guided Reference Generation for Uncertainty-Aware Compliant Assistance
Exoskeletons have shown great potential for augmenting human physical capabilities, yet generating reliable assistance remains challenging because human motion patterns and interaction requirements vary substantially in practice. To address this problem, a dual-loop assistance strategy is proposed to couple multimodal exoskeleton trajectory generation with variable admittance control, where trajectory variance links the outer-loop generator and the inner-loop controller. The generator synthesizes future exoskeleton trajectories through denoising, primarily using whole-body human motion and exoskeleton proprioception, with task descriptions as auxiliary semantic cues. To reduce cross-modal representation mismatch, a three-stage training pipeline is introduced to progressively optimize the human motion encoder, the exoskeleton proprioception encoder, and the trajectory generator. In the inner loop, multiple future trajectories are sampled from the generator to estimate the trajectory mean and variance. The mean is used as the control reference, while the variance provides an uncertainty measure for prediction reliability. This measure is further incorporated into a stiffness regulation law, enabling compliant assistance with input-to-state stability under bounded lumped interaction input. Experimental results demonstrate that the proposed method significantly reduces muscle activation levels compared with standard baselines, while also decreasing users' metabolic energy expenditure relative to the unassisted condition.
★ Learning Robot Behavior Priors from Non-Robotic Synthetic Data
Real robot data, rich across tasks and morphologies, is often treated as the bottleneck for training robot learning models, including embodied foundation models that aim for zero-shot physical transfer. In this paper, we investigate the degree to which such models can acquire necessary structure using data from systems without embodiment. To this end, we test if inexpensive, procedurally generated synthetic data can yield useful priors before a model observes any real-world behavior. We study this through robot action prediction, a natural pretraining objective and a direct test of whether structure learned from arbitrary synthetic tasks transfers to embodied settings. We introduce three data generation pipelines that model mathematical abstractions of robotic control, and use them as pretraining priors for a Transformer. We demonstrate that a model trained purely on this non-robotic data attains non-trivial zero-shot prediction performance on unseen physical robot tasks via an in-context learning mechanism, and that our specialized control priors notably improve upon generic causal priors. We also compare against a compute-matched baseline trained on real robot data, and find that it outperforms our model but only on the datasets it was trained on. These findings suggest that procedurally generated, mathematical abstractions can yield useful robotic priors, offering a potential complement to expensive embodied data collection.
★ Consistent Distributed Invariant EKF SLAM with Point and Object Features
Multi-robot SLAM allows robots to improve their estimates by exchanging map information, but repeated exchanges create unknown correlations between local estimators. Covariance intersection (CI) can fuse estimates without tracking these correlations. In right-invariant extended Kalman filter (RI-EKF) SLAM, however, point features obtain their invariant errors through the robot rotation, so a neighbor's map estimate is expressed in its own error coordinates and is correlated with its pose, and cannot be fused by CI directly. This paper presents Joint-CI, a unified fusion method for distributed RI-EKF SLAM with point, object, and mixed point--object maps. Using a neighboring robot's local posterior and an inter-robot observation, a joint estimate of the receiving robot's pose and the common features is constructed in the receiving robot's error coordinates, retaining the robot--feature and feature--feature correlations. Partial-state CI then fuses this estimate with the receiving robot's local posterior. We show that the unified formulation and the fusion preserve the six unobservable directions associated with global rotation and translation, and that communication and fusion preserve first-order covariance consistency given consistent local posteriors. Monte Carlo simulations and a real-world dataset-derived experiment show that Joint-CI improves estimation accuracy while maintaining consistent, mildly conservative uncertainty.
comment: Submitted to IEEE Robotics and Automation Letters (RA-L)
★ Inside-JEPA: From Prediction to Relational Grounding in Latent World Models
Joint-embedding predictive architectures (JEPAs) learn world models in latent space, enabling efficient planning but leaving their states opaque. We introduce Inside-JEPA, which places relational channels for ordered entity pairs inside the JEPA latent state and shows that an inverse-dynamics loss can ground these channels without relation labels when its head reads them, either alone or as part of the full state. On Something-Something V2, latent prediction alone performs no better than a random relation head, while inverse-dynamics supervision that recovers actions from channel changes improves relational AUROC by .065-.103 across ten settings with annotated boxes, by .081 on RH20T robot episodes with held-out tasks, and by .050-.061 across five relations using off-the-shelf detection and tracking. An inverse head on entity latents decodes actions more accurately but barely improves relational grounding; shuffled labels, two label-free objectives, and action labels on only 5% of the pairs also fail to ground the channels. Given an action, predicted relational changes approach the quality of observed future relations (AUROC .599 vs .621). For one-step relational goals, a supervised readout of the 16 channels ranks the correct action among the top five of 65 in 43% of cases, compared with 24% for prediction-only latents and 17% for a frequency prior.
★ Memory before Goal: Role of Pre-History Memory for Humanoid Loco-Manipulation
Humanoid loco-manipulation in dynamic scenes is non-Markovian: cues vanish as objects change or are hidden, or leave the view as the robot turns and walks, yet whole-body vision-language-action (VLA) policies see only the present. We present WholeBodyMem, an object-guided perceptual memory that keeps the pre-history of object events seen before the robot acts on its goal. SAM3 tracks the task objects, and their events causally select key frames within an adaptive budget, which the action expert reads by cross-attention together with object tokens encoding appearance, geometry and time. We also contribute HuMeGo, a humanoid memory benchmark with six simulated and six real-world Unitree G1 tasks. Keeping the right frames rather than more frames, WholeBodyMem surpasses memoryless and uniform-memory baselines, raising its base policy's success from 29.1% to 72.6% in simulation and from 36.7% to 88.3% on the real robot, while also improving the standard SIMPLE tasks (70.2% to 86.9%).
comment: 8 pages, 4 figures, 5 tables. Submitted to IEEE Robotics and Automation Letters (RA-L). Project page: https://memory-before-goal.github.io
★ Calibration-free IMU-based Relative Orientation Estimation under Dynamic Human Motion
Recent magnetometer-free inertial measurement unit (IMU) methods can reduce drift in joint motion estimation using rigid-body constraints, but they typically require pre-calibrated sensor locations, assumptions of specific joint models (e.g., hinge joints) or limited motion types, or an external reference for global orientation. Furthermore, challenges related to high-frequency impacts (e.g. at foot strikes during gait) and numerical differentiation of noisy measurements have not been explicitly addressed. We propose an extended Kalman filter that estimates relative IMU orientation and sensor location simultaneously, requiring neither offline sensor location calibration nor a global orientation anchor. A systematic simulation study shows that the quality of the relative orientation and the sensor location estimates depends on different motion-intensity sources: the joint center acceleration and the relative angular velocity, respectively. A low-pass filter (LPF) is used to mitigate real-signal outliers before applying the rigid-body constraint. When applied to the OpenSense dataset with real IMU measurements across four subjects with three lower-body joint types during walking and complex tasks including various motion types, the proposed method achieves an overall relative orientation error of 2.61°, compared with 10.40° for a baseline using offline-estimated sensor locations and 7.43° for a baseline using marker-derived sensor locations. These results show that the proposed calibration-free estimation framework handles real IMU data effectively and achieves accurate relative orientation without offline sensor location calibration.
comment: 8 pages, 7 figures, 1 table. Submitted to IEEE Sensors Journal
★ Navigating human-robot interaction challenges in private and residential aged care homes
This chapter explores how robotic systems and ambient intelligence can enhance care for older adults in residential aged care (RAC) homes and private homes. Although these settings differ, we frame both as extreme and challenging environments for human-robot interaction (HRI), characterised by cluttered and dynamic layouts, diverse resident needs, and limited on-site technical support. We review embodied and non-embodied systems that assist with daily living, safety monitoring, and social engagement, highlighting hybrid deployments where robots complement fixed and wearable sensors. To ensure these technologies are usable and effective, design considerations consider the cognitive, physical, and social needs of older adults. Interaction design prioritises accessibility for older adults with low digital literacy, using robust but simple voice interfaces, small sets of non-verbal cues, accessible screen-based companions, and safe object handovers. Privacy, trust, and acceptability are addressed via practical strategies such as on-device inference, data minimisation, visible consent controls, and transparent data handling aligned with emerging artificial intelligence (AI) and privacy regulation. Evaluation emphasises real-world metrics including false alarms per day, time to recover from faults, staff workload, and long-term system reliability. We close by outlining open research challenges and future directions for scalable, regulation-ready deployments that support aging with dignity, safety, and independence.
comment: Author-accepted manuscript of a book chapter published online in Elsevier's Reference Module in Materials Science and Materials Engineering (2026). Scheduled for inclusion in Comprehensive Robotics for Extreme and Challenging Environments (June 2027)
★ RGraph-Eval: Task-Grounded Scene Graphs for Evaluating Robot Video Generation
Generated robot videos can look convincing while depicting the wrong interac- tion: a different object is moved, a grasp disappears, or a placement ends out- side its destination. We introduce RGraph-Eval, a reference-based framework that makes these discrepancies explicit through task-grounded temporal scene graphs. Unlike approaches that construct separate evidence representations for different evaluation dimensions, we organize task entities, interactions, and motion within a unified, interpretable scene graph representation. A foundation-model pipeline identifies manipulation roles, grounds persistent object masks, infers and verifies state transitions and action intervals, and reconstructs manipulated- object motion in three dimensions. Applying this representation to a generated video and a reference demonstration supports an interpretable RGraph-Eval score with visible-object, relation-change, and 3D path-shape components. We also introduce ShareRobot-SG, a collection of 1,000 manipulation scenes from ten ShareRobot sources annotated with the multimodal representation. Our eval- uation separates graph-generation quality from agreement with human video judgments, and includes comparisons with adapted SVG2 and SG-Ego, a bench- mark of three video generators. The framework connects robot-video assessment to the observable entities and interactions underlying a task, while making the graph annotations reusable beyond evaluation. Code will be publicly available at https://github.com/SashaDance/rgraph-eval.
★ To Route or to Adapt? Multi-Expert Diffusion Planners in Unseen Regions
Diffusion trajectory planners with domain-specific experts face a practical question in unseen regions: can they rely on selecting among existing experts, or must the planner itself adapt? We study this question using a planner with a shared adapter and per-domain expert adapters. Across three real-world held-out maps, none of eight test-time routing schemes improves the fleet-mean lateral offset over the static default, which uses the same expert for every scene. Even an oracle that selects the best expert for each scene reduces the mean offset by only 0.9 m. In contrast, reward-based fine-tuning on just 20 target-region scenes yields substantially larger gains. On matched evaluation slices, joint replay achieves a fleet-mean offset of 6.9 m, compared with 12.4 m for the static default. Comparisons under the same target-region training budget show that target-region training drives the improvement, while replaying training scenes limits forgetting on the original ODDs. On nuPlan, fine-tuning across training cities also improves open-loop offsets, although closed-loop performance deteriorates. Further diagnostics reveal that two test-time adaptation baselines achieve low offsets by producing degenerate trajectories. These results show the limited headroom for selecting among existing experts and the greater open-loop gains available from adapting the planner.
comment: Accepted to the 10th Conference on Robot Learning (CoRL 2026). 36 pages. Code and weights: https://github.com/tlab-wide/RouteOrAdapt
★ AstroBench: Benchmarking Free-Floating Capture and Space Robotic Manipulation
Capturing a loose object inside a spacecraft requires a robot to approach a target that can translate and rotate, and whose motion changes when contact is made. These conditions motivate evaluation beyond manipulation of supported objects. We introduce AstroBench, an open benchmark for free-floating capture and space robotic manipulation with a fixed-base arm. The task suite combines four initial-motion tracks with sample-storage operations based on MELFI hardware and cap and captive-screw servicing based on the Robotic Refueling Mission. The two workcells provide nine task definitions and 29 task-target combinations. A metadata-filtered Objaverse inventory supplies 8,414 object instances for asset expansion, alongside five cabin visual contexts and versioned task instructions. We report two acquisition-focused experiments using existing checkpoints. A frozen $π_{0.5}$ DROID joint-position policy captures 2 of 200 static targets and 4 of 200 moving targets across twenty object identities. A cube-specific ACT policy captures 8 of 10 static targets and 6 of 10 moving targets in a separate experiment. Each experiment retains its recorded observation, action and capture configuration. The results document difficult direct transfer for the tested generalist checkpoint and successful task-specific capture behavior, while identifying concrete settings for studying motion-aware perception and contact-sensitive control. Code, task definitions and editable workcell assets are available at https://github.com/AstroUpLab/AstroBench.
comment: 12 pages, 5 figures, 9 tables. Code and assets: https://github.com/AstroUpLab/AstroBench ; project website: https://astroup.com/astrobench/
★ Tilt-Rotor Quadcopter with Learned Residual Control and Mode-Dependent State Estimation
Tilt-rotor quadcopters redirect thrust for translation without banking, but complicate modeling, state estimation, and disturbance rejection. This paper presents a simulation-based flight stack for a \SI{2.00}{\kilo\gram} tilt-rotor quadcopter. The system uses a six-degree-of-freedom nonlinear model with first-order motor dynamics and two-axis rotor tilt, parameterized using a 3D CAD model. A cascaded PD controller commands forward and lateral tilt angles. A 14-state EKF fuses sensor data across hover, early-transition, and transition modes using a hysteretic (Schmitt-trigger) classifier to prevent covariance discontinuities. Because the proportional position loop leaves a steady-state offset under constant wind, a small bounded neural residual is added on top of the unmodified PD controller. It is trained by behavioral cloning from a teacher with privileged access to the true wind, while the deployed network never observes wind. In simulation, the EKF achieves a post-settling position root mean square error (RMSE) of \SI{0.226}{\metre} (horizontal) and \SI{0.03}{\metre} (vertical) and an attitude RMSE below \SI{0.8}{\degree}, and it estimates a wind velocity of $(\num{2.0}, \num{-1.2})\,\unit{\metre\per\second}$ to within \SI{0.15}{\metre\per\second} per axis. The residual reduces steady-state station-keeping error under wind by approximately \SI{49}{\percent}, from \SI{0.117}{\metre} to \SI{0.060}{\metre} at \SI{1.5}{\metre\per\second} and from \SI{0.319}{\metre} to \SI{0.162}{\metre} at an out-of-distribution \SI{3.9}{\metre\per\second}. The framework provides a basis for hardware-in-the-loop and flight-test studies.
comment: 5 figures
★ Multimodal Progress-Residual Fusion for Anticipating Navigation Retreat in Robotic Flexible Endoscopy
Flexible robotic endoscopes provide access through tortuous anatomy, where friction, deformation, and wall interaction separate commanded insertion from useful distal progress. Operators interpret this mismatch and retreat to recover a navigable configuration. Recognizing the approach to that decision requires relating motor intent to distal response and the local visual scene. We introduce Multimodal Progress-Residual Fusion (MPRF), a causal model that anticipates operator retreat from commands, electromagnetic (EM) tracking, and endoscopic video. A nominal-progress model predicts the response expected in normal or successful navigation; a temporal predictor combines its EM residual with all three observed modalities. In a 150-episode colon-phantom collection, MPRF achieves 0.580 +/- 0.007 AUPRC across three seeds, compared with 0.511 for the strongest reduced-modality control. Complete multimodal histories provide the principal ranking advantage, while the residual adds an explicit intent-response representation with a small, uncertain increment over raw fusion. Analysis of frozen alarms relates warning timing to the behavioral label and explains the increase in event recall under a longer matching window. These results establish a basis for behavioral assistance during operator-controlled navigation.
★ Demonstration-Based Robotic Feeding Using Dynamic Movement Primitives
This paper presents a demonstration-based robotic feeding framework that integrates human demonstration, Dynamic Movement Primitive-based motion planning, vision-based scene perception, and robot control. A human operator first demonstrates the scooping and feeding motions while the position and Euler-angle orientation of the spoon are recorded using a motion-capture system. The recorded trajectory is segmented into scooping and feeding motions, which are encoded using separate Dynamic Movement Primitives. Before task execution, an RGB-D camera estimates the participant's mouth position, while two AprilTags are used for robot-frame registration and tray localization. The detected tray pose is used to position the scooping motion, whereas the detected mouth position defines the positional goal of the feeding motion. The approach, scooping, and feeding trajectories are then executed sequentially by a Kinova Gen3 manipulator through a ROS~2-based framework. A real-world experiment demonstrates the feasibility of the integrated pipeline and shows that the learned spoon motions can be reproduced within the detected experimental scene.
comment: Accepted on RAAI 2026
★ Multi-Tool Selection for Robotic Systems: Comparing Semantic Retrieval and LLM
Natural-language interfaces allow users to command robotic systems without interacting directly with mission planners or robotics middleware. Their reliable use depends on robot tool routing, defined here as the selection of the smallest complete subset of predefined executable capabilities required to satisfy a request. This task is particularly difficult for compound instructions, where incomplete selection may omit necessary sensing or monitoring capabilities, while excessive selection may expose irrelevant tools. We compare four routing strategies: three progressively refined ChromaDB-based methods - fixed-threshold retrieval, relative-distance retrieval, and clause-aware aggregated retrieval - and a cloud-hosted 120-billion-parameter language-model baseline. The methods are evaluated on 164 manually annotated prompts organised into three complexity levels, from direct single-tool requests to compound requests requiring at least three tools. Using exact-match accuracy, completeness, precision, a completeness-weighted composite score, and routing latency, CDB-v3 achieves the best macro-average score among the retrieval-only methods, with 0.7822, compared with 0.8869 for the large-model baseline. Although the LLM provides higher routing quality, CDB-v3 exhibits substantially lower latency and can be deployed locally without requiring cloud connectivity. These results identify a quality-latency trade-off between direct large-model selection and clause-aware semantic retrieval for privacy-sensitive, connectivity-constrained, and resource-constrained robotic systems.
★ Revisiting Distribution Matching for State-to-Vision Policy Distillation in Laparoscopic Robotic Manipulation
End-to-end control policies trained via Reinforcement Learning (RL) have emerged as an alternative to conventional surgical robotic control approaches, with recent studies exploring state-based policies trained on privileged state observations. However, such approaches rely on precise physical states that are difficult to obtain reliably in surgical environments. Vision-based RL provides an alternative by learning manipulation policies directly from RGB observations, avoiding explicit state estimation and manually engineered perception pipelines while enabling policies to acquire task-relevant visual representations. Nevertheless, obtaining vision-based policies via pure RL methods for surgical manipulation remains challenging. Thus, imitation learning (IL) becomes another solution, yet it requires a large number of expert demonstrations, which are difficult to obtain in minimally invasive surgery. In this work, we propose a framework for state-to-vision policy distillation in laparoscopic robotic manipulation. Specifically, a state-based policy with privileged observations is employed as the teacher, and a vision-based policy learns from RGB observations and robot proprioceptive information, which requires less expert demonstrations than IL and introduces superior performance than RL methods. We conduct extensive experiments on various surgical manipulation tasks in the simulated environment. Experimental results show that, in the context of laparoscopic robotic manipulation, training vision-based policies with general-purpose RL methods is challenging, whereas our proposed state-to-vision policy distillation leads to more consistent and stable performance across tasks.
comment: 8 pages, 3 figures, 2 tables
★ TRACE: Traversability-aware Reactive Navigation via VLM-driven Adaptive Constraints Estimation
This paper presents TRACE, a traversability-aware reactive navigation framework designed for the autonomous operation of ground robots in unstructured forest environments. Unlike traditional approaches, TRACE leverages VLM-based confidence maps from positive and negative natural-language prompt sets and combines them through evidential fusion to produce a dense traversability map that prioritizes the terrain based on signs of previously traversed paths. Traversability-annotated points are then extracted and used to derive the left and right boundaries of the traversable path via percentile binning followed by spline or regularized polynomial fitting. These boundaries are then incorporated as soft barrier constraints within an NMPC formulation, enabling safe and adaptive path following in complex natural environments. Rigorous experimental trials carried out in real-life forest environments demonstrate TRACE ability for smooth navigation in cluttered, deformable, and visually ambiguous terrains.
comment: Presented at IFAC 2026
★ UniFlowWAM: A Unified World-Action Model with Visual Semantics and 4D Geometry
Robot demonstrations record what the robot saw and what it did. They do not specify how the 3D world changed in between. Scene-level 3D flow describes it by tracking 3D scene points, providing the world representation with 4D geometry. Prior work has used flow to improve robot policies. Flow served as an intermediate prediction that a separate model converts into actions, as a prior injected at inference time, or as an extra output learned together with the action. In all of these designs, flow stays outside the policy, as a separate module or as an extra prediction head. We introduce UniFlowWAM, a Unified World-Action Model that makes 3D scene flow a third modality on par with vision and action. Its world representation therefore covers both visual semantics, through future visual features, and 4D geometry, through 3D flow, and flow receives its own supervision. UniFlowWAM is a diffusion transformer with one expert per modality, and the three experts exchange information through joint attention. We train UniFlowWAM under a ten-objective role grid, where each objective assigns every modality exactly one of three roles, namely target, condition, or absent. The same model thus serves as a policy, as forward and inverse dynamics, and as a predictor of future visual features or 3D flow. We evaluate UniFlowWAM on RoboCasa-GR1, LIBERO, and LIBERO-Plus in simulation, and on four real-world bimanual manipulation tasks. On RoboCasa-GR1, UniFlowWAM raises the success rate of its base model without flow from 52.1% to 56.4%. It preserves in-distribution success on LIBERO at 98.1% and reaches 87.7% zero-shot on LIBERO-Plus. On the real robot, it achieves 87.0% average success, compared with 68.5% for the strongest baseline.
comment: 14 pages, 5 figures, 4 tables. Project page: https://kksham111.github.io/UniFlowWAM_ProjectPage/
★ In-Context Learning World Models for Robot Dual-Arm Manipulation of Soft Garment with Partial Observations
Robust manipulation of soft objects, such as garments, is a critical skill for service and household robots to adopt, but remains challenging due to its high-dimensional state space, and material-dependent deformations. Humans address this challenge by using mental world models that "imagine" the shape and material properties from partial observations, which enables them to predict how actions affect future states. As such, embedding such world models in robots allows for better generalization in manipulating a wide variety of garments in the real world. However, learned world models typically require large amounts of real world data that is strenuous to collect, and a mechanism to infer latent properties that govern dynamics. We present a Soft-Object World Model that performs generative full-state estimation and action-conditioned dynamics prediction using a transformer backbone trained on large-scale simulated soft object interactions, with measures to mitigate the sim-to-real gap. By learning to infer latent shape and material cues through in-context learning over a variable-length partial point-cloud history, the model generalizes across robot embodiments, material parameters, garment types and shapes, and manipulation coordination modes without weight updates. The world model is then integrated with a Goal-Conditioned action planner for motion synthesis. We validate the approach in simulation and real world for bimanual garment folding, reporting improvements in reconstruction fidelity, prediction accuracy, and task success.
★ LAST-V2X: Language-Driven Adaptive Scenario Synthesis and Testing for Communication-Induced Failures in V2X Safety Applications
Vehicle-to-everything (V2X) communication enables connected vehicles to make safety-critical decisions, but delayed, missing, or incorrect information can compromise driving safety. Systematically testing such failures is challenging because traffic interactions, communication conditions, and application responses must be jointly configured to produce meaningful scenarios. This paper presents LAST-V2X, a language-driven, adaptive framework for automated scenario synthesis and safety testing of V2X applications. LAST-V2X leverages large language models (LLMs) to translate natural-language test descriptions into executable scenarios grounded in roadway geometry, V2X communication semantics, and vehicle behavior. The framework integrates physical traffic simulation with spatially aware communication modeling and message-level fault injection to evaluate how information degradation affects vehicle decisions. To establish whether an observed safety failure is caused by communication rather than the underlying traffic conditions, LAST-V2X employs matched executions under degraded and ground-truth information. Furthermore, an execution-guided refinement mechanism combines parameter search and LLM self-reflection to identify ineffective test configurations, improve scenario effectiveness, and produce minimal reproducible failure cases. Experiments across diverse CARLA environments and V2X safety applications demonstrate the framework's ability to generate reliable scenarios, distinguish communication-induced failures from unrelated safety violations, and substantially improve failure discovery through adaptive refinement. These results establish LAST-V2X as a systematic approach to language-driven, causally verified, and adaptive safety testing for connected vehicle systems.
★ IORA: Irregular Object Immobilization under Restricted Actuation
Stable grasping under task-specific disturbances is important for robotic manipulation. However, most dexterous grasp synthesis methods are developed for highly actuated hands in free space, and low-DoF hands may not be able to realize the contact topologies required by these grasps. Meanwhile, support surfaces can provide useful contact forces, but are often treated only as geometric constraints. We present a support-aware grasp synthesis framework for object immobilization with low-DoF coupled hands. Given irregular object geometry, our method represents both hand and object local surface regions as graphs and searches for kinematically feasible matchings between them. We then refine the wrist pose and motor state for geometric contact feasibility and validates the refined grasps through physical rollouts. Starting from the validated grasp pose, an optional task-guided local search explores nearby grasp configurations by varying the wrist pose and motor state, and selects one according to the specified task condition. Experiments on diverse objects show that our method generates various grasp configurations that achieve high grasp success and strong disturbance resistance. Real-world experiments further demonstrate physically executable grasps which enable object immobilization in cooking scenario.
comment: Preprint
★ A Transparent 3-DoF Assistive Device for Upper-Limb Tremor and Ataxia Compensation: Design and Experimental Validation
Assistive devices for patients with neurodegenerative movement disorders must combine high mechanical transparency with the ability to generate corrective interaction forces. Achieving both properties simultaneously remains challenging. This paper presents the completed iAssistADL platform, a 3-DoF assistive device for future tremor and ataxia compensation during activities of daily living. The system employs a transparency-oriented mechanical architecture with all actuators located at the base, resulting in low moving mass, decoupled kinematics, and high mechanical transparency. To compensate for friction-induced transparency limitations along the vertical axis, a hybrid control strategy is proposed that combines open-loop force rendering in the horizontal plane with closed-loop force control in the vertical direction. The platform is experimentally evaluated through transparency measurements and static and dynamic force-rendering experiments at representative configurations. Results demonstrate resistive torques below 0.1 Nm in the horizontal plane, end-effector forces of up to approximately 10 N throughout the workspace, and accurate horizontal force rendering up to 4 Hz. While the vertical axis remains limited by friction-induced dynamics at higher frequencies, the proposed system successfully combines high transparency with controlled force generation, establishing a promising hardware platform for future tremor and ataxia compensation.
comment: 8 pages, 7 figures, 3 tables, Preprint
★ LocoManiLab: A Unified Testbed for Data, Policies, Controllers, and Deployment in Humanoid Loco-Manipulation
Humanoid robots must coordinate locomotion and manipulation in everyday environments, yet progress is difficult to assess across differing policies, controllers, and task settings. LocoManiLab provides an integrated testbed spanning data collection and generation, policy training, evaluation, and physical deployment. Its task suite covers 234 atomic and compositional tasks across five household scene categories, supported by 51.5 hours of simulated teleoperation. The evaluation covers twelve policies with a shared controller across task and scene splits and perturbation levels, two policies across five controllers, and studies of demonstration scale and data-source mixtures. Success rates fall on unseen tasks and scenes and generally decline as visual perturbations increase, with differences in generalization and robustness across policies. For data generation, a humanoid motion generation model is fine-tuned for loco-manipulation to study the alignment of its outputs with task execution requirements and its integration into the data generation pipeline. Three simulated loco-manipulation tasks closely match real-world setups, enabling evaluation of three policies in both domains. Similar policy rankings support the reliability of simulation-based evaluation.
★ ProTrav: Probabilistic Traversability with Elevation Uncertainty Propagation for Risk-aware Off-road Navigation
Reliable traversability estimation is essential for autonomous navigation on unstructured terrain. State-of-the-art approaches typically evaluate traversability from deterministic terrain elevation, thereby overlooking spatial correlations in the terrain and failing to account for how elevation uncertainty propagates through nonlinear traversability mappings. We propose ProTrav, a probabilistic terrain representation that uses function-space sampling to generate spatially correlated terrain realizations and propagate elevation uncertainty to traversability cost. First, a Sparse Variational Gaussian Process (SVGP) reconstructs a probabilistic elevation field from observations. Then, posterior function-space sampling generates spatially correlated terrain realizations, preserving spatial dependencies between neighboring locations while propagating elevation uncertainty through geometric features into traversability cost. Consequently, a distribution of traversability cost at arbitrary location is established, where the mean and standard deviation quantify the expected traversability cost and associated uncertainty, respectively. A risk term is defined by moment-based Upper Confidence Bound (UCB) and incorporated into a Model Predictive Path Integral (MPPI) controller for risk-aware navigation. Simulation experiments demonstrate navigation success rates exceeding 90\% on different terrains, compared with below 40\% for a baseline, while reducing attitude variations. Furthermore, the proposed approach is validated on a robot traversing real-world off-road terrain.
comment: 9 pages, 7 figures
★ TEAR: Task-Evidence Authority Routing for Robotic Manipulation
Reliable robotic manipulation in field-deployed settings requires more than executing a capable controller. As the execution conditions change, the system must decide, online and on the robot side, which control source should hold authority. Human teleoperation adapts readily to contact-rich and semantically ambiguous situations but depends on a stable communication link. Local autonomy is executed without continuous human intervention, but may fail under semantic ambiguity, long-horizon drift, and unforeseen contingencies. These two control sources exhibit complementary reliability profiles. We formulate this complementarity as a control authority routing problem and present Task-Evidence Authority Routing for Robotic Manipulation (TEAR), an edge-deployable framework that allocates authority between human teleoperation and local autonomy. TEAR arbitrates on the robot side using network-state evidence, which governs the usability of teleoperation, and task-semantic evidence, which reflects whether autonomous execution remains consistent with the current task. A lightweight temporal arbitrator observes both forms of evidence together with robot state and current authority to time authority transfers. Across simulated and real-world trials under dynamic network conditions, TEAR achieves task success rates close to pure teleoperation while limiting teleoperation to 19.8-27.4% of control steps and improves success by 30-40% over autonomy-only execution in our settings. These results demonstrate the potential of complementary authority routing to sustain successful manipulation with substantially reduced dependence on continuous human control.
comment: 9 pages
★ Robot Motion Transition with Human Motion Prior
Humanoid motion-tracking policies can reproduce diverse behaviors, yet connecting independently acquired motions into a continuous sequence remains challenging. Interpolation connects boundary poses without considering whether their relative position, heading, and timing provide suitable conditions for a transition. A generative motion prior can produce richer motion, but poorly arranged boundaries may cause it to generate excessive locomotion, turning, or another unintended action. We present Spatio-Temporal Alignment with a Motion Prior (STAMP), a framework for connecting independently acquired humanoid motions through a Robot-to-Human-to-Robot process. STAMP uses human motion space to draw on rich motion priors and reuse transition knowledge across robot embodiments. Its key component is a learned alignment model that determines the boundary conditions for motion inbetweening. Given the two motions retargeted to a human rig, the alignment model ranks candidate positions, headings, and transition durations for the following motion. A human motion prior generates the transition under the selected alignment. The completed sequence is retargeted to the robot and used as a reference to train a motion-tracking policy. We evaluate the tracking success of the generated transitions on Unitree~G1 and~ROBOTIS K1. Physical robot demonstrations further show that STAMP connects independently acquired motions into long-horizon sequences. Project page: https://childtoy.github.io/STAMP-project
comment: 9 pages, 4 figures
★ Adaptive Aerial Loco-Manipulation Policy Learning via LLM-Assisted Proactive Task Generation
Loco-manipulation extends the workspace of conventional manipulation by combining mobility with physical interaction, while aerial loco-manipulation further expands this capability into three-dimensional space, enabling robots to reach environments inaccessible to ground-based systems. Moving to an aerial platform, however, gives rise to a demanding control problem, since the vehicle must track trajectories accurately in three dimensions while the motion of the arm and its interaction with the environment continuously shift the center of mass, alter the inertia, and impose force and moment disturbances on the body. Conventional controllers address these effects through explicit models and estimation, whereas learning-based controllers infer dynamics from experience, yet control rewards provide no direct target for the response information retained from history, and fixed task sampling can leave persistent tracking difficulties insufficiently explored. We present ALMA, an aerial loco-manipulation framework for adaptive flight control under external arm commands. ALMA employs a contextual flight policy that adapts from recent measurements and actions, using action-conditioned prediction of future measurement representations to train its context to retain response information relevant to flight control. To expand the experience available for this adaptation, an evolutionary process uses a large language model (LLM) to analyze observed control difficulties and propose new structures of arm motion for subsequent training. In simulation, ALMA reduces position root-mean-square error (RMSE) by up to 24.7% relative to the strongest baseline, while the same policy transfers zero-shot to the aerial manipulator and reduces hover position RMSE by 79.6% during periodic arm motion relative to the real-world baseline.
★ EBV-CAN: Vision--Language Grounding for Contact-Aided Endoscope Navigation
Flexible endoscopes exploit structural compliance to traverse narrow lumens, pipes, and cavities, where navigation involves distributed environmental contact. This compliance makes tip motion depend on the evolving support configuration. We present external bird's-eye-view contact-aided navigation (EBV-CAN), a vision-language policy that combines one initial global static bird's-eye image with evolving endoscopic observations and motor state. Three supervised semantic queries locate the tip, next waypoint, and instructed target on the cached global image; their residual representations predict bending and insertion commands. We introduce the Vision-Language Contact-Aided (VLCA) dataset, linking 517 demonstrations and 75k action samples to 958k spatial entries and 93k remaining-route records. On 50 held-out episodes, tip-coordinate and waypoint localization reach 97.5% and 94.8%. Swapping the target instruction under identical observations changes target attention and motor output across 17 dual-target episodes. Across 1,920 physical trials of eight policies trained under a shared protocol, EBV-CAN achieves 183/240 successes (76.25%) under automatic blue-area scoring. Gated recurrent unit behavior cloning (GRU-BC) achieves the lowest matched-task offline mean absolute error (MAE), 0.056, and 1/240 physical successes, establishing distinct rankings for action imitation and task completion. Successful multi-guide executions and representative failures connect spatial progress, steering transitions, and visibility to physical navigation.
★ Causal Analysis of Failures of Cyber-Physical Systems with Continuous Inputs
Modern autonomous Cyber-Physical Systems (CPSs), such as self-driving cars, operate in increasingly complex environments while expected to maintain high reliability. The black-box nature of many of these systems makes it impossible to fully verify their behavior prior to deployment. When a failure occurs, it is important to analyze it to guide improvements to system robustness. A recent work introduced the novel concept of leveraging the framework of actual causality for CPS failure explanation. However, that work considered only a discrete, binary causal model that cannot capture continuous changes in system inputs. In this paper, we address these limitations by extending the actual causality framework with the notion of distance to account for the continuous characteristics of the system. Specifically, we introduce the novel concepts of distance-weighted actual cause and distance-weighted degree of responsibility, enabling causal influence to account for the magnitude of changes required to avoid a failure, rather than only its cardinality. While motivated by the continuous nature of CPSs, this constitutes a general extension of the theory of actual causality and enables more fine-grained analysis of continuous inputs in any causal model. Building on these notions, we present an efficient, practical, system-agnostic explanation-derivation algorithm that uses distance-weighted responsibility to guide the explanation search. We experimentally evaluate our approach across a number of environments from two autonomous-driving simulators with black-box neural controllers. The results show that the proposed extension provides more effective guidance than the current discrete approach.
★ When One Step Is Enough: An Empirical Study of Euler Integration in Flow-Matching Policies
Flow matching is now the dominant approach for vision-language action (VLA) robot policies, thanks to its expressiveness and simple training objective. However, inference for standard flow matching requires iteratively solving an ODE, limiting the control frequency at deployment. While several works have proposed novel methods to allow for single-shot action generation, we ask whether such additional objectives are actually necessary. We find that across six flow matching VLAs, one-step action generation largely matches multi-step sampling at scale: multi-step integration confers little to no consistent advantage at inference. We empirically find that this holds as long as models are sufficiently trained and have enough capacity. Practically, the implication is that for large VLAs, the single-step policy comes essentially for free from the multi-step model, enabling low-latency deployment without retraining, distillation, or auxiliary objectives.
comment: Accepted at the 10th Conference on Robot Learning (CoRL 2026)
★ Live Thread Reconstruction for Autonomous Manipulation
Autonomous robotic surgery has drawn increased attention in recent years due to its potential to relieve surgeons of time-consuming, mentally straining, and tedious tasks, particularly during prolonged operations. One suitable task for automation is suturing, which involves tracking the thread as it deforms during needle retrieval, de-tangling, knot tightening, and other tasks. Unlike rigid-object manipulation, deformable thread manipulation has no fixed pose to track and effectively infinite degrees of freedom, making its configuration difficult to predict and requiring the thread's state to be re-estimated rather than assumed. At the same time, thread manipulations involve sequential motions that depend on past and current thread contact points, which must be accurately and continuously tracked. This paper presents an image-to-manipulation pipeline that concurrently generates live thread reconstruction while conducting safe and optimized sequential manipulations to achieve long-horizon tasks.
★ Co^2Gen: Collaboratively Generating Task Plans for Decentralized Multi-Robot and Human Collaboration
Visual Language Action (VLA) models are designed to be generalist robot models that can be implemented across embodiments, tasks and environments. However, their ability to collaborate with other robots and humans remains underexplored. In this work, we propose a framework to automate the generation of multi-robot collaborative data for decentralized VLA training without requiring human teleoperation. Our framework first utilizes a decentralized collaborative planning approach to generate multi-robot collaborative robot plans in the form of an executable function. We then utilize the collaborative plan to produce collaborative demonstration data to finetune VLA models in a decentralized manner. We show that our approach produces demonstration data that when used to finetune VLA models on decentralized collaboration tasks matches the task performance of VLA models finetuned on human-teleoperated collaborative data.
★ Continuous-Time Motion Planning and Steering with Global Convergence
Motion planning is central to robotics, and it is increasingly used to correct the motions proposed by learned planners and policies. In both uses, constraints must hold at every instant of the motion, not only at sampled times. To our knowledge, no algorithm provably converges to a globally optimal motion once a smoothness cost is combined with such constraints. We cast planning and steering as a single continuous-time program: robots avoid one another and static or moving obstacles, and a learned proposal is steered by keeping the motion close to it. The program is elementary to state, yet nonconvex. We propose GlobMP (Global Motion Planning), a gradient projection method over piecewise-polynomial curves, on which each constraint is one polynomial per piece that must stay nonnegative over the whole motion; it alternates cost reduction with feasibility restoration. Under mild assumptions, we show that, from a fine grid start, its cost approaches the global optimum as the number of polynomial pieces grows, with every constraint met to any chosen tolerance, or exactly on stable instances. We demonstrate its superior performance over strong baselines in real and synthetic experiments, including teams of up to $200$ robots, multi-robot planning on the Robotarium testbed and an extended benchmark, maze navigation, and steering tactile-reactive policies on a bimanual humanoid.
★ CMAR-search: Commonsense and Memory Augmented Reasoning for Object Search in Dynamic Interactive Environments ICRA 2026
Dynamic interactive object search in large-scale human environments presents substantial challenges for existing methods. Current scene representations like 3D Scene Graphs (3DSG) only provide coarse-grained spatial segmentation and cannot identify functional areas such as storage or leisure areas. Without functional area understanding, existing methods are constrained to exhaustive sequential exploration at large scales, resulting in inefficient search behaviors -- particularly in open-layout environments with numerous interactive objects such as drawers and cabinets. Moreover, these methods lack adaptability to environmental dynamics such as object relocations. To address these limitations, this paper proposes CMAR-search, a novel framework built upon Commonsense and Memory Augmented Reasoning (CMAR). Our approach leverages commonsense about area functionalities and aggregates environmental memory to construct a Functional 3D Scene Graph (F3DSG), which organizes the environment into functional areas with their associated containers. Through this structured representation, CMAR enables hierarchical action planning at both macro-area and micro-container levels, empowering the system to efficiently identify and inspect semantically relevant areas for effective object search. Notably, CMAR continuously integrates real-time perception, accumulated memory, and commonsense to dynamically relocalize objects in changing environments. Extensive experiments in simulation and real-world settings demonstrate that CMAR-search significantly surpasses state-of-the-art baselines in both success rate and search efficiency for object search in dynamic interactive environments.
comment: Published in ICRA 2026. 8 pages, 5 figures
★ PATH: Preference-Aware Traversability Estimation for Robotic Teleoperation in Off-Road Navigation
Traversability estimation is critical for ground vehicle teleoperation, yet existing methods often neglect the human operator's preferences for speed and risk tolerance. To address this gap, we propose PATH-Net, a Preference-Aware Traversability estimation Hierarchical Network that adaptively generates preference-conditioned traversability cost maps and speed profiles for off-road teleoperation. Our framework integrates visual and geometric bird's-eye-view features with short-term historical vehicle states through a multi-head shared-backbone architecture. Our PATH-Net infers traversability cost maps while optimizing speed recommendations by using maximum entropy inverse reinforcement learning and sparse speed supervision. Our method incorporates operator preference via an asymmetric feature aggregation mechanism, splitting inference into a cost branch that prioritizes high-risk traversability regions and a speed branch that outputs preference-adjusted recommended speeds. Experiments with 15 operators across 225 teleoperated trajectories show alignment with user-specific navigation strategies, achieving modified Hausdorff distance of 0.17 m (aggressive) and 0.15 m (conservative). In the questionnaire assessment, participants reported lower NASA-TLX workload scores with PATH-Net recommendations than without recommendations, decreasing from 34.7 to 22.2. These results support preference-conditioned trajectory and speed recommendation for off-road teleoperation.
★ OASIS: Online Anchor-centric Scene Inference System for Open-Vocabulary Retrieval and Navigation in Dynamic Environments ACM MM
Reliable robotic mapping in dynamic real-world environments is essential for embodied intelligence and long-term autonomy. However, existing methods often involve a trade-off: lightweight online mapping is efficient but captures limited semantic context, while dense mapping provides richer semantics at higher computational cost and is more vulnerable to distractors. They also struggle to relocalize objects after scene changes. To address these challenges, we propose the Online Anchor-centric Scene Inference System (OASIS), a unified system for online mapping, complex open-vocabulary retrieval, and navigation in dynamic environments. Motivated by the observation that indoor scenes are organized around stable semantic anchors, OASIS represents the environment using a sparse set of anchor nodes, each maintaining a unified multimodal representation that integrates semantic descriptions, visual features, and geometric states for reasoning, verification, and navigation. Its compact anchor-centric memory supports on-demand evidence extraction and hierarchical retrieval, reducing redundant storage while improving retrieval reliability. To handle scene dynamics, OASIS further reuses transient observations collected during navigation to relocalize moved objects. Experiments in both simulation and real-world environments, including standard retrieval benchmarks and navigation tasks under both static and dynamic settings, demonstrate that our system can perform complex open-vocabulary retrieval and navigation, while also enabling relocation of displaced objects after scene changes.
comment: Accepted for oral presentation at ACM Multimedia (ACM MM) 2026. 17 pages, 10 figures, including supplementary material
★ Know When to Escalate: Critic-Guided Adaptive Execution for Cloud-Local Embodied Agents
Recent advances in foundation-model agents have enabled increasingly capable embodied systems for long-horizon planning and interaction. However, embodied agents require fast, closed-loop decisions as they continuously perceive and act on a changing environment. Deploying strong cloud agents leaves the platform idle for seconds between consecutive actions, while lightweight local models respond quickly but are more vulnerable to errors. Existing hierarchical architectures typically assign models fixed roles, whereas conventional model routers make stateless per-request decisions. We present CARE, a critic-guided adaptive execution framework that dynamically determines when to escalate control from a local executor to a stronger cloud model. A lightweight critic model monitors execution reliability and triggers escalation when local execution becomes unreliable; once escalated, control persists until the agent recovers, avoiding step-wise switching. To support such control transfer, a shared handoff memory maintains execution context and task-level progress across executors, enabling the model to intervene without restarting from scratch. Across diverse embodied benchmarks, CARE improves task success by 6.3-18.3 percentage points over the strongest evaluated collaboration baseline. CARE further approaches or exceeds Cloud-only success while reducing median time-between-action latency by 28.8-82.6%.
♻ ★ Reflection-Based Relative Localization for Cooperative UAV Teams Using Active Markers
Reflections of active markers in the environment are a common source of ambiguity in onboard visual relative localization. This work presents a novel approach that exploits these typically unwanted reflections for onboard relative localization in heterogeneous multi-UAV teams operating over specular, near-planar surfaces such as calm water and flat indoor floors. Using only a camera, active markers, and a single-beam rangefinder, the method requires no prior knowledge of robot size, marker configuration, or surface reflectance properties, and propagates the uncertainty introduced by surface irregularities, including those of dynamic water surfaces. We further characterize, analytically and experimentally, how surface tilt biases the estimate, and identify the slope range over which the level-surface assumption holds. Indoor multi-UAV and outdoor experiments validate the localization accuracy, demonstrating reliable operation across varying lighting conditions and greater effective range than state-of-the-art vision-based active-marker methods, with higher accuracy beyond 30 m. The project website with the video is available under the following link: https://timlakemann.github.io/reflection-based-relative-localization/.
♻ ★ CENIC: Convex Error-controlled Numerical Integration for Contact
State-of-the-art robotics simulators operate in discrete time. This requires users to choose a time step, which is both critical and challenging: large steps can produce non-physical artifacts, while small steps force the simulation to run slowly. Continuous-time error-controlled integration avoids such issues by automatically adjusting the time step to achieve a desired accuracy. But existing error-controlled integrators struggle with the stiff dynamics of contact, and cannot meet the speed and scalability requirements of modern robotics workflows. We introduce CENIC, a new variable-step integrator that brings together recent advances in convex time-stepping and error-controlled integration, inheriting benefits from both continuous integration and discrete time-stepping. CENIC runs at fast real-time rates comparable to discrete-time robotics simulators like MuJoCo, Drake and Isaac Sim, while also providing guarantees on accuracy and convergence.
comment: 20 pages with 20 figures. Submitted to IEEE Transactions on Robotics (T-RO). The supplemental video is available publicly at https://www.youtube.com/watch?v=9ZZ15MfCgtI
♻ ★ Replan for Error Recovery: Training Embodied VLMs through Verifiable Interaction with Action-Grouped Turn-GSPO
Embodied VLMs can execute increasingly complex manipulation tasks, yet execution errors can leave the environment in states that require replanning for recovery. Existing recovery methods either rely on inference-time mechanisms with fixed policies or learn from failure examples and reference corrections rather than verified action outcomes. To enable recovery learning through verifiable interaction, we introduce the Embodied Replanning for Recovery Environment (ERR-Env), which provides feedback on task progress and execution errors. We further construct ERR-Bench, a benchmark with held-out tasks spanning nominal, intermediate, and error states for evaluating recovery capability and execution reliability. We then propose Action-Grouped Turn-GSPO (AGT-GSPO), an online reinforcement learning method combining turn-level rewards for task progress and error states with action-grouped advantage estimation, and use it to train ERR-VLM, a compact 2B embodied VLM. Evaluation on ERR-Bench reveals a mismatch between task progress and execution reliability across diverse VLMs. AGT-GSPO improves ERR-VLM by 10.25 percentage points in Strict Success Rate for error-free completion and 22.95 percentage points in Recovery Rate, while ablations and zero-shot transfer further validate the effectiveness of our method.
♻ ★ HEX: Humanoid-Aligned Experts for Cross-Embodiment Whole-Body Manipulation
Humans achieve complex manipulation through coordinated whole-body control, whereas most Vision-Language-Action (VLA) models treat robot body parts largely independently, making high-DoF humanoid control challenging and often unstable. We present HEX, a state-centric framework for coordinated manipulation on full-sized bipedal humanoid robots. HEX introduces a humanoid-aligned universal state representation for scalable learning across heterogeneous embodiments, and incorporates a Mixture-of-Experts Unified Proprioceptive Predictor to model whole-body coordination and temporal motion dynamics from large-scale multi-embodiment trajectory data. To efficiently capture temporal visual context, HEX uses lightweight history tokens to summarize past observations, avoiding repeated encoding of historical images during inference. It further employs a residual-gated fusion mechanism with a flow-matching action head to adaptively integrate visual-language cues with proprioceptive dynamics for action generation. Experiments on real-world humanoid manipulation tasks show that HEX achieves state-of-the-art performance in task success rate and generalization, particularly in fast-reaction and long-horizon scenarios.
comment: Project page: https://hex-humanoid.github.io/
♻ ★ Super Agents and Confounders: Influence of surrounding agents on vehicle trajectory prediction
In highly interactive driving scenes, trajectory prediction is conditioned on information from surrounding traffic participants such as cars and pedestrians. Our main contribution is a comprehensive analysis of state-of-the-art trajectory predictors, which reveals a surprising and critical flaw: many surrounding agents degrade prediction accuracy rather than improve it. Using Shapley-based attribution, we rigorously demonstrate that models learn unstable and non-causal decision-making schemes that vary significantly across training runs. Building on these insights, we propose to integrate a Conditional Information Bottleneck (CIB), which does not require additional supervision and is trained to effectively compress agent features as well as ignore those that are not beneficial for the prediction task. Comprehensive experiments using multiple datasets and model architectures demonstrate that this simple yet effective approach not only improves overall trajectory prediction performance in many cases but also increases robustness to different perturbations. Our results highlight the importance of selectively integrating contextual information, which can often contain spurious or misleading signals, in trajectory prediction. Moreover, we provide interpretable metrics for identifying non-robust behavior and present a promising avenue towards a solution.
♻ ★ V2-STRep: VLM-Grounded Structured Task Representations for Reusable Robot Skills Acquired from Generated Videos
Human manipulation videos provide rich motion and interaction cues for acquiring robot skills without robot demonstrations. Video generation models synthesize such demonstrations from an initial scene image and task instruction, avoiding the need to record demonstrations for each task. However, the recovered motion captures only one scene-specific realization, leaving task structure, geometric relations, and constraints implicit. We present V2-STRep, a zero-shot framework that converts generated video motion into reusable robot skills through VLM-grounded structured task representations. The representation specifies motion phases, references, and task-relevant constraints, with targets described by minimal geometric structures: points, point-normals, axes, planes, and full 6D poses. VLM-provided 2D image-space cues are lifted into 3D using RGB-D observations to reconstruct task geometry and candidate grasp poses. Geometry-specific rules transfer motion to new scenes, while task-constrained trajectory optimization couples grasp selection with complete robot motion planning. It preserves task requirements while using remaining rotational freedom to accommodate joint limits. Updating deployment grounding and constraints enables reuse under new compatible instructions without generating another video. Experiments on six real-world manipulation tasks demonstrate improved execution success over baselines, reliable cross-scene transfer of successfully acquired skills, and adaptation to changed deployment instructions. Details are available on the project website: https://tuwien-asl.github.io/V2-STRep/.
♻ ★ Full 3D Radiation Pattern Approximation in Multi-Rotor UAVs
In-flight outdoor approximation of full 3D Radiation Patterns (RPs) in non-holonomic multi-rotor Uncrewed Aerial Vehicles (UAVs) enables efficient communication-aware robotic missions. Our method identifies the Spherical Harmonics series coefficients, decoupling the observed signal power gains into respective RPs. The proposed calibration trajectory provides training and testing samples of signal strength at an obstacle-free altitude. Simulation verifies the proposed novel approximation of the RPs, and real-world validation achieves an RMS prediction error under 5dB.
comment: 6 pages, 9 figures
♻ ★ AlignDrive: Aligned Lateral-Longitudinal Planning for End-to-End Autonomous Driving NeurIPS 2026
Practical autonomous driving requires models that generalize by reasoning through spatial-temporal possibilities to exclude unsafe outcomes. While state-of-the-art (SOTA) methods use parallel planning architectures, they fail to explicitly couple speed decisions with agent behavior along the driving path, leading to suboptimal coordination. To address this, we propose a cascaded framework that transforms longitudinal planning from an independent prediction task into a path-conditioned reasoning process. On the model side, we introduce an anchor-based regression design that conditions longitudinal prediction on the lateral drive path, and reformulate longitudinal planning as 1D displacement prediction along the path. This reduces geometric uncertainty and sharpens the model's focus on interaction-driven dynamics. On the data side, we introduce a planning-oriented data augmentation strategy that simulates rare safety-critical events by programmatically inserting agents and relabeling longitudinal targets to enforce collision avoidance. Evaluated on the challenging Bench2Drive benchmark, our method achieves SOTA performance with a driving score of 89.07 and a success rate of 73.18%, demonstrating significantly improved coordination and safety. Further evaluation on Fail2Drive confirms strong generalization to rare edge cases where parallel formulations typically fail. Project page:https://yanhaowu.github.io/AlignDrive/.
comment: NeurIPS 2026
♻ ★ Where Success Breaks: Failure-Boundary Learning for Robust Vision-Language-Action Models
Vision-language-action (VLA) models adapted through supervised fine-tuning (SFT) inherit a structural asymmetry: expert demonstrations teach the policy where success behavior lies, but provide no signal about where it ceases to be reliable. We argue that robust VLA adaptation should therefore be viewed not as further demonstration fitting, but as Failure-Boundary Learning -- the problem of Discovering, Localizing, and Shaping the boundary between recoverable deviations and task failure. To instantiate this view, we propose DLS: built on a real-grounded behavioral prior from few real demonstrations and simulated co-training, DLS discovers failure boundaries at scale through on-policy digital twin rollouts. Rather than reducing each rollout to a binary label, semantic progress localization uses privileged simulator states to assign progress-aware signals that capture where the failure boundary is crossed, not merely whether. These signals drive directional boundary shaping in the flow dynamics -- reinforcing success-producing denoising directions and suppressing failure-producing ones, without action likelihoods or auxiliary critics. Across real-robot manipulation tasks, DLS improves robustness over SFT and online RL baselines, especially under randomized initial states and unseen visual conditions.
comment: 23 pages, 16 figures
♻ ★ Hearing the shape of an arena with spectral swarm robotics
Swarm robotics relies on robots with minimal sensing and computation that achieve self-organization at the level of the collective to address a variety of tasks. Yet swarm robotics still struggles with global tasks that require understanding the broader spatial context in which robots operate, such as identifying the shape of the arena in which they are embedded. Here, we introduce spectral swarm robotics, a framework in which robots recover global spectral descriptors of the swarm's geometric structure through strictly local interactions. To do so, robots emulate diffusion to locally estimate Laplacian spectral invariants, quantities normally obtained only through centralized computation. These spectral values serve as compact descriptors of the environment. In the present proof of concept, robots estimated the second Laplacian eigenvalue and used it to classify arenas from a predefined set of shapes with distinct spectral signatures. We show that estimating a single eigenvalue is sufficient for one-shot classification among selected confined environments with distinct spectral signatures. We uncovered a scaling law linking swarm size (a global parameter) and interaction radius (a local parameter) that ensures robust estimation. We validated the approach experimentally on two distinct hardware platforms, Kilobots and Pogobots, achieving reliable classification of arena shapes under realistic noise and sparse-connectivity conditions. Last, we discuss how locally recovering Laplacian spectral invariants opens avenues for swarm robotics in settings where producing efficient collective behaviors strongly depends on the global structure of the swarm.
♻ ★ Learning to Drive in New Cities Without Human Demonstrations
While autonomous vehicles have achieved reliable performance within specific operating regions, their deployment to new cities remains costly and slow. A key bottleneck is the need to collect many human demonstration trajectories when adapting driving policies to new cities that differ from those seen in training in terms of road geometry, traffic rules, and interaction patterns. In this paper, we show that self-play multi-agent reinforcement learning can adapt a driving policy to a substantially different target city using only the map and meta-information, without requiring any human demonstrations from that city. We introduce NO data Map-based self-play for Autonomous Driving (NOMAD), which enables policy adaptation in a simulator constructed based on the target-city map. Using a simple reward function, NOMAD substantially improves both task success rate and trajectory realism in target cities, demonstrating an effective and scalable alternative to data-intensive city-transfer methods. Project page: https://zerlinwang.github.io/NOMAD/
comment: Autonomous Driving, Reinforcement Learning, Self-play, Simulation, Transfer Learning, Data-efficient Adaptation. Project page: https://zerlinwang.github.io/NOMAD/
♻ ★ ReactiveBFM: Reactive Closed-Loop Motion Planning Towards Universal Humanoid Whole-Body Control
While current Behavior Foundation Models (BFMs) provide robust control priors for humanoids, they only execute pre-defined reference motions. As a result, they are vulnerable to environmental shifts and incapable of reactive whole-body coordination. Naively cascading them with generative motion planners fails to achieve true reactivity, as inevitable tracking discrepancies induce fatal cumulative exposure bias. To bridge this gap, we propose ReactiveBFM, a real-time closed-loop planning-control framework. At its core, we effectively mitigate exposure bias via a scheduled prefix sampling curriculum, forcing the generative planner to actively learn error-recovery behaviors from imperfect physical states rather than ground-truth trajectories. Systematically, to reconcile the severe latency mismatch between auto-regressive planning and high-frequency tracking, we introduce an asynchronous replanning mechanism. Combined with trajectory chunking to temporally ensemble spatial references, our system guarantees spatio-temporally fluid execution without physical jitter. Deployed on the Unitree G1 humanoid, ReactiveBFM demonstrates unprecedented physical agility across a vast repertoire of text-conditioned closed-loop motions. Notably, ReactiveBFM achieves zero-shot moving target reaching, showcasing intricate whole-body coordination and on-the-fly replanning. In sim-to-sim benchmarking under severe perturbations, ReactiveBFM achieves a 93.1% success rate, significantly outperforming cascaded open-loop baselines by 28.6%.
comment: Accepted by CoRL 2026. Project page: https://xiao-chen.tech/reactivebfm/
♻ ★ ThermoAct:Thermal-Aware Vision-Language-Action Models for Robotic Perception and Decision-Making
In recent human-robot collaboration environments, there is a growing focus on integrating diverse sensor data beyond visual information to enable safer and more intelligent task execution. Although thermal data can be crucial for enhancing robot safety and operational efficiency, its integration has been relatively overlooked in prior research. This paper proposes a novel Vision-Language-Action (VLA) framework that incorporates thermal information for robot task execution. The proposed system leverages a Vision-Language Model (VLM) as a high-level planner to interpret complex natural language commands and decompose them into simpler sub-tasks. This approach facilitates efficient data collection and robust reasoning for complex operations. Unlike conventional methods that rely solely on visual data, our approach integrates thermal information, enabling the robot to perceive physical properties and proactively ensure environmental safety. Experimental results from real-world task scenarios validate the feasibility of our proposed framework, suggesting its potential to enhance task success rates and safety compared to existing vision-based systems.Project page: https://youngchae912.github.io/ThermoAct/
comment: 2026 RA-L
♻ ★ Scale and Selection: What Makes Automatic Harness Evolution Work for Visual-Interface Robot Agents
When an off-the-shelf coding agent is used directly as a robot policy, observing a browser-based 3D interface through screenshots and acting by posing a virtual target gripper through a few tools, the agent's harness, its prompts, tools, and control rules, largely determines success, and until now it has been written by hand. We show that this harness can be improved automatically by another coding agent, the optimizer agent, and report two findings about what makes it work. First, the number of rollouts the optimizer agent sees per round governs whether the evolved harness generalizes: with rounds fixed, held-out success rises as the training batch grows from 5 to 100 rollouts, while small batches overfit the training tasks. Second, the optimizer agent must not be given free rein. Accepting every revision it proposes degrades the harness within a few rounds; a simple Champion-Challenger rule that promotes a revision only if it beats the incumbent on the same fixed cases turns the same loop into one that raises held-out success on LIBERO tasks from 51% to 67% over 30 rounds. The evolved harness is also a natural place to attach a pretrained policy, through one tool that hands it control. Distilling the rollouts of that combination back into the policy with multi-teacher on-policy distillation yields a standalone policy at 62.2% on RoboCasa365 under the official protocol, 4.8 points above its starting point with no harness at inference; the evolved harness on top of the distilled policy reaches 71.2% on the RoboCasa365 leaderboard benchmark, 13.1 points above the best published entry, and the loop closes when harness evolution resumes on the stronger policy. Automatic harness evolution for visual-interface robot agents is thus feasible, but its gains hinge on the rollout scale behind each decision and on how the optimizer agent's revisions are selected.
comment: 15 pages, 5 figures
♻ ★ Prior Evolution and Task Alignment for Aerial Grasping
Aerial grasping is a remarkable capability exhibited by predatory birds, allowing them to capture prey through highly coordinated maneuvers in flight. Inspired by this capability, researchers have developed various formulations to reproduce such maneuvers through trajectory optimization. However, two limitations remain in practice. First, the resulting optimization problem is highly nonconvex and sensitive to initialization, making high-quality solutions difficult to obtain under a limited computational budget. Second, prescribed numerical objectives are human-designed abstractions that describe successful grasping through a limited set of mathematically tractable quantities and may not fully capture what determines task success. We investigate how learning can address these limitations within an analytical planner. Accordingly, a trajectory prior is first learned from optimized motions and then evolved through a CEM-based process that evaluates sampled initializations with the deployed optimizer and retains favorable ones as new supervision. An Execution-Aware Critic learns from contact, lift, and completion outcomes to assess whether the optimized trajectories are likely to succeed in physical execution. Its frozen energy can further serve as a differentiable grasping cost, allowing execution data to directly shape trajectory generation. Simulation and real-world experiments demonstrate improved optimization reliability, trajectory consistency, and grasping performance.
comment: 20 pages, 14 figures
♻ ★ Dual-Arm Box Manipulation via Online Inertial Estimation and Convex Wrench Optimization
Dual-arm robotic manipulation often requires lifting and transporting objects whose mass and center of mass are not known in advance. These uncertainties affect the force-moment balance required for stable lifting, and improper regulation of contact wrenches can lead to slip, object drop, orientation deviation, or excessive squeezing. This paper presents a friction-aware dual-arm box-manipulation framework for objects with unknown inertial properties. The framework combines online inertial estimation with convex contact-wrench optimization. The inertial estimation module estimates the object mass and center of mass from measured contact wrenches, while the optimization module computes friction-feasible contact forces and torsional moments through a second-order cone program (SOCP) under ellipsoidal friction-limit-surface constraints. An offline trajectory refinement stage is also included to reduce undesired object-environment contact when geometric constraints are present. By enforcing friction feasibility as a hard constraint and minimizing contact effort within the feasible region, the framework achieves stable lifting without treating slip avoidance and excessive squeezing as separately tuned objectives. Experiments on a real dual-arm robotic system under different center-of-mass configurations demonstrate that the method lifts objects with unknown inertial properties while maintaining stable frictional contact.
comment: Submitted to Robotics and Autonomous Systems 2026
♻ ★ Beyond Imitation: Reinforcement Learning-Based Sim-Real Co-Training for VLA Models
Simulation offers a scalable and low-cost way to enrich vision-language-action (VLA) training, reducing reliance on expensive real-robot demonstrations. However, most sim-real co-training methods rely on supervised fine-tuning (SFT), which treats simulation as a static source of demonstrations and does not exploit large-scale closed-loop interaction. Consequently, real-world gains and generalization are often limited. In this paper, we propose an RL-based sim-real Co-training (RL-Co) framework that leverages interactive simulation while preserving real-world capabilities. Our method follows a generic two-stage design: we first warm-start the policy with SFT on a mixture of real and simulated demonstrations, then fine-tune it with reinforcement learning in simulation while adding an auxiliary supervised loss on real-world data to anchor the policy and mitigate catastrophic forgetting. We evaluate our framework on four real-world tabletop manipulation tasks using two representative VLA architectures, OpenVLA and $π_{0.5}$, and observe consistent improvements over real-only fine-tuning and SFT-based co-training, including +24% real-world success on OpenVLA and +20% on $π_{0.5}$. Beyond higher success rates, RL co-training yields stronger generalization to unseen task variations and substantially improved real-world data efficiency, providing a practical and scalable pathway for leveraging simulation to enhance real-robot deployment.
♻ ★ Do VLAs Really Follow Instructions? Diagnosing How-Level Instruction Following in Vision-Language-Action Models
Vision-language-action (VLA) models are expected to follow instructions like their language backbones, yet prior work has focused on What-level instructions that specify the task rather than How-level instructions that specify how to carry it out. Through our diagnoses, we find that How-level instructions do not help current VLAs and lower task success. We hypothesize that pairing each training episode with a single instruction limits How-level instruction following, and show that training with segment-level enriched instructions makes actions respond to such instructions without consistently raising task success. To measure How-level instruction following beyond standard benchmarks, we introduce SIMPLER-IN, where success requires conditions given only in the instruction. It shows that the gained responsiveness does not extend to demanding or out-of-distribution How-level instructions. How-level instruction following thus remains an open challenge.
comment: Preprint
♻ ★ Robustness Verification of an Autonomous Underwater Vehicle-based Plankton Classifier
The assessment of planktonic standing stocks and microorganism structures is critical for understanding upper ocean biological processes. Currently, autonomous underwater vehicles (AUVs) equipped with in-situ optical imaging and artificial intelligence (AI) methods offer a promising solution for persistent surveillance, mapping and monitoring of planktonic life. However, current AI methods often lack robustness in dynamic, unstructured environments, where environmental noise and non-biological artifacts lead to frequent misclassifications. Standard convolutional neural network (CNN) classifiers often struggle with such conditions, leading to misclassifications that require time-consuming manual validation by marine biologists. To address this issue, we propose a novel robustness verification framework for in-situ plankton classifiers based on reachability analysis. We also introduce a continuous-time neural ordinary differential equation (neural ODE) classification model leveraging the high-resolution imaging capabilities of the SilCam particle imager. In this paper, we demonstrate the effectiveness of the proposed framework by formally verifying the robustness of the neural ODE model against environmental perturbations. We demonstrate that our verification framework acts as an automated filter providing formal guarantees of model stability against ambiguous data, thereby improving the reliability of autonomous sampling and reducing the post-processing workload.
comment: 6 pages, 2 figures, Accepted in IEEE OES AUV Symposium 2026
♻ ★ FluxVLA Engine: A One-Stop VLA Engineering Platform for Embodied Intelligence
Vision-language-action (VLA) models, world-action models (WAMs), and offline reinforcement learning methods are rapidly expanding the design space of embodied policies, yet turning these algorithms into reliable robot systems remains constrained by fragmented data formats, training stacks, evaluation protocols, inference runtimes, and embodiment-specific interfaces. We present $\mathrm{FluxVLA}$ Engine, an open, configuration-driven platform that turns heterogeneous embodied-policy components into a reproducible data-to-deployment workflow. Rather than introducing another policy model, $\mathrm{FluxVLA}$ standardizes interfaces for datasets, visual-language and world models, action heads, reward- or advantage-weighted learning, distributed training, simulation evaluation, optimized inference, and robot operators. The engine further integrates compositional dual-arm simulation, scalable automatic data generation, and model-decoupled human-in-the-loop rollout, takeover, correction collection, and reward annotation. For responsive physical execution, it combines Real-Time Chunking (RTC) with accelerated inference backends, lightweight remote GPU serving, and configurable trajectory post-processing. Together, these capabilities connect offline learning, simulation validation, online correction, and real-robot execution through shared and auditable contracts. $\mathrm{FluxVLA}$ therefore targets the engineering bottlenecks separating promising embodied-learning algorithms from reproducible evaluation and dependable deployment. Code is available at https://github.com/FluxVLA/FluxVLA
♻ ★ RangeNav: Task-Aligned LiDAR Range-Image Pretraining for Robot Navigation
Autonomous mobile robots operating in dense crowds and cluttered sites require $360^\circ$ perception to react to surrounding hazards. While panoramic LiDAR offers full-surround coverage, learning representations within reinforcement learning (RL) is sample-inefficient under sparse rewards, and existing 3D pretrained point-cloud encoders are too memory-heavy for massively parallel RL simulation. We present RangeNav, a compact (183k parameters) LiDAR range-image encoder pretrained entirely on synthetic data. RangeNav uses anisotropic convolutions and circular padding to handle the extreme aspect ratio and azimuthal seam of range images. We pretrain the network using a navigation-aligned objective combining range reconstruction, semantic segmentation, and bird's-eye-view (BEV) prediction, then freeze it for policy learning. In simulation, adding semantic and BEV supervision increases success rates over a reconstruction-only baseline by an average of 32.03 and 23.73 percentage points (pp) in procedurally generated training and out-of-distribution realistic test environments, respectively. Zero-shot real-world deployments on a Unitree B2-W quadruped demonstrate robust navigation through dense crowds, low-light settings, and construction zones where limited-field-of-view depth policies struggle. Code and models will be open-sourced. Project website: https://rangenav.github.io/.
comment: 9 pages, 6 figures, 5 tables. Added project website to abstract and updated author affiliation; research content unchanged
♻ ★ Proprioceptive Safe Active Navigation and Exploration for Planetary Environments IROS 2026
Deformable granular terrains introduce significant locomotion and immobilization risks in planetary exploration and are difficult to detect via remote sensing (e.g., vision). Legged robots can sense terrain properties through leg-terrain interactions during locomotion, offering a direct means to assess traversability in deformable environments. How to systematically exploit this interaction-derived information for navigation planning, however, remains underexplored. We address this gap by presenting PSANE, a Proprioceptive Safe Active Navigation and Exploration framework that leverages leg-terrain interaction measurements for safe navigation and exploration in unknown deformable environments. PSANE learns a traversability model via Gaussian Process regression to estimate and certify safe regions and identify exploration frontiers online, and integrates these estimates with a reactive controller for real-time navigation. Frontier selection is formulated as a multi-objective optimization that balances safe-set expansion probability and goal-directed cost, with subgoals selected via scalarization over the Pareto-optimal frontier set. PSANE safely explores unknown granular terrain and reaches specified goals using only proprioceptively estimated traversability, while achieving performance improvements over baseline methods.
comment: Accepted to IEEE/RSJ IROS 2026. 8 pages, 7 figures
♻ ★ TRACE: Typed Reference-Aware Conversational Editing for Humanoid Motion Diffusion
Language-conditioned humanoid motion generation enables users to create and refine robot movements through natural language. The central challenge is to connect an underspecified request to concrete motion structure that can subsequently be revised relative to an accepted reference. We present TRACE, a generation and editing framework for the Unitree G1 humanoid robot built around typed, measurable kinematic quantities. Our insight is to ground generation conditions in paired training motions and editing goals in the current motion. A language model learns to predict support, root, and hand events from motion-derived supervision; their time-aligned encodings complement the original text in a diffusion generator. Gravity-referenced states and trajectory--contact objectives improve kinematic consistency. For editing, typed requests define body--time scopes and source-relative goals. A separate reference-conditioned diffusion editor uses these goals and revised events to realize changes, while sampling-time projection preserves protected state coordinates. Quantitative evaluation and human assessment show improved semantic alignment and motion quality over the compared methods. Multi-turn editing evaluations demonstrate more reliable execution of successive relative requests under shared preservation constraints. Code is available at https://github.com/Multimodal-Perception-Action/TRACE.
♻ ★ Perceive, Predict, and Plan: a Unified World Action Model for Egocentric Dexterous Manipulation
We introduce P$^{3}$-WAM, a unified world action model for egocentric dexterous manipulation that learns 3D perception, action-conditioned prediction, and goal-directed planning through three complementary modes: Perceive, which reconstructs articulated hand motion and camera trajectories from video; Predict, which generates visual futures under prescribed actions; and Plan, which jointly generates action trajectories and their anticipated visual consequences conditioned on an initial image, a language instruction, and an optional visual goal. Our model integrates a compact, structured 3D action representation into a pretrained video diffusion transformer through a mixture-of-transformers architecture with shared base weights, modality-specific adaptation, and bidirectional visual-action attention. Randomized modality visibility lets video and 3D actions serve as either conditioning context or prediction targets across the three modes. Experiments show that unified training improves all three capabilities over single-mode training, including gains in planning quality under a vision-language model evaluation. Through unified training, our model outperforms specialized baselines on 3D hand reconstruction and action-conditioned video prediction across four datasets, while producing goal-directed manipulation videos with stronger instruction adherence and action progression than video-generation baselines. Project Page: https://p3-wam.pages.dev/
♻ ★ Event-based SLAM Benchmark for High-Speed Maneuvers
Event-based cameras are bio-inspired sensors with pixels that independently and asynchronously respond to brightness changes at microsecond resolution, offering the potential to handle visual tasks in high-speed maneuvering scenarios. Existing event-based approaches, although successful in mitigating motion blur caused by high-speed maneuvers, suffer from many limitations. Some of them highlight a success of pose tracking for a fronto-parallel fast shaking camera closed to the structure, while others assume pure (optionally aggressive) three-degree-of-freedom rotations. The former requires persistent local map visibility within the field of view (FOV), whereas the latter fails to generalize to six-degree-of-freedom (6-DoF) motions where both linear and angular velocities may be large. Consequently, current successes do not fully demonstrate that event-based state estimation under arbitrary aggressive maneuvers is a fully solved problem. To quantitatively assess the extent to which the potential of event cameras has been unlocked, we conduct a thorough analysis of state-of-the-art (SOTA) event-based visual odometry (VO)/visual-inertial odometry (VIO) methods and report shortcomings in current public datasets. Furthermore, we introduce a benchmarking framework for event-based state estimation, called EvSLAM, characterized by sufficient variation in data collection platforms, diverse extreme lighting scenarios, and a wide scope of challenging motion patterns under a clear and rigorous definition of high-speed maneuvers for mobile robots, along with a novel evaluation metric designed to fairly assess the operational limits of event-based solutions. This framework benchmarks state-of-the-art methods, yielding insights into optimal architectures and persistent challenges.
♻ ★ Learning Cooperative Bimanual Dexterous Grasp of Large Objects with Tactile Sensing IROS 2026
Bimanual dexterous grasping of large objects is a critical challenge in robotic manipulation. However, most existing studies focus on sequential manipulation rather than cooperative grasping, and methods addressing such bimanual tasks have largely been limited to simulation. These limitations stem from the difficulty of acquiring full 3D object models and generating physically plausible grasping actions. To fill this gap, we propose a real-world bimanual grasping framework that includes: a multimodal dataset capturing joint angles, visual observations and force signals; a Denoising Diffusion Probabilistic Model (DDPM)-based module that generates joint-level grasp configurations from segmented point clouds; and an execution strategy that integrates motion planning with online grasp refinement to ensure physical stability and feasibility. Our approach enables the synthesis of executable bimanual grasps from single-view inputs, reducing dependence on complete 3D object models and ensuring stable real-world performance. Experiments on a dual-arm robot demonstrate high success rates across unseen objects with varying geometries and poses, and ablation studies confirm the contributions of key components of our system.
comment: Accepted to the 2026 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2026)
♻ ★ Robot Generalization Beyond Volume: The Diversity Path via Zero-Shot Capabilities
General-purpose robots must handle tasks, objects and environments they were never trained on, yet zero-shot generalization remains one of the least solved problems in physical AI. In digital AI, zero-shot capability grew with the volume and, above all, the diversity of training data, much of it sourced from the crowd. Robotics instead relies on paid collection by a few well-funded teams, which yields volume without diversity. We formulate zero-shot generalization in physical AI as coverage of the deployment distribution, and show that more samples from the same collectors shrink the unseen region only slowly, while each new contributor can extend it. Building on this and on zero-shot learning, we propose a shared latent space for robot actions that combines multi-modal alignment, generative augmentation, and noisy-label learning. As a scalable source of diverse data, we propose a community of developers building applications on one standard, approachable robot, Origami. We close with a testable hypothesis and an evaluation protocol relating real-robot zero-shot success to the diversity of contributed data.
comment: 13 pages, 7 figures, 2 tables, including supplementary material
♻ ★ NavBind: Diagnosing Instruction-Action Binding at Route Forks in Navigation VLAs
Vision-language-action (VLA) navigation models translate visual observations and instructions into actions. Navigation success assesses whether an agent reaches the goal. However, reaching the goal successfully does not establish whether the model genuinely follows the instructions or relies on visual cues or navigation history. We introduce NavBind to diagnose instruction-action binding: whether changing the instruction influences the probabilistic distribution of the actions, with the navigation context fixed. We construct 3,252 route-fork contexts by recombining route-aligned instruction segments from R2R-CE and RxR-CE. Each context pairs full instructions that share the description of the route already traveled but specify different branches. Both are evaluated under the same agent state and navigation history. We compare generated local action sequences, the likelihoods of fixed reference action sequences under both instructions, and changes in visual attention and attribution. Across the evaluated models, higher reported navigation success does not imply that the model more faithfully follows the instructions.
♻ ★ OEL-AD: An Online Evolutive Learning Framework for Cross-Region Adaptive Autonomous Driving
Autonomous driving models have achieved great progress in recent years, yet most existing approaches remain static, following a train-then-deploy paradigm that lacks the ability to adjust internal parameters when encountering new environments. This limitation is particularly critical in cross-region scenarios, which are common in real-world deployments in autonomous driving where differences in road layouts, traffic patterns, and environmental conditions introduce distribution shifts. In this paper, we argue for a paradigm shift towards Online Evolutive Learning (OEL), a learning paradigm designed to enable adaptive, online model optimization through the interaction and coordination of intelligent components in dynamic environments. We introduce OEL-AD, a novel framework that instantiates this paradigm for autonomous driving. Specifically, OEL-AD leverages trajectory uncertainty from the planning module to determine whether adaptive updates are necessary. When triggered, the framework uses discrepancies between predicted agent behaviors and subsequent observations as a self-supervised learning signal to refine perception and prediction modules, thereby indirectly improving planning accuracy and safety. Experiments on the nuScenes dataset with a cross-region split between Singapore and Boston demonstrate that OEL-AD consistently improves perception and prediction performance. This enhancement in perception and prediction consequently results in improved planning capabilities, thereby confirming the promise of online evolutive learning as a fundamental approach for building adaptive and robust autonomous driving systems.
Multiagent Systems
★ Decentralized Bi-Level Optimization for Cooperative Orbital Transportation of a Rigid Payload
The rapid growth of the space economy, alongside increasing orbital congestion and debris, is driving demand for reliable on-orbit transportation of large and non-cooperative objects. Performing such transportation with a swarm of small spacecraft distributes risk and removes the single point of failure inherent to monolithic architectures, but requires solving a non-convex trajectory optimization coupling every agent's cone-constrained thrust to the payload's rotational dynamics. Existing approaches span decentralized control laws with asymptotic guarantees but limited constraint enforcement, and centralized and partially centralized planners that reintroduce a single point of failure. In this work, we present a decentralized bi-level framework driven by exact gradients: the problem is split into an inner convex thrust allocation and an outer search over payload torque trajectories. This decomposition yields fuel-efficient solutions quickly, without requiring full NLP convergence, and at short timescales its performance scales with the number of agents. Theoretical convergence guarantees are established, and the method is validated through extensive Monte Carlo simulation, including closed-loop recovery from in-flight faults. These properties make the framework well suited to spacecraft swarm applications.
comment: 10 pages, 6 figures, 4 tables
★ Contestable Agents for Data Governance: Recording Human Stewardship of AI-Generated Metadata Across Structured and Unstructured Systems
Industrial organizations hold operational knowledge in structured records, such as telemetry, point lists, and transactions, and in unstructured sources, such as manuals, service notes, communications, and inspection reports. Governance programs often manage these sources separately. Classifying, linking, and documenting them depends on a small group of employees with deep domain knowledge. Agents built on large language models can propose metadata, cite documents, and flag quality issues, but unreviewed proposals can spread errors into downstream systems. A perfunctory approval leaves no record of what a reviewer checked. We present contestable governance: specialist agents produce scored, cited arguments for disputed metadata decisions; a steward can accept, reject, modify, or add arguments before commit; and a governance record retains the evidence, interventions, and outcome. A versioned domain glossary supplies model-specific terms and acronym definitions to the agents. We describe the workflow, a triage path that reserves full argumentation for uncertain proposals while preserving review before commit, the scoring and record models, and a gate that controls which verified decisions become precedent. Two HVAC use cases illustrate the pattern: point classification using Brick Schema class labels, supported by an open reference implementation, and an illustrative warranty failure-mode decision linking claim fields and telemetry to service notes and revisioned bulletins.
comment: 15 pages, 2 figures, 2 tables
★ Dispersive Tree Agreements for Multi-Robot Coordination
Multi-robot coordination of nonlinear systems is naturally formulated as joint-space optimal control subject to constraints over groups of robots. Existing methods that approach joint optimality remain largely centralized, offline, and confined to collision-free motion planning, while decentralized, real-time counterparts target safe navigation without pursuing the joint optimum. This paper develops a decentralized, real-time planner that approaches the joint optimum as an anytime process. We extend a recent single-robot framework that searches dispersive trees of forward-propagated commands for differentially flat systems to coordinate a multi-robot team near-optimally. We first show that the inter-robot coupling can be effectively replaced by some local constraints, under which locally grown dispersive trees contain a near-optimal agreement. It is this structure that the robots pursue by decentralized column generation, negotiating rents on the shared space that guide the search of each robot's tree. The negotiation settles on an agreement whose near-optimality is certified at quiescence. Our protocol, WarpWeaver, implements this negotiation as an asynchronous real-time loop over a receding horizon, and demonstrates effective closure to quiesced agreements even when real-time deadlines cut the negotiation short. In simulation, WarpWeaver reduces total travel time by up to 22% against centralized offline planners on the Dynobench suite and by up to 37% against a decentralized real-time baseline, and nearly doubles the success rate of a task-specific planner on graph connectivity. Hardware experiments on six robots planning onboard demonstrate real-world implementability.
comment: 19 pages, 5 figures, 7 tables. Submitted to IEEE Transactions on Robotics
★ When Is a Memory Safe Enough to Act On? Action-Conditioned Verification for Long-Lived Agents
Long-lived agents act on memory carried across sessions, and many of their actions are irreversible, so a stale memory can turn directly into harm. Existing work improves the memory itself --- retrieving, refreshing, or reconciling entries --- yet whether an entry must be re-checked before acting is not a property of the entry: the same remembered address can be used unchecked for a routine shipment but should be re-confirmed before a valuable one. We therefore study \emph{action-conditioned memory verification}: given a candidate action, decide which of its memory premises to re-verify and when to proceed or abstain, balancing execution risk, abstention, and verification cost. We formulate this as a sequential stopping problem, in which a verification is worth its cost only if it can change what the agent does next; under suitable verifier reliability and cost conditions, increasing action consequence can move the same memory through proceed, verify, and direct-abstention regimes. Building on this formulation, \textsc{AMV-Guard} is a practical verification gate that extracts the premises a candidate action rests on, calibrates beliefs about whether they are wrong, and selects checks by bounded value of information (VOI). On ToolSandbox, relative to a static policy given the same inputs, it lowers mean verification cost by 33.9\% while keeping 100\% safe-proceed recall and 0\% Gate harmful, and raises Proceed coverage from 38.83\% to 44.17\%; in a gate-core transfer to AppWorld, which carries over the stopping/VOI core but not the full structured pipeline, it lowers verification cost by 21.3\% at a small cost in safe-proceed recall and task success. These results suggest that verification effort should be set by what an agent is about to do, not only by how reliable a memory appears.
comment: 22 Pages, 5 Figures, 12 Tables
★ Conservative Meta-Agent Search: Architecture Adaptation under Non-Stationarity
Automated agentic-system design is typically evaluated on static task suites with immediate scalar rewards. Deployed systems face persistent shift, delayed dependent feedback, correlated component failures, and costly architecture changes. We formalize meta-agent architecture search as sequential incumbent replacement with switching costs and hard risk constraints and introduce Conservative Meta-Agent Search (CMAS). CMAS retrieves archived architectures and proposes typed graph edits, then scores each against the incumbent by paired shadow replay under common random numbers, with interventional component credit. Promotion requires an empirically calibrated, variance-penalized simultaneous score above a pre-specified margin, and is followed by a simulated canary stage and pre-specified rollback. Since backtests cannot certify meta-level skill, we introduce MetaInvest-Bench, a regime-switching simulator whose specified worker competence makes a surrogate-optimal architecture computable, with leakage-controlled replay, adversarial governance tracks, and five negative controls. Financial streams serve as a risk-sensitive testbed for meta-level generalization; we claim no alpha. Across 96 streams, CMAS cuts normalized mean surrogate dynamic-oracle regret from 0.116 to 0.104 and raises held-out utility from 0.44 to 0.48 over the strongest adaptive baseline; when extra workers add no value, it contracts to near-single-agent teams. Its deployment inference cost is ~6% lower, though its search-inclusive token cost is the highest of the methods compared. The observed per-update false-promotion rate is 3.4% against a nominal 5% target, and 4.8% in a sealed rerun. The calibration is empirical: it concerns an update-level replay statistic on an author-designed benchmark, and guarantees neither full-utility improvement, nor absence of simulated canary-stage harm, nor sequence-level error control.
comment: 24 pages, 8 figures, 17 tables
★ Probabilistic Energy Modeling Unifies Versatile Swarm Control
Robotic swarms must coordinate across diverse missions, yet existing controllers often require redesign, retuning, or retraining when objectives change. Here we show that probabilistic energy modeling provides a unifying representation for versatile swarm control, in which diverse missions emerge from compositions of shared interaction principles rather than task-specific control formulations. By exploiting the correspondence between Gibbs random field energy minimization and structured reward maximization, we encode fundamental swarm behaviors as reusable energy terms within a unified probabilistic objective. Decentralized policies trained within this representation are composed into a single controller through lightweight policy gating, without retraining. An interpretable action-attention mechanism captures non-uniform local interactions, while dimensionless normalization supports transfer across swarm scales and platforms. Demonstrated on aerial swarms, the unified policy generalizes zero-shot across six mission classes, 10-100 agents, two hardware platforms, and indoor cluttered and GPS-denied woodland environments. Large-scale simulations achieve 0.96-1.00 task success rates with 1-10 ms per-agent inference latency, while physical deployments confirm zero-shot transfer to real platforms and unstructured environments. These results establish probabilistic energy modeling as a scalable and transferable framework for decentralized swarm control.
comment: 25
★ Muffin - A Shared Memory Buffer for Asynchronous AI Agents
The rise of Agentic AI systems has changed the landscape of artificial intelligence by enabling strong reasoning capabilities through specialized multi-agent interaction with structured self-reflection. However, in target-driven reasoning tasks, single- and multi-agent systems are often on par in terms of performance gains. A major challenge is that failures often arise from an ineffective exchange of critical information during inter-agent interaction and coordination. Different from existing multi-agent debate paradigms that rely on synchronous turn-taking and sharing information across different agents, we propose an inter-agent protocol based on shared Memory bUFFer INference - referred to as MUFFIN - in an asynchronous agentic framework. In the first step, an orchestrator routes queries based on their reasoning demands and delegates demanding ones to a multi-agent setting, where a shared memory buffer provides asynchronous exchange of intermediate findings while an agent's private reasoning remains local. Our experiments show that our framework Muffin outperforms established competitors, achieving the best accuracy in 17 out of 20 of our benchmark settings, and provides a better quality-cost trade-off than multi-agent debate: Muffin improves accuracy by 19.2 accuracy points while using 77.4% fewer tokens, without requiring full-trace sharing.
★ Automating Feature Flag Cleanup at Scale with Multi-Agent LLM Systems: An Industrial Experience at DoorDash
Feature flags are runtime configuration mechanisms used for gradual feature rollouts, experimentation, and kill switches. Once a feature is fully rolled out, the flag and its gated code become technical debt and can introduce operational risk through inadvertently enabling dormant code paths. At DoorDash, over 56,000 feature flags span 623 repositories, with 1,659 currently stale as of May 2026, with internal estimates of one to two hours per manual cleanup. Existing rule-based cleanup tools proved insufficient in DoorDash's codebases. Our contribution is an end-to-end automated LLM-based feature flag cleanup system that integrates querying live feature flag rollout data using MCP, Jira ticket processing, and end-to-end pull request creation. Our system successfully produced engineer-approved flag removal PRs for 45 (90%) of the 50 stale flags it was evaluated on, proving the system's operational viability at scale with each cleanup averaging only $4.79 and 13.8 minutes. We evaluated the system's quality and reliability boundary based on success rates, cost per cleanup and time metrics. Furthermore, we discuss our lessons learned when architecting this system, to provide practical guidance for developing similar multi-agent systems.
comment: 6 pages, 2 figures. Accepted at the 2026 IEEE International Conference on Software Maintenance and Evolution (ICSME), Industry Track
★ Consistent but Biased: Chair Personas Trade Noise for Bias in Multi-Agent Clinical Decisions
In clinical systems built from several large language model (LLM) agents, one agent often chairs the discussion and makes the final decision. Whether the chair's persona prompt changes that decision, or the agreement between repeated runs that some systems treat as reliability, has not been tested with the case held constant. We ask whether the persona changes the decision or only its timing, how it affects agreement between runs, and which behaviors appear when the prompt describes only personality traits. A simulated department of seven agents discussed 20 published cases of degenerative cervical myelopathy, with the case reports replayed unchanged in every run. Only the prompts of the chair and its deputy changed (no persona, or a consultative, autocratic or avoidant one), written either as rules of conduct or as personality traits. With personas written as rules, the chair chose to operate now in 93% of runs when autocratic and in 2% when avoidant, against 57% without a persona. Surgery was still planned in nearly all decisions, so the personas mainly changed whether to operate now or after more tests. These effects held with a second model and reworded prompts. The autocratic and avoidant personas made the three runs of a case agree more often, but only about as often as chance predicts, and Fleiss $κ$ did not rise. Prompts that described only personality traits kept more than half of the autocratic and avoidant effects on operating now. They reproduced quick decisions, delays and extra tests, but not ignoring red flags, dropping backup plans or passing the decision to the family. Studies should report and vary the chair's persona as they do the model, and correct agreement between runs for chance before calling it reliability.
comment: 32 pages, 8 figures
★ Agri-SCHOLAR: Simulation-Grounded In-Season Rolling Advisories for Agricultural Crops Using LLM-Bounded Optimization
Existing agricultural advisories to farmers suffer from several limitations: (a) remain largely expert-driven, (b) do not adequately account for in-season weather, and (c) do not scale to individual farms. In this paper, our objective is to close these gaps by developing Agri-SCHOLAR (Simulation-Grounded Crop Harvest Optimization with LLM-Guided In-Season Advisory Rollouts). Agri-SCHOLAR is a rolling, in-season advisory framework that issues, at intervals of every few days or every week, a personalized advisory using the following information: soil and crop properties, observed weather to date, short-range forecast of weather, and a season extrapolation of weather based on historical analogs. The novelty and depth of Agri-SCHOLAR lies in each advisory being produced by a closed-loop system which couples multiple LLM agents with a powerful crop simulator (such as APSIM in our case). The simulator crucially serves as an evaluation oracle. In Agri-SCHOLAR, the LLM agents propose agronomically sound and feasible bounds on actionable parameters and an optimizer searches within those bounds against a multi-objective reward spanning yield, resource utilization, and economic feasibility. We conduct a detailed 11-year analysis of the maize crop in two selected locations of South Asia and show that Agri-SCHOLAR consistently outperforms the region's standard packages of practice and satisfactorily addresses the limitations of existing agricultural advisories.
★ Consensus-Based Distributed Filtering for Angles-Only Tracking in Cislunar Space
This paper investigates consensus-based distributed filtering strategies for multi-observer angles-only navigation in the cislunar space. The objective is to estimate the inertial state of a target spacecraft using only line-of-sight measurements collected by a network of observer spacecraft. This problem is particularly relevant for autonomous cislunar navigation, where ground support may be limited, and centralized fusion architectures may be problematic due to communication and computational constraints, and single-point failure risks. Three distributed filtering approaches are considered: the Distributed Extended Kalman Filter, the Information-Weighted Consensus Filter, and the Kalman Consensus Filter. All methods share the same local prediction and measurement-processing steps, but differ in the way information is exchanged and fused across the network. Their performance is evaluated and compared to a centralized Extended Kalman Filter, adopted as benchmark, and independent local filters in a high-fidelity cislunar scenario. Numerical simulations show that the distributed approaches significantly improve estimation accuracy with respect to independent filters. Monte Carlo sensitivity analyses are then conducted to characterize the robustness of the considered methods to finite consensus iterations and prior misalignment among the agents.
comment: Accepted for presentation at IEEE NAVICON 2026
★ ADAPT for Science: A Multi-Agent System for Self-Evolving Scientific Optimization
Scientific discovery often requires identifying a small number of high-performing designs in vast, constrained search spaces. This is difficult when evaluations are costly, feasible solutions are sparse, parameters are strongly coupled, and physical constraints are strict. Recent AI for AI methods can automatically improve algorithms and optimization procedures. However, general purpose approaches are not designed around scientific priors, physical constraints, and limited evaluation budgets. We introduce ADAPT for Science, a multi-agent framework that treats scientific optimization as two coupled decisions: where to search and how to search. ADAPT uses historical designs to identify promising and diverse initial conditions, progressively concentrates computation on favorable search directions, and evolves the optimization procedure from execution feedback. We evaluate ADAPT on the design of gravitational wave detectors using a differentiable physics simulator and eight unseen detector topologies. Under matched scientific evaluation budgets, ADAPT achieves a mean loss of 0.204 +/- 0.101, reducing loss by 54.9% relative to the best classical optimizer and by 62.2% relative to the best general purpose agentic evolution baseline. ADAPT achieves the lowest loss on all unseen topologies among the evaluated baselines. ADAPT provides a step toward scientific optimization systems that continuously learn from prior experience and their own search outcomes.
comment: Includes supplementary material
★ ABRA: Agent-Bound UI for Human-Agent Communication beyond Chat with Reusable Views EMNLP 2026
Agents can create rich representations, yet these representations are rarely used as a medium for expressing user intent. We present Agent-Bound UI, an interaction model that separates reusable interaction structures from task-specific meanings. Agents bind task-specific meanings to interactions in reusable views and publish the resulting bindings as shared state, enabling humans and agents to understand and act on shared surfaces whose meanings vary across tasks. We implement ABRA, a framework-agnostic prototype of Agent-Bound UI, in which users and agents collaborate through chat and shared interactive canvases. We demonstrate ABRA through a representative use case and evaluate four LLM-backed agents on canvas authoring, owner-side consequence resolution, and cross-agent invocation. The code and demo video are available at https://github.com/megagonlabs/abra.
comment: EMNLP 2026 System Demonstrations
★ LLM Multi-Agent Systems Are Vulnerable to Herding and Runaway Cascades
Large language models (LLMs) increasingly operate in multi-agent systems where they collaborate by responding to others' choices, creating endogenous feedback loops with vulnerability to social influence, herding behavior and runaway cascades. Following the design of the classic Music Lab experiment, we tested this vulnerability in large populations of LLM agents that recommended unfamiliar songs, with or without exposure to cumulative recommendation counts. Exposure led to runaway cascades that generated extreme and unpredictable collective recommendations increasingly divergent from a song's intrinsic appeal. Compared to humans, LLM populations were generally far more susceptible to herding behavior, and unlike humans, LLM cascades were less able to self-correct. This vulnerability has implications for multi-agent collaboration in recommender systems, automated evaluation, complex problem-solving, and unauthorized conspiracies.
★ Governed Self-Evolution for Autonomous Investing: A Multi-Agent LLM System with an Invariant Core
LLM agents that act on live Web data must combine heterogeneous evidence, remember what they have done and improve from outcomes without weakening their own safeguards. We study this problem in autonomous investing, where every action is irreversible and based on untrusted Web content. We present Steward, a multi-agent LLM system that draws the boundary of self-evolution in its type system. First, nine role-specialised agents decide jointly: four analysts read news feeds, SEC filings, macroeconomic series and market data and emit typed reports, from which a Portfolio Manager sets target positions that a protected Risk Manager audits. Second, a seven-layer shared memory, assembled by code from the system's own record, gives each decision its position context, the market narrative, calibration and the agent's past decisions. Third, the system improves itself by review: code grades every trade and forecast nightly and writes the grades into memory, and each quarter a reflector proposes prompt learnings that a deterministic gate applies reversibly and replay of recorded inputs verifies. All three operate around an invariant core of order arithmetic, risk rules and discipline-agent prompts that no self-edit can reach. In a five-month live deployment (102 trading days, 1,632 LLM calls), the system contained a 24.6% sector drawdown to a 0.3% loss; in replay of its recorded inputs, a single model given the same raw data traded more and lost 1.35 points against the market over five days where the joint design showed no significant loss, and no injected instruction was followed; and a 2015-2026 backtest shows the core alone lowering the maximum drawdown under all 50 random entry policies. The reflector traced the dominant failure to its root cause, and the gate applied a learning that the agent adopted on trigger evenings and on no control evening. Code: https://github.com/yebof/quant-agent
★ The Helpful Agent Problem: When Emergent Cooperation Among LLM Agents Bypasses Intended Workflows
LLM agents are increasingly deployed in collaborative settings where they pursue separate tasks while sharing resources and operating under review or oversight processes. Such settings create safety risks not only when agents are malicious, but also when they are simply trying to help. Prior work has largely studied adversarial or explicitly collusive behavior; we show that benign agents pursuing separate objectives can spontaneously cooperate in ways that bypass intended workflows. We call this the \emph{helpful agent problem}. We study long-horizon shared-resource environments in which agents sometimes depend on resources owned by their peers and cross-owner changes are normally mediated by a reviewer. When review becomes unavailable, agents begin making changes on one another's behalf despite having no shared reward or explicit incentive to cooperate. Crucially, these actions remain within each agent's local permissions while circumventing the system's intended oversight structure. We introduce \textsc{FavorBench}, a controlled benchmark with measurable downstream side effects, and adapt \textsc{CooperBench} to test the phenomenon in realistic repositories. Across two benchmarks and four models, we find that out-of-workflow favors emerge frequently. Further, in \textsc{FavorBench}, favors from at least three models cause significant harm to parties outside the interaction. These results reveal a distinct multi-agent safety challenge: cooperation itself can create pathways around system-level safeguards.
★ Agentic AI Defense: Decode-Time Provenance Binding for MCP
Agentic AI systems using the Model Context Protocol (MCP) are vulnerable to indirect prompt injection through untrusted tool outputs. We demonstrate cross-modal trust laundering, in which instructions concealed in an email-signature image using least-significant-bit (LSB), discrete cosine transform (DCT), or metadata steganography bypass text scanners and enter a multi-agent workflow when decoded by a tool. To counter this threat, we propose decode-time provenance binding, which combines HMAC-authenticated trust labels with an MCP-maintained provenance ledger and context-taint tracking. The mechanism labels decoded content as untrusted and blocks critical tool calls influenced by that content, even when a compromised agent substitutes a valid trusted label. We evaluate the approach using a real MCP testbed with three steganographic carriers and 24 adaptive payloads. The undefended pipeline exhibits 100% attack success, while keyword and regex defenses each permit 62.5% of attacks. Our mechanism blocks all attacks (0% success), including adversarial label substitution across mediated agent boundaries. A four-component ablation validates the roles of decode marking, cryptographic authentication, lineage tracking, and sink enforcement. An audited trusted-promotion mechanism increases legitimate-task completion from 67% to 100% without compromising protection. The approach provides carrier-independent containment within fully mediated MCP execution environments. Code and data are publicly available.
comment: 12 pages, 5 figures, 13 tables, 2 algorithms. Code and data available at https://github.com/ericyoc/stego_provenance_defense
★ Knowing Whom to Trust is not Enough: The Trust--Agreement Gap in Repeated Multi-Party LLM Negotiation
Large language model (LLM) agents increasingly participate in negotiations on behalf of individuals and organizations, yet they remain vulnerable to manipulation, particularly in repeated multi-party interactions. Classical multi-agent systems address this challenge through explicit trust and reputation models, whereas existing work on LLM negotiation largely treats trust as an emergent phenomenon. We investigate whether LLM agents can form useful trust estimates from prior interactions and leverage them to improve future negotiation outcomes. We introduce a repeated, multi-party, multi-issue negotiation framework in which recurring agents participate in a sequence of negotiation games while one participant covertly attempts to prevent agreement. After each game, agents assess their counterparts and maintain pairwise trust estimates that carry over to subsequent negotiations. We compare trust-aware and trust-agnostic agents and analyze how explicit trust influences negotiation outcomes. Our results reveal a \emph{trust--agreement gap}: trust-aware agents can identify the adversary within a few games, but accurate trust estimates do not consistently translate into better agreements. Trust increases agreement rates by up to 23 percentage points, but only when agents are explicitly guided to use it in their decisions. Explicit trust can therefore improve repeated LLM negotiations, but its value depends critically on how trust is assessed and incorporated into agents' reasoning.
★ Distributed Swarm Morphogenesis with Neural Cellular Automata
Swarm morphogenesis requires robots to assemble desired shapes through local interactions. We present a modular framework integrating neural cellular automata (NCA) with existing site selection and collision-aware control. Within this framework, we introduce a decentralized attachment-site discovery mechanism. Robots already in the formation update and share local NCA states to identify new attachment sites, enabling progressive growth without a predefined list of sites during deployment. To bridge the mismatch between conventional NCA growth and gradual physical assembly, we additionally propose offline sparse-retention fine-tuning that simulates partial attachment, together with a false-positive loss that penalizes invalid site activations. Simulations demonstrate formation of diverse lattice morphologies, scaling to several hundred robots, and substantial improvements in morphology accuracy from the proposed fine-tuning. Experiments with Crazyflie quadrotors validate the integrated execution pipeline using robot-relative observations reconstructed from external tracking.
★ Emergent Cooperative Thermoregulation in Lunar Swarm Robotics
Swarms of small robots are an attractive way to survey lunar caves and lava tubes. Missions such as NASA's CADRE will fly cooperating rovers for distributed mapping, but only in daylight. The 354 h lunar night drops the surface towards 95 K, and the radioisotope heaters that let single rovers survive it cannot be supplied at swarm scale. In this study we investigate whether a swarm can instead use its own members as thermal shielding. We train one policy, shared by all twenty robots, by reinforcement learning with an evolution-strategies estimator. The policy receives a single reward, once, at the end of the night: the mean temperature of the five coldest robots. Nothing in that reward refers to position, spacing, exposure or motion. Two behaviors nonetheless emerge. The swarm aggregates, and robots circulate between the sheltered core and the exposed rim, the same pair of behaviors with which emperor penguins survive the Antarctic winter. We measure the survival threshold, the smallest stored energy per robot at which every robot is still alive at sunrise. Across ten runs it is 155.6 +/- 1.3 pcm-h, where one pcm-h is 7.2 kJ of stored heat. That is 37% below a motionless huddle and within 13% of a physics-based lower bound, and at a budget where the motionless huddle loses six robots, every robot survives. The behavior needs no shared spatial reference and no robot-robot contact, and transfers from twenty robots to forty without retraining.
comment: 7 pages, 4 figures. Submitted to IEEE Robotics and Automation Letters. Video in ancillary files
★ VibeGame: Prompt-to-Game Development with AI-Native Engine and Self-Evolving Adversarial Agent Team
Large language model agents have enabled software development through natural-language interaction, but extending this paradigm to end-to-end game development remains challenging. Building a playable game requires coordinating code, scenes, assets, animations, physics, and configurations, while allowing developers to validate their integrated behavior through interactive testing. We introduce VibeGame, a framework for creating complete 2D game projects from natural-language prompts and optional reference images. Its AI-native game engine represents project state as schema-validated structured text and provides frame-synchronous runtime control, enabling agents to inspect game states, execute actions, and verify behavior at their own pace. Built upon this interface, we design an Adversarial Agent Team of eight specialized agents to separate implementation from review, so that problems are found and fixed through multi-round loops. For each conducted and accepted project, we further introduce a training-free self-evolution mechanism to distill it into reusable skeletons, modules, and collaboration contracts for future development. We evaluate prompt-assets-to-game (PA2G), where agents are given an asset library, and prompt-to-game (P2G), where agents generate their own art. On a GameCraft-Bench subset, VibeGame in P2G consistently outperforms a range of agent harnesses in either P2G or PA2G with the same underlying model, including GPT-6.1-Sol.
★ Toward an Agentic AI Approach to Public Health Data Collection
Disease surveillance increasingly integrates automated data collection, yet predefined sources and procedures require manual adaptation as reporting practices change. We explore agentic artificial intelligence (AI) as an approach to coordinating surveillance data collection across heterogeneous sources. Specifically, we built a multi-agent framework that integrates source discovery, source assessment, evidence structuring, event linking, and validation. Three retrospective applications illustrate its promise. The workflow recovered all 13 individual cases in a hantavirus outbreak, with links to 85 distinct publisher domains compared with 28 in the reference benchmark, reconstructed Canadian measles counts with a final total 0.33% below the national reference and higher cumulative counts on 79.8% of comparable days, and increased the share of days with recent Sierra Leone mpox reports from 28.2% to 65.6% by incorporating attributed media. These demonstrations also reveal limitations in record completeness and accuracy, while prospective gains in timeliness remain untested. We envision agentic AI as a complementary layer of public health data infrastructure, coordinating evidence-linked collection under surveillance goals and standards defined by public health professionals.
comment: 46 pages, 2 figures; Supplementary Information included
♻ ★ CollabFlow: Recursive Self-Improvement of Agent Collaboration
Recursive self-improvement (RSI) lets a system improve from its own outcomes; in LLM-based multi-agent systems, Agents refine one another within a task, and outcomes improve how they collaborate across tasks. However, existing multi-agent collaboration leaves this loop open: collaboration is pre-defined at the operator level, topology-only learning keeps verbatim exchange that propagates errors, and reward maximization on a system's own outcomes concentrates on a few teams. To address these challenges, we propose CollabFlow, an RSI system of Learned Agent Collaboration: a trainable Collab-Director constructs teams of complete Agents, a frozen executor runs them, and each round's outcomes retrain the director. Within each round, the edges of a collaboration graph carry protocols of Evidence-Conditioned Communication: a receiver adopts a differing answer only when the sender's evidence is stronger by a margin, so the director learns who communicates and how. Across rounds, we further propose Collaborative Trajectory Balance (CTB), a flow-based objective that credits each team once across its construction orders and targets a reward-proportional distribution over teams, so several good teams stay in play. We also bound how far this self-generated target moves between rounds, which shrinks as records accumulate. On twelve datasets, CollabFlow outperforms all baselines and keeps improving across rounds. Code is available at https://anonymous.4open.science/r/CollabFlow-631E.
♻ ★ Learning to Drive in New Cities Without Human Demonstrations
While autonomous vehicles have achieved reliable performance within specific operating regions, their deployment to new cities remains costly and slow. A key bottleneck is the need to collect many human demonstration trajectories when adapting driving policies to new cities that differ from those seen in training in terms of road geometry, traffic rules, and interaction patterns. In this paper, we show that self-play multi-agent reinforcement learning can adapt a driving policy to a substantially different target city using only the map and meta-information, without requiring any human demonstrations from that city. We introduce NO data Map-based self-play for Autonomous Driving (NOMAD), which enables policy adaptation in a simulator constructed based on the target-city map. Using a simple reward function, NOMAD substantially improves both task success rate and trajectory realism in target cities, demonstrating an effective and scalable alternative to data-intensive city-transfer methods. Project page: https://zerlinwang.github.io/NOMAD/
comment: Autonomous Driving, Reinforcement Learning, Self-play, Simulation, Transfer Learning, Data-efficient Adaptation. Project page: https://zerlinwang.github.io/NOMAD/
♻ ★ S$^2$-MAS: A Society-Scale Multi-Agent Simulation Framework with 500 Million Household Persona Agents
Simulating economic responses at society scale requires agents whose decisions reflect household circumstances and the effects of earlier actions. We introduce S$^2$-MAS, a society-scale multi-agent simulation framework built on 500 million synthetic household personas grounded in real-survey data. Its Genese engine separates decisions from state updates, while reusable financial scenarios and application ledgers track consumption, cash, debt and payments. Trained with SFT + RLCD on 2.93 million examples, sharp-421m-mlx achieves 74.55% accuracy and 38.37% question-macro F1 on one-shot consumer financial-choice tasks derived from held-out survey responses. Its accuracy exceeds GLM-5.3-Flash, Jev 1.13, DeepSeek-V4.1-Flash, GPT-6-Luna and GPT-6-Astra in the common benchmark (full test for the first four; matched 500-group sample for Astra). We evaluate eight financial task families and simulate 12 million distinct household agents in paired purchase-funding conditions. Jointly raising purchase goals and approved credit limits increases consumption, with responses differing across age and income groups. Education, employment and household-size differences persist after age-income standardization and mainly reflect transitions from deferral to purchase. Asset and debt groups differ in cash drawdown, repayment and closing debt. A response-stratified 24-month panel varies purchase goals under a common credit product, with repayment and arrears updating subsequent offers. S$^2$-MAS links heterogeneous household decisions to evolving consumption, cash and repayment paths, supporting economic experiments across household groups, population scales and time horizons.
comment: 58 pages, 13 main figures and 7 supplementary figures. Project resources: https://github.com/KataDavidXD/s2-mas
♻ ★ OACap: Caption-Guided Active Observation for Detailed Video Captioning
Unlike question answering, where a query specifies a perceptual target, open-ended detailed captioning must determine what evidence to acquire and how to integrate it into a coherent global description. We propose Observation-Adaptive Video Captioner (OACap), a caption-centered agent that turns unsupported or missing information into targeted observation requests, uses spatiotemporal tools to acquire evidence, and iteratively verifies and revises the caption. OACap captions consistently improve performance across general video-understanding benchmarks using a fixed text-only reader. To better diagnose the information they preserve, we introduce Factual-Semantic Captioning Bench (FSCap-Bench). Its Core-Support Factuality Score (CSFS) combines claim precision with importance-aware fact coverage and responds as expected to four controlled perturbation types on 93.0-98.3% of valid caption pairs. Its Cross-Modal Semantic Round-Trip Score (CSRS) measures semantic preservation through a video-caption-video-caption round trip by matching atomic facts from source and recovered captions. Together, they connect downstream textual-memory utility with factual and semantic preservation.
comment: 14 pages, 4 figures, 4 tables. Technical report
♻ ★ Stop Drawing Scientific Claims from LLM Social Simulations Without Robustness Audits NeurIPS 2026
The scientific claims drawn from LLM social simulations should be no stronger than the robustness audits that support them. Generative agents bring new expressive power to agent-based modeling, enabling simulations of collective social processes like cooperation, polarization, and norm formation. Yet they also introduce complexity through additional architectural choices, such as agent specification, memory representation, interaction protocols, and environment design. Small perturbations that appear minor to researchers can cascade into macro-level outcomes through repeated interaction, creating a "butterfly effect." Consequently, scientific claims drawn from LLM social simulations may reflect implementation artifacts rather than the social mechanisms being modeled. We support this position with two case studies: a repeated Prisoner's Dilemma and a social media echo chamber simulation. Across multiple models, minor perturbations in persona format and game-instruction framing shift cooperation rates by up to 76 percentage points, while network homophily and hub assignment produce significant and consistent shifts in polarization metrics. We also find that sensitivity is unevenly distributed across both architectural choices and model families: the same perturbation that produces the 76 pp shift in one frontier model only shifts another by 1 pp. Robustness is therefore a property that should be measured per claim and per model, not assumed. To address this validation gap, we introduce TRAILS (Taxonomy for Robustness Audits In LLM Simulations), a robustness-audit taxonomy spanning three levels of simulation design: agent (micro-level), interaction (meso-level), and system (macro-level). We call for robustness to become a first-order validation requirement before LLM social simulations are used to explain mechanisms, evaluate interventions, or inform decisions.
comment: NeurIPS 2026 Workshop on Large Language Models for Social Reasoning and Simulation (SocialAgent@NeurIPS 2026, Oral Presentation)
♻ ★ ROAR: Benchmarking LMM Agents under Real-Time Constraints in Multiplayer Action-Shooter Games NeurIPS 2026
Large multimodal models (LMMs) are increasingly deployed as agents that perceive, reason, and act in interactive multi-agent environments. Yet most existing benchmarks for LMM-based agents in multi-agent settings remain turn-based: the environment and other agents pause while an agent performs inference. Such protocols obscure a fundamental challenge of real-time constraints, where the world continues to evolve and delayed decisions may become outdated before execution. Motivated by this limitation, we introduce ROAR, a multiplayer action-shooter game environment for evaluating LMM-based agents in multi-agent settings under real-time constraints. ROAR covers cooperative, competitive, mixed cooperative-competitive, and basic single-agent scenarios. We further propose a Multi-Agent Asynchronous Gym interface that decouples environment evolution from agent reasoning by running the environment and each agent in separate threads, enabling the environment to advance at fixed time intervals without waiting for LMM inference and allowing agents to reason independently without blocking one another. ROAR also features convenient deployment, multimodal observations, and customizable levels. ROAR offers a practical testbed for advancing LMM-based agents beyond static, turn-based evaluation toward real-time multi-agent decision-making.
comment: NeurIPS 2026
♻ ★ Under the Influence: Evaluating and Improving LLM Vigilance in Agentic Tasks
Large Language Models (LLMs) are becoming increasingly autonomous assistants, capable of accomplishing series of complex tasks by interacting with open world domains such as software databases and the internet. This presents great opportunity but also great risk: open domains routinely present opportunities for autonomous agents to be influenced by others, e.g. deciding whether to accept pull requests or whether to use advice on a message board in order to accomplish a task. Critically, agents must be capable of deciphering when to incorporate benevolent advice into their reasoning, and when to reject malicious advice. This involves the social capacity of vigilance: the ability to determine which information to use, and which to discard, for effective communication. While well-studied in humans, the extent to which LLM agents exhibit vigilance is relatively unknown. Here, we examine LLM agent vigilance in three realistic tasks: navigating dangerous websites, avoiding e-commerce scams, and patching an unsafe code repository. We find that all frontier models are highly susceptible to malicious influence across all environments. We also find that model performance itself is surprisingly dissociable from vigilance. In light of these results, we propose a vigilance harness which uses Bayesian reasoning over a model's estimates of the value of an advisor's advice to decide whether to act on that advice. The vigilance harness helps recover baseline performance while retaining the benefit of positive influence on agents, paving the way for safer agentic AI systems.
Systems and Control (EESS)
★ OpenTac: An Open Ecosystem for Customizable, Scalable, and Transferable 3D Tactile Sensing
We present OpenTac, a novel sensor and a full-stack ecosystem for tactile skins that are highly customizable, scalable, and readily deployed in downstream robot control and learning applications. Our core contribution is twofold. First, we develop a scalable resistive tactile skin with a structured deformable overlay that converts tangential loading into spatially varying pressure. Differential responses across neighboring "taxels" support decoding of local shear vectors, and the common-mode response encodes normal force. Second, we provide a highly customizable sensor design tool that is able to automatically generate fabrication files for these tactile skins from 2D outlines and developable 3D surfaces, including from abstract descriptions such as text (e.g., "design a sensor for a size 9 shoe insole"). We develop an ecosystem around the OpenTac sensor, including the CAD design tool, a sensor simulator, cross-sensor transfer, and fast/adaptive data acquisition. We demonstrate the broad applicability of the OpenTac-CAD sensor design tool by designing and fabricating more than 30 sensors across a variety of robot embodiments. We characterize the three-dimensional tactile fields produced by OpenTac with a commercial, high-precision force/torque sensor. We demonstrate the broad applicability of 3D tactile sensing across contact-rich feedback control and manipulation policy learning on four real-world robot embodiments. Additionally, adaptive acquisition enables kilohertz-rate partial readouts, exceeding full-array raster scanning on the same sensor by 19.2X at 16 sampled taxels per update for a 1024-taxel skin. We provide all hardware, software, and result videos are at opentac.github.io
comment: 14 pages, 14 figures. Project website, open-source hardware and software, and demonstration videos: https://opentac.github.io
★ Workload-Aware Monitoring for AI-RAN
AI-RAN platforms are expected to share CPU and GPU resources between radio workloads and AI services. Aggregate telemetry reports operating conditions, but timing changes may reflect different workloads, processors, or measurement boundaries. We propose workload-aware monitoring that associates radio-work descriptors with processing paths, timing boundaries, and operating context before comparing like with like. In 16 over-the-air downlink experiments, hybrid dispatch (CPU for transport blocks (TBs) below 4,096 bytes and GPU otherwise) has 29.66% and 22.93% lower mean host-callback durations than direct GPU dispatch, without and with a synthetic GPU co-runner, respectively. Grouping locates the separation in small TBs, while large-TB callback-duration ranges overlap. These observations identify a candidate for complementary CPU/GPU use.
★ Extending Harmonic Function Methods via Root Locus for Coordinated Path Generation
We consider the offline generation of coordinated paths for N interchangeable robots and N goals in planar environments with polygonal obstacles. After a conformal map to a circular domain, starts and goals become the poles and zeros of a rational function R, and the paths are the branches of its Generalized Root Locus (GRL), i.e. the streamlines of the logarithmic potential -log R. We show that, for every phase outside a finite set of critical phases and a set of boundary phases, the induced gradient motion drives each robot to a distinct goal along pairwise disjoint paths that never reach the obstacles; the robot-goal assignment is produced by the construction, and with the gain used as a clock all robots move as the closed-loop roots of the GRL and arrive together at a prescribed time. Obstacles become streamlines through successive Milne-Thomson reflections, and for a finite number of reflections a finite test certifies safety for the potential that is actually computed. Reciprocal root-locus planning is recovered as a special case. The method is illustrated on two polygonal workspaces, 100 random instances compared with independent harmonic planning and artificial potentials, and a scene with ten robots and ten obstacles.
comment: 16 pages, 6 figures. Extended version of a paper submitted to Automatica. Corresponding code: https://github.com/mfrascarelli/grl-certified-planning
★ Controlled Radio Fault Injection and Telemetry Evaluation for Open 5G RANs
Radio access network (RAN) telemetry reveals radio conditions, protocol behavior, and their consequences for applications. Evaluating its usefulness requires controlled impairments and observations of the same execution. We present C and Python channel brokers and a fault injection workflow for software 5G testbeds connecting an OCUDU base station and a software UE through ZeroMQ. The brokers apply scheduled, directional transformations to I/Q samples and restore the link, while the workflow jointly records applied faults, native and programmable (codelet) telemetry, and application delay. We demonstrate this capability through 45 comparative trials of brief uplink interruptions and stepped additive noise. These use cases distinguish recognizing a radio change from measuring its service consequence: native missing-feedback events retrospectively identify interruptions, whereas noise changes decoding-error measurements without producing those events. Median on-time round-trip times remain near 30 ms, but the prespecified comparisons show mean increases in on-time 95th-percentile delay of 354 and 604 ms for one and five interruptions, and 113 ms for noise. Adding receive-side radio link control (RLC) codelet summaries did not improve interruption classification on held-out trials. The framework provides a reusable way to evaluate measurement choices under known radio faults, before using them in monitoring or learning systems.
★ Decentralized Bi-Level Optimization for Cooperative Orbital Transportation of a Rigid Payload
The rapid growth of the space economy, alongside increasing orbital congestion and debris, is driving demand for reliable on-orbit transportation of large and non-cooperative objects. Performing such transportation with a swarm of small spacecraft distributes risk and removes the single point of failure inherent to monolithic architectures, but requires solving a non-convex trajectory optimization coupling every agent's cone-constrained thrust to the payload's rotational dynamics. Existing approaches span decentralized control laws with asymptotic guarantees but limited constraint enforcement, and centralized and partially centralized planners that reintroduce a single point of failure. In this work, we present a decentralized bi-level framework driven by exact gradients: the problem is split into an inner convex thrust allocation and an outer search over payload torque trajectories. This decomposition yields fuel-efficient solutions quickly, without requiring full NLP convergence, and at short timescales its performance scales with the number of agents. Theoretical convergence guarantees are established, and the method is validated through extensive Monte Carlo simulation, including closed-loop recovery from in-flight faults. These properties make the framework well suited to spacecraft swarm applications.
comment: 10 pages, 6 figures, 4 tables
★ Space Solar Applications: Simulation with MATLAB-Simulink Environment and Transmission of Solar Power Generated
In this paper a photovoltaic (PV) array suitable for use in outer space is simulated in the MATLAB/Simulink environment. The simulation of the PV array is first carried out with maximum power point tracking (MPPT) and then without MPPT. In a second step, the response of the system to different irradiation levels is observed. A brief overview of the transmission of solar power generated in space down to the earth via microwaves is also given. Selected simulation results are provided for a comparative study of both cases.
comment: 6 pages, 16 figures, 4 tables. Work done at Illinois Institute of Technology, 2010
★ Direct Inference on Admissible Sets (Diona)
We propose a new regression method named Direct Inference on Admissible Sets (Diona) that directly uses samples to estimate the state and its derivatives at a query point, along with their admissible bounds, without long global training. By casting the values and derivatives at the query and neighbouring samples as latent variables in a constrained optimisation problem, the burden of long training loops becomes one of on-demand local solves. We establish Diona's theoretical properties including finite-sample coverage and, in the noiseless case, convergence to the true state and derivatives under dense sampling. In a nonlinear regression benchmark on six functions of three to eight dimensions, Diona was more accurate than a Gaussian process (GP) on noiseless data especially with denser data sets. Furthermore, as with GP, it produced uncertainty bounds that contained the true value at more than 95% of query points in four functions. Diona's time to a first prediction after a data set update was three to four orders of magnitude shorter than that of the GP, and two to three orders shorter than that of a neural network, which was more accurate but offers no uncertainty quantification. This low latency is paid for by on-demand predictions, so this latency advantage applies when relatively few predictions are needed between data updates.
comment: 27 pages, 4 figures
★ History-Aware Control Barrier Functionals with Nonsingular Fading-Memory Kernels
Safety-critical protection often depends not only on the state and physical limits of the system, but on how that state evolved over time. Here, the safety margin measures the remaining allowance before a physical limit is reached. In this context, the objective is to enforce both physical safety and history-dependent retention of this margin for integer-order control-affine systems. To this end, changes in this margin are converted into memory through fading-memory kernels with finite nonnegative weights. A history-aware control barrier functional (HA-CBFal) is constructed explicitly, with nonnegativity equivalent to retaining a prescribed fraction of this margin's weighted historical value. This condition is enforced by an affine input constraint and proved to imply physical safety under stated kernel, initialization, and feasibility conditions, without finite-dimensional memory realization. In 36 calibration cases on the ADMIRE flight-control benchmark, mean per-case tracking-RMSE reductions of 27.2\% and 7.0\% are obtained relative to calibrated state-only CBF and moving-average control barrier functional (MA-CBFal), respectively, with both constraints preserved to numerical tolerance. In a shorter-recovery maneuver outside calibration with frozen gains, both constraints are preserved to tolerance by HA-CBFal and state-only CBF, but the retention requirement is violated despite MA-CBFal nonnegativity.
★ Barrier Function Control for the Interconnection of DC Electronic Loads to Utility Grid
This paper proposes a nonlinear control law that facilitates the safe integration of DC electronic loads to the utility grid. Specifically, by employing suitable reciprocal barrier functions, it is rigorously proven that the closed-loop system guarantees constrained operation within a desired operating region, despite the influence of fast converter dynamics and fluctuating constant power load demand. This way, the proposed controller guarantees safety of the electronic components from surges in power demand and high transients. The theoretic results are validated in simulation scenarios, where the performance of the proposed controller is compared against the conventional saturated-PI controller. Contrary to the latter, the proposed method guarantees stable and safe operation, enhancing the system robustness.
comment: preprint for paper to appear in IEEE ISGT Europe conference 2026
★ Certified Execution Harness for Agentic AI: Safe and Least-Intrusive Petri-Net Supervisory Control Between Reasoning and Action
This paper studies the problem of safe tool execution for large language model (LLM) agents under prompt injection. Using Petri nets, we formalize the execution harness between the LLM and its tools as a supervisory control problem, in which marks represent capabilities, approvals and budgets. Since a transition fires only if the marks it needs are present, an injected request whose preconditions are unmet or already spent is refused regardless of its wording. Treating tool outcomes as uncontrollable, we enforce safety by the conventional maximally permissive supervisor, and this safety transfers to the running system under four assumptions. Moreover, to capture an agent that may propose again after a refusal, we introduce the notion of a replanning closed loop, in which each retry costs an inference call. We show that, in this loop, the conventional supervisor can be arbitrarily more intrusive than a stricter safe one. Furthermore, for a given proposer model, the maximally permissive least-intrusive supervisor is characterized as a value-threshold refinement of the conventional one. Under precise conditions, this refinement orders two concurrent actions by at most one monitor place. On an autonomous release agent, we find by synthesis that safe deployment requires a step that the supervisor can refuse after a rollout starts.
comment: 17 pages, 9 figures, 2 tables. Extended version of a paper submitted for publication
★ Transfer Meta-Learning for State Space Autoencoders
Nonlinear system identification often requires costly experimental campaigns and separate model fitting for each different instance (target system) of a given system family. To reduce this burden, meta-learning approaches reuse information acquired across related systems. In particular, in-context meta-learning enables zero-shot prediction from a brief target system context (i.e., a short input-output trajectory), but generally yields large black-box sequence predictors without exposing an explicit state space model. Conversely, compact structured state space models are suitable for simulation, estimation, and control, but are usually fitted or adapted separately for each target system. To bridge this gap, we introduce a transfer meta-learning extension of state space autoencoders. The method reuses a pretrained encoder, whose weights remain fixed, to extract a compact representation of the target system from its context. An affine parameterization maps this representation to the parameters of a compact neural state space model without target-specific adaptation; the resulting model contains orders of magnitude fewer parameters than the reference in-context predictors from the literature. We evaluate zero-shot prediction and simulation performance on held-out system instances from Buck-Boost converter and Bouc-Wen oscillator system families, comparing performance against individually fitted baseline models.
comment: 8 pages, 4 figures. Submitted to IFAC for possible publication
★ Model Predictive Control of Wind Turbines: A Model-Oriented Review of Prediction Models, Control Objectives, and Nonlinear Aerodynamic Formulations
Model predictive control (MPC) has been applied to wind-turbine regulation with markedly different internal prediction models, ranging from linearized low-order turbine models to nonlinear, aeroelastic, floating, and data-driven predictors. This review organizes the literature primarily by the model propagated by the controller and treats model order, predictor representation, MPC formulation, control objective, and validation plant as separate descriptors. Primary studies show that low-order linear or piecewise-affine predictors are attractive for constrained power and speed regulation, whereas flexible drivetrain, structural, and floating models are needed when drivetrain twist, tower motion, blade loads, or platform dynamics are explicit control quantities. Nonlinear and economic MPC studies further demonstrate coordinated pitch/torque control across broad operating regions, while preview, repetitive, and data-driven formulations address wind prediction, periodic loads, and model mismatch. Particular attention is given to aerodynamic pitch dependence: variable-pitch turbine dynamics can be non-affine with respect to applied pitch, and the term NMPC alone does not prove that the original nonlinear pitch map is retained inside the optimizer. The review therefore adopts a conservative, equation-based classification for direct nonlinear applied-pitch propagation. Here, direct nonlinear applied-pitch propagation means that the applied pitch at each prediction step is evaluated inside the original nonlinear aerodynamic map, rather than replacing that dependence by a linearized, scheduled, piecewise-affine, or learned surrogate. The literature supports a remaining integration challenge at the intersection of flexible drivetrain prediction, nonlinear aerodynamic pitch treatment, explicit constraints and load objectives, and validated adaptive or learned model correction.
comment: Khalili-Amirabadi, R., Jalaeian-Farimani, M., & Ghane, M. (2026). Model Predictive Control of Wind Turbines: A Model-Oriented Review of Prediction Models, Control Objectives, and Nonlinear Aerodynamic Formulations. Journal of Intelligent Monitoring and Control Systems, 1(1): 35-41. https://doi.org/10.65098/jimcs.01.2026.35.41
★ Evolving Federated Density Peak Clustering with Shuffler-Based Anonymization
In this paper, we present a novel one-shot federated clustering approach based on evolving fuzzy clustering and shuffler-based anonymization. In federated clustering, each round reveals additional model information to the server and can increase the opportunity to infer properties of local datasets, even when raw samples are never shared. Consequently, fewer client-server exchanges and unlinkability are important, so transmitted information cannot be attributed to any single client. Data-stream clustering algorithms, which learn in a single pass over the data, align well with the one-shot federated learning paradigm, where a single communication round can be sufficient. The proposed Evolving Federated Density Peak Clustering (eFDPC) method combines evolving fuzzy data-stream clustering and density peak clustering in a federated setting. Each client independently evolves micro-clusters from private data mini-batches through cluster addition and merging. To limit communication, density peak clustering is applied to the learned micro-clusters and only the selected peaks are transmitted, rather than the full local models. A shuffler-based layer is introduced between the clients and the server to anonymize contributions prior to reconstruction and global peak selection. We consider a peak-only variant, eFDPC-P, as well as eFDPC-G with server-side label propagation. Experiments on synthetic and real-world benchmark datasets show that eFDPC outperforms recent federated clustering baselines in one-shot settings and scales to higher-dimensional, non-IID data.
comment: 30 pages, 4 figures. Published open access (CC BY 4.0) in Evolving Systems 17, 123 (2026). Code: https://github.com/mihaozbot/eFDPC
★ Feeder-Aware Service-Conditioned Flexibility of Grid-Interactive Buildings
Grid-interactive buildings can adjust demand, but building-feasible power budgets may differ from accepted feeder-head offers. This study addresses a practical question: to what extent can the adjustable electricity consumption of buildings be translated into deliverable flexibility services while satisfying feeder-head operating requirements? To enable a fair comparison, we adopt the same baseline operating schedule, equipment conditions, and post-service recovery requirements for indoor temperature and battery state of charge (SOC). Flexibility is then evaluated under two settings: one considering only building-level constraints, and the other additionally accounting for feeder-head constraints. We first use a simplified network model with explicit security margins to determine a preliminary amount of flexibility that can be committed. The resulting schedule is then validated using a more detailed network simulation via Modelica. If network losses cause the actual response to deviate from the committed response, the electricity consumption of individual buildings is rescheduled. If the required response still cannot be achieved, the committed flexibility level is reduced. The experiments cover multiple service activation times, regulation directions, and service durations. For comparative purposes, we retain the initial optimization result as the reference estimate of the maximum potentially available flexibility, rather than interpreting the failure of a particular subsequent realization attempt as evidence that a larger flexibility level is inherently infeasible. The results demonstrated that 24.51\% of nonzero flexibility contracts were rejected due to feeder-head constraints, 67.04\% of contracts were accepted with some rescheduling and building redispatch, and only 8.09\% of contracts were accepted without any changes.
★ Constrained Sparsest Input Selection for Structural Controllability is NP-Hard
We study the problem of selecting the fewest admissible input--state links that make a structured linear system structurally controllable. This objective counts nonzero entries from prescribed positions of the input matrix and is different from minimizing its number of nonzero columns. Prior work gives polynomial-time algorithms for several restricted cases, including single or dedicated inputs and prescribed input patterns satisfying additional graph-theoretic conditions. We give a polynomial-time reduction from perfect three-dimensional matching, establishing for the first time that the decision problem is NP-complete and the optimization problem is NP-hard. A refinement of the reduction yields an exact optimum relation and shows that the problem is APX-hard. An example also exhibits a strict integrality gap for the natural linear programming relaxation.
comment: 6 pages, 1 figure
★ Large Language Models Meet Bayesian Optimization: Towards Self-evolving Wireless Digital Twins
Wireless digital twins (DTs) are increasingly integrated with artificial intelligence to support online modeling, prediction, and network optimization. However, most existing approaches perform adaptation within predefined learning or optimization structures, which may become unsuitable when wireless tasks and operating conditions change. In this article, we investigate the integration of large language models (LLMs) and Bayesian optimization (BO) to extend wireless DTs from online adaptation towards algorithm-level self-evolution. We introduce an LLM-assisted BO framework in which BO provides a stable and sample-efficient optimization backbone, while the LLM determines whether selected BO components should be retained or adapted based on the current task and accumulated optimization experience. In simulations, we apply the framework to Gaussian process (GP) kernel discovery for a multi-user wireless beam-selection problem. Simulation results show that the LLM-discovered kernel improves the final BO performance by 19.7% on the training tasks compared with the strongest standard kernel baseline. The results demonstrate the potential of combining LLM flexibility with structured BO for self-evolving wireless DTs.
★ Calibration-free IMU-based Relative Orientation Estimation under Dynamic Human Motion
Recent magnetometer-free inertial measurement unit (IMU) methods can reduce drift in joint motion estimation using rigid-body constraints, but they typically require pre-calibrated sensor locations, assumptions of specific joint models (e.g., hinge joints) or limited motion types, or an external reference for global orientation. Furthermore, challenges related to high-frequency impacts (e.g. at foot strikes during gait) and numerical differentiation of noisy measurements have not been explicitly addressed. We propose an extended Kalman filter that estimates relative IMU orientation and sensor location simultaneously, requiring neither offline sensor location calibration nor a global orientation anchor. A systematic simulation study shows that the quality of the relative orientation and the sensor location estimates depends on different motion-intensity sources: the joint center acceleration and the relative angular velocity, respectively. A low-pass filter (LPF) is used to mitigate real-signal outliers before applying the rigid-body constraint. When applied to the OpenSense dataset with real IMU measurements across four subjects with three lower-body joint types during walking and complex tasks including various motion types, the proposed method achieves an overall relative orientation error of 2.61°, compared with 10.40° for a baseline using offline-estimated sensor locations and 7.43° for a baseline using marker-derived sensor locations. These results show that the proposed calibration-free estimation framework handles real IMU data effectively and achieves accurate relative orientation without offline sensor location calibration.
comment: 8 pages, 7 figures, 1 table. Submitted to IEEE Sensors Journal
★ PriMa-Causa: Causal Prescriptive Maintenance
Automated manufacturing requires data-driven decisions under uncertainty and sparse observational records, and unplanned downtime is a dominant cost of getting them wrong. Prescriptive maintenance recommends a maintenance action against an impending failure rather than merely predicting it. Established reference architectures couple predictive models with rule- or optimisation-based recommendation logic. Selecting a corrective action, however, requires an interventional quantity: how the line behaviour would change if the action were set. Manufacturing data is observational, so a model fitted to predict the outcome estimates an associational quantity and may misdirect the choice of action without the prediction error revealing it. Causal estimators target the interventional quantity but require an expert-led causal study per case, for which the bounded decision window of operation leaves no room. This dissertation analyses how causal reasoning can be embedded into prescriptive maintenance so that actions are ranked by their estimated interventional effect. The result is PriMa-Causa, built from three components: a causal reasoning layer that integrates the estimate into the PriMa reference model, a parametric simulator that supplies the interventional ground truth absent from manufacturing data, and a graph-conditioned tabular causal foundation model that, pre-trained once on this synthetic production-line prior, answers interventional queries in a single forward pass. PriMa-Causa attains a lower prescription error than associational baselines, with the largest margin under weak overlap and sparse context, and a partially disclosed causal graph reduces this error further. The causal inductive bias of the synthetic prior and the graph conditioning can thus replace per-case estimator fitting, a step toward automating causal inference for autonomous and self-healing maintenance.
comment: PhD thesis, University of Luxembourg, 2026. 203 pages. Also available at https://hdl.handle.net/10993/69383
★ Cross-Region Cooperation of UAV Swarms Towards Air-Ground ISAC via Hierarchical Multi-Agent Reinforcement Learning
The limited communication resources and sensing perspectives of a single uncrewed aerial vehicle (UAV) motivate UAV swarms for integrated sensing and communication (ISAC) in air-ground networks. However, in multi-region scenarios, regional UAV allocations may not match vehicle demand, while local observations may lack sufficient diversity for accurate sensing. To address these issues, this paper proposes a cross-region cooperation framework for UAV swarms. For communication, cross-region UAV reallocation reduces service disparities by matching resources to vehicle demand. For sensing, combining UAV observations from different regions increases observation diversity, while adaptive handshaking guided by marginal Cramér--Rao bound (CRB) utilities balances phase synchronization gains against handshaking energy consumption. Joint optimization of UAV positions and transmit power within each region then further improves communication service and sensing performance. These coupled decisions are coordinated by a hierarchical multi-agent reinforcement learning (H-MARL) approach through slow-timescale inter-region UAV reallocation and fast-timescale intra-region UAV control. Under the considered simulation settings, reallocation can improve quality of service (QoS) by over 20 percentage points compared with retaining the initial allocation under severe resource-demand mismatches. Cooperative sensing with adaptive handshaking incurs only about a 5.0% increase in mean CRB relative to perfect synchronization.
comment: 13 pages, 12 figures
★ Stability-Preserving Reinforcement Learning for Heterogeneity-Aware Composite Max-Pressure Traffic Signal Control
Reinforcement learning (RL) provides a flexible mechanism for adapting traffic signal control to heterogeneous traffic and sensing conditions, but directly learned signal actions do not automatically inherit the analytical properties of structured traffic-control methods. This paper proposes Com-MP, a theory-guided RL framework that learns a context-dependent composition of multiple max-pressure (MP) control preferences while retaining decentralized MP phase selection. The learned policy adapts the relative influence of constituent MP formulations according to local traffic and sensing conditions, allowing data-driven adaptation to operate within a structured traffic-control framework. We show that such intersection-specific and time-varying composition preserves strong queue stability for every demand in the strictly stabilizable region under any admissible non-anticipative composition policy, thereby separating the performance-adaptation role of learning from the stability property of the underlying controller. In a representative implementation combining detector-supported queue-based MP and connected-vehicle-supported travel-time MP, experiments on a real-world 45-intersection network show that Com-MP reduces average delay by 15.6\% and 8.6\% relative to the two constituent controllers, respectively, and by about 7.0\% relative to the best fixed composition. The learned composition responds systematically to local traffic and sensing conditions, remains effective under heterogeneous and degraded sensing, and can be transferred without retraining to an unseen network with an average delay only 2.4\% higher than that of a policy trained directly on the target network. These results show how learning-based adaptation can be introduced within a stability-preserving traffic-control structure, providing a theory-guided pathway toward trustworthy learning-enabled traffic signal control.
★ Archive coverage constrains abstaining retrieval for transient stability screening more than conditional accuracy
Power-grid transient stability is assessed by simulating credible faults, but repeating simulations as operating conditions change is costly, motivating faster critical clearing time estimators. Within the AI-ASSIST project, we developed a retrieval-based estimator around three adopted requirements: estimates retain their supporting states, retrieval abstains without compatible support, and support statistics are tested against observed error. Abstention turns estimator uncertainty into a demand on the archive. Exact switching-configuration equality is used. The dense synthetic BUS39 archive covers 99.9996% of 1,044,797 records; the operational ELES archive covers 73.47% of 322,823 records. At query time, 59.37% of ELES states have an earlier same-topology candidate. Under causal availability, retrieval covers 56.34% of records; on records covered under both protocols, mean absolute error increases by 10.0%. No tested support diagnostic ranks first on both grids. Most tested supervised regressors have lower error than retrieval on BUS39, all higher on the matched retrieval-covered ELES population. On ELES, gradient boosting becomes more accurate than retrieval when temporally adjacent states are excluded, so the accuracy trade depends on the archive. Under the adopted compatibility key, archive availability constrains deployment more than conditional accuracy on ELES. Compatible coverage and chronological availability are auditable before an estimator is fitted.
comment: 34 pages, 4 figures, 9 tables, including Supplementary Information. Submitted to Scientific Reports. Code and BUS39 data: https://github.com/sensorlab/AI-ASSIST
★ Consensus-Based Distributed Filtering for Angles-Only Tracking in Cislunar Space
This paper investigates consensus-based distributed filtering strategies for multi-observer angles-only navigation in the cislunar space. The objective is to estimate the inertial state of a target spacecraft using only line-of-sight measurements collected by a network of observer spacecraft. This problem is particularly relevant for autonomous cislunar navigation, where ground support may be limited, and centralized fusion architectures may be problematic due to communication and computational constraints, and single-point failure risks. Three distributed filtering approaches are considered: the Distributed Extended Kalman Filter, the Information-Weighted Consensus Filter, and the Kalman Consensus Filter. All methods share the same local prediction and measurement-processing steps, but differ in the way information is exchanged and fused across the network. Their performance is evaluated and compared to a centralized Extended Kalman Filter, adopted as benchmark, and independent local filters in a high-fidelity cislunar scenario. Numerical simulations show that the distributed approaches significantly improve estimation accuracy with respect to independent filters. Monte Carlo sensitivity analyses are then conducted to characterize the robustness of the considered methods to finite consensus iterations and prior misalignment among the agents.
comment: Accepted for presentation at IEEE NAVICON 2026
★ Numerical Solution of Large-Scale Low-Rank Coupled Generalized Sylvester Equations
This paper develops projection methods for computing low-rank solutions to large-scale coupled generalized Sylvester equations. The dominant subspaces of these solutions are shown to be spanned by specific resolvent directions, motivating a Petrov--Galerkin projection framework. Conditions for the unique solvability of the projected coupled generalized Sylvester equations are also discussed. Based on these dominant subspaces, two algorithms are proposed: an extended Krylov-based projection algorithm and a rational Krylov-based projection algorithm. The former uses fixed shifts, whereas the latter uses adaptive shifts. Numerical experiments on large-scale benchmark problems demonstrate that the proposed methods achieve high accuracy and fast convergence.
★ Anomaly Score Preserving Data Poisoning Attacks on Data-Driven Controller Design
This paper investigates stealthy data poisoning attacks against data-driven controller synthesis and develops an attack generation method. During data collection, an attacker can sequentially manipulate state measurements, while the defender attempts to detect such manipulation using a residual-based anomaly score based on the Mahalanobis distance. Under suitable assumptions on the defender's prior knowledge, we characterize the class of time-varying affine transformations that preserve the anomaly score for arbitrary data and parameterize this class by the orthogonal group. Based on this characterization, we formulate attack design under the stealthiness condition as an optimization problem over the orthogonal group and propose a Riemannian gradient method. Numerical experiments demonstrate that the proposed attack can degrade closed-loop performance while remaining stealthy.
comment: 8 pages, 3 figures, 2 tables
★ Incentive Mechanisms for Welfare Maximization Under Individual Rationality in Noncooperative Dynamical Systems
Human societies are shaped by strategic interactions among self-interested decision-makers, where conflicts between individual and social interests naturally arise. This paper investigates the design of incentive mechanism for multi-agent game systems to reconcile social welfare maximization with individual benefits under economic constraints. We develop a linear-quadratic incentive framework to reshape Nash equilibria while reconciling the constraints of Pareto efficiency, individual rationality, and budget balance. The optimal target state is identified as the solution of a constrained optimization problem associated with the social welfare and aggregated payoff functions. Our theoretical analysis characterizes the conditions of incentive functions that steer self-interested agents towards the optimal target state as a Pareto-efficient Nash equilibrium for achieving the maximum attainable social welfare. We further characterize how the choice of social welfare function constrains achievable individual outcomes, revealing a structural paradox associated with priority weighting under individual rationality. Several numerical examples are presented to demonstrate the efficacy of our results.
comment: 16 pages, 10 figures
★ Half-Slot Link Scheduling for Open GNSS with $x$-Tx $y$-Rx Multibeam Phased Array Antennas
Global Navigation Satellite Systems (GNSSs) are evolving from closed navigation constellations toward open space infrastructure capable of serving extensive spacecraft users. This evolution requires the constellation to support navigation ranging, telemetry (TM) return, and telecommand (TC) delivery for users while preserving its own ranging and communication functions.Existing phased-array payloads onboard BeiDou satellites provide one transmit (Tx) beam and one receive (Rx) beam. Such a configuration was designed mainly for intra-constellation operation, and can become a resource bottleneck when a large number of users share the same link opportunities. Upgrading GNSS satellites to phased-array terminals with $x$-Tx and $y$-Rx beams is therefore a natural enhancement. This paper develops a staged dynamic half-slot link scheduling framework (SDHSS) for open GNSS with $x$-Tx $y$-Rx multibeam phased-array antennas. The framework models navigation, TM, and TC as three different services, reserves the essential ranging and ground-access opportunities required by the GNSS constellation, and then allocates the remaining half-slot beam resources according to current service demand. To support multibeam concurrency, feasible links are selected through dynamic service-layer ordering and beam-copy matching. Simulations based on a BeiDou constellation show that the proposed method protects GNSS baseline operation and provides stronger external navigation, TM, and TC performance than fair contact planning. The results further show that additional Tx and Rx beams can be effectively converted into multi-service capability. This work establishes a link-scheduling approach for extending GNSS from constellation-internal operation to large-scale spacecraft service support.
comment: 20 pages and 10 figs
★ Behavioral Mapping of Hierarchical Dynamical Models Accelerates Bayesian Parameter Inference
Bayesian parameter inference for complex nonlinear dynamical models is often limited by the cost of repeated model simulation. We present a behavioral mapping framework that uses the inferred posterior of a higher-level model to accelerate inference of a lower-level model without requiring state or parameter correspondence between them. Building on approximate simulation relations for hierarchically related models, we define the notion of behavioral compatibility and quantify the behavioral support of detailed model parameters using the reduced model posterior. We learn a level set of behavioral support using a Gaussian process and use it to filter detailed model parameter proposals before expensive simulations during Markov Chain Monte Carlo (MCMC) inference. We derive conditions under which the behavioral support level set retains all detailed model parameters that fit the experimental data within a prescribed tolerance. We apply this method to a hierarchy of gene-expression models fitted to cell-free experimental data. After initial setup costs, our approach speeds up MCMC by 8.02% for the phenomenological model and 15.21% for the mechanistic model.
★ On the Structure of Quantum Finite-Level Systems: Decoherence-Free Subsystems, Quantum Nondemolition Variables, and Back-Action-Evading Measurements
The purpose of this paper is to conduct a comprehensive structural analysis of quantum finite-level systems in the Heisenberg picture, where the evolution of system observables is modeled by bilinear quantum stochastic differential equations (QSDEs). Firstly, real coordinate transformations that preserve commutation and anticommutation relations are characterized, and relative admissibility between different orthonormal Hermitian bases is formulated. Secondly, strongly aligned subspaces are introduced, based on which decoherence-free subsystems (DFSs) are constructed in general orthonormal Hermitian basis as well as the generalized Gell-Mann basis. Thirdly, conditions for the existence of quantum nondemolition (QND) variables are given in terms of both linear systems theory and spectral separation by output-signal observables. Finally, a sufficient condition for quantum back-action-evading (BAE) measurements is established. A four-qubit nearest-neighbor Heisenberg chain with collective dephasing illustrates these structural properties of quantum finite-level systems.
comment: 16 pages, comments are welcome
★ Learning-Oriented Fault-Tolerant Control for UAVs with Time-Varying Fault Estimation
The increasing deployment of unmanned aerial vehicles (UAVs) in safety-critical applications has created a growing demand for control strategies that remain reliable in the presence of actuator and sensor faults. This paper presents a fault-tolerant control framework that combines constructive mathematical modeling, proportional--integral (PI) fault estimation, and learning-oriented control. A nonlinear quadcopter model is first developed using both the Newton--Euler and Euler--Lagrange formulations, from which a discrete-time linear model is derived for observer and controller design. A PI observer is then proposed to estimate actuator and sensor faults, with exponential estimation-error convergence established for constant faults in the absence of disturbances and ultimate boundedness established for slowly time-varying faults under bounded disturbances. The estimated fault signals are incorporated into an active fault-tolerant control architecture together with a Deep Deterministic Policy Gradient (DDPG) controller, which is trained using only the yaw channel and subsequently applied to improve closed-loop tracking performance. Numerical simulations illustrate the observer's estimation performance under constant and slowly time-varying faults, including bounded disturbances, and the tracking performance of the integrated controller under multiple actuator and sensor fault scenarios. The proposed framework combines model-based fault estimation with learning-based control to improve fault accommodation and tracking performance in UAV systems.
comment: This is an extended version of the paper published in the proceedings of IEEE CoDIT
★ Exact Characterization of Admissible Bandwidths for Linear Disturbance Observers
This paper characterizes the entire set of admissible Q-filter bandwidths, equivalently parameterized by the time constant $τ$, that guarantee robust internal stability of a linear disturbance observer (DOB) under parametric plant uncertainties. Classical small-$τ$ stability results provide a rigorous foundation for Q-filter bandwidth design. Building on this foundation, we formulate the $τ$-dependent robust internal-stability problem as a semialgebraic feasibility problem in bivariate polynomials and prove that the complete admissible set is a finite union of open intervals under the standing assumptions. A constructive procedure with exact membership tests recovers all stabilizing intervals. A numerical example with an uncertain plant family containing non-minimum-phase members illustrates a disconnected admissible set.
comment: 5 pages, 1 figure
★ Using Finite-Time Control to Synthesize Dynamic Backup Control Barrier Functions
This paper presents a novel approach for the construction of control barrier functions (CBFs) for controllable linear systems subject to multiple state and input constraints. The approach is based on the Dynamic Backup CBF (DB-CBF) framework, where the control policy is parametrized using the equilibrium subspace of the system. By using a minimum-energy finite-time controller, we show that the terminal set constraint trivially reduces to having an admissible equilibrium point. The proposed DB-CBF is easy to tune (one parameter), fully takes advantage of the input range (by minimizing the control effort), and guarantees constraint enforcement (by tracking a backup equilibrium point that can be reached safely in finite time).
★ Practical Complete Tracking Through Gimbaled Unidirectional Actuation with Application to Multirotor UAVs
This paper addresses the problem of practical tracking control for robotic vehicles with gimbaled unidirectional actuators, with special application to multirotor unmanned aerial vehicles. A widely recognized obstacle to tracking control for multirotors is the free-fall singularity --- when a multirotor is commanded to free fall, the vehicle's target attitude becomes undefined. This singularity is a pathology pertinent to all robotic vehicles with gimbaled unidirectional actuators. This paper considers the general theoretical problem for vehicles with gimbaled actuators. Starting from a baseline robust controller that assumes unconstrained inputs, we redesign the control law to be compatible with the gimbaled actuator. This is accomplished by driving the output of the gimbaled actuator to a ball centered at the target input value. The baseline controller's tracking guarantees are recovered in a practical sense. We apply our results to synthesize a global practical complete position tracking controller for multirotor unmanned aerial vehicles. The theory is demonstrated in simulation.
comment: 16 pages, 10 figures
★ On Sample Complexity Bounds for Frequency-Domain System Identification
We study the sample complexity of frequency-domain system identification with bounded measurement noise. We focus on interpolatory identification algorithms, which return models consistent with both the data and a priori constraints. For equally spaced samples, we sharpen previous worst-case bounds on the consistency-set radius. Its excess above the noise level decays as $N^{-2}$, where $N$ is the number of samples, and a matching lower bound shows that this rate is tight. We then examine whether bounded stochastic noise allows identification below the noise level. We show that high-probability error bounds can converge to zero. Their convergence rate depends on how strongly the noise distribution is concentrated near the boundary of the noise set. For example, noise uniformly distributed on a disk gives an upper bound of order $(\log N/N)^{2/3}$. Numerical experiments examine these convergence rates and the effect of the noise distribution on the identification error.
★ Silence Is Not Assurance: When Production Lane Keeping Assist Does Not Warn, and What It Costs
Production lane keeping assist (LKA) systems tell the driver whether they detect the lane markings, but whether this displayed status reflects what the system perceives, and what happens when it does not, has not been examined in real-world driving. This study introduces CANDID-LKA, a CAN-based Diagnosis of the Indicated Detection status in LKA. It evaluates a proprietary driver-facing status using three signals: the displayed status and the steering torque of the driver, both recorded from the controller area network (CAN) bus of the vehicle, and the lane perception confidence of a third-party vision system observing the same road. The framework examines how the status responds to driving conditions and estimates how completely it reports degraded lane perception by comparing it with the other two signals in a latent-class model. We apply it to 39,800 seconds of real-world driving from 13 vehicle models of three manufacturers. The results show that the status does not respond consistently to the conditions that make markings harder to see, and that the three makes differ widely in how often they report not detected. The latent-class model shows that the displayed status is incomplete: it reports only 47% of degraded seconds as not detected, compared with 80% for the third-party low perception confidence, and in half of the low-confidence episodes it never reports not detected. In the silent seconds in which the status reads detected while the perception confidence is low and the driver is not in control, the standard deviation of lateral position over the next 3 s is about 60% larger than when the status and the perception confidence agree. A detected status thus does not guarantee reliable lane perception, and silence with the driver not in control is the case that most needs attention.
★ Model-Free Generative Stealthy Attacks on Safety-Critical Vehicle Cyber-Physical Systems
Existing stealthy attacks presuppose white- or grey-box access to the target system, unrealistic in practice. This paper develops a model-free stealthy attack framework that learns an attack policy directly from nominal measurement data, without knowledge of the system dynamics, estimator, or detector. We establish a certified residual-impact bound and an observable stealth budget around a stealth-consistent anchor, and show that recurrence and context conditioning are structural necessities for closed-loop, multivariate cyber-physical systems (CPS). The resulting attack generator, trained against a Wasserstein critic under a stealth-constrained impact objective, induces a hazardous 2.51 m lane departure while evading five intrusion detection systems, and, in real-time hardware-in-the-loop tests, a speed deviation of approximately 8 km/h while staying below the detection threshold. Results show that safety-critical CPS remain exposed without attacker access to privileged information, motivating offense-informed defenses.
★ A graph decomposition approach for the synthesis of EMPs in directed acyclic networks
This paper considers the identifiability of dynamical networks whose topology is a Directed Acyclic Graph (DAG), in the context where not all nodes are excited and not all nodes are measured. The objective is to construct a valid and parsimonious Excitation and Measurement Pattern (EMP), i.e. a selection of excited and measured nodes, that allows to identify the whole network with a small number of excitations and measurements. We show first that a DAG can be decomposed into a set of trees and parallel paths networks. We propose an algorithm that achieves this decomposition and computes an EMP that combines EMPs for the individual trees and parallel paths networks. We formally show that this algorithm yields an EMP capable of identifying the whole network.
comment: Paper submitted to Automatica
★ Exposing Noise-Deficiency Signatures of Deep Learning-Based False Data Injection Attacks Through AC State Estimation
Deep learning (DL)-based false data injection (FDI) attacks can resemble normal measurements and evade both conventional bad data detection (BDD) and measurement-space data-driven detectors. This paper identifies a previously overlooked signature of such attacks in state-estimation (SE) residual space. The residuals of reconstruction-based DL attacks exhibit deficient stochastic variation, manifested as abnormally small residual energy and reduced directional dispersion. We establish the conditions under which this signature emerges and explain why the attacks carrying it can evade conventional BDD. We show that SE can reveal residual-space differences obscured by operating-state variation in measurement space. We then explain the origin of the signature through the noise-suppression mechanism of DL attack generation. A generator can learn the reproducible structure of historical measurements but cannot reproduce stochastic measurement components that are unpredictable from its input. This signature enables label-free detection using a simple two-sided test on the conventional BDD statistic, while the full residual vector can also be exploited when labeled attack samples are available. Experiments on the IEEE 14-bus and 118-bus AC systems show that the label-free test detects up to 99% of three representative DL attacks at a false-alarm rate comparable to that of BDD, which detects at most 5%. Ablation studies further examine the effects of residual representation, decision rule, and measurement-noise magnitude and distribution.
★ Asset-Normalized Distribution Reliability Budget Optimization Using Historical Outage Benefits and Work-Type Learning
Reliability investments compete for constrained capital but are often evaluated in separate programs. This paper presents a data-driven framework that translates historical work orders, geographic asset records, outage events, and costs into comparable customer-minute and customer-interruption benefits at the circuit-zone level. A normalization process maps historical device names and non-isolating assets to current automaticisolating-device zones and links investments to pre- and postconstruction outage histories. Utility data covering more than $250 million in investments completed from 2010 to 2020 are used to estimate benefit factors for seven work types. Those factors feed a mixed-integer portfolio model that selects at most one applicable action per zone under a capital budget. Observed reductions range from 32% to 94% for customer-minutes interrupted and from 43% to 93% for customer interruptions. The framework provides a traceable path from heterogeneous utility records to zone-level reliability investment decisions
★ Inverse Modeling for Laser Pulse Shape Design in Inertial Confinement Fusion NeurIPS 2026
The achievement of practical fusion energy remains one of the most pressing unsolved scientific challenges, with immense implications for carbon-free power. A critical factor in the success of Inertial Confinement Fusion (ICF) is the design of a Laser Pulse Shape (LP) that can optimally drive implosions under stringent physical constraints. Traditional LP design relies on computationally expensive simulations and manual iterative refinement. We introduce the ICF Laser Pulse Shape Design System (LPDS), a generative inverse modeling framework that maps desired outcomes and target pellet configurations directly to optimized LPs. Crucially, we design a multi-objective loss function to ensure the generated LPs satisfy fundamental physical constraints and experimental requirements. Furthermore, we present constraint-conditioning, inpainting, and gradient-based LP editing mechanisms for fine-grained control over specific pulse characteristics during generation. Moreover, we study the transfer of our framework to a limited set of real experimental shots. Our method establishes a data-driven inverse design framework for LP design in ICF, contributing to the advancement of practical and sustainable fusion energy.
comment: Initial version submitted to NeurIPS 2026 pending publication
♻ ★ Dissipation Bounds and Hardness in Globally Asymptotically Stable Quadratic Systems
Approximating the maximum retained energy fraction in globally asymptotically stable quadratic systems of growing dimension is NP-hard at every fixed positive additive error below $1/2$, even when strict quartic Lyapunov functions and strictness proofs are constructed in polynomial time and included in the input. Polynomial-factor approximation is also NP-hard for the least dissipated fraction, energy-halving time and structured coefficient distance to loss of asymptotic stability. For a partially damped family, minimum unit-sphere coupling norm equals that distance, with two-sided inverse-square halving-time bounds at distances in $(0,1/2]$ and explicit coefficient factors. Additive hardness persists under structured coefficient perturbations and direct damping, even with global exponential stability, a strict quadratic Lyapunov function and a Hurwitz Jacobian.
comment: 7 pages, 3 figures; revised proof exposition and conclusion
♻ ★ Calibrated uncertainty quantification for prosumer flexibility aggregation in ancillary service markets
Reliable forecasting of prosumer flexibility is critical for demand response aggregators in frequency controlled ancillary service markets, where reliability requirements such as the P90 standard are enforced. Limited behavioural data and dependence on exogenous forecasts introduce uncertainty into flexibility estimates, which can cause reserve overestimation and financial penalties. This study systematically evaluates established uncertainty quantification approaches based on Monte Carlo dropout (MCD), conformal prediction (CP), and conformalised quantile regression (CQR) to construct prediction intervals for aggregated prosumer flexibility and support uncertainty-aware bidding. The approaches are evaluated for a behind-the-meter aggregator in the Danish manual frequency restoration reserve capacity market, using a large-scale synthetic dataset generated with an industry-grade home energy management system and publicly available load, solar, price, activation, and device-level data. Multiple multivariate CP strategies are applied to MCD predictions, alongside CQR using a separately trained quantile regression model. Under nominal test conditions, conformal methods substantially improve joint coverage over uncalibrated MCD baselines. More conservative methods satisfy the 90% lower-tail reliability requirement, while CQR and multivariate CP (MCP) yield narrower intervals and higher economic value, retaining approximately 69% and 68% of perfect-information profit, respectively, under the stricter penalty interpretation. A controlled robustness analysis shows that increasing perturbations of load, solar, price, and activation inputs increase overbidding and reduce coverage and economic performance. The results highlight the trade-off between reliability, economic value, and robustness to input uncertainty in aggregator bidding.
♻ ★ Physically Constrained Agentic AI for Energy Scheduling
Agentic AI extends energy management beyond fixed-form interaction by translating natural-language requests into coordinated scheduling actions. We present a hierarchical ReAct Energy Management System (EMS) in which one orchestrator coordinates specialist agent types for shiftable appliances, EV charging, and thermal control. Physical authorization is separated from language generation: a deterministic critic reconstructs each integrated day-ahead candidate and checks its schema, appliance cycles, device power, thermal comfort, and, when active, the whole power feeder limit. Across Qwen 3.5 checkpoints, single-appliance mixed-integer schedules were feasible in 83.3 percent of runs. Localized feedback produced no accepted coupled schedule, whereas a multi-step policy authorized 6/6 current coupled runs: 3/3 for 27B and 3/3 for 35B-A3B. The standard occupied-window policy permits pre-conditioning, enforces comfort from 09:00-18:00. Every accepted schedule passed an independent final replay. Feasible costs were 2522.499 JPY for 27B and 1592.697 JPY for 35B-A3B, which are slightly higher than the mathematical optimization optimum of 1343.380 JPY. These results establish a fail-closed workflow for agentic MIP and MILP energy scheduling under the declared physical model.
comment: 5 pages, 2 fiugres, Accepted by IEEE PESIM 2027
♻ ★ Evolutionary Deep Nets for Non-Intrusive Load Monitoring
Compact appliance classifiers can support local electricity-load monitoring where memory and computation are constrained. We evaluate adaptive sparse evolutionary training and warm-start compression using synthetic aggregate windows derived from household appliance recordings. A completed convolutional network study trains 120 models over four targets and three seeds with validation-only model selection. Two analyses examine compression with approximately preserved binary accuracy and joint improvement in positive-class F1 and sparsity. Relative to a newly trained fixed sparse evolutionary baseline, the selected compressed policy changes accuracy from 91.53%, while increasing macro F1 to 92.40%. Stored weights decrease by 17.36% and active first-layer connections by 23.93%. Refinement requires additional training. Three seeds and unverified source-window independence prevent claims of statistical equivalence or unseen-house performance.
♻ ★ Unifying Goal-Conditioned RL and Unsupervised Skill Learning via Control-Maximization NeurIPS 2026
Unsupervised pretraining has driven empirical advances in goal-conditioned reinforcement learning (GCRL), but its theoretical foundations remain poorly understood. In particular, an influential class of methods, mutual information skill learning (MISL), discovers behaviorally diverse skills that can later be used for downstream goal-reaching. However, it remains a theoretical mystery why skills learned through MISL should support goal-reaching. A subtle challenge is that both GCRL and MISL are umbrella terms: Different GCRL tasks use distinct criteria for measuring goal-reaching performance, while different MISL methods optimize distinct notions of behavioral diversity. We address this challenge and unify GCRL and MISL as instances of control maximization. We identify three canonical GCRL formulations and prove that they are fundamentally inequivalent: They can induce incompatible optimal policies even in the same environment. Nevertheless, they all share a common interpretation: A well-performing goal-conditioned policy is one whose future trajectory is highly sensitive to the commanded goal, with the precise notion of sensitivity determined by the GCRL formulation. Noting that MISL objectives can be understood as measures of skill sensitivity akin to goal sensitivity, we show that MISL objectives are bounded by formulation-specific downstream goal sensitivities. These bounds establish a precise correspondence between MISL methods and downstream GCRL tasks: For every GCRL formulation, there exists a matching MISL objective for which more diverse skills afford greater downstream goal sensitivity. Our results thus lay a theoretical foundation for RL pretraining and have important practical implications, such as suggesting which pretraining objectives to use when a user cares about a specific class of downstream tasks.
comment: Accepted at NeurIPS 2026 (Oral Presentation)
♻ ★ Adversarial Reinforcement Learning for Robust Planetary Landing
Deep reinforcement learning (DRL) policies for planetary landing and other aerospace guidance, navigation, and control tasks are commonly trained under randomized disturbances to improve robustness. However, typical stochastic perturbation sampling assumes independent variations, whereas real mission disturbances often exhibit spatial and temporal correlation and cannot be reproduced through independent random draws. This work investigates adversarial reinforcement learning to discover challenging off-nominal conditions. A secondary agent generates physically constrained perturbations that degrade the performance of a landing controller, forming an asymmetric sequential zero-sum game. Training proceeds in three stages: baseline controller learning, robustness enhancement with stochastic perturbations, and an iterative adversarial phase in which the controller and disturbance generator are alternately optimized. Preliminary experiments in a three-degree-of-freedom landing environment show that adversarial disturbances reveal vulnerabilities missed by conventional randomization and provide early indications that, with a more extensive training campaign, the approach can yield improved robustness and stronger generalization under previously unseen perturbation regimes.
♻ ★ Prior Evolution and Task Alignment for Aerial Grasping
Aerial grasping is a remarkable capability exhibited by predatory birds, allowing them to capture prey through highly coordinated maneuvers in flight. Inspired by this capability, researchers have developed various formulations to reproduce such maneuvers through trajectory optimization. However, two limitations remain in practice. First, the resulting optimization problem is highly nonconvex and sensitive to initialization, making high-quality solutions difficult to obtain under a limited computational budget. Second, prescribed numerical objectives are human-designed abstractions that describe successful grasping through a limited set of mathematically tractable quantities and may not fully capture what determines task success. We investigate how learning can address these limitations within an analytical planner. Accordingly, a trajectory prior is first learned from optimized motions and then evolved through a CEM-based process that evaluates sampled initializations with the deployed optimizer and retains favorable ones as new supervision. An Execution-Aware Critic learns from contact, lift, and completion outcomes to assess whether the optimized trajectories are likely to succeed in physical execution. Its frozen energy can further serve as a differentiable grasping cost, allowing execution data to directly shape trajectory generation. Simulation and real-world experiments demonstrate improved optimization reliability, trajectory consistency, and grasping performance.
comment: 20 pages, 14 figures
♻ ★ Robustness Verification of an Autonomous Underwater Vehicle-based Plankton Classifier
The assessment of planktonic standing stocks and microorganism structures is critical for understanding upper ocean biological processes. Currently, autonomous underwater vehicles (AUVs) equipped with in-situ optical imaging and artificial intelligence (AI) methods offer a promising solution for persistent surveillance, mapping and monitoring of planktonic life. However, current AI methods often lack robustness in dynamic, unstructured environments, where environmental noise and non-biological artifacts lead to frequent misclassifications. Standard convolutional neural network (CNN) classifiers often struggle with such conditions, leading to misclassifications that require time-consuming manual validation by marine biologists. To address this issue, we propose a novel robustness verification framework for in-situ plankton classifiers based on reachability analysis. We also introduce a continuous-time neural ordinary differential equation (neural ODE) classification model leveraging the high-resolution imaging capabilities of the SilCam particle imager. In this paper, we demonstrate the effectiveness of the proposed framework by formally verifying the robustness of the neural ODE model against environmental perturbations. We demonstrate that our verification framework acts as an automated filter providing formal guarantees of model stability against ambiguous data, thereby improving the reliability of autonomous sampling and reducing the post-processing workload.
comment: 6 pages, 2 figures, Accepted in IEEE OES AUV Symposium 2026
♻ ★ Network-Aware Progressive Replacement Sequence Planning for Laser-Enhanced BeiDou Navigation Constellations
The early satellites of the third-generation BeiDou Navigation Satellite System (BDS-3) are approaching the end of their design lifetime, making progressive constellation replacement an inevitable engineering task. Meanwhile, laser inter-satellite links (LISLs) offer high-precision time transfer, ranging, and high-rate data forwarding, providing a promising upgrade path for future BeiDou satellites. This paper investigates the replacement-sequence planning problem for progressively replacing legacy microwave satellites with laser-enabled satellites from a networking perspective. Since practical replacement decisions remain constrained by legacy satellite lifetime, on-orbit condition, laser satellite manufacturing readiness, and launch scheduling, networking information is used as an auxiliary criterion in the actual engineering replacement process. An integer linear programming model is developed to evaluate the network value of intermediate laser--microwave hybrid states. Based on this common state evaluator, two complementary planning methods are developed. A priority-aware exact method supports round-wise network-assisted launch decisions, while a visibility-guided adaptive large-neighborhood search is used for full-sequence planning and provides a full-horizon network-oriented near-optimal reference. Simulations using the actual BDS-3 constellation show that stronger network participation leads to earlier and larger ground-connected laser subnetworks, fewer microwave-dependent satellites, better coordination between laser-backbone growth and high-rate ground access, and improved residual microwave-layer performance. These results provide a quantitative basis for incorporating network performance as an auxiliary criterion in future BeiDou replacement planning.
comment: 18 pages, 7 figures
♻ ★ Lightweight Onboard Routing and Topology Co-Construction for Autonomous GNSSs
Global navigation satellite systems (GNSSs) can already plan their inter-satellite link (ISL) topology onboard, leaving routing as the last step toward autonomous networking. In GNSS ISLs, each satellite links to at most one peer per time slot. We exploit this structure to replace end-to-end path search with a single backward sweep over the slots. The proposed method, DS-SMFP, propagates earliest-arrival times backward from each destination and outputs a STORE/FORWARD decision for every slot, satellite, and destination, in time linear in the number of slots and with working memory independent of the planning horizon. Because each slot needs only its own topology and the delivery state of the next one, routing can steer link selection while the topology is built. This slotwise dependency enables Master-Oriented Topology--Routing Co-Construction (MTRCC), which favors traffic to and from a master satellite performing onboard orbit determination. In a 24-h, 30-satellite BeiDou scenario, DS-SMFP matches the delays of Contact Graph Routing (CGR) and shortest path tree (SPT) routing while running over 180 times faster and using over 36 times less peak routing working memory. Compared with fair contact plan (FCP) topology, MTRCC reduces mean master-related traffic delays by over 10\%, with an approximately 3\% increase in mean PDOP. This paper provides a methodological foundation for efficient autonomous GNSS networking.
comment: 15 pages, 6 figs
♻ ★ Opinion Dynamics in Social Multiplex Networks with Mono and Bi-directional Interactions in the Presence of Leaders
We delve into the dynamics of opinions within a multiplex network using coordination games, where agents communicate either through one-way or two-way interactions, and where a designated leader may be present. By employing graph theory and Markov chains, we illustrate that despite zero diagonal elements in transition probability matrices or decomposable layers, opinions converge under specific conditions on the layers, leading to a consensus, and we show by a counterexample that a spanning tree in the union of the layers does not suffice. We further scrutinize the convergence rates of opinion dynamics in networks with one-way versus two-way interactions. We find that in networks with a designated leader, opinions converge towards the initial opinion of the leader, whereas in networks without a designated leader, opinions converge to a convex combination of the opinions of agents. Moreover, we emphasize the crucial role of designated leaders in steering opinion convergence within the network. Paired experiments on 1,200 random multiplex networks show that a leader slows down convergence, and that two-way interactions slow it down in the presence of a leader but speed it up in its absence.
♻ ★ Operator-in-the-Loop Rolling Dispatch for Damage-Assessment Crews Under Partial Storm Information
Natural disasters often inflict severe damage on distribution grids. Rapid, reliable damage assessment (DA) is essential for storm restoration, yet most optimization work targets repair dispatch after faults are identified. This paper presents a production, rolling horizon DA crew allocation system deployed across multiple U.S. states in Eversource Energy's service territory and used during live storms. The method implements a sequential k-job assignment policy per available crew, executed on a fixed cadence and on operators' control. The objective jointly prioritizes critical facilities and customer impact while controlling travel time on the actual road network via the Google Maps API. A key constraint is the absence of live crew GPS; we infer crew locations from the last confirmed DA site and robustify travel estimates for staleness, yielding stable recommendations without continuous tracking. The operator remains in the loop with controls to limit churn and to publish a feasible plan. Using data from the March 7 New Hampshire storm with 90 moderate outages and seven DA crews, we observe shorter time to first assessment, fewer revisits with reduced distance traveled. To our knowledge, this is among the first multi-state enterprise integrated deployments to treat DA crews as a first-class optimized resource in storm restoration.
Robotics
★ Agentic World Action Models
Recent world action models provide visual foresight and future prediction for robot manipulation, but lack explicit mechanisms for compositional skill reuse. Agentic robotics combines code-based planning and reasoning with reusable skills, but often lacks predictive grounding in physical dynamics. We introduce AW-0, an agentic world action model framework that integrates code-based planning with joint video and action prediction. We jointly model code, future video, and robot actions as three coupled generative streams within a unified mixture-of-transformers architecture. Within this architecture a code expert generates parameterized skill routines and control logic while video and action experts generate future visual states and low-level motor commands. Code serves two complementary roles: it structures multi-step prediction and control within the world model and represents skills as reusable programs for subsequent agentic execution. We further introduce an automated curation pipeline that labels robot and human video trajectories with aligned programs and consolidates recurring routines into reusable skill libraries. Evaluations on RoboTwin 2.0 and physical bimanual manipulation tasks show improved task performance over VLA and WAM baselines, alongside gains in language following, semantic reasoning and robustness, demonstrating that agentic code enables connection between planning, prediction and execution within WAMs.
comment: Project page: https://agentic-wam.github.io/
★ PhysiCalWorld: In-Context Physics Calibration for World Models
Existing world models typically perceive a scene through current visual frame or a narrow history window. However, these observations may not reveal the physics properties that determine transition dynamics, such as an object's mass distribution, joint damping, or surface friction. Without calibrating these physics factors for world models, predictions tend to regress toward an empirical prior and induce severe drift in novel test scenes. To this end, we propose PhysiCalWorld, which calibrates action-conditioned world models to the instance-specific dynamics via in-context prompting, without fine-tuning or analytical physics modeling. Our method encodes an exploratory interaction clip collected within the test scene into a compact physics descriptor token, which grounds subsequent predictions in the scene's actual physical behavior. PhysiCalWorld is trained end-to-end with the latent prediction loss alone, without physics labels. At inference time, the descriptor token is extracted once and can be reused across rollouts with negligible compute overhead. We evaluate PhysiCalWorld across four environments covering diverse physics variations and show that it substantially improves rollout prediction accuracy and visual planning success rate. The learned descriptors also form a physically structured latent space and generalize beyond the physics ranges seen during training.
comment: Project page: https://physical-world.github.io/
★ CW-WAM: A Structured Contact-Wrench World Action Model for Force-Sensitive Manipulation
Force-sensitive manipulation requires contact and load information that is not provided explicitly by visual observations. We present CW-WAM, a world action model that jointly predicts video, motion, and contact wrench. Inspired by hybrid motion-force control, it contains a unified interaction module that treats both motion and contact wrench as modeling targets and predicts time-aligned motion-wrench chunks as control references. A pair-causal attention mechanism structures the motion-wrench dependencies, while a long fixed-length wrench memory retains past interaction information. The predicted wrench chunk acts as a force reference for a 20-Hz outer-loop admittance controller within a slow-fast control system that maintains force feedback and smooth motion between model queries. CW-WAM improves contact prediction and increases the insertion success rate from 25% to 70% for square pegs with 1-mm clearance per side and from 40% to 95% for power adapters, compared with WAM without wrench input. It also improves lifting-height accuracy across different loads. Our framework with an admittance controller reduces force-reference RMSE from 4.8 to 2.7 N for peg insertion and from 2.0 to 1.3 N for wiping, respectively, with smaller force fluctuations during successful execution. Project website: https://cw-wam-project.github.io/
comment: Project website: https://cw-wam-project.github.io/
★ Multimodal Embodied Agents: A Unified Capability-centric Perspective from Computer-Use to Robot-Use
Multimodal agents (MMAs) sustain interaction through reasoning, memory, tools, and feedback, most visibly as computer-use agents, while robotic systems (RSs) couple sensing and actuation under physical dynamics. We define multimodal embodied agents (MMEAs) as goal-directed systems that couple multimodal task reasoning with physical action and revise decisions from the resulting feedback. This raises a central question: what changes when a multimodal agent moves from computer-use to robot-use? We introduce PAPAV, a unified capability-centric framework of five recurring functions: Perceive the current state, Anticipate action effects, Plan a feasible course, Act through an interface or body, and Verify the outcome. Defined by function rather than architecture, these capabilities let PAPAV identify what all three share and then compare MMEAs with MMAs and with RSs. Five physical constraints, from partial observability to unverifiable outcomes, leave fewer choices fixed in advance and less room to reverse errors. Across 64 benchmarks, explicit evaluation centers on Act; only 5 assess Anticipate and 3 assess Verify, leaving both largely hidden behind task success. These constraints also frame the open challenges we identify. PAPAV therefore provides a common basis for designing reliable physical agents and measuring progress across all five capabilities rather than task success alone.
comment: Project website: https://showlab.github.io/Awesome-Multimodal-Embodied-Agent/
★ Asymmetric On-Policy Distillation Learns In-Context Exploration
Sequential decision-making under partial observability is difficult because agents must explore to infer hidden, task-relevant information before acting effectively. Learning such information-gathering behaviors tabula rasa via online reinforcement learning is sample-inefficient, whereas offline imitation averages expert actions across hidden states and fails to learn exploration. In this work, we propose ASTEROID, a simple and effective student--teacher supervised learning framework that can learn test-time exploration. Although a privileged expert never needs to explore, we demonstrate that a history-conditioned student can learn to do so when trained with on-policy contexts: the student first collects on-policy context under partial observability, and the privileged expert provides action labels conditioned on that context. By distilling expert actions directly conditioned on the student's on-policy history, the student learns to systematically resolve partial observability, i.e., search while uncertain and exploit once the hidden state is revealed. This enables efficient test-time exploration (similar to Bayesian posterior sampling) without incurring the impractical training-data costs of reinforcement learning. Across a diverse range of eleven different environments, including discrete exploration benchmarks, robotic manipulation, navigation, and Procgen environments, ASTEROID achieves ~2x higher returns under fixed exploration budgets, uses ~100x fewer interactions than RL baselines, and outperforms frontier models at exploration. Finally, our method transfers policies trained in simulation to vision-denied real-world robot manipulation, exhibiting efficient exploration and increasing success by +40% over standard distillation. Overall, it provides a simple yet surprisingly effective framework to learn efficient test-time exploration.
★ Implementation of Dynamic Microsegmentation for a Multi-Robot Network
This report describes a laboratory instantiation of a dynamic microsegmentation function for an Internet-of-Things network, specifically a multi-robot network, using a commercial software-defined microsegmentation tool. Fine-grained dynamic segmentation policies are defined, where segments are adapted based on asset functional and health tags. Automated implementation of the dynamic policies in software is also undertaken. The microsegmentation solution is shown to automatically isolate an indicated threat in real time, while maintaining unaffected operations. The demonstration indicates that microsegmentation can be implemented in IoT networks, in a way that allows for robust operation in the face of threats and disruptions.
★ AtomicRL: Atomic Reinforcement Learning for Long-Horizon Robotic Manipulation
Vision-language-action (VLA) models have shown promise in robotic manipulation, but imitation learning (IL) alone often struggles to master complex long-horizon skills. Reinforcement learning (RL) offers an effective avenue for further improving VLA policies. However, existing VLA-RL methods typically suffer from delayed and biased credit assignment due to trajectory-level sparse rewards, while imbalanced subtask mastery creates bottlenecks that limit overall performance. To address these challenges, we propose AtomicRL, an RL post-training framework that leverages atomic task structures for reliable credit assignment and adaptive exploration. AtomicRL introduces Semantic-Aware Policy Optimization (SAPO), which uses semantic-aware gating to constrain TD residuals from failed atomic subtasks from propagating across task boundaries, mitigating credit contamination while preserving intra-subtask credit assignment and value bootstrapping. In addition, Bottleneck-Adaptive Target Exploration (BATE) dynamically coordinates task sampling and local exploration incentives based on the mastery of individual atomic subtasks, directing learning toward the remaining bottlenecks. Finally, we develop a Real2Sim2Real pipeline that bridges online RL in simulation with real-world deployment, facilitating practical transfer to physical robots. On LIBERO-LONG, AtomicRL achieves absolute performance gains of 3.6% and 4.0% over PPO across two VLA architectures. On CALVIN ABC-D, it improves full-task performance by up to 9.5% over PPO. Combined with the Real2Sim2Real pipeline, AtomicRL further improves real-world task performance by an average of 21.3%. These results demonstrate that AtomicRL effectively improves the learning efficiency and manipulation performance of VLA policies on long-horizon tasks.
★ Do Earth-Trained Humanoid Controllers Walk on the Moon? Gravity, Terrain, and Soil Transfer
The Moon changes gravity, terrain, and soil at once. Learned locomotion has studied reduced gravity on rigid ground and granular terrain mostly at Earth gravity, but not humanoids under both, generalist-controller transfer, or the Earth cost of lunar adaptation. On a simulated Unitree G1, we train 12 policies with one recipe, varying gravity, terrain geometry, and ground contact (rigid or three-dimensional resistive force theory, 3D RFT), and including Earth-to-Moon fine-tuning and gravity randomization. We test them at both gravities on rigid ground and 3D RFT, and at their training gravity on three held-out soil models. We also test the frozen SONIC, a released generalist humanoid controller, at both gravities on rigid ground only. Gravity transfer is asymmetric. Earth-trained policies keep 0.64 to 0.75 survival at the Moon on flat rigid ground, Moon-trained and fine-tuned policies cannot stand at Earth, and SONIC survives at most 0.10 at the Moon. Rough training lifts combined-tile survival from 0.53 to 0.98 at Earth and 0.80 to 0.99 at the Moon. In 3D RFT, soil training helps on Earth sand, on tiles and for rough-trained policies also on flat sand. On the stiffer lunar soil it helps only flat-trained Earth policies at Earth and changes the survival of Moon-trained policies by at most a few hundredths. In the sampled rollouts of the held-out soil models, Moon-trained policies keep most of their rigid-ground progress on lunar regolith at the low G1 foot pressure, with per-seed progress ratios of 0.84 to 1.39 apart from one seed with a short rigid reference. A gravity-randomized policy trained on rough 3D RFT terrain has a seed-mean survival of at least 0.84 at Earth and 0.96 at the Moon on every key test cell up to 20 degrees. Lunar results are simulation predictions conditional on the contact model and soil stiffness.
comment: 11 pages, 4 figures, 5 tables. Supplementary material (13 pages) included as an ancillary file
★ Test-Time Motion Steering for Perceptive Humanoid Fall Recovery
Humanoid fall recovery requires selecting a motion that suits both the surrounding scene and the task. The same fallen pose can admit several ways to stand, with different routes, support transitions, and final headings. Yet reliable recovery alone does not provide control over which of these alternatives the robot executes. Steering this choice requires anticipating the physical consequences of a requested movement. We present RISE, an approach to perceptive humanoid recovery that lets task objectives steer how the robot returns to standing at inference time. It builds on two insights: diverse recoveries can be composed from shared movements, and anticipating how these movements unfold allows recovery to be steered toward a task objective. We learn a recovery vocabulary that connects intended movements to motor actions through sensory feedback. A visual diffusion planner learns from executions of this vocabulary to predict motion intent as future body states. Task costs guide this prediction toward a desired heading, rise speed, or path. The planner then generates motion latents conditioned on the steered intent, allowing the requested movement to shape its execution. Simulation experiments demonstrate diverse recoveries, terrain adaptation, and test-time steering without retraining, complemented by real-world demonstrations on a Unitree G1. Project page: https://osheraz.github.io/RISE/
★ From Reasoning to Action: Benchmarking Frontier AI Models on LIBERO-10 Robot Manipulation Tasks
Recent advances in frontier AI models have demonstrated remarkable capabilities in visual understanding, reasoning, planning, and tool use, raising the possibility of using general-purpose AI models for robot manipulation without task-specific training. However, their manipulation capabilities have not been systematically evaluated on standardized robotics benchmarks. In this report, we present a unified closed-loop manipulation framework for evaluating frontier AI models on the LIBERO-10 benchmark. Our framework enables models to control a robot directly through visual observations, robot proprioception, and structured tool calls, primarily by specifying target end-effector poses and gripper commands, without task-specific fine-tuning or pretrained manipulation skills. We evaluate six frontier AI models, including GPT-6 Astra, GPT-6.1 Sol, GPT-6 Luna, Claude Opus 5, Claude Opus 5.5, and Gemini-4 Argon, across three reasoning effort levels, resulting in 18 configurations and 1,800 evaluation episodes. Our results show that GPT-6 Astra achieves the highest success rate of 44%, followed by Gemini-4 Argon at 43%. Increasing reasoning effort generally improves manipulation performance, but the benefits vary across models and come at additional inference cost. We also observe substantial differences across tasks, with models performing relatively well on object placement tasks but struggling with precise spatial arrangements and complex multi-step interactions. These findings demonstrate both the potential and limitations of frontier AI models for robot manipulation, highlighting the challenges of translating reasoning capabilities into reliable physical actions. We release our evaluation framework and experimental results to support reproducible benchmarking and future research on reasoning-based robot manipulation.
comment: 17 pages, 6 figures, 8 tables. May Espinola and Qifan Zhang contributed equally. Project page: https://irvlutd.github.io/LLM-LIBERO/ Code: https://github.com/IRVLUTD/LLM_LIBERO
★ SP-HLM: Sparse-Pose-Driven Motion Generation and Whole-Body Control for Humanoid Loco-Manipulation
Humanoid loco-manipulation requires an effective interface between high-level task policies and whole-body control. Existing interfaces typically rely on a single action representation for both task intent and whole-body coordination. This may burden the high-level policy or leave one aspect underspecified. We present SP-HLM, a framework built on a sparse-pose-driven interface. The interface combines five task-critical body poses with a learned latent motion prior to represent spatial intent and whole-body coordination in complementary forms. SP-VLA, a vision-language-action model, first predicts task-relevant sparse poses and finger joint positions from multimodal observations and language instructions using asynchronous action-chunk inference. Given these sparse targets, SP-MGM, a motion generation model trained through two-stage distillation, reconstructs coordinated full-body motion in real time. A pretrained motion encoder maps this motion to a compact latent representation. SP-GMT, a general motion tracker, then integrates the explicit sparse targets with this implicit motion prior for whole-body control. Experiments in simulation and on a real humanoid robot demonstrate improved motion tracking, whole-body coordination, and loco-manipulation success over existing approaches. Additional results and videos are available on our \href{https://sp-hlm.github.io/}{\textcolor{blue}{project page}}.
★ TARSphere: A Robot-Oriented Full-Stack Framework for Long-Horizon Embodied Agents in Evolving Physical and Social Environments
Robots must act under physical and social constraints while adapting to changes in their environment and human needs. Existing embodied-agent systems often focus on specified goals or abstract execution into symbolic actions, limiting evaluation in an evolving world. We introduce TARSphere, a robot-oriented full-stack framework for physically and socially grounded long-horizon embodied agents. TARSphere-Infra runs robot policies in a physics simulator with embodied NPCs and constrained dialogue. An event-driven finite-state machine (FSM) uses completed actions and delivered information to update task states and evaluate outcomes. TARSphere-Harness, its embodied brain harness, lets a frozen multimodal large language model (MLLM) act through multi-level executable skills, using observations, hierarchical memory, and execution feedback to revise its decisions. On this foundation, agent-in-the-loop curation aligns task semantics, causal dependencies, and scene geometry to construct TARSphere-Bench: 360 episodes across 30 indoor scenes. Five paired conditions assess perceptual robustness, disruption adaptation, task update, human coordination, and intent understanding for navigate-and-report and fetch-and-deliver tasks. Our evaluation of 8 MLLMs shows that general-purpose models can outperform embodied models, while task updates and human interaction remain difficult. Frequent collisions and falls further motivate evaluating execution safety alongside task success, even with a shared action interface and decision budget.
comment: 13 pages, 8 figures, 7 tables. Project page: https://xuyicheng-zju.github.io/TARSphere/ Code: https://github.com/xuyicheng-zju/TARSphere
★ DONIC: Generalist Dexterous Bimanual Tracking Control from Human Demonstrations
Despite recent progress in dexterous reference tracking, existing models focus primarily on single-hand manipulation and a limited range of interactions, leaving the bimanual dexterous behaviors common in real life unaddressed. The scarcity of contact-rich bimanual manipulation data and the increased complexity of coordinating contacts make developing a generalist bimanual tracker particularly challenging. Therefore, we present 1) DONIC, the first generalist bimanual tracker jointly trained across operation categories and human-motion sources, together with 2) BiHOI, a multi-view dataset of 1,575 sequences spanning 229 objects and emphasizing sustained bimanual interaction, where reconstructed interactions are retargeted into robot references. DONIC combines proximal policy optimization with behavioral cloning in a feedback loop that recycles verified successful executions into training demonstrations. An object-centered per-hand reward maintains object-motion feedback while supervising each hand according to its reference contact role. Clip-Centered Advantages remove reference-dependent advantage offsets while preserving within-trajectory differences. Together, these designs enable one single policy to learn coordinated manipulation across heterogeneous references. Without finetuning, a single checkpoint achieves 86.79% success on unseen trajectories of training objects and 57.32% on held-out objects, surpassing the state-of-the-art bimanual tracking baselines by 77.36 and 45.73 percentage points.
★ Object- and Task-Aware Active Perception for Safe Navigation of Mobile Manipulators in Semi-Static Environments
Mobile manipulators must navigate safely in cluttered, partially unknown environments across logistics, manufacturing, and service applications. Semi-static changes between robot visits can lead to outdated scene knowledge and increased collision risk. We propose an active perception framework that enables a high degree-of-freedom (DoF) mobile manipulator to inspect uncertain or safety-critical regions with its hand camera during navigation. A probabilistic, object- and task-aware next-best-view (NBV) formulation prioritizes observations near the desired path and around uncertain or potentially changing objects. The perception objective is integrated into hierarchical-task model predictive control (HTMPC), exploiting kinematic redundancy for viewpoint adjustment while preserving the higher priority of base-trajectory tracking. The system unifies navigation, map maintenance, and active perception in a closed loop. Real-world tests show improved map consistency, reduced task-dependent map entropy, and lower navigation risk than passive or non-task-aware baselines. Additional simulation and real-world sequences demonstrate responses to controlled semi-static scene changes.
★ HuMiT: Low-Latency Whole-Body Humanoid Teleoperation via Compact Reference Tracking
Real-time whole-body humanoid teleoperation remains challenging due to the computational latency introduced by optimization-based retargeting pipelines and future-frame conditioning. Existing motion tracking policies typically require dense reference targets including full joint positions, body-part transformations, and even future frames, which tightly couples the controller to expensive upstream modules. In this work, we present HuMiT, a whole-body teleoperation system built on a unified tracking policy that requires only a compact reference target, consisting of root height, root velocity, and the SE(3) poses of four sparse keypoints at the current frame, yet achieves competitive tracking fidelity compared to methods relying on more diverse reference states including complete joint positions, dense body-part transformations, and even future frames. Experiments show that our policy achieves a communication-to-inference latency of $\sim$20 ms measured between headset streaming and policy output and a video-based optical flow latency of 65 ms (median) across different motion types. We validate the system on a Unitree G1 humanoid, showcasing split-second reactive responses such as catching a tossed ball, as well as coordinated loco-manipulation capabilities. Project page: https://xiaohu-art.github.io/Track/.
comment: 7 pages, 3 figures, 4 tables. Project page: https://xiaohu-art.github.io/Track/
★ RangeNav: Task-Aligned LiDAR Range-Image Pretraining for Robot Navigation ICRA 2027
Autonomous mobile robots operating in dense crowds and cluttered sites require $360^\circ$ perception to react to surrounding hazards. While panoramic LiDAR offers full-surround coverage, learning representations within reinforcement learning (RL) is sample-inefficient under sparse rewards, and existing 3D pretrained point-cloud encoders are too memory-heavy for massively parallel RL simulation. We present RangeNav, a compact (183k parameters) LiDAR range-image encoder pretrained entirely on synthetic data. RangeNav uses anisotropic convolutions and circular padding to handle the extreme aspect ratio and azimuthal seam of range images. We pretrain the network using a navigation-aligned objective combining range reconstruction, semantic segmentation, and bird's-eye-view (BEV) prediction, then freeze it for policy learning. In simulation, adding semantic and BEV supervision increases success rates over a reconstruction-only baseline by an average of 32.03 and 23.73 percentage points (pp) in procedurally generated training and out-of-distribution realistic test environments, respectively. Zero-shot real-world deployments on a Unitree B2-W quadruped demonstrate robust navigation through dense crowds, low-light settings, and construction zones where limited-field-of-view depth policies struggle. Code and models will be open-sourced.
comment: 9 pages, 6 figures, 5 tables. Submitted to ICRA 2027; under review
★ Spacetime Splines for Adaptive Robot Motions
Movement primitives commonly represent robot mo- tions through basis functions indexed by time or phase. Inspired by spacetime representations employed in physics, we propose a spline-based formulation in which spatial geometry and temporal progression are represented within a common set of continuous variables. The proposed method jointly estimates a shared Bezier curve, while allowing spatial and temporal model complexity to be adapted independently (e.g., demonstration-specific phase modulation). As use case, we consider an alternating optimization process that simultaneously estimates path intervals observed in partial demonstrations. Simulation experiments demonstrate accurate joint recovery, refinement through modifications in model complexity and continuity, and improved robustness to variance in the demonstration set. Finally, we demonstrate online retiming and synchronization capabilities in a patient-turning task assisted by a robot, evaluated with a physical manikin.
★ Robot Generalization Beyond Volume: The Diversity Path via Zero-Shot Capabilities
General-purpose robots must handle tasks, objects and environments they were never trained on, yet zero-shot generalization remains one of the least solved problems in physical AI. In digital AI, zero-shot capability grew with the volume and, above all, the diversity of training data, much of it sourced from the crowd. Robotics instead relies on paid collection by a few well-funded teams, which yields volume without diversity. We formulate zero-shot generalization in physical AI as coverage of the deployment distribution, and show that more samples from the same collectors shrink the unseen region only slowly, while each new contributor can extend it. Building on this and on zero-shot learning, we propose a shared latent space for robot actions that combines multi-modal alignment, generative augmentation, and noisy-label learning. As a scalable source of diverse data, we propose a community of developers building applications on one standard, approachable robot, Origami. We close with a testable hypothesis and an evaluation protocol relating real-robot zero-shot success to the diversity of contributed data.
comment: 13 pages, 7 figures, 2 tables, including supplementary material
★ OMAR: Open-World Multi-Attribute Reasoning for Scene-Conditioned Shelf Organization
With the advent of foundation models, robots are increasingly capable of understanding and executing explicitly specified human tasks. Extending these capabilities to open-ended tasks, however, requires robots to infer appropriate actions from the observed scene. Household organization exemplifies this challenge, as robots must often act without detailed instructions, predefined object vocabularies, or known object-receptacle associations. While robots may follow instructions or learn organization preferences from demonstrations, inferring the latent organization of an occasionally changing scene and placing a novel object consistently with it remains challenging. We propose an open-world multi-attribute reasoning pipeline that grounds unknown objects across views and constructs an instance-level 3D scene graph for placement reasoning. First, a vision-language model proposes labels together with attributes such as content and packaging type, which are grounded through segmentation, depth, and multi-view association. Then, an LLM reasons over this representation to infer the latent shelf organization and rank scene-consistent object placements without explicitly user-provided preferences. The resulting placement rankings guide subsequent geometric and kinematic feasibility checks for execution. Our experimental evaluation shows that our approach leads to improved instance mapping and placement coherence over single-view language-grounding and geometry-only baselines, enabling autonomous placement of previously unseen objects without predefined object-receptacle mappings or class-specific retraining.
★ Post-hoc Epistemic Uncertainty for Vision-Language-Action Policies
Vision-language-action (VLA) policies need to recognize gaps in their training knowledge to request useful demonstrations or assistance in new situations, especially when failures are costly. Disagreement within an ensemble of independently trained policies can guide these decisions, but requires training and running multiple policies. We propose post-hoc kernel-variance heads that estimate epistemic uncertainty from the embeddings of a single policy. Each head computes the conditional variance of a Gaussian process conditioned on embeddings from the training demonstrations. Fitting requires no regression targets or gradient updates and leaves the policy unchanged. In experiments with SmolVLA on LIBERO-10, the kernel variance heads perform competitively with two-policy ensemble scores in task-level correlation with task success, in demonstration acquisition during active learning, and in referral to a stronger supervisor policy as a proxy for human assistance. For SmolVLA, the head on the vision-language backbone reduces uncertainty-scoring time by up to approximately a factor of 50 compared with ensemble scores, while also substantially reducing setup costs. We also show that the established evaluation protocol on LIBERO-10 primarily rewards recognition of coverage differences between tasks, while execution variability weakens assessment of within-task rankings. Thus, controlled perturbations provide a complementary evaluation: modality-specific heads identify unfamiliar camera, instruction, or state inputs, while fused heads can detect unfamiliar combinations of individually familiar inputs. With a supervisor proxy trained on the evaluated perturbation families, the decoder head improves within-task referral over ensemble disagreement. These findings show that useful uncertainty estimates for adaptive robot learning can be obtained from a single policy.
comment: 30 pages, 5 figures, 11 tables. Code: https://github.com/post-hoc-vla-uncertainty/post-hoc-vla-uncertainty
★ Sufficient and Necessary Causal Explanations for the Analysis of Autonomous Systems
Understanding failures of autonomous systems is challenging, particularly when their behavior depends on black-box components such as neural networks. In this work, we make a distinction between two objectives of failure analysis: reproduction, which seeks to identify conditions sufficient to reproduce an observed failure, and avoidance, which seeks to identify the changes required to prevent it. Previous work on causal explanations of system failures is based on the concept of sufficient explanations and naturally supports reproduction. Nevertheless, we observe that it does not address the problem of avoiding the failure. In this paper, we introduce and formalize the concept of necessary explanations for system analysis as the correct formalism for failure avoidance. We place necessary explanations in the framework of actual causality and show that, together with sufficient explanations, they provide a unified causal framework for addressing both objectives of failure analysis. We present an efficient approximation algorithm for generating necessary explanations based on the notion of causal responsibility. We evaluate the efficiency of our approach across multiple environments. We demonstrate the practical utility of necessary explanations for failure avoidance through counterfactual generation and show that it outperforms existing methods.
★ Reconstruction-Augmented Visuomotor Pre-Training of 3D-Grounded Bimanual Dexterity from RGB-Only Human Videos
We present SpatialDex, a framework that learns 3D-grounded bimanual dexterity from RGB-only human videos through reconstruction-augmented visuomotor pre-training. The central idea is to learn human behavior together with its spatial context. From each video, we reconstruct the scene geometry and estimate the hand motion, align them at a shared video-level scale using the hand-size prior of 3D hand estimates. We retain the reconstructed geometry as both a policy input and a prediction target. The policy grounds the token grid of a pre-trained vision-language model in this geometry through gated residual fusion, is trained to predict future scene geometry alongside wrist and finger actions, and generates finger motion conditioned on the predicted trajectories of both wrists. Pre-trained on 1M human-video clips and fine-tuned with a small number of robot demonstrations, SpatialDex outperforms the state-of-the-art methods by 16.5 points on nine simulated dexterous tasks across two embodiments and by 38.3 points on six real-world bimanual dexterous tasks. Under a matched comparison, pre-training with reconstructed geometry improves success by 14.2 points over pre-training on hand motion alone.
comment: Project page: https://spatial-dex.github.io/
★ CausGround: Causal Interaction Grounding for Adaptive Embodied Planning
Multimodal large language models (MLLMs) show promising reasoning capabilities, but reliable physical interaction requires planning that accounts for when actions are executable and how they change the world. These preconditions and effects are environment-dependent and often not captured by MLLMs, making adaptation challenging. Yet existing approaches either require costly environment-specific training or rely on unstructured interaction experience that is difficult to reuse. Therefore, we introduce CausGround, a non-parametric causal interaction grounding framework that distills interaction experience into reusable action-centered causal knowledge. CausGround identifies action preconditions and effects and organizes them in a factorized world-state representation capturing object attributes and inter-object relations. At planning time, task-relevant causal dependencies are grounded in current observations to support backward reasoning over preconditions and forward reasoning over effects for action selection and multi-step planning. We evaluate CausGround on simulated high-level planning, interaction reasoning, and real-world robot manipulation. On adapted VirtualHome and RoboCasa benchmarks, CausGround improves vision-based high-level planning across MLLM backbones, increasing average task success from 24.9% to 53.3% on VirtualHome and closed-loop planning success from 43.0% to 57.0% on RoboCasa. On SWITCH, average interaction reasoning accuracy increases from 25.8% to 31.9%, indicating improved action selection, recovery, and verification. Moreover, real-world experiments show that CausGround reasons backward from MLLM-interpreted goals to derive executable high-level action sequences for sequential manipulation. These results demonstrate that reusable action preconditions and effects support long-horizon embodied planning across diverse environments without parameter updates.
comment: Project Page: https://causground.github.io
★ AirPlayground: Benchmarking Aerial Navigation across Diverse Reconstructed Environments
Large-scale aerial resources provide a foundation for learning navigation across diverse environments. However, existing resources remain limited in providing (1) densely sampled flight demonstrations for learning different policy formulations and (2) broad environment coverage for standardized closed-loop evaluation. To address these limitations, we introduce AirPlayground, a unified benchmark for training and evaluating aerial navigation policies across diverse environments. We first construct 94 photorealistic 3D Gaussian Splatting (3DGS) environments spanning indoor, urban outdoor, and vegetation settings, with aligned collision geometry. Building on these environments, we develop a scene-adaptive pipeline for automated expert flight collection, yielding 94K demonstrations and 46.5M synchronized observation-state-action samples at 50 Hz. We further provide an extensible API that supports training different policy formulations on this shared corpus and evaluating them under unified protocols in the corresponding environments. We benchmark seven baselines spanning four policy paradigms across three navigation settings in seen and unseen environments, and further examine scaling behavior with ACT. Scaling ACT's training corpus from 5 to 70 scenes raises median success rates on seen and unseen environments from 30.6% and 16.8% to 41.2% and 29.4%, respectively. Through this shared training and evaluation framework, AirPlayground supports analysis of how offline prediction errors relate to closed-loop outcomes. Our analyses highlight the roles of prediction errors, execution protocols, and scene interactions in closed-loop performance.
★ FlowVLA: Learning Flow-Aware Physical Completion for Quantitative Robotic Pouring
During steady pouring, images and robot pose may remain nearly unchanged while transferred mass accumulates, making the return decision ambiguous for a whole-task vision-language-action (VLA) policy. We present FlowVLA, which separates motor generation from physical-completion estimation. A shared $π_{0.5}$ action expert learns approach, pour, and return with boundary-aware supervision. A target-independent estimator fuses visual, joint, wrist-wrench, and recurrent force-belief histories to maintain an estimate of transferred mass. The controller initiates return when this estimate plus predicted residual outflow reaches the requested mass. In coffee-pot/water experiments, FlowVLA achieves 13.1 g goal-balanced mean absolute error (MacroMAE), lower than all tested VLA baselines. Across all attempts, 53.7% both complete the task and finish within +/-10% of the requested mass. The same model interpolates to held-out 150/250-g targets without retraining. Independently trained models further evaluate the formulation in three additional container-fluid settings.
comment: 9 pages, 6 figures
★ Toward Recursive Self-Improvement in Physical AI
Recursive self-improvement (RSI) describes a system that improves not only its behavior but also the process by which it improves. Recent digital agents test this idea by rewriting their own code and checking each version with executable tests. Bringing these mechanisms into the physical world requires learning from partial observations through costly trials, while accounting for changing operating conditions and potentially irreversible consequences. We position RSI within robot learning and trace pathways from improving robot capabilities to improving the processes that generate them. We examine how physical experience can strengthen these improvement mechanisms and when the resulting gains can support further self-improvement of the mechanisms themselves. Gains carry forward only when feedback addresses remaining learning needs, the receiving component can use the change, its value survives changes in tasks or hardware, and continued practice remains safe and affordable. Existing studies, concentrated in manipulation, support behavioral improvement, benefits to later learning, and procedure improvement. Some systems also change the improver's knowledge or tools, but the physical studies reviewed here have yet to isolate gains attributable to recursion. We synthesize these findings into evaluation principles and research directions for machines that, trial by trial, safely improve the way they improve.
comment: 93 pages, 15 figures, 21 tables. Project page: https://physical2rsi.github.io/
★ DriveMLP: When Training Strategy Outperforms Model Complexity in Autonomous Driving
Learning motion planning policies that remain reliable under closed-loop execution is a central challenge in Autonomous Driving (AD). Imitation Learning (IL) scales well to large AD datasets but suffers from distribution shift, causal confusion, and compounding errors under closed-loop execution. In contrast, Reinforcement Learning (RL) typically requires handcrafted rule-based rewards and massive simulation. We study a simple closed-loop IL framework that uses RL purely as an optimization tool, guided by a trajectory-distance reward derived from logged expert data. We refer to this training strategy as Expert-Guided RL (EGRL). Through a controlled empirical study we find evidence that the training procedure, rather than model size and complexity, accounts for a substantial part of the performance gains. We then introduce DriveMLP, a lightweight MLP policy trained with EGRL, as a simple instantiation of these findings. It achieves a score of 86.51 on the nuPlan Val14 Reactive benchmark, a +3.2 point improvement over the previous best IL planner (Flow Planner), with >15x faster inference. Code is available at: https://github.com/alessandro-canevaro/DriveMLP .
comment: Accepted at 2026 Conference on Robot Learning (CoRL)
★ From Acceptance and Avoidance to Affective Viscosity: Designing Social Agents Intermediating in Human Relationships
As social agents become embedded in human relationships, they are expected not only to provide smooth assistance but also to handle rejection, discomfort, jealousy, persistence, and other forms of social tension. This paper connects two research directions: acceptance and avoidance in response to an agent's gaze, approach, and contact, and affective viscosity as a model of how negative affect accumulates, persists, is suppressed, and is expressed over time. We argue that avoidance should not be treated merely as a failure of interaction, but as a relational signal that indicates boundaries between humans and agents. Building on this view, affective viscosity provides a design perspective for agents that maintain, suppress, or express negative states depending on social relationships and interaction histories. We discuss several ongoing design cases, including triadic jealousy, discomfort suppression, patience timing, and spatiotemporal stickiness, and consider how these cases lead to broader questions of blame and accountable intervention. Rather than designing agents as simple scapegoats or artificial villains, we ask how agents can intermediate in problematic social situations while making their intervention traceable, limited, contestable, and repairable.
comment: The 2nd International Workshop on Trust and Blame in Multi-Person Multi-Agent Societies (TBSA_2nd) in HAI 2026, 5 pages, 2 figures
★ Reading the Room: Scale-ambiguous Semantic Registration of Sparse VSLAM Maps to Floor Plans in Partially Built Environments
Many robotic construction applications, such as progress tracking, require registering a sensor trajectory to a prior 2D floor plan. Visual capture is attractive for its low cost and availability, but aligning a visual simultaneous localization and mapping (VSLAM) map to a floor plan is difficult; VSLAM recovers geometry only up to an unknown similarity transform, the sparse point cloud is cross-modal to the symbolic floor plan, sites are only partially built during capture, and construction sites are visually repetitive, degrading map quality while removing distinctive features. Existing methods address at most two of these challenges. We thus introduce Reading the Room(RR), a global-local approach to register scale-ambiguous, sparse, semantic VSLAM maps to partially built floor plans. Our key idea is to define a single robust, differentiable, class-balanced semantic-geometric energy function that governs all stages of the pipeline. We develop two global initialization schemes, leveraging a single inaccurate user-provided correspondence if provided, or estimating the initialization globally using a cross-modal matching network. Both schemes are designed to find initial candidates with respect to the energy function, which is then directly optimized for local refinement. We evaluate RR on 33 sequences across three active construction sites against classical and learning-based baselines, achieving a 4.5x improvement in mean absolute trajectory error over the strongest baseline in the general unconstrained setting, which increases to 7.6x when an anchor is provided.
★ Watch Your Step: Anticipated Perception for Humanoid Locomotion
Perceptive locomotion policies typically observe a dense robot-centric elevation map, although only the terrain beneath and around the next footsteps informs the control decision. Learned encoders compress the whole visual input, leaving the policy to work out which inputs matter while learning the baseline locomotion task. Learning both at once slows training and, without computationally expensive attention mechanisms, makes generalization to new terrain harder. We instead remove redundant inputs from the elevation map before the policy sees it. A lightweight foothold prediction model (FPM) predicts future footholds from proprioception and the velocity command alone, and we sample the elevation map only in small patches around these predicted footholds. We train the prediction model alongside the locomotion policy, periodically retraining it on new policy rollouts as the robot's gait matures. This allows the perception to anticipate where the policy will step, while the policy learns to leverage terrain information around those steps. With the proposed lightweight preprocessing, the policy reaches a given terrain difficulty in roughly a third of the training iterations required with the complete map. It outperforms policies that observe the complete map where footholds are sparse, including in narrow walled corridors, and on unseen terrain, and it matches an attention-based encoder trained at several times the cost. The policy transfers zero-shot to a Unitree G1 humanoid traversing various obstacles, stairs, gaps, and step-downs beyond the training distribution in both indoor and outdoor environments.
comment: 8 pages, 5 figures, 3 tables. Project page: https://leggedrobotics.github.io/watch-your-step
★ End-to-End Reinforcement Learning for Quadrupedal Jumping in Lunar Gravity
Legged robots can extend planetary exploration to rugged and discontinuous terrain that is inaccessible to conventional rovers. In lunar gravity, jumping further expands this mobility by enabling robots to cross obstacles and gaps that cannot be traversed by walking alone. We present a multistage reinforcement-learning approach that produces a single end-to-end control policy for commanded point-to-point jumping under lunar gravity. The resulting policy has a simple architecture and relies exclusively on mission-compatible onboard sensing, making its deployment on a real lunar mission plausible. In simulation, the learned policy performs accurate point-to-point jumps with landing errors smaller than the robot's body size. We analyze the torque and power demands of lunar jumping and characterize performance degradation under extreme outlier events, including control-update freezes of up to \SI{1}{s} during flight. Hardware transfer demonstrates that the policy can successfully execute a vertical jump under lunar-gravity offloading. Furthermore, the measured torque and power profiles are consistent with the simulation results and remain within the robot's hardware limits.
★ Joint-Angle Invariance of Jacobian-Based Performance Indices for Serial Robotic Manipulators IROS 2026
This article investigates the joint-angle invariance (JAI) of Jacobian-based performance indices for serial robotic manipulators. We identify kinematic conditions under which representative performance indices, including manipulability, stiffness, and other Jacobian-derived indices, remain independent of the first and last joint angles. This property allows a systematic reduction in the effective configuration-space dimensionality, thereby simplifying the performance evaluation. The utility of JAI is demonstrated through two application scenarios: workspace performance map construction and task layout optimization. Comparative results show that the proposed JAI-based strategy reduces computational effort and storage requirements without compromising the effectiveness.
comment: 9 pages, 6 figures, 4 tables. Accepted at the 2026 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2026)
★ Mixture of Flow Matching and Particle Filter for Proprioception-Based Online Contact Estimation
In this article, we propose ContactFlowFilter, a novel approach for contact estimation that determines the number of active contacts, their locations, and the associated force vectors. We consider a robot manipulator equipped with joint torque sensors and a base force/torque sensor; such a proprioceptive-only setup presents a challenging inverse problem characterized by (i) multi-modality from non-unique solutions, (ii) temporal dependencies in state evolution, and (iii) discrete contact events involving contact addition/removal. Existing approaches, such as particle filters (PFs) and learning-based methods, often struggle to address these challenges simultaneously. To overcome this, ContactFlowFilter integrates a generative model into the PF framework. The recursive structure of the PF captures temporal dependencies, while the flow matching model serves as an auxiliary proposal distribution to track multi-modal beliefs and detect contact events. Extensive validation in both simulation and real-world experiments confirms the effectiveness of the proposed method. In real-world settings, ContactFlowFilter achieves the lowest dissimilarity from the target distribution, with errors of only 0.36 cm and 0.88 cm for single- and dual-contact scenarios, respectively, while operating at 0.69 ms and 2.69 ms.
comment: Author accepted manuscript; 21 pages (including a cover page), 21 figures, 5 tables
★ Preserving Operator Intent Under Actuation Constraints: Teleoperation of Manipulators with Coupled Transmissions IROS 2026
Teleoperation enables intuitive control of remote engineering robots for precision tasks. Existing direct joint-mapping methods have two limitations: independent motor saturation disrupts pitch-roll coordination in differential wrists, while fixed mappings struggle with abrupt tracking changes caused by gravitational loads and actuation constraints. We propose an actuator-constraint-aware leader-follower teleoperation method that jointly models leader gravity compensation, leader-follower joint mapping, and follower differential-wrist control. Specifically, we perform coordinated saturation in motor space according to the available gravity margin and integrate continuous angle unwrapping, feedback-state monitoring, and feasible-reference updates. This preserves the operator's motion direction, reduces coordination distortion, and suppresses recovery transients during saturation or brief communication failures. Real-robot experiments demonstrate smoother wrist trajectories and more continuous recovery than direct mapping.
comment: 6 pages, 5 figures. Contribution to the IROS 2026 workshop "5th Sensorimotor-Augmented Teleoperation: Human-Interaction-Oriented Paradigm and Real-World Concerns"
★ From VLM Planners to Robotic Agents: Self-Evolving Harnesses for History-Dependent Manipulation
Vision-language-action (VLA) models achieve strong performance across embodied manipulation tasks by unifying visual understanding, language conditioning, and action generation. However, long-horizon tasks can involve non-Markovian dependencies: successful decisions may require retaining task progress and past interaction outcomes, while partial observability and execution errors compound over time. Existing work has begun to address these challenges by augmenting foundation-model policies with memory and by introducing hierarchical architectures that separate high-level planning from low-level execution. We organize the high-level VLM as a robotic agent through an editable harness to maintain task context and improve planning through execution experience. We instantiate this approach as SERA (Self-Evolving Robotic Agent), with a fixed VLA executor providing low-level control. The harness maintains and updates task context using observations and execution feedback to guide subsequent planning. Self-evolution uses environment rollouts to propose and evaluate harness revisions, with the selected configuration fixed before held-out testing. On RoboMME, SERA achieves 51.38% overall success with autonomous selection, compared with 44.38% without self-evolution. Our exploration suggests that agentic planning is a promising direction for history-dependent robotic manipulation.
★ Make it Walk: Computational Design of LEGO Walkers
LEGO walkers are walking automata constructed from modular LEGO Technic bricks. They exhibit intriguing locomotion and are also highly reconfigurable through the modular Technic bricks, making them widely applicable in education, recreation, and robotics research. However, existing LEGO walkers are designed mostly by reusing existing single-actuator walking mechanisms. Although the locomotion is stable, their fixed walking gaits limit their adaptability and versatility. This paper presents the first computational solution capable of designing LEGO walkers that not only walk stably, but also support a variety of walking gaits. Our idea is twofold. First, we control the foot motion trajectory of each leg mechanically via a specifically-designed gear-linkage mechanism, in which we incorporate a pair of 3D-printed non-circular gears to work with a LEGO Technic mechanism. Second, we drive each leg by an independent motor and synchronize the timings of multiple leg motions via a simple yet adaptable control algorithm, enabling stable and versatile locomotion. Importantly, we take this modeling as a foundation to co-design the morphology and control of the LEGO walkers to meet the target locomotion and appearance. Leveraging our approach, we created a variety of LEGO walkers, demonstrating various walking gaits and appearances, and fabricated some of them to show their locomotion capability in the real world.
★ S-Rolling: Steerable Rolling Gaits for Twistable Snake Robots in Low-Clearance Environments
Snake robots can traverse complex environments with limbless locomotion, but maneuvering in low-clearance environments remains challenging because a gait must maintain a small swept height while also providing planar steering. Existing rolling gaits preserve a low swept height body configuration but generally provide limited steering, whereas steering gaits often introduce vertical undulation and require additional clearance. With that in mind, this paper develops S-rolling, a steerable rolling gait that combines low swept height locomotion with planar maneuverability for twistable snake robots. S-rolling is designed using an S-shaped planar backbone curve, whose head and tail segments roll with independently controlled angular velocities, producing different motion modes. We derive its joint space gait equation, in which a central twist joint realizes the relative rotation between the two segments. We also develop a quasi-static kinematic model relating the segment rolling speeds to the linear and angular velocities of the robot. Real-robot experiments demonstrate the various motion modes, traversal through a 10 cm clearance test track, and maneuvering tasks beneath a large workpiece and a car chassis. In these low-clearance scenarios, S-rolling successfully performs approximately 90$^\circ$ and 180$^\circ$ heading changes, whereas the baseline rolling arc gait cannot change the robot heading.
★ Extending Affordance Segmentation for Wearable Robots to Multi-Object Images
This paper proposes a refinement to existing vision-based portable pipelines for the recognition of graspable object parts. Such pipelines can effectively support automatic grasp selection and preshaping in upper-limb prosthetic devices. However, realistic scenarios in which multiple objects are present still represent a challenge. The proposed method exploits a minor modification to the conventional hardware setup and a custom training approach to extend the applicability of vision-based pipelines. A laser pointer is introduced in the setup as an effective mechanism for object disambiguation. Accordingly, a suitable retraining of the deep convolutional neural networks supporting recognition of graspable parts is implemented. A mixed reality strategy allows the reuse of available high-quality segmentation masks to generate realistic images procedurally, extending the range of scenarios supported by the final device. Experimental results confirm that the proposed pipeline improves target disambiguation accuracy by up to 9% compared with existing solutions, while preserving the computational cost.
★ Linear Twisting Sidewinding Gaits for Twistable Snake Robot Motion Planning on Complex Terrains
Sidewinding is an efficient gait for snake robots to locomote on various terrains, by undulating the backbone curve and shifting the contact segments along it. However, relying solely on undulatory motion limits the locomotive capability of snake robots, and combining it with twisting motion can improve their speed and robustness. This hypothesis has recently been verified by the rolling vector approach, which locomotes the robot like a differential-drive car. Yet its twisting is concentrated in a short buffer segment, imposing most of the workload on the two twist joints in the middle, which can often overburden the motors and limit its locomotion. With that in mind, this paper develops a new family of twisting sidewinding gaits that distribute the workload over the entire body with our proposed linear twisting function. Under matched turning speed and total output power, our gaits reduce the peak output power of a single twist joint by 42%. Real-robot experiments show that, with the reduced workload, our new gaits can locomote the robot in various terrains that the existing rolling vector approach cannot. Besides, our new gaits can move up to 49% faster than conventional sidewinding without twisting.
★ Distance Field Driven Reinforcement Learning for Reactive and Robust Collision Avoidance IROS
Robust and reactive collision avoidance remains a central challenge for deploying Reinforcement Learning (RL) policies on real robotic manipulators, particularly when transferring behaviours learned in simulation to the physical world. This paper presents an RL framework that integrates the Euclidean Distance Field (EDF) directly into the observation space, providing the agent with continuous geometric cues that support stable and responsive avoidance behaviours. By leveraging the rich spatial information encoded in the EDF, the method enables a streamlined reward formulation that achieves effective obstacle avoidance both with and without Artificial Potential Field guidance. The proposed approach is validated on a 7-DoF redundant manipulator, demonstrating zero-shot transfer from simulation to real hardware. Despite being trained exclusively with randomised static objects, the deployed policy generalizes to previously unseen real environments and exhibits robust online avoidance of dynamic obstacles. These results highlight the potential of EDF-based representations to mitigate the sim-to-real gap and support reliable and real-time collision-aware behaviour in complex and changing environments.
comment: Accepted to 2026 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
★ A2I-Calib++: A Kinematics-Driven Anti-noise Active Calibration Framework for Whole-Body Multi-IMU Systems on Legged Robots
Recently, multi-node inertial measurement unit (multi-IMU) systems have shown promise for robust odometry in legged robots. Multi-IMU fusion can improve legged robot odometry but requires accurate spatiotemporal calibration among the sensors. Existing multi-IMU calibration methods, designed for aerial or handheld platforms, face challenges on legged robots because routine locomotion may provide insufficient excitation, while foot slips, impacts, and vibrations introduce measurement disturbances. To address these challenges, we propose A2I-Calib++, a kinematics-driven anti-noise active calibration framework for body-mounted IMUs covered by the robot kinematic model. It estimates temporal offsets, extrinsic rotations and translations, and inertial biases through temporal correlation search followed by analytic spatial and bias estimation, without artificial calibration targets or complex parameter tuning for the target IMU. A2I-Calib++ comprises three core modules: 1) a Noise-Isotropy-Inspired Trajectory supported by a finite-horizon conditioning proposition that generates sufficient excitation to limit noise sensitivity; 2) a constraint-aware Adversarial Motion Priors controller that combines the anti-noise excitation with a rolling-contact reward for contact-aware execution; and 3) a contact-constrained virtual IMU that fuses pre-calibrated calf IMUs and joint kinematics under a rolling-contact model to construct the calibration reference. Furthermore, evaluations on both simulation and real-world quadruped platforms demonstrate notably reduced noise sensitivity and simulation calibration errors compared to conventional methods, together with repeatable real-world calibration and improved proprioceptive odometry accuracy for legged robots.
★ PRISM: Learning Realistic Depth via Physics-Grounded Noise Disentanglement with Semantic-Geometric Collaboration
Real-world physical sensing exhibits complex, heterogeneous noise patterns that deviate significantly from idealized simulation, posing a fundamental bottleneck for sim-to-real transfer. Existing sensor modelings typically treat depth noise as a monolithic black-box process, overlooking the distinct physical mechanisms that govern different error modalities. In this work, we introduce a physics-grounded paradigm that disentangles monolithic noise into two complementary modalities: \textbf{sensing invalidation} and \textbf{measurement inaccuracy}, enabling a tailored treatment of noise sources based on their physical origins. Building on this insight, we propose \textbf{PRISM}, a tripartite framework that distills 3D Visual Foundation Model features as rich spatial-semantic priors for physics-based reasoning. To address the inherent sparsity and class imbalance of invalidation regions, we develop Hierarchical Positive-Prioritized Supervision, integrating multi-scale positive-weighted objectives with a positive-preserving dynamic hard mining strategy to enforce precise artifact delineation. Extensive benchmarks demonstrate that PRISM achieves state-of-the-art fidelity in noisy depth synthesis. Furthermore, downstream robotic experiments show that PRISM facilitates a \textbf{92.1\%} average success rate in the real world, marking a significant improvement over monolithic baselines. Our project page is available at: \resizebox{\linewidth}{!}{\href{https://shawnleung-sii.github.io/prism-icml2026/}{https://shawnleung-sii.github.io/prism-icml2026/}.}
★ Integrating Flow Matching and Model Predictive Control for Safe Robot Navigation
Learning-based path planners based on diffusion and flow matching can generate diverse trajectories from demonstrations but typically lack mechanisms for constraint satisfaction and collision avoidance during deployment. This work proposes a framework that combines flow-matching-based path planning with model predictive path-following control to integrate data-driven path generation with real-time, constraint-aware execution. The flow-matching model captures multimodal path distributions, while the predictive controller adapts motion online to satisfy state and input constraints and avoid obstacles using updated local sensor information. An event-triggered re-planning scheme incorporates predictive control solutions into subsequent path generation, promoting collision-free navigation in environments with previously unseen obstacles. Simulation results for mobile robot navigation demonstrate robust performance across cluttered environments. The framework provides a systematic approach to combining flow matching with predictive control for adaptive, data-driven navigation.
★ AOV-GS: Active Open-Vocabulary Semantic Mapping with 3D Gaussian Splatting
Robots deployed in human-centered environments need maps that support both geometric navigation and open-vocabulary semantic queries beyond a fixed label set. A key challenge is enriching such maps with semantic information without relying on manually annotated segmentation masks or closed-set semantic categories. Existing geometry-driven active mapping maximizes coverage but treats semantics as a passive by-product of the collected trajectory, whereas active semantic mapping typically depends on supervised semantic segmentation. We present AOV-GS, an active 3D Gaussian mapping system that guides exploration using frozen foundation models. During coarse exploration, AOV-GS re-ranks geometrically promising candidate views according to CLIP mask-embedding novelty computed from SAM instance segmentation, biasing exploration toward semantically informative regions. Frozen SAM and CLIP provide instance masks and language-aligned embeddings throughout mapping, while a lightweight scene-specific language field is fitted offline directly to the collected CLIP embeddings rather than manually annotated masks, enabling open-vocabulary text-query segmentation on novel views. In a controlled comparison using the same SAM-CLIP perception stack, AOV-GS improves average language-field mIoU over geometry-only active planning by 1.3 percentage points on Replica (34.6% on eight scenes) and by 1.8 points on Matterport3D (23.1% on five scenes). With CorrCLIP, AOV-GS reaches 35.6% on Replica, 2.0 points above the matched Geometry-based + CorrCLIP baseline. Novel-view PSNR remains comparable to the geometry-only baseline on both datasets while depth L1 improves on Replica. Code is available at https://github.com/OptimusFaber/AOV-GS.
comment: 17 pages, 7 figures, 7 tables. Published at AIST 2026
★ Payload generalization on feedforward control actions for trajectory tracking in robots with rigid and elastic joints
Industries employ robots for repetitive tasks due to their high precision and performance, typically achieved through feedforward control actions based on an accurate robot description. However, an exact dynamic model can be difficult to obtain, especially for robots with elastic elements or when the payload changes. Learning-based feedforward approaches are promising for trajectory tracking under model uncertainties, but typically face generalization issues when task parameters vary. This paper proposes a novel method to generalize the feedforward control action with respect to payload parameters, independently of the underlying controller. Firstly, the minimal number of example objects needed for payload generalization is derived. Then, an algorithm is proposed to execute the same trajectory with a new payload, given only its parameters, without knowledge of the robot's dynamic model and even in the presence of elastic joints. The effectiveness of the proposed algorithm is validated in simulations and experiments on tracking tasks with different robots and controllers. Two sets of experiments are performed: on a standard Franka Emika Panda robot, and on a Franka Emika Panda robot equipped with an elastic wrist.
comment: 18 pages, 17 figures. Accepted for publication in Control Engineering Practice
★ Plan First, Act Later: Streaming Reasoning for Executable Memory in Robot Control
Long-horizon robot manipulation requires a robot to remember task-relevant information beyond its short observation window. Existing memory-augmented policies typically retrieve relevant information from history and reason over it at execution time, placing a heavy reasoning burden on the high-level model. To address this challenge, we propose PAL (\textbf{P}lan first, \textbf{A}ct \textbf{L}ater), a hierarchical memory framework for vision-language robot manipulation. It maintains a short-term visual buffer, a semantic-event memory, and an executable \texttt{subtask\_list}. A high-level vision-language model (VLM) converts streaming visual observations into the \texttt{subtask\_list}. At execution time, the VLM selects the currently relevant subtask from the \texttt{subtask\_list} and passes it to a low-level vision-language-action (VLA) policy. To teach the model what to write to memory and when to update it, we introduce a two-stage data generation procedure that first expands sparse update supervision and then uses self-labeling with task-grounded verification to generate training data that better match the state distribution encountered during execution. We evaluate our approach on memory-dependent benchmarks and real-world manipulation tasks. On RoboMME, PAL achieves a \textbf{56.05\%} success rate, outperforming the strongest baseline, FrameSamp-Modul, at 44.51\%. In real-world experiments across four manipulation tasks, \pal{} reaches \textbf{60.83\%} overall success, surpassing MemER (45.00\%) and FrameSamp-Modul (28.33\%). These results show that proactively writing future-relevant subtasks into hierarchical memory improves long-horizon visual reasoning and robot control. Code and videos can be found on our website: https://qinghangliu.github.io/pal-webpage/
★ Compressing Onboard Language Models for Space Robots under a Behavioral-Fidelity Requirement
Language models are being explored as the interface through which space robots interpret operator commands and select among verified skills. Running them onboard calls for low-bit compression, and flight use adds requirements that generic post-training quantization leaves implicit: a fixed memory budget and storage format, a static artifact under configuration control, and behavior close to the full-precision model validated on the ground. We treat compression as a ground-segment step and study QSelect, which composes one static low-bit model from compatible AWQ, GPTQ, and AutoRound checkpoints by assigning each linear module to one quantizer family and scoring every substitution at the final next-token distribution. The default fidelity mode needs no labels or tuned thresholds and accepts a substitution only when the forward KL divergence from the BF16 reference decreases; an optional target-adapted mode adds answer likelihood on a small labeled set. On DeepSeek-R1-Distill-Qwen-7B, Llama-3.1-8B, and Llama-3.2-3B at 3 and 4 bits, composed models reach lower mean held-out perplexity than every single-family model in all six settings, and the fidelity mode lowers perplexity and KL further. In a matched scaffold, final-output scoring gives lower held-out KL than representation-similarity scoring in all 12 paired runs. With labels from training or development splits of the evaluated task families, the target-adapted mode raises 3-bit pooled accuracy on the two Llama models by 4.3 and 4.5 points over the best single-family model. At 3 bits, unquantized embedding and output layers make up 43-46% of the packed artifact. All evaluations use general-domain benchmarks on a ground GPU; space-domain tasks and flight hardware are left to future work.
★ Relational Geometry Forcing for World Action Models
World Action Models (WAMs) have shown strong potential for robot manipulation by jointly learning visual dynamics and executable actions. However, their video and action objectives do not explicitly constrain the 3D geometry that is crucial for precise manipulation. Recently, geometry-aware WAMs introduce depth or 4D prediction branches, whereas representation alignment methods supervise intermediate WAM features with pretrained 3D foundation models. Yet we find that such pointwise alignment benefits the video features but degrades action learning, since dense geometric feature targets overly restrict compact action features. To address this limitation, we propose Relational Geometry Forcing (RGF), which transfers geometric priors from a frozen 3D foundation model by aligning pairwise feature relations instead of pointwise features. Specifically, RGF applies three complementary relational objectives, where intra-frame and cross-view relations transfer scene geometry and multi-view correspondence to the video features, while action-temporal relations couple action dynamics to geometric scene evolution. Applied only during training, RGF requires no explicit geometry branch or extra inference module. Extensive experiments in simulation and the real world demonstrate the effectiveness of Relational Geometry Forcing, showing consistent gains across diverse manipulation scenarios and WAM paradigms.
★ Training in Deployment: Human-in-the-loop Online Post-Training for Robot Manipulation
Large pretrained robot foundation models provide strong priors for manipulation, yet adapting them to target tasks with offline teleoperated demonstrations often fails to cover the contact-rich, recovery, and precision-critical states encountered during deployment. Existing methods mitigate this issue by augmenting offline demonstrations with human-in-the-loop data or online reinforcement learning. Despite this, offline post-training adapts slowly because it cannot immediately absorb deployment experience, while online RL typically relies on value functions or auxiliary actor-critic models, increasing both training complexity and instability. We present TIDE (Training In DEployment), a human-in-the-loop online post-training system. TIDE introduces a provenance-conditioned supervised learning framework that updates the policy online from human interventions and policy-generated rollouts collected during deployment, without a learned critic, reward model, or action scorer. Across four real-world bimanual manipulation tasks, TIDE improves milestone-normalized task progress and binary full-task completion over the warm-start policy and matched additional offline demonstrations, and performs comparably to the fixed-policy collection baseline while making updated checkpoints available during deployment. Each task-specific online post-training session, from the start of deployment rollouts until the final checkpoint is available, completes in less than one hour while learning and checkpoint handling run asynchronously.
comment: 9 pages, 8 figures. Project page: https://logosroboticsgroup.github.io/TIDE/
★ DepState-JEPA: Learning Depth-Grounded State Representation for Visual Perception and Manipulation
Predictive representations for robot manipulation should connect scene geometry with the state changes induced by actions. Our intuition is that understanding what an action will change starts with understanding where things are: complementary RGB and depth observations ground a common scene representation from which to infer those changes. Guided by this intuition, we introduce \textbf{DepState-JEPA}, a dual-stream RGB--Depth joint-embedding predictive architecture that connects spatial understanding with action-conditioned prediction through a shared latent state, \textbf{DepState}. Built on V-JEPA~2.1, the framework uses depth both as a complementary observation stream and to ground both modalities in metric 3D space, aggregating shared information into DepState while retaining modality-private features for decoding. Ray-conditioned readouts predict local features without target depth; after an action-conditioned transition, the same decoders predict future RGB and depth features using the predicted state and fixed initial private features. Robot-state readouts provide recursive proprioceptive inputs; object-trajectory supervision and calibrated forward kinematics shape state learning through motion targets and joint--TCP consistency. Comparisons show stronger RGB-encoder transfer after RGB-D adaptation, feature completion, conditioned visual forecasting, and manipulation success. Within-model ablations identify contributions from depth content, metric coordinates, and future-state matching, with additional control gains from FK regularization.
★ Failure Modes of Vision-Language Models for Human Activity Understanding in Thermal HRI
Thermal cameras enable robots to perceive human activities in darkness while revealing less personally identifiable appearance information and fewer details of the surrounding environment than RGB cameras. As vision-language models (VLMs), originally developed for RGB imagery, are increasingly integrated into embodied systems for scene understanding and high-level planning, thermal sensing offers a promising way to extend activity perception to low-light and privacy-sensitive settings. However, the failure modes of VLMs in thermal human activity understanding for human-robot interaction (HRI) remain underexplored. We fill this gap by evaluating seven locally deployable VLMs, including models developed for physical AI and robotics, on two person-centric thermal datasets. Our diagnostic tasks examine fine-grained action recognition, activity timeline reconstruction, and the ability to withhold unsupported answers. Paired RGB recordings serve as a performance reference, while controlled variations in frame sampling, thermal input presentation, and reasoning are used to assess whether the observed errors can be mitigated. Our results show that models can recognize broad activity categories but struggle with fine-grained actions such as coughing, sneezing, and yawning. They also produce inaccurate event timelines and hallucinate actions involving objects. Simple thermal input preprocessing and additional reasoning offer only limited, model-dependent gains. Abstention behavior also varies across models even under explicit instructions, limiting their reliability for practical robotic use.
comment: 9 pages, 6 figures, 2 tables
★ NavBind: Diagnosing Instruction-Action Binding at Route Forks in Navigation VLAs
Vision-language-action (VLA) navigation models translate visual observations and instructions into actions. Navigation success assesses whether an agent reaches the goal. However, reaching the goal successfully does not establish whether the model genuinely follows the instructions or relies on visual cues or navigation history. We introduce NavBind to diagnose instruction-action binding: whether changing the instruction influences the probabilistic distribution of the actions, with the navigation context fixed. We construct 3,252 route-fork contexts by recombining route-aligned instruction segments from R2R-CE and RxR-CE. Each context pairs full instructions that share the description of the route already traveled but specify different branches. Both are evaluated under the same agent state and navigation history. We compare generated local action sequences, the likelihoods of fixed reference action sequences under both instructions, and changes in visual attention and attribution. Across the evaluated models, higher reported navigation success does not imply that the model more faithfully follows the instructions.
★ SMR: Learning streaming motion retargeting for humanoids from human semantic
Retargeting human motion to humanoids requires preserving motion semantics while accounting for the robot's physical constraints. Kinematic pipelines prescribe a robot trajectory through geometric matching, without establishing whether it can be executed under the robot's dynamics. Prior physics-supervised neural methods train bidirectional full-sequence models on physics-refined kinematic trajectories, remaining tied to prescribed kinematic solutions and requiring noncausal context. We propose SMR, a streaming neural motion retargeter learned from semantic-conditioned physics experts. We convert human motion into robot-aware task-space targets for body landmark positions, orientations, and velocities. Physics experts track these targets directly in simulation under contact, dynamics, and actuator constraints. We train a single strictly causal Transformer on their executed state trajectories that complete the motion and pass semantic and physical validity checks. At each incoming frame, the student uses only current and past human observations to generate a robot motion reference for a downstream whole-body controller. Experiments demonstrate that semantic-conditioned physics experts generate motion with higher semantic fidelity and physical validity than direct optimization-based retargeting, while the resulting streaming retargeter achieves improved closed-loop performance over kinematic and physics-refined baselines under a common whole-body controller.
comment: 17 pages, 6 figures. Project page: https://logosroboticsgroup.github.io/SMR/
★ How Representational Priors Shape Generalization in World Action Models
World action models (WAMs) are shown to substantially diverge under generalization performance based on their generative backbones. We investigate this gap through the representational geometry that WAMs inherit from their backbones and how policy training reorganizes it. Our central hypothesis is that a useful representational prior would structure geometry that separates nuisance information from the in-distribution manifold in advance of training as a policy. We quantify this with nuisance orthogonality; orthogonality of the feature under visual variation outside the empirical in-distribution manifold. Across various image and video backbones, we show that a higher orthogonality from the representational prior is associated with stronger robustness to visual variations after WAM training, enabling policy generalization measurable prospectively for world models. In addition, analyzing the resulting robust WAMs reveals nuisance suppression; relative suppression of feature displacement caused by such variations and robust action generation of the policy. Together, these findings provide a geometric perspective on how the representational prior of the world model shapes the generalization performance of WAMs.
★ SimPT: An Automated Simulation Framework for Intervention-Based VLA Post-Training
Human-in-the-loop (HIL) post-training improves vision-language-action (VLA) policies through expert corrections, but costly human supervision and repeated real-robot evaluation limit algorithm exploration and development. We introduce SimPT (Simulation-based Post-Training), a simulation framework that automates intervention-based VLA post-training by integrating policy rollouts, corrective data collection, policy training, and standalone evaluation. To automatically determine intervention timing and recovery strategies, we develop an Agent-Compiled Progress Graph that tracks task progress using privileged simulator state and retains stable snapshots during policy execution. After a rollout fails, a CuRobo-based expert attempts recovery from these snapshots, starting at more advanced task stages and falling back to earlier ones as needed to generate corrective trajectories. We implement and compare four post-training methods, RECAP, HG-DAgger, DPO+SFT, and FlowPRO, across seven RoboTwin manipulation tasks. Relative to a fixed-interval snapshot baseline, the Progress Graph reduces recovery attempts by 75.0% and increases recovery success from 93.4% to 98.9%. Using SimPT, we develop RECAP-CF-LPF, a refinement of RECAP that combines cross-fitting and low-pass filtering of value predictions. This refinement increases the overall success rate from 76.50% to 79.36% in held-out simulation trials. In real-robot HIL post-training, it increases the success rate from 72% to 96% across 25 scenarios of an Ethernet cable-insertion task. These results demonstrate the utility of SimPT for exploring and developing VLA post-training algorithms.
comment: 11 pages, 6 figures
★ State-Dependent Objective Coordination for End-to-End Balance and Navigation
Traditional robotic navigation systems separate mapping, planning, and low-level control through cascaded modules, introducing computational overhead and accumulated latency. End-to-end reinforcement learning offers a compact unified policy that maps onboard observations directly to control actions. However, on inherently unstable platforms such as two-wheeled inverted-pendulum robots, navigation progress and obstacle avoidance can conflict with attitude stabilization in a strongly state-dependent manner. We propose a lightweight state-dependent objective coordination method for integrated navigation and control. A pitch-conditioned scalarization coefficient $β(s)\in[0,1]$ adjusts the relative contributions of balance stabilization and navigation objectives during training, without extra control hierarchies or auxiliary attitude-stabilization controllers, such as proportional-integral-derivative (PID) or linear-quadratic regulator (LQR) controllers. To support zero-shot deployment, simulation training models execution delay, action hold, asynchronous and noisy observations, and dynamics uncertainty. Gradient analysis reveals persistent directional interference and state-dependent variation in the objectives' relative scales. Comparisons with alternative reward-weighting, constraint, training, and control-structure strategies show that PPO+$β(s)$, where PPO denotes proximal policy optimization, achieves 98% and 95% success rates with small attitude excursions on the fixed obstacle map and the randomly generated high-obstacle-density map, respectively. Zero-shot hardware experiments further demonstrate bounded attitude behavior and consistent obstacle avoidance using only onboard observations. Training and evaluation code will be made publicly available upon acceptance.
comment: 8 pages, 7 figures, 3 tables
★ Learning to Absorb Corrections: Critic-Weighted Residual Flow for Vision-Based Insertion
Vision-based insertion requires correcting alignment errors while retaining useful behavior learned from demonstrations. We present ACT--Flow, an online learning framework that couples an adaptive Action Chunking Transformer (ACT) with a conditional residual flow policy. Following anchor-guided warmup, ACT absorbs successful executed actions through behavioral cloning, while residual targets are recomputed relative to the updated prior. A chunk-level critic learns from successful and failed interactions and evaluates final composed actions. Its estimates weight generated residual candidates for flow matching, complemented by supervision from successful real executions. Candidate scoring is confined to training; deployment uses RGB observations and samples one residual per action chunk. We evaluate the framework on power-plug, USB, and key insertion, together with cable routing and needle threading. Results show a five-task mean success rate of 66% at 20,000 post-initialization interaction steps, exceeding the strongest baseline by 18 percentage points at this checkpoint. Evaluations at matched checkpoints with the residual disabled show ACT's mean success increasing from 16% to 76%. Component ablations further support the complementary roles of behavior absorption, value weighting, and successful-action supervision.
comment: 9 pages, 7 figures
★ Causal world models under physical parameter shifts
World models predict how embodied systems evolve under actions, but their accuracy can degrade when physical conditions move outside the training distribution. We study two components of causal modeling: which inputs a predictor may use, and how its mechanism computes their effects. In a synthetic benchmark with shortcut features, a frozen dependency mask derived from paired interventions reduces mean out-of-distribution (OOD) RMSE from 1.769 for a fully connected predictor to 0.697, close to the generator-oracle mask (0.694). For robot link-mass extrapolation in CausalWorld, with disjoint training and test mass ranges, we develop finger-local world models in which link masses enter inertia and dynamics-bias terms through explicit affine coefficients, actions act through either a learned torque network or the known PD controller, and learned residuals capture contact-related effects. Compared with a separately trained control whose mass inputs are fixed at training means, the known-PD model turns out-of-range masses into a consistent predictive gain: median time-averaged joint R2 over 150-step rollouts rises from 0.609 to 0.727, with improvement in all ten task-seed runs (exact two-sided sign test, p = 0.002). In a shared 100-step forecast, the learned-torque model is competitive with the strongest recurrent robotic world model (RWM) configuration (median joint R2 of 0.755 - 0.768 versus 0.752), whereas supplying out-of-range masses sharply degrades the MLP and recurrent predictors. These results indicate that how a physical parameter enters the dynamics mechanism determines whether its out-of-range value helps or harms prediction.
comment: 2 figures
★ PHD-Hand: A Printable Humanoid Dexterous Hand with Linkage-Driven 3-DOA Digits
Currently, dexterous humanoid robot hands are widely considered the final hardware bottleneck that hampers generalist robots from physically assisting people at home and work. As conventional tendon-driven designs suffer from high cost, demanding maintenance, poor reliability, and short durability, alternative linkage-driven designs have increasingly gained attention. Despite their potential to avoid the weaknesses of tendon-driven hands, integrating multiple degrees of actuation (DOA) in a single digit with linkage drivetrains is a challenge for design and fabrication. Only a few studies have managed to present functional humanoid hands that are completely linkage-driven and have at least three DOAs per digit without compromising actuator placement and other design preferences. Our work introduces a new linkage-driven hand featuring a unique flexure hinge. Together with elastic drive rods made of spring steel, the linkage drivetrains are significantly simplified. Unlike other linkage-driven designs of similar dexterity, our design does not require many miniature moving parts. As a result, the hand can be fabricated easily with a budget 3D printer. This not only significantly lowers the barrier to entry for using and developing dexterous humanoid robot hand functions, but also allows the hands to be sized much closer to regular human hands. The design offers desirable kinematic advantages as well. A prototype is evaluated for grasping and object manipulation. In addition, this paper presents our latest findings on kinematics and control of the hand.
comment: Submitted and Accepted at IEEE-RAS International Conference on Humanoid Robots (Humanoids 2026). 8 pages. Supplementary video: https://lnkd.in/p/er-NRfMn
★ Motion Retargeting with Spatiotemporal Preferences for Humanoid Loco-Manipulation
Human motion retargeting bridges the embodiment gap between humans and humanoid robots by converting human demonstrations into reference trajectories for whole-body control and loco-manipulation. For object-interaction tasks, successful execution depends on accurately preserving body--object relations at critical stages. However, uniform per-frame iteration budgets do not account for these localized accuracy requirements, limiting the computational efficiency of retargeting. To address this limitation, we present Spatiotemporal Retargeting (STR), which uses keyframes and key body parts to allocate optimization effort. Specifically, a vision-language model selects relevant motion signals and body parts, and keyframes are identified from changes in the selected signals. Based on these selections, STR assigns additional solver iterations to keyframes and higher interaction weights to the selected parts. To further improve contact fidelity, we introduce Mechanics-Aware Contact Anchoring (MCA) to refine object-surface contact positions using an approximate object-wrench model while retaining the selected surface directions. Building on the resulting references, we develop Spatiotemporal Reinforcement Learning (STRL), which combines adaptive key-stage sampling with a contact reward based on target proximity and object contact forces for downstream control. Simulation experiments on a Unitree G1 and real-world experiments on a Unitree R1 demonstrate that STR improves geometric contact coverage and continuity at lower computational cost, while the resulting references improve downstream task completion across diverse object-interaction motions.
★ Decision Making and Ride Comfort Evaluation Based on Lane-Change Prediction for Autonomous Vehicles
Predicting surrounding vehicles' lane-change intentions enables autonomous vehicles to anticipate potential hazards and respond proactively rather than relying solely on reactive maneuvers. This study investigates the benefits of integrating lane-change prediction, based on knowledge graph embeddings and Bayesian inference, into autonomous vehicle decision-making. Three driving strategies are evaluated in risky merging scenarios simulated in CARLA: reactive driving, prediction-based longitudinal action, and prediction-based lateral action. The proposed prediction model anticipates the surrounding vehicle's lane change approximately 3-4 seconds before its occurrence, allowing the ego vehicle to adjust its behavior accordingly. In the representative simulation records, reactive driving results in a collision accompanied by aggressive deceleration. In contrast, prediction-based longitudinal action initiates early braking to establish a safe gap, while prediction-based lateral action performs a lane change when a safe adjacent lane is available. Both predictive strategies avoid collisions in the evaluated records. Compared with reactive driving, peak deceleration before impact is reduced by 31.5% with longitudinal action and 91.0% with lateral action. Additionally, an ISO 2631-1-based horizontal vibration estimate is used to provide a ride comfort assessment. Overall, the results highlight the potential of lane-change prediction to support safer and smoother driving through proactive decision-making, with additional insights into ride comfort provided by vibration-based estimates.
★ PhysibleMDM: Enforcing Inference-Time Physical Feasibility Constraints on Motion Diffusion Models via Noise Optimization
Motion Diffusion Models (MDMs) can generate diverse and visually appealing human motion; yet, the resulting trajectories frequently violate physical constraints, limiting their downstream use in simulation and control. We present PhysibleMDM, a framework that promotes physical feasibility of MDM output by optimizing at inference time the diffusion noise without retraining the generative model. We introduce a set of surrogate feasibility constraints that can be evaluated without full trajectory simulation. These constraints combine kinematic checks (e.g., ground penetration, slip, and self-collision) with dynamic feasibility tests computed via an inverse-dynamics inner loop formulated as a cone program. We solve the resulting black-box noise optimization problem using the Separable Covariance Matrix Adaptation Evolution Strategy (Sep-CMA-ES), thereby avoiding reliance on computationally expensive and discontinuous gradients. An autoregressive scheme then chains optimized segments to synthesize long motion sequences. Across 100 motion styles on an SMPL humanoid, PhysibleMDM reduces mean ground penetration, self-collision penetration, and ground slip by 75%, 92%, and 74%, respectively, relative to unoptimized diffusion output. Across 11 sequences for a custom bipedal robot, it achieves median reductions of 89.5% in per-sequence mean acceleration-reconstruction error and 79.5% in the mean inverse-dynamics residual.
★ Externalise the Memory: A Pre-Registered Study of Where an Embodied Agent's Experience Should Live NeurIPS 2026
Should an embodied agent's experience be baked into its policy weights, or kept external and retrieved at inference? A memory-augmented teacher (a VLM that retrieves lessons, with their episode images, from its own past attempts) produces trajectories on a vision-conditioned reach task; from the same trajectories we distil LoRA student policies and evaluate them under matched protocols, including one that re-attaches the teacher's frozen retrieval buffer to a fixed set of student weights. We pre-registered the hypotheses, arms, tests, and analysis rules before training and before any p-value. Distilling the memory-augmented behaviour into weights beat an action-only control by +1 pp (n.s.); the registered test proves underpowered for its own minimum effect, so we withdraw - rather than refute - the memory-in-weights claim. Attaching retrieval to a fixed set of student weights added +34 pp (z=5.15), reaching the pooled memory-augmented teacher reference (49.3%); the pre-registered protocol contrast agrees (+23 pp, z=3.36); under a pre-registered auto-reset sensitivity count the two are +36 and +29 pp. An exploratory behavioural probe with pre-registered per-arm predictions is consistent with behavioural distillation transferring the marginal correction but not the conditional, situation-specific correction that retrieval supplies. The recipe that follows is to externalise the memory: in follow-up controls registered before they ran, the same frozen buffer lifts the base model with no adapter to 48% (sensitivity 48%) and the other three adapters from 19-26% to 53-55% (sensitivity 17-21% to 50-55%), and the adapter adds nothing detectable once the buffer is attached. On a second, harder task, the retrieval advantage points the same direction but falls short of significance (+6 pp, 95% CI [-4.5, +16.4]); exploratory diagnostics tie the attenuation to memory coverage.
comment: 28 pages. A 5-page version appears at the CL4FMAgents workshop, NeurIPS 2026 (non-archival). Code and pre-registration: https://github.com/TPEmist/AionGenos
★ Integrated Respiratory Sensing and Gated Control for Autonomous Needle Insertion Via CT Bore Robotic Arm
Currently, CT-guided lung biopsy is a highly manual procedure requiring patient breath-holding, iterative manual needle insertions, and repeated CT confirmation scans to accurately target a tumor. Automating this process within the CT bore--without relying on manual needle placement or patient breath-holding--has the potential to reduce procedure time, patient discomfort, and radiation exposure. In this work, we present an integrated sensing and control system capable of performing a gated needle insertion within the CT bore. The experiments demonstrate the system's ability to track and predict patient breath cycles, perform autonomous respiratory-gated needle insertion, and detect breath cycle anomalies.
★ Tactile-Based Robotic Manipulation: From Mechanics-Driven to Data-Driven Perspectives
Tactile sensing provides robots with direct access to physical interactions at the contact interface, making it a critical modality for robust and dexterous manipulation. Over the past several decades, tactile-based manipulation research has evolved from mechanics-driven approaches, which explicitly interpret tactile signals through contact mechanics and task-specific reasoning, to data-driven approaches that learn tactile representations and manipulation policies directly from data. This paper reviews the development of tactile-based manipulation from these two complementary perspectives. We first introduce the fundamentals of tactile sensing, and then review, from a mechanics-driven perspective, how those signals form manipulation-relevant state representations and subsequently influence the robot action. We then examine the recent advances in data-driven tactile manipulation, covering imitation learning, reinforcement learning, vision-language-action models, and predictive models. Finally, we discuss open challenges and future opportunities toward scalable, generalizable, and robust tactile-enabled robotic systems.
comment: 30 pages. Author accepted manuscript. Posted with permission from the Annual Review of Control, Robotics, and Autonomous Systems, Volume 10; copyright 2027 the author(s), https://www.annualreviews.org. Published version: https://doi.org/10.1146/annurev-control-032125-030330
★ Integrated Longitudinal and Lateral Control for High-Speed Platooning of Articulated Vehicles
This paper presents an integrated control framework for high-speed platooning of articulated vehicles, combining advanced longitudinal and lateral control strategies to address the challenges of safe and efficient autonomous driving. A novel proof of string stability for the proposed longitudinal control algorithm, based on a PID framework, ensures robust inter-vehicle distance regulation even under dynamic operating conditions. Additionally, an arc-length-based longitudinal error formulation enhances the applicability of the control system for curved roads, making it realistic for diverse driving scenarios. For lateral control, significant improvements to the clothoid-based path tracking algorithm are introduced to mitigate oscillations and enhance stability at high speeds. These enhancements include a tangential criterion for oscillation reduction, trajectory segment classification for adaptive lookahead distance tuning, a dual-adaptive mechanism for regulating the rate of lookahead distance adjustments based on both cross-track error (CTE) and anticipated road curvature, and extensions for articulated vehicle dynamics. The proposed integrated control framework is validated through simulations, demonstrating its effectiveness in maintaining stability, minimizing tracking errors, and ensuring robust performance in high-speed and challenging platooning scenarios.
comment: Original manuscript of an article published in Vehicle System Dynamics on September 20, 2026. Published version: https://doi.org/10.1080/00423114.2026.2733699
★ MPBRS: Multi-Potential-Based Reward Shaping for Unified Reward Configuration in Humanoid Locomotion
Designing and tuning reward functions remains one of the main challenges in building effective reinforcement learning (RL) pipelines for humanoid locomotion. While shaping rewards can accelerate learning, naively formulated terms often conflict with the desired behavior, overfit to a specific robot, or become unstable under small weight changes. Potential-Based Reward Shaping (PBRS) offers an alternative solution: shaping the reward via differences of a potential function to guide exploration without changing the optimal policy. In this paper, we study Multi-PBRS for humanoid locomotion and propose a unified reward configuration. We then evaluate the same reward structure and weights across nine humanoid robots, requiring only robot-specific metadata (joint/link mappings and default poses) rather than per-robot reward redesign. Our experiments show that, compared to standard shaping, MPBRS provides more consistent scaling behavior and reduced tuning sensitivity across robots, making it easier to tune without altering the optimal policy.
★ Self-Distilling Future Video Representations for World Action Models
World Action Models (WAMs) have emerged as a promising approach to learning generalist robot policies using rich spatiotemporal priors from pre-trained video diffusion models. A recent line of WAMs improves inference efficiency by conditioning a separate action decoder on intermediate future video representations extracted from a single denoising step, avoiding costly iterative denoising required for video generation. This design makes action prediction depend directly on the quality of these future representations. In this paper, we introduce Future REpresentation learning via self-Distillation with temporal context (FRED), a framework that trains WAMs to improve the quality of future video representations from noise in a single denoising step. Specifically, FRED uses self-distillation to predict future representations from noise, using representations extracted from ground-truth future observations as targets. We further condition the video model on historical observations, providing temporal context for predicting future representations better. Across diverse real-world manipulation scenarios, FRED consistently outperforms competing WAMs. It achieves particularly large gains on long-horizon and out-of-domain tasks under zero-shot environment generalization, and substantially improves human-to-robot skill transfer, reaching 50.0% task success on tasks absent from robot training data and demonstrated only in human videos, compared with 19.4% for the best competing WAM.
comment: 27 pages; Project page: https://huiwon-jang.github.io/FRED/
★ TerraTransfer++: Demonstration-Free End-to-End Planner
End-to-end autonomous driving has achieved state-of-the-art performance on benchmarks and real-world deployments. Its standard training recipe, however, is expensive across all existing paradigms: imitation learning requires fleet-scale driving logs whose behavior distribution only thinly covers the safety-critical tail; closed-loop reinforcement learning in a photorealistic simulator, while offering diverse behaviors, is bottlenecked by sample-efficiency, per-step cost of rendering, and a forward pass through a large vision backbone. Self-play in vectorized simulators changes the economics: at millions of rollout steps per second, one can obtain a state distribution with natural diversity in collisions, near-misses, and recoveries that no driving log contains, and a highly reactive policy emerges from a basic driving-quality reward. Our approach exploits this asymmetry. We pretrain a single policy by self-play, distill its rollouts into a flow-matching trajectory teacher that also plans, and train lightweight adapters on a frozen pretrained vision backbone to reproduce the latent space of either teacher through the action KL divergence, a batch-relational low-rank structural loss, and, for the planner, a trajectory-matching loss. Every target traces back to the self-play policy, so alignment never supervises against a logged trajectory: a paired dataset of (image, scene-state) frames suffices, with no need for the curated expert demonstrations that imitation pretraining is built on. From this one self-play teacher we obtain two end-to-end students: a closed-loop policy that outperforms every published imitation-based end-to-end method on the photorealistic 3D Gaussian splatting closed-loop benchmark HUGSim, and a planner that reaches 59.4 EPDMS on NAVSIM navhard, above every published method including DriveZero-Scale (57.1), and 95.5 PDMS on navtest, without human demonstrations.
♻ ★ OSCAR: Obstacle Survival Curves for Adaptive Robot Navigation
A mobile robot following a graph of known routes can make costly navigation errors when a temporary obstacle blocks a critical edge: waiting too long behind a parked cart wastes time, but immediately rerouting around a person who would move in a few seconds is also inefficient. Standard reactive obstacle avoidance addresses local motion around obstacles, while fixed wait-or-reroute rules ignore how long different obstacle types tend to persist. We propose OSCAR: an adaptive survival-modeling framework for graph-based navigation with temporary blockages. Assuming obstacle class labels are available at encounter time, the robot learns class-conditioned residual clearance-time distributions from online experience, including right-censored observations when it reroutes before observing clearance. These survival models are integrated into a time-dependent graph planner that maintains obstacle memory and computes a patience threshold at each blocked edge: how long to wait before taking an alternate route. The method continuously updates its clearance estimates across episodes and uses them to balance waiting against rerouting. We evaluate the approach in simulation and on a real mobile robot in a university atrium with obstacles including people, chairs, bins, and tubes. In simulation, the learned policy's time-to-goal converges to within 1% of an oracle with access to ground-truth clearance distributions after fewer than 20 observations per obstacle class, outperforming all heuristic baselines. Real-world deployment confirms that the policy improves online, adapting its patience thresholds from experience across 50 navigation episodes.
comment: 8 pages main text, appendices included
♻ ★ Physical Words: A Vocabulary for Structuring Latent Reasoning in Robot Manipulation
Multimodal reasoning provides robotic foundation models with predictive context needed for reliable action generation. However, language-based reasoning captures task semantics but struggles to express fine-grained physical dynamics, while latent reasoning models future state evolution but offers limited structure for organizing these dynamics into reusable reasoning primitives. To bridge this gap, we introduce PhysWord, a reasoning-before-acting foundation model built on Physical Words, which use atomic action verbs as shared semantic anchors for continuous representations of physical dynamics. Specifically, PhysWord predicts upcoming atomic action words and directly aligns their contextual hidden states with latent targets derived from the corresponding future observations. Building on these primitives, we construct a temporally structured PhysWord space with visual and optional tactile variants of each action word. Autoregressively predicting Physical Words across future timesteps captures transitions between execution stages, providing structured multimodal context for action generation. PhysWord achieves state-of-the-art success rates of 90.3\% on visuotactile UniVTAC and 99.6\% on vision-based LIBERO-Long, and outperforms competing methods on real-world tasks using bimanual grippers and dexterous hands. Further evaluations demonstrate that our structured latent reasoning framework generalizes to unseen scenarios and enables long-horizon manipulation. Project page: https://nvidia-isaac.github.io/video_to_data/physicalword/
♻ ★ FoveaVLA: Foveation Actions Replace the Wrist Camera in VLAs
We present FoveaVLA, a Vision-Language-Action (VLA) model that replaces the wrist camera with learned foveation actions. Wrist cameras provide close-up detail but restrict the view to the end-effector's surroundings. External cameras see the whole scene, but the policy receives them at low resolution, so the fine detail needed for manipulation is lost. FoveaVLA recovers this detail by predicting, alongside each action chunk, the interaction region to observe at the next policy query. The policy then receives crops of these regions, taken from the original-resolution images, together with the full views. We build FoveaVLA on $π_{0.5}$ in two configurations, Single and Dual, using one or two external stereo cameras with full and foveal RGB and depth inputs, and post-train both on DROID with RoboInter interaction annotations. Across four real-world task suites, FoveaVLA-Single achieves 70.8% success with one external RGB-D view, and FoveaVLA-Dual achieves 77.1%, versus 52.1% for the wrist-equipped $π_{0.5}$. Both match the baseline on plain pick-and-place and improve success with distractors, cloth manipulation, and far-away objects.
comment: Project website: https://tml.stanford.edu/FoveaVLA/
♻ ★ Onboard UAV State Estimation for Fast Flights Without External Localization Systems: A2RL Drone Racing Finalist Approach
Fast flights with aggressive maneuvers in cluttered GNSS-denied environments require fast, reliable, and accurate UAV state estimation. In this paper, we present an approach for onboard state estimation of a high-speed UAV using a monocular RGB camera and an IMU. Our approach fuses data from Visual-Inertial Odometry (VIO), an onboard landmark-based camera measurement system, and an IMU to produce an accurate state estimate. Using onboard measurement data, we estimate and compensate for VIO drift through a novel mathematical drift model. State-of-the-art approaches often rely on more complex hardware (e.g., stereo cameras or rangefinders) and use uncorrected drifting VIO velocities, orientation, and angular rates, leading to errors during fast maneuvers. In contrast, our method corrects all VIO states (position, orientation, linear and angular velocity), resulting in accurate state estimation even during rapid and dynamic motion. Our approach was thoroughly validated through 1600 simulations and numerous real-world experiments. Furthermore, we applied the proposed method in the A2RL Drone Racing Challenge 2025, where our team advanced to the final four out of 210 teams and earned a medal.
comment: Visit our webpage for more details: https://mrs.fel.cvut.cz/papers/onboard-uav-state-estimation-a2rl
♻ ★ Feedback World Model Enables Precise Guidance of Diffusion Policy NeurIPS 2026
World models aim to improve robotic decision making by predicting the consequences of actions. However, in practice, their predictions often become unreliable once the robot encounters states outside the training distribution, limiting their effectiveness at deployment. We observe that execution itself provides a natural but underutilized signal: after each action, the robot directly observes the true next state, revealing the mismatch between predicted and actual outcomes. Building on this insight, we propose feedback world model, an inference-time mechanism that closes the loop between prediction and observation at inference time. Instead of treating the world model as a static open-loop predictor, our method maintains a lightweight feedback state that is updated online to iteratively correct future predictions, compensating for model errors using real-time observations without additional training data or parameter updates. We show that this process can be interpreted as a latent-space observer and admits convergence guarantees under bounded, asymptotically convergent dynamics residuals and a stable feedback gain. We further introduce action-aware guidance to better translate corrected predictions into control by emphasizing action-controllable components while suppressing irrelevant variations. Experiments on LIBERO-Plus, Robomimic, and real-world manipulation tasks demonstrate that our method substantially improves both prediction accuracy and policy performance under distribution shift, reducing world model prediction error by up to 76.4%. The same feedback mechanism improves the predictions of the pretrained foundation model Fast-WAM-Joint without updating its parameters, demonstrating its plug-and-play applicability. These results show that incorporating real-time feedback at inference time provides a simple yet powerful alternative to static world modeling.
comment: Accepted by NeurIPS 2026
♻ ★ Physical AI for Intralogistics: Benchmarking the NVIDIA Sim-to-Real Ecosystem for Cosmos-Enhanced COMPASS Navigation on Industrial Mobile Robots
Classical robot navigation stacks are reliable but often require re-engineering for each new robot type and site. Learning-based navigation shifts this effort from hand-tuned parameters to data generation and training, where a policy adapts to each new embodiment or facility by fine-tuning in simulation. Whether policies adapted this way stay reliable on real industrial robots has seen little systematic testing. We study this for NVIDIA's X-Mobility and COMPASS workflow for training learned navigation policies on Idealworks robots. In simulation, we extend the COMPASS benchmark with two embodiments, the differential-drive iw$\mkern-1.5mu.\mkern-1.5mu$hub and an omnidirectional Humanoid Mobile Platform (HMP), and with a digital twin of the Idealworks R&D center. For the real world, we fine-tune X-Mobility on data from this digital twin, with and without sim-to-real variants generated by NVIDIA Cosmos, train COMPASS on top of it from scratch in the digital twin, deploy the resulting policies on the iw$\mkern-1.5mu.\mkern-1.5mu$hub's onboard Jetson AGX Xavier, and benchmark them in the Idealworks R&D center against COMPASS trained only on NVIDIA's environments and the existing Idealworks navigation stack on task success, efficiency, safety, smoothness and onboard compute. Running every static scenario both in a digital-twin replica and in the real facility measures the sim-to-real gap and how far Cosmos-augmented data closes it. Training in the digital twin with Cosmos sim-to-real augmentation raises the real-world success rate of learned navigation from 65.3% to 91.7% across four scenarios and nearly halves its weighted travel time compared with simulation-only training, bringing it within 8.3 points of the production navigation stack, fully onboard; with an integrated global planner, it also reaches far, occluded goals in every trial.
♻ ★ TrackDeform3D: Markerless and Autonomous 3D Keypoint Tracking and Dataset Collection for Deformable Objects IROS
Structured 3D representations such as keypoints and meshes offer compact, expressive descriptions of deformable objects, jointly capturing geometric and topological information useful for downstream tasks such as dynamics modeling and motion planning. However, robustly extracting such representations remains challenging, as current perception methods struggle to handle complex deformations. Moreover, large-scale 3D data collection remains a bottleneck: existing approaches either require prohibitive data collection efforts, such as labor-intensive annotation or expensive motion capture setups, or rely on simplifying assumptions that break down in unstructured environments. As a result, large-scale 3D datasets and benchmarks for deformable objects remain scarce. To address these challenges, this paper presents an affordable and autonomous framework for collecting 3D datasets of deformable objects using only RGB-D cameras. The proposed method identifies 3D keypoints and robustly tracks their trajectories, incorporating motion consistency constraints to produce temporally smooth and geometrically coherent data. TrackDeform3D is evaluated against several state-of-the-art tracking methods across diverse object categories and demonstrates consistent improvements in both geometric and tracking accuracy. Using this framework, this paper presents a high-quality, large-scale dataset consisting of 6 deformable objects, totaling 110 minutes of trajectory data. Project page: https://roahmlab.github.io/trackDeform3D-core-tracking/
comment: Accepted to the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2026
♻ ★ DEFT: Differentiable Branched Discrete Elastic Rods for Modeling Furcated DLOs in Real-Time
Autonomous wire-harness assembly requires precise, reliable manipulation of branched cables, whose deformation is difficult to predict. Models for single Deformable Linear Objects (DLOs) do not extend directly to Branched DLOs (BDLOs), whose junctions couple bending, twisting, and stretching across branches. We introduce Differentiable discrete branched Elastic rods for modeling Furcated DLOs in real-Time (DEFT), a framework combining differentiable physics and learning for accurate BDLO dynamics, real-time inference, and dexterous manipulation planning. DEFT introduces momentum-conserving junction constraints to capture cross-branch coupling, supports mid-structure grasping, uses analytical gradients and branch-parallel computation for efficient parameter learning and inference, and incorporates a residual Graph Neural Network (GNN) to reduce long-horizon prediction errors. Experiments on six physical BDLOs with different topologies and materials demonstrate formulation-level generalization: the same formulation and residual-GNN architecture apply without redesign, with physical and learned parameters fitted separately for each BDLO. DEFT outperforms state-of-the-art physics-based and learning-based methods, achieving RMSEs of 0.93-2.14 cm over five-second recursive open-loop predictions and inference above 100 Hz. In real-world planning, DEFT achieves success rates of 36/40 for shape matching, 40/40 for parent-branch insertion, and 16/20 for child-branch insertion. These results demonstrate accurate, efficient, planning-compatible BDLO prediction for autonomous wire-harness manipulation. Project website: https://roahmlab.github.io/DEFT/.
comment: Accepted for publication in The International Journal of Robotics Research (IJRR)
♻ ★ Multi-Resolution Tactile Imitation Learning for Contact-Rich Robotic Manipulation
Touch sensing is beneficial for solving a wide variety of manipulation tasks. While there exists a wide range of tactile sensors with different properties, exploiting the fusion of multiple heterogeneous tactile sensors to improve manipulation learning remains underexplored. We present Multi-Resolution Tactile Sensing (MiTaS), a representation framework that leverages multiple tactile sensors operating at different temporal resolutions in order to solve complex contact-rich manipulation tasks. We propose an effective architecture using modality-specific convolutional stems and transformer-based fusion that effectively integrates information from an RGB camera stream, a vision-based GelSight Mini sensor and a high-frequency event-based Evetac sensor. This multisensory representation then conditions a flow-matching policy for solving downstream tasks. Our systematic study across five contact-rich manipulation tasks demonstrates the effectiveness of multi-resolution tactile features in imitation learning wherein MiTaS achieves an average success rate of 80 \%, while vision-only (35.5\,\%) and visuotactile (55.5\,\%, 63\,\%) baselines cannot solve the task reliably. Project Page: http://mitas-touch.github.io.
comment: 20 pages, 5 figures, CoRL 2026
♻ ★ WorldOracle: Benchmarking Action-Conditioned World Models against Real-World Executions
We introduce WorldOracle, a benchmark specifically designed to evaluate action-conditioned world models (ACWMs) against paired real-world executions. Given the same initial observation and low-level action sequence, WorldOracle asks whether a predicted rollout reproduces what happens in the real world, including physical consequences and task-state transitions. We evaluate models along four complementary dimensions: physical state correctness, interaction spatial alignment, key interaction area fidelity, and multi-view action coherence. To support this evaluation, we curate real-world training and evaluation suites that cover different interaction dynamics and embodiments. The evaluation suite contains long, multi-segment episodes with both successful and failed executions, enabling evaluation across varied action outcomes and behavior distributions. Experiments on seven representative ACWMs reveal distinct model strengths and failure modes. WorldOracle provides a systematic foundation for measuring and advancing action-conditioned world modeling for manipulation. We release the datasets, evaluation code, and leaderboard: https://worldoracle-benchmark.github.io/.
♻ ★ AgriVLN: Vision-and-Language Navigation for Agricultural Robots
Agricultural robots are serving as powerful members across a wide range of tasks, nevertheless, they are still heavily relying on manual operations or railway systems for movement. In contrast, Vision-and-Language Navigation (VLN) enables navigating robots from starting points to target positions following natural language instructions, having shown strong performances across various domains. However, none of the existing benchmarks or methods are specifically designed for agricultural robots. To bridge this gap, we build our A2A benchmark, which consists of 1,560 episodes across 6 classes of agricultural scenes using the step-by-step format for instructions and the continuous environment for actions. We evaluate several existing state-of-the-art VLN methods on A2A, then we observe that they perform well on short instructions while perform poorly on long instructions. To alleviate this phenomenon, we design our STL module, which decomposes a complex instruction into a list of simple subtasks, thereby assisting the decision-maker to focus on the current short subtask rather than the entire long instruction at any time. We integrate STL into the NavGPT backbone to propose our AgriVLN method, achieving a sufficient improvement on SR from 0.33 to 0.47 with a slight sacrifice on NE from 2.76 m to 2.91 m. We further implement the ablation studies and the case study, discussing the effectiveness and limitations of AgriVLN on different subtask quantities and scene classes. Repository: https://github.com/AlexTraveling/AgriVLN.
♻ ★ Let It Be Simple: One-Step Action Generation for Vision-Language-Action Models
One-step generation from standard diffusion or Flow-Matching training is rarely effective for text-to-image or video generation without distillation or specialized one-step objectives. We find that VLA action generation behaves strikingly differently. Biasing only the training timestep distribution toward the noise endpoint yields strong one-step policies across LIBERO, LIBERO-Plus, LIBERO-Pro, and real-world robot tasks. Moreover, for open-source VLA models with different architectures, simply replacing iterative decoding with a single forward pass often performs comparably to the original multi-step decoding on benchmarks such as LIBERO, CALVIN, and SimplerEnv. We attribute this difference to the condition-target structure of VLA generation, which is closer to image-to-text than text-to-image: rich observations strongly constrain a comparatively compact target. We formalize this intuition through the irreducible velocity loss $R_v(t,c)$ of standard Flow Matching, which becomes much smaller toward the noise endpoint for VLA, whereas image-generation models usually exhibit a U-shaped profile. The geometry induced by the learned generative map shows the same contrast: the Jacobian of this map with respect to the initial noise has a substantially smaller largest singular value for action generation than for image generation. Together, these results suggest that VLA's condition-target structure makes one-step generation unusually accessible to standard Flow Matching. Code and training configurations for this paper are available at https://github.com/bondtesty/let-it-be-simple.
comment: 23 pages, 18 figures, 15 tables
♻ ★ SafeDojo: Safe Reinforcement Learning for VLA via Interactive World Model
Safe control is a prerequisite for real-world embodied intelligence, for which safe reinforcement learning has emerged as a promising paradigm. However, existing safe reinforcement learning methods either require costly real-world exploration or depend on hand-crafted safety functions. Neither scales to vision-language-action models deployed in open-world physical environments. We propose SafeDojo, a model-based safe reinforcement learning framework that improves vision-language-action policies through world-model imagination. SafeDojo performs online reinforcement learning on top of an interactive video world model. A ResNet success classifier scores predicted future frames, while a lightweight safety head predicts per-step contact probabilities from the current latent context and proposed action chunk, jointly assessing task execution and trajectory safety. These decoupled signals are balanced through Lagrangian-constrained GRPO. On SafeLIBERO Level I, SafeDojo improves average safe-success rate by 8.25 percentage points over the strongest baseline; on the zero-shot Level II setting, it is competitive with the strongest model-free baseline. Real-world Franka deployment further shows the best average task and safe-success rates across five tasks. These results support world-model-based safe reinforcement learning as a promising path toward safer embodied intelligence.
comment: CoRL 2026
♻ ★ UGV-Conditioned Multi-UAV Informative Planning on a Shared Exposure Belief
Safe ground navigation in large, threat-augmented environments requires aerial support that actively reduces the risks that a ground vehicle faces along its route. Existing aerial reconnaissance systems focus on mapping or covering the environment, but do not direct sensing toward regions that are most relevant for ground vehicle safety. In this paper, we address the problem of coordinating a team of unmanned aerial vehicles (UAVs) to improve the safety of an unmanned ground vehicle (UGV) navigating through unknown threat zones. A key aspect of our approach is a risk-averse shared exposure belief for UGV routing and a mission-conditioned UAV reward that directs sensing toward uncertain, exposure-relevant regions of the current UGV corridor. UAV observations update the shared belief online, allowing the UGV to replan around newly revealed threats while the evolving route in turn reshapes UAV sensing priorities. For multiple UAVs, we use region-conditioned spatial assignment to reduce redundant sensing. Simulation experiments show that our approach reduces cumulative UGV exposure by 38% compared with terrain-semantic replanning, and reduces redundant aerial coverage from 38.8% to 3.7% under our multi-UAV coordination scheme.
comment: 8 pages, 6 figures
♻ ★ Action-Conditioned World Model for Goal Plane Guidance in Robotic Neck Ultrasound
Ultrasound examinations typically follow specific scanning protocols that require navigating through an ordered sequence of anatomical landmarks and target imaging planes. We present a local action-conditioned world model framework for goal plane probe guidance in robotic ultrasound. Autonomous ultrasound tasks often require large numbers of probe-motion trajectories for training, but collecting high-quality demonstrations is labor-intensive and explicit simulators are difficult to build because ultrasound appearance depends on contact, tissue deformation, and view-dependent acoustic artifacts. We address this problem with a two-stage model-based learning pipeline. First, a latent conditional diffusion world model predicts future ultrasound observations from recent context frames, probe motions and temporal offset. Second, a goal-conditioned temporal transformer predicts ordered probe motions and is fine-tuned using rewards from the frozen world model. Experiments on the self-collected dataset show that the world model preserves action-dependent anatomical structure on target-directed scans. In real-world closed loop experiments, the framework achieves success rates of 70.0\% for carotid guidance and 65.0\% for thyroid guidance. These results demonstrate the potential of learned ultrasound dynamics for training goal-directed robotic probe navigation within protocol-constrained organ specific tasks.
♻ ★ Beyond Appearance Shifts: Task-Semantic Action Calibration for VLA Models
Vision-language-action (VLA) models have achieved strong performance in embodied manipulation, but still struggle to distinguish between changes that should preserve behavior and those that require different actions. We identify two complementary failure modes. Under task-preserving changes, where task semantics remain unchanged but scene appearance varies (e.g., style, illumination, clutter, or paraphrasing), policies often exhibit unnecessary action drift. Conversely, under semantic-breaking changes, where key task semantics such as the target object or constraint are altered, policies frequently continue following the original trajectory instead of adapting to the new task. To address these failures, we propose BAS-VLA, a task-semantic action calibration framework built on top of a frozen base VLA. BAS-VLA adopts a breaking-centered calibration core as its default path, complemented by a selective evidence-gated preserving auxiliary that corrects appearance-induced drift when task-preserving evidence is strong. Across four target-object swap tasks, BAS-VLA improves changed-task success from 2.3% to 68.5% while maintaining strong clean and semantics-preserving performance and suppressing outdated behavior. On validated style-preserving shifts, it further improves success from 42% to 70% without degrading clean performance. These results demonstrate the importance of calibrating VLA policies to remain stable when task semantics are preserved and adapt when task semantics change.
comment: 27 pages (10-page main text), 15 figures, 24 tables. Project page: https://nebulis-lab.com/BAS-VLA
♻ ★ Hilti-Trimble-Oxford Dataset: 360 Visual-Inertial Benchmark with Floor Plan Priors for SLAM and Localization
Automated progress monitoring on construction sites is an active area of research and development. Robot and human-carried mapping systems have been developed to build 3D maps of building and infrastructure projects. While LiDAR-based mapping systems achieve high accuracy, the cost of LiDAR can be prohibitive. Consumer-grade cameras with wide field of view ("360 cameras") combined with embedded inertial measurement units (IMUs) provide a cost-effective alternative. To support change detection and progress monitoring, highly accurate visual simultaneous localization and mapping (SLAM) and floor plan localization systems are required. In this paper, we present a high-quality dataset collected at an active construction site, which captures realistic challenges such as variable lighting conditions, moving workers, fast motions, and repetitive structures. The dataset offers thirty visual-inertial sequences recorded across ten floors over an eight-month period of the construction project. Ground truth trajectories are collected using a high quality LiDAR-inertial SLAM system rigidly attached to the 360 camera. Additionally, we report the results of an open research challenge evaluating the best visual SLAM and localization systems from around the world. The Challenge attracted substantially higher participation in SLAM, with 62 teams compared to 22 in floor plan localization, which suggests that this task receives less attention from researchers and motivates further exploration through proposed dataset and challenge, available at: https://hilti-trimble-challenge.com/dataset-2026.
♻ ★ Learning Altruistic Collaboration in Heterogeneous Multi-Team Systems
This paper studies heterogeneous multi-team collaboration through dynamic robot allocation, where the robots themselves are treated as transferable resources. Leveraging Hamilton's rule from ecology as an altruistic decision-making mechanism, we propose a multi-team collaborative resource allocation framework with heterogeneous capabilities, transfer costs, and capability-dependent contributions. The resulting general heterogeneous resource assignment problem is combinatorial and is shown to be NP-hard. To address scalability, we develop a graph neural network policy under centralized training and decentralized execution that approximates the altruistic allocations based on Hamilton's rule. The model operates over the team interaction graph and predicts robot-level transfer decisions and subsequent robot-to-team assignments. The proposed approach is evaluated in a fire-fighting scenario against exact optimization and multi-step simulations, quantifying allocation quality, decision computation, and fire suppression. Five paired physical testbed trials with 15 robots compare learned reallocation with fixed team membership; the GNN reaches the modeled fire stopping threshold in all five trials, compared with two for static allocation.
♻ ★ $λ_0$: Scaling Open-World Egocentric Human Data for Humanoid Loco-Manipulation
Humanoid loco-manipulation requires coordinated whole-body interaction with the physical world, yet collecting robot demonstrations at scale remains costly. Human demonstrations offer a scalable alternative, but existing egocentric human supervision often omits locomotion and posture, while embodiment differences and controller dynamics hinder accurate end-effector execution. We introduce HumanVerse, a 500-hour dataset of open-world human loco-manipulation pairing egocentric video with synchronized body and hand motion. Our human-to-humanoid pipeline aligns camera viewpoints and arm appearance, and refines motion references and controller inputs offline to reduce end-effector errors while preserving whole-body coordination. We validate the pipeline through zero-shot transfer to Unitree G1 on representative whole-body manipulation and long-range loco-manipulation tasks, without target-task robot demonstrations. Building on this pipeline, we develop $λ_0$, a humanoid foundation model for whole-body loco-manipulation, with robot-aligned human demonstrations providing supervision for mid-training. Increasing whole-body human data consistently improves downstream task progress. $λ_0$ achieves 90.0% success on SIMPLE and 62.5% across real-world loco-manipulation tasks. We expect this study to guide scalable whole-body human data collection toward humanoid foundation models for richer interaction with the physical world. Homepage: https://lambdahumanoid.github.io/
comment: Tech report, 24 pages
♻ ★ SSIO: Self-Supervised Inertial Odometry
Learning inertial odometry for a new platform usually requires ground-truth trajectories. We show that an inertial odometry model can instead be trained from scratch by a myopic teacher that integrates the IMU over each short window independently, a scheme we call integration self-supervision. Interestingly, although supervised only by short-window velocity changes, the student learns to directly predict velocity, and does so more accurately than its integration teacher. Pose graph distillation then refines the model with sequence-consistent targets from a pose graph over many windows. Training on a new platform therefore no longer requires collecting ground-truth trajectories, but only calibrating the IMU and recording data during normal operation. Across six datasets spanning pedestrians, drones, quadrupeds, and a humanoid, the same recipe with no velocity or position labels trains models that come within 0.07~m/s average velocity error of a same-architecture supervised baseline.
♻ ★ rMuscle: Robotic Muscle Memory for Efficient Vision-Language-Action Model Inference
Factory work is a promising early scenario for embodied AI: assigning repetitive manual jobs to robots has clear economic payoff, and a structured station keeps the jobs tractable for current policies. Vision-Language-Action (VLA) models now dominate as the policy paradigm for these robots. The inference latency of VLA models directly affects robot responsiveness and motion smoothness. However, existing VLA inference frameworks do not fully exploit the characteristics of embodied workloads or account for the distinct bottlenecks across different stages of VLA inference. In this paper, we first characterize embodied workloads and identify substantial task similarity across repeated robot executions. We further find that such similarity extends beyond observations and action trajectories to internal model states. Drawing on these observations, we present rMuscle, a real-time VLA inference framework inspired by human muscle memory. It exploits cross-execution similarity through a dual-phase muscle-memory cache. The Context Cache reuses visual-token outputs to reduce computation, while the Action Cache reuses neuron activation patterns to reduce weight accesses. We keep both the cache memory footprint and access overhead low through online cache recomputation, sliding-window cache retrieval, and mask sharing across consecutive denoising steps. rMuscle achieves 1.29-1.42X speedup on RTX 4090 and Jetson Thor across LIBERO, RoboTwin, and physical manipulation tasks, while maintaining the original success rates on real-world robots.
♻ ★ PaceVLA: Adaptive Temporal Action Quantization in Vision-Language-Action Models
Fast and reliable manipulation is crucial for practical robot deployment. Although Vision-Language-Action (VLA) models have recently shown remarkable progress as generalist robot policies, their execution often remains temporally inefficient. A key source of this inefficiency is that policies learn fine-grained temporal patterns from teleoperated demonstrations, requiring many low-level action steps. Temporal action quantization offers a simple way to accelerate execution by aggregating consecutive low-level actions into temporally coarser actions. However, applying action quantization without considering when fine-grained control is required introduces a speed-accuracy trade-off: accelerated actions may efficiently reduce redundant steps in some phases, while discarding critical motion details in precision-sensitive interactions. To address this, we propose PaceVLA, a unified framework for adaptive temporal execution in VLAs that integrates two coupled components: (1) mode-specific action decoders built on a shared action expert, supporting multiple temporal granularities and prediction horizons, and (2) a quantizability-guided mode selection scheme which decides whether coarse execution is appropriate, and a context-conditioned router that selects an execution mode at each inference step. Experiments across three multi-task simulation benchmarks and six real-world tasks show that PaceVLA improves execution efficiency while maintaining task success rate across multiple embodiments, including long-horizon and dexterous manipulation.
comment: 25 pages, 7 figures, 10 tables. Project page: https://pacevla.github.io/
♻ ★ Learning from Fleet Experience: Collective Experience Learning Across 96 Robots
Robot deployment generates experience that can support policy improvement, but continued reliance on expert corrective actions makes learning difficult to scale across growing fleets. We propose learning from fleet experience: using deployed robots' own executions as a source of supervision for further policy improvement. We call this paradigm Collective Experience Learning (CEL), in which robots pool execution experience to update a shared policy that returns to the fleet. Following expert-supervised initialization, we realize CEL through failure-guided experience retrieval fine-tuning (FERT). FERT uses failure contexts to retrieve relevant segments from successful trajectories and prioritizes their recorded actions for supervised updates, without collecting additional expert demonstrations or corrective actions. Operators provide binary episode-outcome labels and environment resets. A fault-tolerant asynchronous training system (FATS) coordinates robot execution, cloud inference and policy optimization despite local interruptions. Across five real-world manipulation tasks, the resulting policies achieve success rates of 93.3-99.2%, evaluated over 120 trials per task on robots excluded from training data collection. With collection time per robot held fixed, larger fleets provide more experience and generally improve performance across the evaluated sizes. On multi-stage phone packaging, learning with 96 robots raises success from 79.2% to 98.3%. These results support CEL as a scalable approach to improving robot policy reliability through experience generated during deployment.
♻ ★ Cartender: Verifiable Language-Conditioned Driving by Learning the Drives Not Taken
Every driving scene is logged once, but driving is one-to-many: the scene determines which behaviors are feasible, while an instruction selects among them. A log records only the action taken, leaving no direct supervision for alternative feasible actions or instructions the scene cannot support. Consequently, a policy that ignores language and one that follows it without regard to scene constraints can fit the same data. We present Cartender, which learns from the drives not taken. Counterfactual Branching constructs matched and mismatched instruction-action pairs from real recordings without a simulator: a vision-language model proposes alternative futures and their selecting instructions, while unsupported instructions provide negative labels. A learned scene-language compatibility score regulates language conditioning in trajectory generation, and a reward-trained language residual re-ranks generated candidates over a scene-based score. The branches are used for training; instruction following is evaluated by swapping instructions under identical recorded inputs and measuring behavior rather than agreement with generated targets. On WOD-E2E, the best of four proposals achieves an RFS of 8.642 across 3,603 feasible instruction conditions. Across 8,974 admitted proposals, mean RFS is 8.203 versus 7.906 for matched logged trajectories, and 67% score higher than their logged counterpart. Compared with matched-only imitation using the same backbone, Cartender improves same-scene paired instruction success from 23.67% to 57.00% and RFS from 6.32 to 7.10; its gate reduces infeasible-instruction execution from 53.36% to 14.18%.
♻ ★ HarnessPAI: An Evolving Harness for Physical AI
Physical AI aims to build embodied agents that perceive the world, understand and reason about it, and decide how to act. Yet the field has focused primarily on the last component: the action model that maps observations to low-level controls. The prevailing training recipe can erode the perceptual and reasoning capabilities needed for robust behavior, leaving even strong action models vulnerable to scene perturbations and long-horizon tasks. We introduce HarnessPAI, a model- and embodiment-agnostic Harness framework for Physical AI that treats code as the executable and evolvable interface that organizes the underlying action primitive. The framework separates two timescales: within a rollout, it executes open-loop at the program level, with a fixed program guiding and checking execution; across rollouts, it evolves closed-loop, using execution feedback to revise the program and distill failures into reusable skills. Across desktop robot arms, household robots, a robot vacuum, and a legged walking agent, HarnessPAI demonstrates consistent improvements against baselines. Beyond the harness, the converged program also acts as a low-cost, reliable source of expert data: it generates training data for fine-tuning VLA and WAM models on tasks where it is hard for these models to obtain successful trajectories on their own. Our results suggest that the frontier of Physical AI depends not only on stronger action models, but also on executable harnesses that integrate perception, task understanding and reasoning, and action execution into a unified, verifiable, and feedback-driven system. Website: https://darwin-agent.github.io/HarnessPAI
comment: 45 pages, 24 figures, 15 tables
♻ ★ VLA-Rolling: Fewer Denoising Steps, Better Control via Adaptive Action Reuse
Flow-matching vision-language-action (VLA) models generate an action chunk by iteratively denoising Gaussian noise, but typically execute only a short prefix before replanning with updated observations. Under this receding-horizon paradigm, the unexecuted remainder---the action tail---is discarded, and the next chunk is regenerated from fresh noise. We show that this discarded tail retains transferable denoising progress: action-tail initialization can match or outperform a native 10-step Gaussian baseline using fewer denoising steps. However, the optimal denoising budget varies substantially across samples, indicating that tail reuse should be adapted to the updated environment. Based on this observation, we introduce VLA-Rolling, a lightweight inference framework that reuses action tails through three stages: action-tail inheritance, per-action confidence estimation, and confidence-guided refinement through renoising and denoising budget allocation, while the underlying VLA remains frozen. Across $π_{0.5}$, SmolVLA, X-VLA, and MolmoAct2 on LIBERO, MetaWorld, LIBERO-Plus, and RoboTwin 2.0, VLA-Rolling reduces action-chunk inference cost in all 16 model--benchmark pairs, achieving up to a $1.48\times$ speedup, while improving success rate in 14 pairs by up to $15.08$ percentage points. These results show that action tails are not merely discarded predictions, but reusable intermediate states that enable more efficient and accurate VLA inference. These results demonstrate that the action tail can support the generation of better actions with fewer denoising steps. Our project is available at https://chenbao123ww.github.io/VLA-Rolling/.
♻ ★ Inference-Time Robot Behavior Steering through Physically-Aware Reconfiguration of Task-Structure
A central challenge in deploying learned robot policies is inference-time behavior steering: redirecting a policy at test time to satisfy user preferences not anticipated during training, without retraining. Existing methods fail in two modes: end-to-end methods require fine-tuning or expert-level guidance, while neuro-symbolic methods rely on predefined symbols whose edits can result in logically reasonable but physically infeasible plans. To address this challenge, we propose ReStruct, which builds upon a neural automaton policy that decomposes a visuomotor policy into a high-level state-machine skeleton capturing task structure and a low-level continuous controller represented as a residual policy. Specifically, ReStruct adopts the automaton to represent the preference and incorporates it into the skeleton through a synchronous product, thereby reconfiguring the task structure. With the controller kept frozen, the action priors provided by the skeleton are updated accordingly to enable physically-aware control under a modified task structure. Extensive experiments from simulation and real-world show that ReStruct steers a wide range of preferences, from object-centric specifications to temporal-logic constraints, and after steering surpasses existing methods, exceeding VLA models in both task success and preference-following by up to 25%.
comment: Accepted to CoRL 2026
♻ ★ Planning under Noise with Neural Operators
Planning under uncertain control execution noise requires reliable feedback policies across diverse environments. We introduce Planning Under Noise (PUN), a neural operator that learns noise-aware value fields for planning in unseen geometries. Building on an established screened-Poisson formulation of stochastic optimal control, we address two challenges: specifying boundary conditions that support navigation across varying geometries and efficiently computing fields with reliable gradients. Zero-Dirichlet obstacle boundaries terminate sampled paths at walls, weakening the goal signal along long routes and making it harder to estimate reliably. We instead impose reflecting Neumann conditions on obstacles and a Dirichlet condition at the goal, allowing sampled paths to continue exploring free space until they reach the target. To solve the resulting PDE without a configuration-space mesh, we extend a Monte Carlo (MC) solver, Walk on Stars (WoSt), to higher dimensions. However, local MC fluctuations can reverse estimated gradient directions and cause planning failures, making direct planning require costly sampling. PUN amortizes this computation by learning geometry-to-field mappings from noisy, partially converged Monte Carlo labels, avoiding expensive convergence for every training example. At inference, test-time scaling uses bounded local search guided by the predicted global field. We evaluate PUN on planning tasks across dimensions. Across four noise levels, PUN matches or outperforms all compared planners, achieving 95-100% success on the 4D replanning benchmark while maintaining the lowest runtime. At the highest tested noise level in 4D, PUN achieves 10.7-60.6% relative gains in success rate over other methods while running 3.29-32.15 times faster.
♻ ★ Move-Then-Operate: Behavioral Phasing for Human-Like Robotic Manipulation ICML 2026
We present Move-Then-Operate, a Vision language action framework that explicitly decouples robotic manipulation into two distinct behavioral phases: coarse relocation (move) and contact-critical interaction (operate). Unlike monolithic policies that conflate these heterogeneous regimes, our architecture employs a dual-expert policy routed by a learnable phase selector, introducing a structural inductive bias that isolates phase-specific dynamics. Phase labels are automatically generated via an MLLM-based pipeline conditioned on lightweight contextual cues such as end-effector velocity and subtask decomposition to ensure alignment with human motor patterns. Evaluated on the RoboTwin2 benchmark, our method achieves an average success rate of $68.9\%$, outperforming the monolithic $π_0$ baseline by $24\%$. It matches or exceeds models trained on $10\times$ more data and reaches peak performance in $40\%$ fewer training steps, demonstrating that architectural disentanglement of move and operate phases is a highly effective and efficient strategy for mastering high-precision manipulation. The code is available at: https://github.com/healenrens/Move-then-Operate
comment: Accepted by ICML 2026. 15 pages, 10 figures
♻ ★ WARPED: Wrist-Aligned Rendering for Robot Policy Learning from Egocentric Human Demonstrations
Recent advancements in learning from human demonstration have shown promising results in addressing the scalability and high cost of data collection required to train robust visuomotor policies. However, existing approaches are often constrained by a reliance on multiview camera setups, depth sensors, or custom hardware, and are typically limited to policy execution from third-person or egocentric cameras. In this paper, we present WARPED, a framework designed to synthesize realistic wrist-view observations and actions from human demonstration videos to facilitate the training of visuomotor policies using only monocular RGB data. With data collected from an egocentric RGB camera, our system leverages vision foundation models to initialize the interactive scene. A hand-object interaction pipeline is then employed to track the hand and manipulated object and retarget the trajectories to a robotic end-effector. Lastly, photorealistic wrist-view observations are synthesized via Gaussian Splatting to directly train a robotic policy. We demonstrate that WARPED achieves success rates comparable to teleoperation on four of five rigid-object tabletop tasks and one articulated-object task, while requiring 4-9x less data collection time.
♻ ★ ExtremControl: Low-Latency Humanoid Teleoperation with Scaled Velocity Feedforward
Building a low-latency humanoid teleoperation system is essential for collecting diverse reactive and dynamic demonstrations. However, end-to-end latencies in existing approaches consistently hover around 200ms, roughly equal to human reaction time, that severely limits responsiveness and prevents tasks requiring rapid feedback. We identify position-only PD control as the root cause of this latency floor and propose ExtremControl, a low-latency whole-body control framework that: (1) incorporates scaled velocity feedforward in low-level control, reducing tracking delay by approximately 70ms with a provably optimal feedforward ratio; (2) operates directly on SE(3) poses of selected rigid links, primarily humanoid extremities, to avoid joint-space retargeting in the control loop. To minimize retargeting time, we derive a closed-form Cartesian-space mapping that uses a single anthropomorphic calibration step to translate human motion into humanoid link targets, accommodating operators with varying body proportions. Furthermore, we provide a theoretical framework for selecting control hyperparameters, and validate the effectiveness of velocity feedforward control in the reinforcement learning for locomotion setting through experiments in both simulation and the real world. Building on ExtremControl, we implement a teleoperation system supporting both optical motion capture and VR-based tracking, achieving end-to-end latency as low as 50ms --- well beneath the threshold at which system delay adds a full human reaction cycle to the control loop. This enables a qualitatively new class of reactive behaviors, including ping-pong ball balancing, juggling, and real-time return, that are beyond the capability of existing teleoperation systems.
comment: Project website: https://extremcontrol.github.io/
♻ ★ HEHA: Bounded Suboptimal Planning for Constrained Routing and Its Use in Heterogeneous Multi-Robot Exploration of Unknown Environments
This paper considers the path planning problem for autonomous exploration of an unknown environment using multiple heterogeneous robots such as drones, wheeled, and legged robots, with different capabilities to traverse complex terrains. A key challenge there is to intelligently allocate the robots to the unknown areas to be explored and determine the visiting order of those areas subject to traversability constraints, which leads to a large-scale constrained multiple hamiltonian path problem (mHPP) that needs to be quickly and iteratively solved every time when new space is explored. To address the hallenge, we propose HEHA (Hierarchical Exploration with Heterogeneous Agents) using a recent hierarchical method that decomposes the exploration into global planning and local planning. The major contribution in HEHA is its global planning, where we propose a new routing algorithm RCH (Routing with Constraints and Heterogeneity) that can quickly find bounded sub-optimal solutions for mHPP to minimize the maximum path length (i.e., makespan) among the agents subject to traversability constraints. Additionally, the local planner in HEHA also considers heterogeneity to avoid repeated and duplicated exploration among the robots. The experimental results show, in the presence of heterogeneous traversability, our RCH can solve mHPP with up to 47% smaller makespan than several baselines, and HEHA can reduce the exploration time by up to 30% compared with existing approaches.
comment: 16 Figures
♻ ★ Principles of Robot Autonomy
Autonomous robots are moving rapidly from research labs into everyday life - on roads, in the air, in warehouses, and in space. Robot autonomy is no longer solely an academic pursuit, but a collection of mature, field-tested methods and tools that practitioners rely on in real-world deployments. This book offers a clear, unified introduction to the methods that make this possible. Built on decades of teaching at Stanford, the text develops the core elements of modern autonomy stacks within a single conceptual framework, bridging classical robotics and modern physical AI. Every major topic is paired with hands-on Jupyter notebooks and implementation-driven exercises, so readers build practical intuition alongside theoretical understanding. The result is a principled, accessible, and deployment-aware foundation for anyone seeking to design, analyze, or contribute to the next generation of autonomous systems. This is a comprehensive resource for students, engineers, and researchers entering one of today's fastest-growing fields.
comment: 531 pages. Pre-publication version of a book forthcoming from Cambridge University Press, posted with the permission of the publisher
♻ ★ UniMem: Unifying Visual and Textual Memory in VLAs for Non-Markovian Tasks
While Vision-Language-Action (VLA) models have leveraged internet-scale pretraining and task-specific finetuning to achieve strong performance on long-horizon tasks, they often struggle with non-Markovian tasks that require memory. Existing approaches to memory typically involve additional Vision-Language-Models (VLMs) for long-term memory management, introducing a memory bottleneck and a fractured training pipeline. Conditioning on multiple historical frames can provide the VLA with access to more descriptive features of past scenes, but can degrade performance if frames are chosen at arbitrary, fixed intervals. To address these limitations, we present UniMem, a framework that unifies high-level visual and textual memory and low-level control under one backbone. UniMem employs an event classifier to initiate textual and visual memory updates, a keyframe encoder for dense visual memory, and a keyframe caching technique to minimize overhead during policy rollouts. We evaluate UniMem across five simulation and four hardware tasks targeting sequential and spatial memory, demonstrating that our unified, single-model system outperforms fixed-interval image sampling baselines (93.4% vs. 68.2%) in simulation and hierarchical VLM-VLA baselines (80.0% vs. 43.5%) in hardware, while offering faster inference and a simple training pipeline for easy adoption. Project website: https://losterberg3.github.io/unimem-vla/
♻ ★ The Bitter Lesson Comes to Robotics
General purpose multimodal language models (LMs) are increasingly capable of directly controlling robotic arms (e.g., by commanding end effector poses) to manipulate their environment. Tracking the development of these capabilities is important for understanding their potential benefits, such as improving industrial automation, and risks, such as enabling malicious users to produce dangerous chemical or biological agents. To this end, we investigate the ability of frontier multimodal LMs to directly control physical robotic arms. We evaluate nine models at multiple reasoning-effort and service-tier settings on six manipulation tasks taken from the RoboDojo benchmark. To our knowledge, this is the first systematic comparison of a broad range of frontier LMs across a common suite of manipulation tasks on physical hardware. We find that current frontier models can directly control real robotic arms to perform a variety of manipulation tasks but are not yet precise or efficient enough to complete them within a practical action budget. However, these capabilities are improving rapidly: performance increases on our benchmark between July and October 2026 (inclusive) correspond to a performance doubling time of 53.9 days, while the best recorded execution rate increased by a factor of 4.4 and the lowest recorded execution cost fell by a factor of 6.2. These trends raise the possibility that the Bitter Lesson is coming to robotics; the capabilities of embodied general purpose LMs may one day be comparable to those of more specialized models.
comment: Submitted to the Workshop on The Action Gap (TAG) at CoRL 2026, updated with Gemini Flash 3.6 and Gemini 4 Argon
♻ ★ What Does It Take For A Generalist Robot Policy To Adapt Using Minimal Data?
A central goal in robot learning is to move beyond task-specific human data collection toward robots that improve through autonomous interaction. Yet fully autonomous learning remains difficult with current policies: sparse rewards and weak zero-shot exploration often prevent online learning from discovering successful behavior. As a first step towards fully autonomous learning, we propose the minimal-data adaptation regime, in which a pretrained robot policy must learn a new task from as little as one demonstration followed by autonomous online interaction. We establish that, although pretrained policies fail zero-shot on many new tasks, a single demonstration suffices to direct their behavioral prior to coarse task-following behavior, albeit with unreliable performance. From this insight, we isolate the key design principles required to convert coarse task-following into reliable task success. These include (1) the surprising efficacy of pretrained VLA representations to support sample-efficient value learning from online interaction (2) the sufficiency of residual policy learning to learn corrective behavior and (3) the need for careful data-balancing to prevent exploration collapse. As a representative of these principles, we propose MiDAS, a simple offline-to-online RL recipe that combines few-demonstration behavior cloning with value-based residual learning over frozen VLA representations. Across LIBERO and RoboCasa tasks, MiDAS substantially improves performance from single-demonstration initializations, including policies that succeed only rarely, validating the principles highlighted above. On a bimanual YAM platform, starting from a brittle initial policy, MiDAS improves task success after approximately six hours of online interaction.
♻ ★ Multilaminate piezoelectric polymer actuators for insect-scale softrobots
Piezoelectric ceramics enable fast, precise, and powerful actuation with integrated sensing in insect-scale robots. However, they are brittle, failing at low strains limiting its applications involving high impacts, unexpected loading, or soft robotics. To address this limitation, we investigate polyvinylidene fluoride (PVDF) as a compliant alternative: it tolerates substantially larger strains, but its low piezoelectric coefficient limits the output of conventional single-layer actuators. We demonstrate a novel manufacturing process for multilayer PVDF actuators that utilizes off the shelf materials and provides parallel bussing of signal voltage to reduce integration complexity. Using first principles models validated with experiments, we show a tradeoff between force (linear or superlinear increase with layer count) and displacement (cubic decrease with layer count) for the actuator geometry chosen. Our work positions PVDF actuation an optimization problem for future robotic applications with clear design knobs available to meet performance demands. Further, we show that using thinner PVDF layers within a stack increases the specific power density of the actuators making them ideal for size, weight, and power (SWAP) limitations in small scale robotics. Among the actuators fabricated, >9 mm of free deflection, >20 mN of blocked force, and >500 Hz bandwidth were achieved. To illustrate their potential in future robust robotic platforms, we integrate our actuators into a low-profile insect-scale robot that leverages resonance to achieve planar locomotion while being robust to large external impacts.
♻ ★ Soft Robotic Exogloves for Dexterous Mobility -- Towards Personalized Rehabilitation
Soft robotic exogloves can provide hand rehabilitation and assistance. Fitting these gloves often relies on standardized measurements not tailored to the individual, limiting their effectiveness, especially for fine articulation necessary for dexterous manipulation. We present the design, fabrication, modeling, and testing of a personalized pneumatically-actuated soft robotic exoglove. The glove was fit to a user's hand with topological scans and fabricated with silicone mold casting. Finite element analysis (FEA) was performed to evaluate actuator bending and forces from physical human-robot interaction (pHRI) between an actuator and a simplified personalized biomechanical finger model. Pneumatic pressure control experiments were conducted to flex the user's finger with static and dynamic references. Fabrication results show that topological scans enable precise tailoring to hand anatomy. Simulations showed that anatomical personalization enables analysis of pHRI contact forces, and results indicate sufficient joint mobilization with non-ideal compression on the proximal phalanx. Pneumatic testing indicates that pressure control allows accurate and targeted mobility of the metacarpophalangeal (MCP) and proximal interphalangeal (PIP) joints with intrinsic stiffness. Testing of multiple designs showed that relaxing the strain-limiting layer improves actuator-to-finger joint alignment during actuation. This work presents personalization to the human hand in structural conformability, joint topology, modeling of pHRI contact, and time-dependent actuation-deformation profiles. This lays the groundwork for informing exoglove design optimization to enable assistance in dexterous manipulation and neuromuscular rehabilitation of fine motor skills.
comment: 8 pages, 14 figures. Accepted in IEEE Transactions on Medical Robotics and Bionics (T-MRB) 2026
♻ ★ AgenticRoboRL: Bringing Coding-Agent Post-Training to Agentic Robotics
Agentic robotics systems combine planners with robot harnesses to perform complex manipulation tasks. Recent work improves these systems through better harnesses and external memory, but relies on powerful closed-source models as planners and does not improve the planners' own capabilities. In contrast, coding agents have improved substantially by post-training the model itself. We therefore investigate whether post-training can also improve robot planners. However, applying it across robot harnesses requires accommodating different execution mechanisms, including VLA skill invocation and control-code generation. To this end, we introduce AgenticRoboRL, a framework for post-training robot planners across different harnesses. The training recipe combines supervised fine-tuning with multi-turn reinforcement learning. To accommodate different execution mechanisms, harness-specific adapters connect recorded planner interactions with task outcomes, enabling a common RL training pipeline while keeping execution components fixed. On LIBERO-PRO, AgenticRoboRL raises the success rate of a 9B-parameter planner from 18.1% to 71.0% without external memory, approaching the 72.1% achieved by Codex (GPT-5.5) with external memory. These results demonstrate post-training as an effective approach for improving robot planners. We will release the framework and trained planner checkpoints.
♻ ★ Real-Time Disturbance-Aware Koopman Model Predictive Control for Quadrotors in Spatially Varying Winds
Reliable quadrotor flight in disturbed environments requires predictive models that remain accurate as unmodeled aerodynamic effects change during deployment. We introduce a disturbance-aware Koopman Control Family (DA-KCF) framework that preserves an offline-identified nominal model while learning only a disturbance-induced correction online. A reference-dependent KCF is first learned from simulation data and calibrated using limited hardware data. During deployment, the nominal KCF remains fixed while a lightweight residual ensemble learns a low-dimensional disturbance correction from causal state-command-state transitions. The Adaptive DA-KCF model is incorporated into MPC, where disturbance corrections are evaluated along the prediction horizon to optimize position and yaw references sent to the onboard stabilizing controller. Hardware experiments on a nano-quadrotor under uniform wind and a spatially varying wake demonstrate that online adaptation improves multi-step prediction and trajectory tracking as disturbance-induced model mismatch increases, while satisfying real-time control requirements.
♻ ★ Initialization-Free LEO Doppler Positioning via Semidefinite Relaxation: A Cold-Start Approach
Low Earth Orbit (LEO) satellite Doppler measurements offer an alternative source of positioning information for Internet of Things devices when Global Navigation Satellite System signals are unavailable. Conventional local optimisation methods depend on an accurate initial position and can fail during a complete cold start. We propose an initialization-free LEO Doppler positioning framework based on semidefinite relaxation. The method reformulates the nonlinear positioning problem into a semidefinite programme to obtain a receiver position estimate without prior location knowledge. We analyse relaxation tightness under noiseless conditions and derive sufficient noise bounds under parameter disturbances. Simulations and field experiments using opportunistic Iridium-NEXT signals evaluate initialization sensitivity, positioning accuracy and computational cost. In the field experiment, the proposed method achieves a three-dimensional positioning error of 140 m without an initial position, whereas local solvers converge to poor solutions under large initial offsets. Using the semidefinite estimate to initialise Gauss-Newton or Dog-Leg reduces the positioning error to 130 m. The framework provides a cold-start initialization stage for subsequent local refinement in opportunistic LEO positioning. The implementation and datasets are publicly available to support reproducibility and further research.
comment: Revised version
Multiagent Systems
★ Multimodal Embodied Agents: A Unified Capability-centric Perspective from Computer-Use to Robot-Use
Multimodal agents (MMAs) sustain interaction through reasoning, memory, tools, and feedback, most visibly as computer-use agents, while robotic systems (RSs) couple sensing and actuation under physical dynamics. We define multimodal embodied agents (MMEAs) as goal-directed systems that couple multimodal task reasoning with physical action and revise decisions from the resulting feedback. This raises a central question: what changes when a multimodal agent moves from computer-use to robot-use? We introduce PAPAV, a unified capability-centric framework of five recurring functions: Perceive the current state, Anticipate action effects, Plan a feasible course, Act through an interface or body, and Verify the outcome. Defined by function rather than architecture, these capabilities let PAPAV identify what all three share and then compare MMEAs with MMAs and with RSs. Five physical constraints, from partial observability to unverifiable outcomes, leave fewer choices fixed in advance and less room to reverse errors. Across 64 benchmarks, explicit evaluation centers on Act; only 5 assess Anticipate and 3 assess Verify, leaving both largely hidden behind task success. These constraints also frame the open challenges we identify. PAPAV therefore provides a common basis for designing reliable physical agents and measuring progress across all five capabilities rather than task success alone.
comment: Project website: https://showlab.github.io/Awesome-Multimodal-Embodied-Agent/
★ Implementation of Dynamic Microsegmentation for a Multi-Robot Network
This report describes a laboratory instantiation of a dynamic microsegmentation function for an Internet-of-Things network, specifically a multi-robot network, using a commercial software-defined microsegmentation tool. Fine-grained dynamic segmentation policies are defined, where segments are adapted based on asset functional and health tags. Automated implementation of the dynamic policies in software is also undertaken. The microsegmentation solution is shown to automatically isolate an indicated threat in real time, while maintaining unaffected operations. The demonstration indicates that microsegmentation can be implemented in IoT networks, in a way that allows for robust operation in the face of threats and disruptions.
★ Parallel AI Agent Collaboration as Graph-Structured Distributed Block Optimization
Parallel AI agents can produce useful local edits yet degrade a shared solution when their edits are combined. We model this coordination problem as discrete block optimization with an exactly evaluable objective and a known dependency graph. Independent sets certify compatible edits whose verified gains, measured against the same current solution, add exactly. A coordinator constructs a safe compatible candidate and can also evaluate interacting edits to capture favorable interactions. We prove that an optimal fractional coloring yields a merger that preserves a $1/χ_f(G)$ fraction of the total verified local improvement, where $χ_f(G)$ is the graph's fractional chromatic number. This guarantee is tight among mergers that select a subset of the same proposals. Under conditional local-improvement guarantees and a block error-bound condition, repeated collaboration converges geometrically in the objective gap. The rate separates task structure, agent allocation, and computation cost, yielding a linear program for agent assignment and an explicit time-cost trade-off. Code-repair experiments show that verified merging resolves conflicts between individually useful edits that cause naive merging to fail. On larger conflict-free tasks, parallel agents reduce modeled completion time at higher input-token-equivalent cost. On three public road networks, fractional integration retains 922 units of verified graph-cut improvement across 90 paired runs. For route-search agents sharing exclusive access to vehicle routes, fractional scheduling reduces measured wall time by a median of 15% across twelve paired runs, with route-cost changes varying across runs.
comment: 24 pages, 8 figures
★ ToM2Vec: Continuous Theory-of-Mind Tokens for Multi-Agent Coordination
Forming an accurate theory-of-mind (ToM) is key to successful multi-agent collaboration and requires reasoning under uncertainty to capture multiple possible partner states. This is hard to do in the text representation space native to large language model (LLM) agents, which more easily describe a single partner state. To address this challenge, we introduce ToM2Vec, which learns to form abstract theories-of-mind in a rich latent space via a fixed budget of continuous reasoning tokens. During training, supervision for these tokens comes from the partner's actual thoughts, which the ToM tokens are trained to abstract and match. At test-time, ToM tokens are generated by each agent and act as conditioners for that agent's thoughts and actions. Across several environments covering collaborative, competitive, and mixed-motive settings (Overcooked-AI, multiparty negotiation, Decrypto, Hanabi, and Collab Escape/Capture), we evaluate ToM2Vec's ability to support multi-agent interactions and collaboration, finding consistent improvements over the base model and a variety of competitive trained and prompted baselines. We show that latent reasoning is critical, with performance dropping when reasoning is expressed in text. Moreover, we show that ToM2Vec training generalizes to partner model families not seen at train-time, including a proprietary model (GPT-5.4). In our analysis, we provide evidence that the latent tokens encode a ToM, rather than task-specific information: ToM2Vec's tokens improve the agent's ability to directly answer questions about partner beliefs, desires, and next actions.
comment: Code: https://github.com/nsivaku/tom2vec
★ Conflicted Records: Typed Adversarial Contracts for Auditable LLM Output Quality Enforcement
LLM-generated records increasingly populate production information systems where they are treated as data assets subject to the same quality expectations as records from operational databases. A single such record can simultaneously satisfy and violate a quality rule: a grounded fact adjacent to a hallucination, a useful answer adjacent to a stereotype. Classical data cleaning cannot resolve this because the two judgments are incompatible over the same span; a single LLM judge cannot either, since it shares the training biases that produced the conflict. Existing interventions such as prompt engineering, chain-of-thought, and RLHF reduce surface error rates but provide no audit trail and cannot distinguish genuine remediation from blanket suppression. The question is whether a deterministic check over structured inter-agent records can catch the defects that referential checks alone miss. We present BiasGuard-D: two adversarial roles drawn from distinct model families exchange typed claim documents, a three-model quorum adjudicates by majority vote, and a combined referential-plus-semantic invariant validates the verdict. The semantic component fires on verdicts referential checks miss; removing the iterative loop drops invariant satisfaction sharply. The deliberation layer transfers from stereotype to hallucination remediation on HaluEval QA with only role-framing changes, raising right-answer recovery by 49 percentage points; the triage gate required a cross-family second auditor to maintain recall on objective hallucination. Every claim, vote, and instruction is stored as a typed record giving operators a provenance trail auditable under emerging AI regulations.
comment: Accepted at the International Conference on Cooperative Information Systems (CoopIS 2026)
★ OACap: Caption-Guided Active Observation for Detailed Video Captioning
Unlike question answering, where a query specifies a perceptual target, open-ended detailed captioning must determine what evidence to acquire and how to integrate it into a coherent global description. We propose Observation-Adaptive Video Captioner (OACap), a caption-centered agent that turns unsupported or missing information into targeted observation requests, uses spatiotemporal tools to acquire evidence, and iteratively verifies and revises the caption. OACap captions consistently improve performance across general video-understanding benchmarks using a fixed text-only reader. To better diagnose the information they preserve, we introduce Factual-Semantic Captioning Bench (FSCap-Bench). Its Core-Support Factuality Score (CSFS) combines claim precision with importance-aware fact coverage and responds as expected to four controlled perturbation types on 93.0-98.3% of valid caption pairs. Its Cross-Modal Semantic Round-Trip Score (CSRS) measures semantic preservation through a video-caption-video-caption round trip by matching atomic facts from source and recovered captions. Together, they connect downstream textual-memory utility with factual and semantic preservation.
comment: 14 pages, 4 figures, 4 tables. Technical report
★ When Are We Near Nash? Certifying Solutions in General-Sum Games
Heuristic solvers for general-sum extensive-form games produce a stream of iterates, and practice reports the NashGap of the returned profile. We ask what a solver can know about its own iterates from the regret accumulators and the terminal-distribution accumulators its updates already maintain. In two-player games the NashGap of a weighted average strategy equals, at the maximizing player, the weighted external regret plus a signed value correction. Bounding the regret by the running counterfactual-regret accumulator gives an online certificate whose cost per checkpoint is linear in the number of terminal histories. The certificate stays sound under alternating updates and in games with three or more players. The mismatch between what alternation evaluates and what the averages record enters as one signed comparator term, maximized in a single backward pass per player whose comparator weight is nonzero. The bound is read off the regret state, with no best response against the average. Run a portfolio of training-free regret-matching solvers, output the iterate with the smallest certificate, and verify that iterate once. On the benchmark's biased Leduc family a pool of sixteen training-free solvers, seven original and nine recent variants, solves $89\%$ of the $64$ instances at threshold $10^{-5}$ this way, matching the trajectory-minimum union of the solvers. The same protocol solves $92\%$ of the $64$ pre-registered held-out instances and $89\%$ again on a third set of $64$ instances that no earlier stage used. The zero-best-response certificate certifies all $57$ in-sample solves and all $57$ fresh ones, and $58$ of the $59$ held-out ones. Its exact-comparator refinement certifies every solve in the three sets. The certificate states both and attributes its bound. Code is available at https://github.com/lbn187/CertNE.
comment: 35 pages, 5 figures
★ Credit Where Credit Is Due: Contrastive Attribution Topology Search for Multi-Agent Systems
An LLM multi-agent system (MAS) is only as good as the agents it chooses to trust: its accuracy and cost hinge on its topology, meaning which agents are included and how they communicate. A single task score, however, does not reveal which agents help or hinder performance. We propose Contrastive Attribution Topology Search (CATS), which searches for compact teams within a fixed supergraph of specialized agents. An LLM attributer compares execution traces across candidate topologies and assigns credit to individual agents. CATS uses these credits to update supergraph edges while progressively pruning weak edges and agents, yielding a small team for deployment. With small open-weight models and matched candidate evaluations, CATS improves mean accuracy on BigCodeBench by 5.3 percentage points over score-only heuristic search while using 54\% fewer deployment tokens. On MBPP, CATS matches the strongest baselines with fewer tokens than the other searches. In a HotpotQA prompt-poisoning stress test, CATS excludes all adversarially prompted agents in all three runs, whereas random search retains some at comparable accuracy. Per-agent credit thus supports both team optimization and the pruning of potentially harmful agents.
★ Design and Tuning of Multi-Agent Systems Using Bayesian Evolutionary Optimization
Designing effective multi-agent systems (MAS) remains a challenging and resource-intensive task. We propose a Bayesian Evolutionary Optimization framework for automatically discovering task-specific MAS topologies. The approach combines evolutionary operators to generate diverse workflow graphs with Bayesian optimization to focus cost intensive evaluations on promising candidates. In contrast to LLM-based candidate selection, our approach uses a graph-aware Gaussian Process and Expected Improvement to provide an explicit selection criterion during evolutionary search. We evaluate the method on BigCodeBench for code generation and on a tabular preprocessing benchmark using TabPFN. On BigCodeBench, evolved architectures achieve the highest overall performance among the evaluated agent systems. On temporal tabular datasets, optimized MAS preprocessing improves over the default TabPFN model on several tasks, while remaining competitive on IID datasets. Overall, the results demonstrate the potential of Bayesian selection for guiding evolutionary MAS search under expensive evaluation budgets.
★ MA-UITD: Offline Multi-Agent Reward Inference for Characterizing Continuous Social Behavior
Multi-agent inverse reinforcement learning offers a powerful framework not only for imitation learning but also for understanding the neural computations supporting social interactions and decision-making. Inferring rewards from interacting agents is challenging because observed choices reflect both personal preferences and expectations about others. Existing methods often require additional environment interaction, repeated policy optimization as an inner loop, or restrictive assumptions about environment dynamics and action spaces, limiting their applicability to fixed behavioral datasets. We introduce Multi-Agent Unilateral Inverse Temporal Difference Learning (MA-UITD), an offline method for inferring explicit agent-specific rewards from joint demonstrations in continuous-state environments with either discrete or bounded continuous actions. Its core idea is to represent the long-term consequences of each agent's own actions through belief-conditioned successor features that average over other agents' behavior. Shared reward features and agent-specific preference vectors connect these unilateral values to the target reward function, while inverse temporal-difference consistency and noise-contrastive behavioral learning enable inference without known dynamics or an inner policy-optimization loop. Across simulated cooperative, competitive, and general-sum tasks, MA-UITD outperforms baselines in discrete-action settings and infers behaviorally compatible reward profiles in continuous-action settings. When applied to experimentally collected social-foraging trajectories, MA-UITD inferred distinct spatial and interaction-dependent reward structures for leaders and followers, providing a quantitative characterization of latent behavioral objectives that may be relevant for understanding the neural computations underlying social behavior.
★ Resonance and Depth: Composing Timbral Identity and Spatial Form in an Artist-in-the-Loop Co-Creative Multi-Agent Percussive Performance
This research-creation artistic report presents Resonance and Depth, a 9-minute immersive electroacoustic/acousmatic performance that treats timbre and space as primary compositional materials. Built from live electronic percussion and a personal corpus recorded, curated, and composed by the author, the work investigates how timbral identity and musical form can emerge through real-time recombination, spatialisation, and performer-system interaction rather than conventional harmonic development. The performance is realised as an artist-in-the-loop co-creative multi-agent system integrating MACAT, MACataRT, a MIDI-based musical agent, a spatialisation layer, and a conductor agent. These components recombine and transform the performer's material through concatenative synthesis, audio mosaicing, symbolic mediation, physically inspired synthesis, and object-based spatialisation. The work approaches spatialisation as an active compositional process in which position, trajectory, depth, diffusion, width, and apparent size participate in musical phrasing and form. Through this practice-based framework, Resonance and Depth examines how performer identity can be retained, transformed, and redistributed across human, algorithmic, timbral, and spatial processes within an immersive co-creative performance ecology.
comment: 4 pages, 1 figure. Proceedings of Timbre 2026, 4th International Conference on Timbre, Montréal, Québec, Canada. Research-creation artistic report
★ Beyond End-to-End Recovery: Towards a Better Understanding of LLM Steganography via Rolewise Decomposition
In large language model (LLM) steganography, an encoder hides a private message in an otherwise ordinary response, as text transmitted to a decoder, who attempts to recover it. The success of this private-message transmission is commonly measured through end-to-end (E2E) recovery, which checks whether the decoder correctly reconstructs the private message. However, E2E recovery cannot distinguish encoding failures from decoding failures or reveal how the private message is represented in the encoder's transmitted text. To address these limitations, we introduce a rolewise decomposition framework that separately examines how private messages are represented in encoder transmitted texts, what decoders recover, and what deterministic verification can establish. Across 7,200 encoder responses, we identify 168 verifiable encodings missed by decoders and 706 responses containing private messages reconstructed through word-length or token-initial patterns not explicitly specified in the encoder's instructions. These findings demonstrate that E2E recovery alone may obscure important distinctions between encoding and decoding, motivating their separate evaluation to better understand and mitigate the safety issues caused by LLM steganography.
comment: 33 pages, 5 figures
★ Learning to Incentivise: Using Reinforcement Learning for Sustainable Urban Mobility
Traffic management has traditionally focused on toll-based and road pricing solutions. However, road pricing often raises concerns about accessibility and public dissatisfaction, leading to its prohibition in some places, such as Finland. This study optimises the dynamic allocation of incentives to drivers, encouraging them to reroute onto alternative (potentially longer) paths to achieve greater societal benefit, reduced total travel time (TTT) and total emissions in the transportation network, contributing to sustainable urban mobility. We employ a multi-agent reinforcement learning approach to dynamically assign incentives to drivers to reduce both TTT and emissions, with travel times estimated using traffic simulation software. We demonstrate that, with an unlimited budget and an objective of minimising travel time, the incentive scheme reduces TTT by 16%, compared to the dynamic User-Equilibrium (UE) with a budget equivalent to about 11% of the UE total time. When the goal is to minimise emissions, a 9% reduction in CO2 emissions is observed under an unlimited budget. We demonstrate a critical trade-off: minimising TTT leads to an increase in emissions, while prioritising emission reductions raises TTT. However, with the right combination of weights in the multi-objective function, both TTT and total emissions are improved beyond the baseline.
comment: 6 pages, 5 figures. Accepted version of a paper published in the Proceedings of the 2025 9th International Conference on Models and Technologies for Intelligent Transportation Systems (MT-ITS), IEEE
★ Agentic Architecture of Information Systems. The AAM/5C Model: A Reference Model for Distributed Autonomy
Large organizations increasingly face what this book calls the cognitive complexity wall: at the scale of hundreds of interacting components, maintaining the dependency map and reacting at the pace of incidents exceeds what a human team can deliver. This conceptual volume proposes AAM/5C, in which five capabilities (Capture, Comprehend, Choose, Coordinate, Control) are instantiated across seven logical layers, L1 to L7. A single authorization rule, R-AUT, separates who proposes, who decides and who executes: a probabilistic planner (L4) proposes; a governance layer (L6) decides under signed policies and records its verdict in a signed evaluation proof; a security layer (L5) issues a short-lived act token bound to that proof; a policy enforcement point verifies both before the target layer materializes the act. Under an explicit guarantee contract, two analytical statements (H7a, H8a) hold by construction. The other statements of its ten hypotheses (H1 to H10), including log completeness (H7b) and non-bypass of the enforcement point (H8b), are refutable, and the protocol that could refute them is fixed before any measurement. The contribution of the book is a prescriptive reference model that articulates, under this single authorization rule, known mechanisms and adds three constructs, all to be validated: the Validation step of the Distributed Autonomy Loop, the ephemeral authority surface and the cognitive dependency plane. The book positions them against autonomic computing, multi-agent systems, enterprise architecture and agent protocols (MCP, A2A), gives an authorization policy in Rego and a partial TLA+ specification of peer consensus, and sets out a validation program with quantified milestones. It reports no empirical results. This English edition is a translation of the French original, produced with the assistance of AI tools and validated by the author.
comment: 120 pages, 19 figures, 15 tables, 3 listings. Conceptual volume. English translation of the French original (version V7.83; HAL: hal-05820640; Zenodo: 10.5281/zenodo.23494375). This English edition is also on Zenodo: 10.5281/zenodo.23494664
★ Credit Where Credit Is Due: Contrastive Attribution Topology Search for Multi-Agent Systems
An LLM multi-agent system (MAS) is only as good as the agents it chooses to trust: its accuracy and cost hinge on its topology, meaning which agents are included and how they communicate. A single task score, however, does not reveal which agents help or hinder performance. We propose Contrastive Attribution Topology Search (CATS), which searches for compact teams within a fixed supergraph of specialized agents. An LLM attributer compares execution traces across candidate topologies and assigns credit to individual agents. CATS uses these credits to update supergraph edges while progressively pruning weak edges and agents, yielding a small team for deployment. With small open-weight models and matched candidate evaluations, CATS improves mean accuracy on BigCodeBench by 5.3 percentage points over score-only heuristic search while using 54$\%$ fewer deployment tokens. On MBPP, CATS matches the strongest baselines with fewer tokens than the other searches. In a HotpotQA prompt-poisoning stress test, CATS excludes all adversarially prompted agents in all three runs, whereas random search retains some at comparable accuracy. Per-agent credit thus supports both team optimization and the pruning of potentially harmful agents.
♻ ★ Learning Large-Scale Competitive Team Behaviors with Mean-Field Interactions NeurIPS 2026
While multi-agent reinforcement learning (MARL) has been proven effective across both collaborative and competitive tasks, existing algorithms often struggle to scale to large populations of agents. Recent advancements in mean-field (MF) theory provide scalable solutions by approximating population interactions as a continuum, yet most existing frameworks focus exclusively on either fully cooperative or purely competitive settings. To bridge this gap, we introduce MF-MAPPO, a mean-field extension of PPO designed for zero-sum team games that integrate intra-team cooperation with inter-team competition. MF-MAPPO employs a shared actor and a minimally informed critic per team and is trained directly on finite-population simulators, thereby enabling deployment to realistic scenarios with thousands of agents. We further show that MF-MAPPO naturally extends to partially observable settings through a simple gradient-regularized training scheme. Our evaluation utilizes large-scale benchmark scenarios using our own testing simulation platform for MF team games (MFEnv), including offense-defense battlefield tasks as well as variants of population-based rock-paper-scissors games that admit analytical solutions, for benchmarking. Across these benchmarks, MF-MAPPO outperforms existing methods and exhibits complex, heterogeneous behaviors, demonstrating the effectiveness of combining mean-field theory and MARL techniques at scale.
comment: Accepted at NeurIPS 2026
♻ ★ Does Centralized Learning Help Embodied Agents Coordinate, and When Can a Tabular Study Tell? AAMAS 2027
Tabular studies are a common way to compare independent and centralized multi-agent learners, because they avoid function approximation. A natural expectation is that centralized learning, which learns joint actions, helps agents coordinate, especially when their bodies make solo success hard. Earlier work tests this only in parts: it measures coordination for one learning method, compares methods with the agents' bodies fixed, or varies speed but reports only how well the team did. We test it in a predator-prey environment, where teammates cooperatively hunt together while two teams compete. We compare independent Q-learning (IQL) with central Q-learning (CQL) in all four predator and prey pairings, under limits on speed and stamina, on a 5x5 and an 8x8 grid trained for 200,000 episodes each, and check that the agents learn on both. We measure coordination directly, with predictions for the main measurement fixed in advance. In this task, where one predator can capture prey alone, we find three things. First, CQL costs its agents in both roles on both grids, and its predators coordinate no more than IQL's. Second, its one benefit comes from acting on the best joint action: this cut capture time by 42% on the small grid with faster prey and unlimited stamina, but helped nowhere on the larger grid, where the best joint action had mostly never been tried. Third, on the larger grid a tie-breaking default decided which method won, even though the agents had passed a learning check. A tabular comparison can therefore be trusted only if it also checks how much of the state space the agents covered and implements the methods alike. We propose simple checks for both.
comment: 10 pages, 2 figures. Under peer review at AAMAS 2027
♻ ★ AI-driven Prices for Externalities and Sustainability in Production Markets AAMAS 2023
Traditional competitive markets do not account for negative externalities; indirect costs that some participants impose on others, such as the cost of over-appropriating a common-pool resource (which diminishes future stock, and thus harvest, for everyone). Quantifying appropriate interventions to market prices has proven to be quite challenging. We propose a practical approach to computing market prices and allocations via a deep reinforcement learning policymaker agent, operating in an environment of other learning agents. Our policymaker allows us to tune the prices with regard to diverse objectives such as sustainability and resource wastefulness, fairness, buyers' and sellers' welfare, etc. As a highlight of our findings, our policymaker is significantly more successful in maintaining resource sustainability, compared to the market equilibrium outcome, in scarce resource environments.
comment: A preliminary version was accepted to the 22nd International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2023). This full version has been accepted at the journal of Autonomous Agents and Multi-Agent Systems (Volume 40, article number 46, 2026)
♻ ★ Group Perspective Matters: Regulating Debate Relationships Can Mitigate Blind Conformity in Multi-Agent Debate NeurIPS 2026
Multi-Agent Debate (MAD) improves the reasoning performance of Large Language Models (LLMs) through multi-round interaction. However, LLMs in MAD are highly susceptible to blind conformity. Existing individual evaluation methods, typically based on confidence or perplexity, fail to reflect the correctness of reasoning and may even exacerbate blind conformity. To address this, we shift the perspective from individual evaluation to group interaction. We define mutual referencing among LLMs as \textbf{Debate Relationships} and recognize that regulating these relationships is the key to mitigating blind conformity. In this paper, we propose a novel framework for \textbf{D}ynamically r\textbf{E}gulating deb\textbf{A}te \textbf{R}elationships (DEAR) from the group perspective. At first, DEAR quantifies consensus and divergence as \textit{group evidence} to capture the debate state. Then, DEAR operates through three stages: 1) What: perceiving group consultation tendency and uncertainty; 2) Who: introducing a Selection RL-Agent to dynamically select reference peers; and 3) How: adopting a Behavior RL-Agent to adaptively adjust generation behaviors. Notably, we formulate the execution of the two RL-Agents as a sequential decision-making process, jointly optimizing via multi-agent reinforcement learning. Extensive experiments demonstrate that DEAR achieves superior performance while significantly reducing token consumption.
comment: NeurIPS 2026 Accept
♻ ★ Tempting the Agent: The Economics of Reputation without Persistent Identity in AI Agent Markets
Reputation is a fundamental mechanism through which markets sustain trust when service quality cannot be perfectly assessed ex ante, constituting a form of intertemporal economic capital by attracting future demand. Its effectiveness as a disciplinary mechanism depends not only on past interactions but also on the persistence of the identity to which reputation is attached. When identities can be abandoned and recreated cheaply, reputational capital may itself become an object of opportunistic exploitation. This paper develops a dynamic economic framework to study when reputation is sufficient to discipline autonomous agents. We model reputation as capital attracting future economic activity. At each point, an agent chooses between operating honestly, investing in quality to preserve future gains, or executing a one-shot deviation to extract its reputation's value and restart from a penalized identity. Our analysis relates the temptation to opportunistic behavior to identity-reset costs, reputation persistence, demand sensitivity, and enforcement design, deriving comparative statics on optimal quality provision. Autonomous AI-agent operating on the blockchain are a relevant application: infrastructures such as ERC-8004, ERC-8183, and x402 combine reputation, identity, and payments in permissionless markets. Nonetheless, our framework applies to any environment where reputation generates future business and identities are replaceable.
♻ ★ S$^2$-MAS: A Society-Scale Multi-Agent Simulation Framework with 500 Million Household Persona Agents
Simulating economic responses at society scale requires agents whose decisions reflect household circumstances and the effects of earlier actions. We introduce S$^2$-MAS, a society-scale multi-agent simulation framework built on 500 million synthetic household personas grounded in real-survey data. Its Genese engine separates decisions from state updates, while reusable financial scenarios and application ledgers track consumption, cash, debt and payments. Sharp-421m-mlx, a compact decision model trained with SFT + RLCD on 2.93 million examples, achieves 74.55% accuracy and 38.37% question-macro F1 in consumer financial-choice scenarios, outperforming Jev 1.13 and GLM-5.3-Flash on both measures. We evaluate eight financial task families and simulate 12 million distinct household agents in paired purchase-funding conditions. Jointly raising purchase goals and approved credit limits increases consumption, with responses differing across age and income groups. Education, employment and household-size differences persist after age--income standardization and mainly reflect transitions from deferral to purchase. Asset and debt groups differ in cash drawdown, repayment and closing debt. A 24-month panel varies purchase goals under a common credit product, with repayment and arrears updating subsequent offers. S$^2$-MAS links heterogeneous household decisions to evolving consumption, cash and repayment paths, supporting economic experiments across household groups, population scales and time horizons.
comment: 58 pages, 13 main figures, 7 supplementary figures
♻ ★ Learning Altruistic Collaboration in Heterogeneous Multi-Team Systems
This paper studies heterogeneous multi-team collaboration through dynamic robot allocation, where the robots themselves are treated as transferable resources. Leveraging Hamilton's rule from ecology as an altruistic decision-making mechanism, we propose a multi-team collaborative resource allocation framework with heterogeneous capabilities, transfer costs, and capability-dependent contributions. The resulting general heterogeneous resource assignment problem is combinatorial and is shown to be NP-hard. To address scalability, we develop a graph neural network policy under centralized training and decentralized execution that approximates the altruistic allocations based on Hamilton's rule. The model operates over the team interaction graph and predicts robot-level transfer decisions and subsequent robot-to-team assignments. The proposed approach is evaluated in a fire-fighting scenario against exact optimization and multi-step simulations, quantifying allocation quality, decision computation, and fire suppression. Five paired physical testbed trials with 15 robots compare learned reallocation with fixed team membership; the GNN reaches the modeled fire stopping threshold in all five trials, compared with two for static allocation.
♻ ★ WaveCommitments: Private and Verifiable Speech-Agent Marketplaces
Modern speech systems are evolving into ecosystems of autonomous speech agents that interact, negotiate, and trade assets such as embeddings, spectrograms, or raw data. In such markets, sellers cannot reveal assets for evaluation due to privacy constraints, and buyers cannot price what they cannot test. Thus, we introduce WaveCommitments, an end-to-end design that couples multivariate polynomial commitments with confidential asset valuation via Data Shapley inside a trusted execution environment. It enables selective openings and tamper-evident provenance to prove that a delivered asset matches the valuation while hiding everything else. Our extensive experiments across three standard datasets and various tasks with state-of-the-art encoders, including Whisper, reveal that we speed up the strongest baselines by up to 18.1x while reducing their proof sizes by up to 43.2x. We also analyze how a speaker's value shifts across tasks and encoders, revealing agent-level market prices.
comment: 5 pages
♻ ★ RuleSmith: Multi-Agent LLMs for Automated Game Balancing NeurIPS 2026
Game balancing is a longstanding challenge requiring repeated playtesting, expert intuition, and extensive manual tuning. We introduce RuleSmith, a framework that couples multi-agent LLM self-play with surrogate-based optimization over a parameterized rule space. As a proof of concept, we build CivMini, a simplified civilization-style game with two asymmetric factions governed by 12 tunable parameters. LLM agents interpret textual rulebooks and game states to propose actions that are validated by the game engine, yielding noisy estimates of faction win rates. Our primary implementation uses Gaussian-process Bayesian optimization with Expected-Improvement-based adaptive sampling to allocate more evaluation games to promising candidates. RuleSmith consistently finds near-balanced configurations: three independent optimization runs reach near-even empirical win rates in independent 200- and 300-game evaluations, and additional experiments with NTBEA and TPE show that the same self-play interface supports different search components. Cross-model evaluations and small human studies provide preliminary evidence of transfer across player populations. RuleSmith produces interpretable numerical rule adjustments without task-specific policy training. Project and code: https://adonis-galaxy.github.io/RuleSmith-website/
comment: Accepted at NeurIPS 2026
Systems and Control (EESS)
★ Implementation of Dynamic Microsegmentation for a Multi-Robot Network
This report describes a laboratory instantiation of a dynamic microsegmentation function for an Internet-of-Things network, specifically a multi-robot network, using a commercial software-defined microsegmentation tool. Fine-grained dynamic segmentation policies are defined, where segments are adapted based on asset functional and health tags. Automated implementation of the dynamic policies in software is also undertaken. The microsegmentation solution is shown to automatically isolate an indicated threat in real time, while maintaining unaffected operations. The demonstration indicates that microsegmentation can be implemented in IoT networks, in a way that allows for robust operation in the face of threats and disruptions.
★ Machine Learning-Enabled Wireless Power Transfer: A Task-Oriented Review of Structural Optimization, State and Parameter Estimation, and Adaptive Control
Wireless power transfer (WPT) systems must maintain efficient and safe energy delivery under variations in geometry, coupling, load, alignment, and operating conditions. Machine-learning methods are increasingly applied to accelerate structural design, estimate operating parameters and states, and support adaptive control and scheduling. This review develops a task-oriented framework organized around three interconnected domains: structural design and optimization, parameter and state estimation, and control and optimization. Supervised learning, reinforcement learning, and hybrid or physics-informed methods are compared in terms of data requirements, dependence on physical models, information sources, computational requirements, accuracy, online response, application scope, and optimization objectives. Supervised learning provides rapid surrogate modeling, regression, and classification when representative labeled data are available. Reinforcement learning addresses sequential control and scheduling but introduces challenges in reward design, safe exploration, and simulation-to-reality transfer. Hybrid methods integrate physical knowledge with data-driven models to improve data efficiency and physical consistency. The principal barriers to engineering deployment include limited and biased datasets, weak out-of-distribution generalization, insufficient physical consistency, unsafe online adaptation, embedded-resource constraints, and the absence of standardized benchmarks. Future research should prioritize uncertainty-aware and physics-informed learning, safe simulation-to-reality transfer, lightweight edge deployment, distributed coordination, reproducible evaluation, and digital-twin-assisted development.
comment: 42 pages, 7 figures; review article
★ Why the Swing Parameters Are Sloppy
Evangelou, Stanković, Kevrekidis and Transtrum compared analytic and data-driven assessments of parameter identifiability for the infinite-bus synchronous generator. Fisher information with the manifold boundary approximation method (MBAM), and output diffusion maps with geometric harmonics, report the same six identifiable parameters on the late window $t \in [3, 5]\text{ s}$, and both place the inertia $H$ and the damping $D$ among the unidentifiable ones. We explain this result from the geometry of the dynamics rather than from that of the parameter-to-data map. The rotor swing is a fast fibre that has collapsed onto a two-dimensional normally attracting invariant manifold (NAIM) of the transient fluxes before the window opens. The zeroth-order NAIM is exactly independent of $H$, $D$, $T_{d0}''$ and $T_{q0}''$, four of the five MBAM limits. We prove, under stated smoothness and rank conditions, that each of the four limits is a boundary of the model manifold at a distance equal, to first order in the parameter, to its current profiled speed. The same analysis predicts how identifiability changes with the experiment: neither a larger initial speed deviation nor a load change restores $H$ or $D$ in the late window, whereas the early window $[0, 2]\text{ s}$ makes $H$ the best-determined parameter. Evaluated in a physical metric fixed by synchrophasor accuracy limits, the late window identifies five parameters rather than six and the early window nine. The spiked-covariance structure of the linearised trajectory map explains why the principal components of whitened, sampled trajectories reproduce the prior-scaled Fisher spectrum, and regression on those components attains the Fisher-predicted recovery error in 113 of 114 parameter--scenario pairs.
comment: 23 pages, 7 figures
★ Dual Effect and Certainty Equivalence Revisited for a Linear-Gaussian Dual Control Problem
This paper revisits the relation between dual effect and certainty equivalence (CE) through a scalar linear-Gaussian model. In dual control, the current input affects not only the immediate cost but also the information available for future decisions, and hence the future cost. Dynamic programming (DP) provides a natural framework for formulating dual control problems. At each step of the Bellman recursion, an optimal input minimizes the sum of the stage cost and the expected cost-to-go. We restrict attention to stage costs minimized by the CE input. However, CE may not minimize the Bellman objective because of the input-dependent expected cost-to-go. Classical no-dualeffect (NDE) conditions guarantee CE optimality in certain problem classes by requiring the conditional distribution of the estimation error to be independent of past controls. In the present model, we observe that CE can remain optimal at the level of a single Bellman recursion even when the classical NDE conditions fail, provided that the expected cost-to-go is independent of the current input. Motivated by this observation, we introduce a notion of NDE formulated directly at the level of the Bellman objective, which we call Bellman-side NDE. Under some mild regularity conditions, we show that Bellman-side NDE can be characterized by a heat equation. This characterization identifies the class of Bellman-side NDE functions, and further yields a decomposition of a generic cost-to-go function into a Bellman-side NDE component and a dual-effect component.
★ Seeking Cost--Effective Incentive for Merchant MTDC Investment in an AC Grid: A Parametric MISOCP Model to VSC Siting and Sizing
Deploying voltage source converter (VSC)-based multi-terminal DC (MTDC) networks in existing AC grids offers a promising solution for alleviating transmission congestion, enhancing power-transfer capability, and improving operational economy. However, constructing an MTDC network requires participation from merchant investors. Therefore, the transmission system operator (TSO) responsible for the AC grid must provide an adequate economic incentive that enables merchant investors not only to recover the MTDC investment cost but also to earn a reasonable return commensurate with their risk and expectations. An insufficient incentive may fail to attract merchant investment, whereas an excessive incentive may unnecessarily reduce the economic benefits retained by the TSO. Determining an appropriate incentive that balances these competing interests therefore becomes a critical issue. To address this issue, this paper formulates a sharing-rate-parameterized mixed-integer second-order cone programming (MISOCP) model that jointly optimizes VSC siting and sizing while coordinating AC/DC system operation. A straightforward search procedure is then employed to identify the benefit-sharing rate that meets the investor's expectations while preserving sufficient benefits for the TSO. Case studies demonstrate the operating-cost benefits of MTDC integration and validate the proposed search strategy for seeking such a benefit-sharing rate.
★ Constrained Koopman Control Parametrization: Convex Controller Synthesis for Tracking under Input Constraints
Koopman operator theory represents nonlinear systems with linear and bilinear terms in the state and input observables, greatly simplifying control design. However, incorporating input constraints into Koopman-based controller synthesis remains challenging, limiting their applicability to constrained control tasks such as tracking. This paper develops a constrained Koopman Control Parametrization (cKoCoP) framework for synthesizing state-feedback controllers that satisfy prescribed input constraints with Lyapunov-certified stability of the identified lifted closed loop. The cKoCoP offers nonlinear state-feedback via linear-in-observables feedback, enabling linear controller design. The proposed controller is obtained through convex optimization: we incorporate ellipsoidal or polytopic input constraints with the S-procedure or Farkas-lemma-based conditions. We also introduce a Control Lyapunov Function (CLF) for trajectory tracking which incorporates input constraints. The approach is validated on two canonical nonlinear systems, a double pendulum and a unicycle, demonstrating data-driven tracking while respecting input constraints. Experiments validate the method's robustness to measurement noise, and runtime improvements over Koopman Model Predictive Control (KMPC) baselines. The results demonstrate that cKoCoP achieves competitive tracking performance with substantially reduced online computational cost, supporting its potential for real-time constrained tracking applications.
comment: 16 pages, 8 figures
★ A Plantwide Simulation Library for Control-Oriented Design: Analytical and Data-driven Use Cases
Control structures and operability indices for plantwide processes are developed and compared on fixed simulated flowsheets, and the configuration of a study is rarely reported in a form that another group can reproduce. In this contribution, plantbench, a library of plantwide process simulations in which the design, the controller tuning and the instrumentation of a plant are declared as hashed plain-data configurations, is presented. On a heat-integrated reactor--separator--recycle plant with an open-loop unstable reactor, a heuristic plantwide control design prices four heat-exchanger networks (HENs) by the least damping ratio of the linearized closed loop, and 2016 closed-loop runs over networks, tuning, analyzer deadtime and upsets test that ranking. Results show that the damping ratio orders the networks as the runs do with the network that recovers the most heat reaching 0.013 against 0.157 for the selected one and cycling under a throughput increase, but that it is a weak detector of which operation will oscillate. At the recall of a random forest on the first 100 min after an upset, 60 of 82 limit cycles, a damping-ratio threshold raises 173 false alarms against 5. The integral time of the temperature loop, to which the damping ratio is nearly insensitive, decides in part whether the plant holds an operating point.
comment: 6 pages, 3 figures, Submitted to the American Control Conference (ACC) 2027
★ Worst-Case Finite-Horizon Analysis of Perturbed LQR via Performance Estimation Problem
We propose an exact formulation to quantify the finite-horizon robustness of a Linear Quadratic Regulator (LQR) subject to norm-bounded Linear Time-Invariant (LTI) parametric uncertainties, using the Performance Estimation Problem (PEP) framework. Originally developed for optimization, this method allows us to reformulate the worst-case performance search as a rank-constrained Semidefinite Program (SDP). By applying operator interpolation, this formulation retains the sequence variables, allowing the worst-case to be explicitly reconstructed when a rank-one solution is found. This approach also opens new possibilities for the finite-horizon certification of broader system classes, such as data-driven models or nonlinear components. Using this framework, we observe that finite-horizon LTI perturbations do not simply follow asymptotic strategies. Instead, they align slowly decaying invariant manifolds with specific initial conditions to drive transient growth, causing a bifurcation in the worst-case behavior as the horizon length changes. Finally, we show numerically that our performance bounds sit between the upper bounds of Integral Quadratic Constraint (IQC) relaxations and a lower bound derived from a pseudospectral heuristic.
★ Recovering Plausible Transmission Outage Scenarios
Expansion planning for extreme weather needs outage scenarios that specify which transmission branches fail. However, public outage records often only report customers without power by county, making it unclear how to exploit this information to create outage scenarios. We propose a method to recover sets of outaged branches that most closely match regional load shed data. This gives us sets of plausible outages for a given hazard event. We formulate a bilevel problem with a load shedding DC optimal power flow in the lower level. We relax the binary branch states to differentiate the lower level problem, replace it with its sensitivity matrix, and use gradient descent to update the upper level. We treat the relaxed branch states as probabilities and perform randomized rounding to restore integrality. On three test systems with synthetic outages, the recovered sets reduce the load shed error to $0.02-4.5\%$. On the California Test System with storm data from December 2025, the error falls to $23\%$. These scenarios give expansion planning a reliable source of outage data tied to observed hazards.
★ Information-Induced Conservatism Frontier of Distributed Stability Certificates
Conservatism in distributed stability certification is often treated as a limitation of the certificate itself, motivating richer Lyapunov functions, tighter relaxations, or stronger optimization. But a more fundamental limitation may arise from the reporting interface: once information is compressed, physically different systems can become indistinguishable. If stable and unstable systems produce the same report, no sound method can separate them. This paper asks where this information limit lies and how much additional information is needed to move it. We model information as an information structure that partitions admissible systems into information fibers. The strongest sound certificate for a fiber is determined by its smallest stability margin, yielding a method-independent benchmark. This separates total conservatism into information-induced and method-induced parts and characterizes information order, task sufficiency, and the stability value of additional information. Three network-node studies show that network-spectrum, scalar/eigenvalue-only passivity, and independent-local reports respectively lose network--node alignment, local directional alignment, and cross-node relation. They show that critical missing information is often relational rather than dimensional. We then move from diagnosis to information design. An information-conservatism frontier gives the smallest irreducible conservatism achievable under a prescribed budget, while its inverse gives the minimum information required for a target conservatism. A witness-pair characterization turns this into a weighted covering problem. Besides, a local sensitivity law provides a computable approximation and identifies valuable unresolved quantities. The framework indicates whether conservatism should be reduced by improving the certificate or enriching the information.
★ Toward a decentralized design framework for discrete-time distributed algorithms via their emergent dynamics
This paper develops a procedure for designing discrete-time distributed algorithms to achieve a desired collective behavior. Each agent combines its node and coupling updates at every iteration, in either explicit or implicit form. For this structure, we derive necessary and sufficient conditions on the local parameters for realizing a desired continuous-time blended dynamics. These conditions yield a decentralized procedure for computing the parameters. A Tikhonov-type approximation result relates the resulting agent trajectories to the blended dynamics on a rescaled time axis. Under a stability condition imposed only on the blended dynamics, we further establish uniform-in-time approximation bounds. We also characterize the trade-off between approximation accuracy and the iteration count guaranteed by these bounds. The framework is illustrated through four applications in distributed computation and estimation.
★ SatCraft: Agentic Optimization of Scheduling Policies for Satellite-Cellular Edge Services
Mobile edge services can combine local computation with satellite and cellular access to deliver time-sensitive results in areas with intermittent terrestrial coverage. Meeting their deadlines and quality requirements calls for joint computation and transmission scheduling that accounts for task dependencies, changing link capacity, and shared resource budgets. However, optimizing resource allocations or tuning parameters within a fixed policy structure offers limited scope to correct failures rooted in the scheduling logic. We propose SatCraft, a framework in which a large language model (LLM) agent uses execution feedback to revise the code of joint scheduling policies. Each revision undergoes policy checks and complete-run evaluation to assess its validity, runtime, and effects on service completion. Because policy performance varies with access conditions, SatCraft retains valid alternatives in a policy library and selects one for each episode using initial access observations and the known resource configuration. Experiments using Starlink and cellular measurement data show that SatCraft improves cumulative completed-service value by 3.61%-16.89% over baselines, including recent state-of-the-art scheduling methods ADPRL, EdgeOPT, and TF-DDRL. It achieves the highest completed-service value in all four resource profiles and averages 25.02 ms per scheduling call. These results demonstrate how LLM-guided policy optimization can improve complete service delivery under the computing, communication, and quality constraints of satellite-cellular networks.
★ Certified Neural Approximation of Robust Nonlinear Model Predictive Control
Model predictive control is a high-performance control method, but it incurs comparatively high online computational costs across many control tasks and is challenging to realize in low-level hardware settings. To address this, we present a framework for generating neural network controllers that provably inherit the safety and stability guarantees of a robust model predictive policy. In contrast to approaches based on post-verification or additional online safety filters, our method provides constructive, a priori design conditions that ensure preservation of recursive feasibility and stability before deployment. We treat the approximation error as a bounded input disturbance, which is addressed in the MPC design. This allows the neural network controller to directly inherit robustness from the underlying model predictive controller, which enables the transfer of guarantees to the approximation. The design process is an offline, constructive synthesis pipeline and can be implemented once Lipschitz bounds are specified. We further demonstrate in simulation that the learned controller reproduces the closed-loop behavior of the original controller while reducing online computation.
comment: 12-page article plus 4-page supplement, 3 figures
★ Safe World Model for Joint Trajectory and Beamforming Design of Mobile Base Stations
Mobile base stations (MBSs), including aerial and ground-based platforms, provide flexible wireless coverage through adaptive positioning. However, jointly optimizing their trajectories and digital beamforming is challenging because movement changes channel conditions and interference, while beamforming determines the resulting signal and interference powers. These coupled decisions also account for how current movements affect future positions and communication performance under mobility and safety constraints. We formulate this problem as a constrained Markov decision process that maximizes the time-averaged sum signal-to-interference-plus-noise ratio (SINR), subject to mobility, restricted-zone avoidance, and transmit-power constraints. To support long-term planning and reduce the extensive environment interactions required by model-free reinforcement learning, we develop a SafeDreamer-based world model framework that learns latent system dynamics from experience and optimizes a joint trajectory and beamforming policy through imagined trajectories. Feasibility-preserving action mappings and safety cost signals incorporate operational constraints into policy learning. Simulation results demonstrate that the proposed framework improves the time-averaged sum SINR by an average of 7\% and 9\% compared with proximal policy and deep Q-network baselines, respectively, while achieving higher sample efficiency and more reliable constraint satisfaction.
★ Dissipation Bounds and Hardness in Globally Asymptotically Stable Quadratic Systems
We study worst-case finite-time energy dissipation in quadratic systems with proved global asymptotic stability (GAS). Estimating the maximum retained energy fraction to any fixed positive additive error below $1/2$ is NP-hard, even when the input includes a strict quartic Lyapunov function and proof of strictness, both constructed in polynomial time. NP denotes nondeterministic polynomial time. For a partially damped family, the unit-sphere minimum norm of the coupling map equals the structured coefficient distance to loss of GAS. Positive distances at most $1/2$ give two-sided inverse-square halving-time bounds with explicit coefficient factors. Polynomial-factor approximation of the least dissipated fraction, halving time or coefficient distance is NP-hard. Additive hardness persists throughout an explicit coefficient neighborhood and under direct damping with exponential stability, a strict quadratic Lyapunov function and a Hurwitz Jacobian. Unless $\mathrm{P}=\mathrm{NP}$, where P denotes deterministic polynomial time, no polynomial-time algorithm attains these guarantees on all inputs.
comment: 7 pages, 3 figures
★ Integrating Flow Matching and Model Predictive Control for Safe Robot Navigation
Learning-based path planners based on diffusion and flow matching can generate diverse trajectories from demonstrations but typically lack mechanisms for constraint satisfaction and collision avoidance during deployment. This work proposes a framework that combines flow-matching-based path planning with model predictive path-following control to integrate data-driven path generation with real-time, constraint-aware execution. The flow-matching model captures multimodal path distributions, while the predictive controller adapts motion online to satisfy state and input constraints and avoid obstacles using updated local sensor information. An event-triggered re-planning scheme incorporates predictive control solutions into subsequent path generation, promoting collision-free navigation in environments with previously unseen obstacles. Simulation results for mobile robot navigation demonstrate robust performance across cluttered environments. The framework provides a systematic approach to combining flow matching with predictive control for adaptive, data-driven navigation.
★ Payload generalization on feedforward control actions for trajectory tracking in robots with rigid and elastic joints
Industries employ robots for repetitive tasks due to their high precision and performance, typically achieved through feedforward control actions based on an accurate robot description. However, an exact dynamic model can be difficult to obtain, especially for robots with elastic elements or when the payload changes. Learning-based feedforward approaches are promising for trajectory tracking under model uncertainties, but typically face generalization issues when task parameters vary. This paper proposes a novel method to generalize the feedforward control action with respect to payload parameters, independently of the underlying controller. Firstly, the minimal number of example objects needed for payload generalization is derived. Then, an algorithm is proposed to execute the same trajectory with a new payload, given only its parameters, without knowledge of the robot's dynamic model and even in the presence of elastic joints. The effectiveness of the proposed algorithm is validated in simulations and experiments on tracking tasks with different robots and controllers. Two sets of experiments are performed: on a standard Franka Emika Panda robot, and on a Franka Emika Panda robot equipped with an elastic wrist.
comment: 18 pages, 17 figures. Accepted for publication in Control Engineering Practice
★ A Decade of Learning-Based Control: From Theoretic Foundations to Practical Deployment
Motivated by ten years of Learning-Based Control Invited Sessions at the IEEE Conference on Decision and Control (CDC), this tutorial paper reviews key advances at the intersection of machine learning and control. We focus on methods that combine the flexibility of machine learning techniques with the structure and theoretical foundations of control to design autonomous systems that are capable, reliable, and safe. We first introduce core notions for establishing stability and safety guarantees, then review approaches that learn dynamics models to improve control under uncertainty, and finally discuss data-driven methods that directly synthesize or learn policies, including reinforcement learning, imitation learning, and Bayesian optimization. We further discuss representative applications and open challenges. The goal is to provide an accessible entry point to the field and to support continued progress toward reliable learning-based control in real-world systems. To support learning and practical exploration, we share implementations and illustrative examples along with our discussion, together with additional resources on our website: https://learning-based-control.github.io/tutorial/.
★ SAFTER: Safety-Aware Filtering Through Explicit Recoverability for Robust Reinforcement Learning under Distribution Shift
Learning safe policies that generalize to unseen environments is difficult under bounded actuation, where an unsafe decision may become unrecoverable. Conventional control barrier function (CBF) filters enforce only instantaneous safety and ignore future recoverability. We propose SAFTER (Safety-Aware Filtering Through Explicit Recoverability), which embeds a differentiable, closed-form backup-CBF filter into Soft Actor-Critic training. SAFTER unifies current safety, finite-horizon backup safety, terminal recoverability, and input admissibility into a single differentiable certificate. Its CBF condition yields an exact minimum-intervention projection without iterative online optimization. In quadruped navigation, SAFTER reaches 90% training success with 30.6% fewer transitions than Vanilla SAC and 34.1-35.1% fewer than current-state CBF-filtered SAC baselines. Across unseen simulated environments, SAFTER improves success by up to 42.4 and 61.4 percentage points over the state-of-the-art methods CBF-RL and SEA-Nav, respectively, and achieves 73.3% and 76.7% success on two real-world navigation maps. Ablations identify accurate differentiation through the safety filter as a key contributor to robustness, in which replacing the projection Jacobian with the identity approximation reduces success rate in a High-Shift environment from 61.4% to 6.2%. These results show that recoverability-aware filtering serves as both a runtime safety mechanism and a learning mechanism for sample-efficient, robust policies.
comment: 24 pages, 6 figures
★ LLM2Cert: LLM Synthesis of Certificates with SMT/Lean Verification
Lyapunov functions and barrier certificates give state-space guarantees of stability and safety, but synthesizing and verifying them---especially for nonlinear, high-dimensional systems---remains an open challenge. Existing pipelines verify candidates with numerical solvers (sum-of-squares programs), interval methods, or satisfiability-modulo-theories (SMT) solvers, which often require relaxations, are conservative, or scale poorly with model dimension and nonlinearity. These limitations have also shaped benchmark design: standard test suites are dominated by low-dimensional polynomial systems chosen to suit what solvers can handle rather than what engineering applications require. We address both sides of this challenge. First, we introduce DynaCertBench, a benchmark of 105 instances: 80 non-redundant systems curated from the literature, extended with high-dimensional and control-constrained variants and new nonlinearly coupled systems, comprehensively spanning stability, region-of-attraction, forward-invariance, and safety tasks. Second, we present LLM2Cert, an LLM-driven framework that synthesizes candidate certificates and verifies them against their proof obligations: each obligation is routed to an SMT solver or directly to the Lean Proof-Repair agent. On DynaCertBench, LLM2Cert verifies 81 of 105 instances ($77.1\%$), a $17.1$ percentage point improvement over one-shot proposal without feedback. On the 81 fixed certificates, LLM2Cert verifies 22 that All-SMT leaves unresolved. Code and data are available at https://github.com/FormalCert-AI/llm2cert.
comment: 59 pages, 3 figures. Equal contribution: Zhipeng Cao and Peiyang Song. Equal advising: Bai Xue and Alessandro Abate
★ Learning to Incentivise: Using Reinforcement Learning for Sustainable Urban Mobility
Traffic management has traditionally focused on toll-based and road pricing solutions. However, road pricing often raises concerns about accessibility and public dissatisfaction, leading to its prohibition in some places, such as Finland. This study optimises the dynamic allocation of incentives to drivers, encouraging them to reroute onto alternative (potentially longer) paths to achieve greater societal benefit, reduced total travel time (TTT) and total emissions in the transportation network, contributing to sustainable urban mobility. We employ a multi-agent reinforcement learning approach to dynamically assign incentives to drivers to reduce both TTT and emissions, with travel times estimated using traffic simulation software. We demonstrate that, with an unlimited budget and an objective of minimising travel time, the incentive scheme reduces TTT by 16%, compared to the dynamic User-Equilibrium (UE) with a budget equivalent to about 11% of the UE total time. When the goal is to minimise emissions, a 9% reduction in CO2 emissions is observed under an unlimited budget. We demonstrate a critical trade-off: minimising TTT leads to an increase in emissions, while prioritising emission reductions raises TTT. However, with the right combination of weights in the multi-objective function, both TTT and total emissions are improved beyond the baseline.
comment: 6 pages, 5 figures. Accepted version of a paper published in the Proceedings of the 2025 9th International Conference on Models and Technologies for Intelligent Transportation Systems (MT-ITS), IEEE
★ Strategic Arbitrage in Electricity Markets: A Multi-Agent Policy Optimization Framework
Grid-scale storage and load aggregators are growing quickly in wholesale electricity markets. If highly consolidated, these new participants may distort the competitive market equilibrium through strategic withholding behavior. Traditional market power mitigation techniques are ill-suited to arbitrage-based participants. We provide a multi-agent policy optimization model to study game-theoretic equilibria for real-time markets with consolidated storage and/or aggregators. The model is designed to facilitate natural first-order equilibrium-seeking update rules. We analyze the theoretical properties of different supply functions and their implications on the convergence of the gradient dynamics. Finally, we train equilibrium policies for a modified version of the IEEE RTS-GMLC benchmark across different levels of storage fleet consolidation, to demonstrate the potential of the framework as a tool to characterize and quantify strategic arbitrage.
comment: Accepted to the 65th IEEE Conference on Decision and Control (CDC 2026). 8 pages, 6 figures
★ Integrated Longitudinal and Lateral Control for High-Speed Platooning of Articulated Vehicles
This paper presents an integrated control framework for high-speed platooning of articulated vehicles, combining advanced longitudinal and lateral control strategies to address the challenges of safe and efficient autonomous driving. A novel proof of string stability for the proposed longitudinal control algorithm, based on a PID framework, ensures robust inter-vehicle distance regulation even under dynamic operating conditions. Additionally, an arc-length-based longitudinal error formulation enhances the applicability of the control system for curved roads, making it realistic for diverse driving scenarios. For lateral control, significant improvements to the clothoid-based path tracking algorithm are introduced to mitigate oscillations and enhance stability at high speeds. These enhancements include a tangential criterion for oscillation reduction, trajectory segment classification for adaptive lookahead distance tuning, a dual-adaptive mechanism for regulating the rate of lookahead distance adjustments based on both cross-track error (CTE) and anticipated road curvature, and extensions for articulated vehicle dynamics. The proposed integrated control framework is validated through simulations, demonstrating its effectiveness in maintaining stability, minimizing tracking errors, and ensuring robust performance in high-speed and challenging platooning scenarios.
comment: Original manuscript of an article published in Vehicle System Dynamics on September 20, 2026. Published version: https://doi.org/10.1080/00423114.2026.2733699
★ Plug-in Estimation Method for Topology and Admittance Estimation in Electric Grids
This paper proposes a scalable method for topology and admittance estimation in electric power grids. Our proposed plug-in estimation method incorporates precise phasor measurement unit data to estimate planned or unplanned grid topology and admittance parameter changes. Our plug-in estimation method decouples the problem into two stages: (i) constructing and averaging voltage coefficient matrices and current vectors from phasor data, and (ii) estimating admittance parameters via a least-squares regression approach with thresholding to recover the network topology. Rigidity matrix theory underpins the design of our plug-in estimation method --- to determine the data requirements (e.g., necessary and sufficient conditions) for accurate estimation of the grid topology and admittance parameters. By means of numerical simulations on the IEEE 13-node, 123-node, and European low voltage test feeders, we demonstrate the scalability of our plug-in estimation method and benchmark the computational time against a structured total least squares method. Our simulation results highlight the practicality of the proposed method for topology and admittance estimation in large-scale electric networks, including electric grids with partial topology information.
★ Computationally Transparent Power Flexibility Aggregation via Directional Support Evaluation
The growing penetration of distributed energy resources (DERs) creates new opportunities for improving resource adequacy, operational flexibility, and power-system reliability. However, efficiently characterizing the aggregate flexibility of heterogeneous DER portfolios remains challenging because of high dimensionality, diverse operating dynamics, and spatio-temporal coupling constraints. This paper proposes a computationally transparent framework that formulates aggregate flexibility characterization as a sequence of linear programming (LP)-based directional support evaluations. The framework accommodates heterogeneous DER technologies and configurable device-, feeder-, and portfolio-level constraints. The convex hull of the resulting boundary trajectories provides a feasible inner approximation of the coupled aggregate power flexibility region. Theoretical analysis establishes feasibility and non-overestimation guarantees and characterizes the computational dependence on portfolio size, scheduling horizon, and direction budget. In the small-scale case, the proposed method produces tighter projected regions than box, zonotope, and Chebyshev inner-ball benchmarks and matches a dense directional reference within numerical tolerance using 16--64 construction directions. Large-scale simulations with up to 2000 DERs demonstrate approximately linear empirical runtime scaling with the portfolio size and direction budget over the tested ranges. The results establish a scalable and theoretically grounded approach for aggregate DER flexibility characterization.
comment: Preprint. Submitted for journal review
★ On the Adversarial Vulnerability of Neural Receivers with Unrolled LDPC Decoders
Neural receivers for 5G New Radio (NR) match classical model-based receivers in link-level performance, but their robustness to adversarial perturbations at the receiver input remains underexplored. We show that standard gradient-based attacks applied end-to-end through an unrolled low-density parity-check (LDPC) decoder fail to significantly outperform a matched-power Gaussian perturbation. We identify the failure as gradient masking caused by tanh saturation in the boxplus operation of the belief-propagation (BP) check-node update, with a per-iteration gradient norm decay of more than four orders of magnitude across ten unrolled BP iterations. Based on this finding, we propose two structurally independent loss functions that restore attack efficacy. L_BP routes the gradient through the unsaturated early iterations via per-iteration supervision, while L_LLR is a margin loss on pre-decoder log-likelihood ratios (LLRs) whose gradient bypasses the LDPC decoder entirely. Both loss functions increase the block-error rate (BLER) by one to four orders of magnitude at matched perturbation power, with the results reproduced on both the Sionna NeuralReceiver and DeepRx. At low PSR, L_LLR-optimized perturbations achieve up to three orders of magnitude higher BLER on unseen channel realizations than LBP-optimized perturbations.
★ Dynamic Active Fleet Sizing of Heterogeneous UAV Relays for Post-Disaster Communication Networks
Uncrewed aerial vehicle (UAV) communication systems are increasingly used to provide temporary connectivity when ground communication infrastructure is damaged by disasters such as hurricanes and wildfires. However, the amount of aerial support required changes as ground networks recover and communication demand evolves. We focus on dynamically adjusting the number and allocation of actively deployed UAVs within a fixed available fleet. We develop a model predictive control (MPC) framework for dynamic fleet sizing of heterogeneous UAV relays during post-disaster communication recovery. The framework accounts for communication demand, ground network recovery, heterogeneous UAV operation, UAV energy availability, and backhaul capacity, which limits how much relayed traffic can be transmitted to the external communication network. The resulting formulation is a convex quadratic program that supports efficient online fleet sizing and deployment decisions. We evaluate the framework using seven historical disaster recovery case studies based on Federal Communications Commission reports, together with four backhaul scenarios and sensitivity analyses. Results show that required active UAV fleet size varies considerably throughout recovery, while backhaul capacity and fleet composition affect communication performance, highlighting the value of adapting active fleet size over time.
♻ ★ Region of Attraction Estimation for Linear Quadratic Regulator, Linear and Robust Model Predictive Control on a Two-Wheeled Inverted Pendulum
Nonlinear underactuated systems such as two-wheeled inverted pendulums (TWIPs) exhibit a limited region of attraction (RoA), which defines the set of initial conditions from which the closed-loop system converges to the equilibrium. The RoA of nonlinear and constrained systems is generally nonconvex and analytically intractable, requiring numerical or approximate estimation methods. This work investigates the estimation of the RoA for a TWIP stabilized under three model-based control strategies: saturated linear quadratic regulator (LQR), linear model predictive control (MPC), and constraint tightening MPC (CTMPC). We first derive a Lyapunov-based invariant set that provides a certified inner approximation of the RoA. Since this analytical bound is highly conservative, a Monte Carlo-based estimation procedure is then employed to obtain a more representative approximation of the RoA, capturing how the controllers behave beyond the analytically guaranteed region. The proposed methodology combines analytical guarantees with data-driven estimation, providing both a formally certified inner bound and an empirical characterization of the RoA, offering a practical way to evaluate controller performance without relying solely on conservative analytical bounds or purely empirical simulation.
♻ ★ Proposals and challenges for zero-knowledge verification of frontier AI training
Proposals for Frontier AI rules often use cumulative training compute as the main criterion for identifying high-impact models. Recent governance analyses call zero-knowledge proofs promising but currently impractical at frontier scale [26, 4]. We argue this barrier reflects the current paradigm, not a fundamental limit. Our frontier dense-pre-training design combines a pre-committed specification, inter-node network observations, and live Merkle commitments to intermediate computation. Our reference zero-knowledge virtual machine (zkVM) provides a succinct proof layer with native BF16/FP32 precompiles. We qualitatively compare zkVMs, custom circuits, and direct interactive audits by transferability, soundness, and cost. Public randomness beacons are a separate audit-selection choice. At challenged positions, proofs check the committed computation against declared BF16/FP32 semantics bit-for-bit. A private training specification targets model-architecture confidentiality. Genesis proofs cover initialisation; then, in-training step proofs verify selected steps across the run. Ex-ante attestations enforce policy-relevant claims as running invariants on every proved step. Together they create a governance-auditable training record. Detection-probability formulas quantify sampling guarantees from the sampling budget and the share of computation altered. We give numerical examples and explain what sampling does not cover. We note, however, that weight smuggling could reuse or merge weights trained elsewhere without charging that work to the current run. We then argue that regimes with global training budgets are more robust than regimes with per-run FLOP caps. This advantage depends on counting all contributing training. We catalogue remaining research and engineering problems for external contributors.
comment: 71 pages, 2 figures
♻ ★ Physics-Informed Koopman Learning for Data-Efficient System Identification of Automated Vehicle Dynamics
Understanding how a particular automated vehicle reacts to nearby traffic can improve motion prediction, control development, traffic simulation, and safety evaluation. However, commercial automated driving systems are proprietary, and single encounters may provide only a few seconds of noisy data. Standard physical models are interpretable but inflexible, whereas neural models are often data-hungry. We propose a physics-informed Koopman framework that combines a calibrated car-following law with a correction learned from the observations. A weak-form estimator learns this correction from velocity measurements integrated over time, avoiding noise-sensitive numerical differentiation. The resulting continuous-time model predicts the follower's response and provides interpretable measures of driving behavior. Experiments on OpenACC and CATS-CF evaluate 5-, 7-, and 11-second histories. Averaged over these histories, the constant-time-headway branch reduces speed prediction error by 16.0% and 15.4% on OpenACC and CATS-CF, respectively, relative to the best neural baselines. The PI-Koopman branches achieve the lowest mean speed and gap errors among the physical and neural comparators. Weak-form estimation reduces acceleration-field reconstruction error by 74.5% in noisy known-law simulations and noise-induced field perturbation by 91.6% in paired real-vehicle measurements. The same PI-Koopman system identification supports following-mode recognition with an approximately eleven-fold difference in median residual variance between human drivers and automated vehicles, and a single residual threshold achieves 85.4% balanced accuracy. The branch informed by the intelligent driver model takes a median 17 to 26 ms for identification and forecasting on the tested central processing units, supporting frequent target adaptation.
comment: 13 pages, 9 figures. The CATS-CF dataset and code are available at https://github.com/CATS-Lab/PI-Koopman
♻ ★ Latency-Neutral Passive Cyber Monitoring for Grid-Edge Operational Technology Networks
Deterministic industrial networks often present an operational trade-off between maintaining microsecond timing of primary control loops and capturing security telemetry. This work investigates the engineering feasibility of the Operational Technology Supervisor (OTSV), a three-tier passive monitoring framework designed to resolve this visibility challenge within timing-sensitive grid-edge infrastructure. By utilizing a Layer-3-unconfigured passive ingestion mode on bare-metal hardware via a physical switched port analyzer (SPAN) loop, the deep packet inspection engine is isolated from the production network, implementing a software-enforced unidirectional export policy. Performance validation conducted over a continuous audit comprising 32,927 active-monitoring round-trip time (RTT) samples yielded stable latency distributions across a laboratory power-monitoring loop running Modbus TCP and IEC 61850 MMS protocols. The measured latency delta between the unmonitored baseline and the active monitoring state was less than 0.12 milliseconds, supporting the feasibility of continuous high-fidelity security observation alongside stringent grid-edge metering timing bounds without measurably degrading the monitored reporting path.
comment: 5 pages, 4 figures, 2 tables. Accepted for presentation at the IEEE PES Grid Edge Technologies Conference and Exposition 2027
♻ ★ The Cheap Hours Don't Stay Cheap: Price-Chasing Data Centers Reshape Grids and Markets
Data centers unsettle the paradigm of location-bound electricity demand: computing workloads can migrate across sites and hours, whereas facilities and grid connections remain fixed. This mobility reduces electricity costs but reshapes the spatiotemporal structure of nodal prices. A model-driven framework couples US wholesale market signals with optimal fleet scheduling, showing how their interaction alters prices and other participants' responses. Greater flexibility enlarges savings yet can erode network benefits: in a Texas study, temporal shifting and cross-site allocation each relieve congestion, but their combination leaves total congestion nearly unchanged. At continental scale, computing demand gravitating toward cheaper electricity narrows price differences and compresses the low-price hours storage arbitrage relies on, threatening storage investment recovery. Cost-minimizing scheduling may therefore undermine grid-supporting resources. Comparing job-level and aggregate fleet representations reveals a trade-off between operational visibility and information disclosure, informing requirements and incentives that could turn workload mobility into a lever for system-wide efficiency.
♻ ★ TNODEV: Toolbox for Neural ODE Verification
Neural ordinary differential equations (neural ODE) gained attention in safety critical settings such as continuous-time controllers for cyber-physical systems and classifiers integrated into automated decision pipelines, raising the question whether their behavior can be formally verified. Existing tools dedicated to neural ODE provide only a single reachability call without iterative input-set refinement, limiting the precision of their verdicts to whatever one reachability call can deliver. We present TNODEV, the first formal verifier for neural ODE that integrates a falsification checker, a fast interval-based reachability backend based on continuous-time mixed monotonicity, a verification and refinement loop with three input-set splitting heuristics, and a parallel scheduler in a single end-to-end pipeline. TNODEV supports safe-set inclusion verification on pure neural ODE, neural ODE in closed loop with a neural network controller and general neural ODE (GNODE), with the safe set specified either as an interval or as the half-space intersection induced by a target classification label. We evaluate TNODEV on a range of benchmarks across safe-set inclusion and classification-robustness properties, including a direct reachability comparison against NNV 2.0 and CORA and a verification comparison against NNV 2.0 on MNIST general neural ODE classifiers.
comment: 44 pages, 10 figures, Accepted in TMLR 2026. Code available at https://github.com/ab-sayed/TNODEV
♻ ★ A Novel Schur Decomposition-Based Weight Projection Method for Stable State-Space Neural-Network Architectures
Building black-box models for dynamical systems from data is a challenging problem in machine learning, especially when asymptotic stability guarantees are required. In this paper, we introduce a novel stability-ensuring and backpropagation-compatible projection scheme based on the Schur decomposition for the state matrix of linear discrete-time state-space layers, as well as an alternative pre-factorized formulation of the methodology. The proposed methods dynamically project the quasi-triangular factor of the state matrix's real Schur decomposition onto its nearest stable peer, ensuring stable dynamics with minimal overparameterization. Experiments on synthetic linear systems demonstrate that, despite a marginal increase in computational complexity, the method achieves accuracy and convergence rates comparable to those of state-of-the-art stable-system identification techniques. Furthermore, the lower weight count facilitates convergence during training without sacrificing accuracy in stacked neural-network architectures with static nonlinearities targeting real-world datasets. These results suggest that the Schur-based projection provides a numerically robust framework for identifying complex dynamics on par with the state of the art while satisfying strict asymptotic-stability requirements.
comment: 32 pages, 13 figures. Source code at https://codeberg.org/sergiovaneg/SchurSS
♻ ★ Learning Altruistic Collaboration in Heterogeneous Multi-Team Systems
This paper studies heterogeneous multi-team collaboration through dynamic robot allocation, where the robots themselves are treated as transferable resources. Leveraging Hamilton's rule from ecology as an altruistic decision-making mechanism, we propose a multi-team collaborative resource allocation framework with heterogeneous capabilities, transfer costs, and capability-dependent contributions. The resulting general heterogeneous resource assignment problem is combinatorial and is shown to be NP-hard. To address scalability, we develop a graph neural network policy under centralized training and decentralized execution that approximates the altruistic allocations based on Hamilton's rule. The model operates over the team interaction graph and predicts robot-level transfer decisions and subsequent robot-to-team assignments. The proposed approach is evaluated in a fire-fighting scenario against exact optimization and multi-step simulations, quantifying allocation quality, decision computation, and fire suppression. Five paired physical testbed trials with 15 robots compare learned reallocation with fixed team membership; the GNN reaches the modeled fire stopping threshold in all five trials, compared with two for static allocation.
♻ ★ Optimal Control of Behavioral-Feedback SIR Epidemic Model
We consider a behavioral-feedback SIR epidemic model, in which the infection rate depends in feedback on the fractions of susceptible and infected agents. Contrary to the classic SIR dynamics, the considered model allows one to account for endogenous adaptation mechanisms of the agents in response to the epidemics, such as voluntary social distancing, or the adoption of face masks. For this model, we formulate an optimal control problem for a social planner that has the ability to reduce the infection rate to keep the infection curve below a certain threshold within an infinite time horizon, while minimizing the intervention cost. Based on the dynamic properties of the model, we prove that, under quite general conditions on the infection rate, the filling-the-box strategy is the optimal control. This strategy consists in letting the epidemics spread without intervention until the threshold is reached, then applying the minimum control that leaves the fraction of infected individuals constantly at the threshold until the reproduction number becomes less than one and the infection naturally fades out. Our contribution enhances the understanding of epidemic management while accounting for adaptive human behavior, offering novel insights for robust containment strategies.
comment: 25 pages, 3 figures
♻ ★ Traffic State Prediction and Control Using a Koopman Operator Framework with Polynomial Observables
In this paper, we develop a Koopman operator framework for modeling and control of traffic flow governed by the Lighthill--Whitham--Richards (LWR) conservation law. We show that the nonlinear dynamics induced by the Greenshields flow--density relation naturally motivate a dictionary of polynomial observables, while the controlled dynamics become bilinear in the lifted Koopman coordinates. A reduced-order Koopman model is identified from spatio-temporal traffic-density snapshots and used as the prediction model in a model predictive control (MPC) framework for ramp metering. The resulting Koopman-MPC is compared with two predictive control baselines that use, respectively, a fixed linearized LWR model and the nonlinear finite-volume LWR model. In a one-hour freeway experiment, the Koopman-based controller maintains the traffic density below the critical Greenshields value and closely reproduces the behavior of the nonlinear PDE-based MPC, with a maximum-density difference below 1%. The Koopman predictor also reduces the mean online model-evaluation cost by approximately one order of magnitude relative to the nonlinear PDE predictor. In contrast, the linearized LWR predictor is computationally cheaper but exhibits substantially larger prediction errors and yields a more conservative metering policy. These results demonstrate that the analytical structure of the nonlinear traffic conservation law can be exploited to construct an interpretable data-driven surrogate for efficient traffic prediction and control
♻ ★ Skew-Induced Insertion Loss Deviation (SILD) and FOM_SILD: Metrics for Quantifying P/N Skew Effects in High-Speed Channels
The rise of AI workloads and growing data center demands have driven the need for ultra-high-speed interconnects exceeding 200 Gb/s. As unit intervals (UI) shrink, even a few picoseconds of P/N skew can degrade serializer-deserializer (SerDes) performance. Traditional methods for quantifying skew fall short in capturing its impact. We introduce two new metrics: 1) Skew-Induced Insertion Loss Deviation (SILD) and 2) its complementary Figure of Merit (FOM_SILD), analytically developed to assess P/N skew effects. Measured S-parameters confirm FOM_SILD reciprocity, while simulations of 224G PAM4 SerDes show strong correlation with bit error rate (BER) trends. This approach offers a robust framework for analyzing skew in next-generation ultra-high-speed interconnects.
♻ ★ Principles of Robot Autonomy
Autonomous robots are moving rapidly from research labs into everyday life - on roads, in the air, in warehouses, and in space. Robot autonomy is no longer solely an academic pursuit, but a collection of mature, field-tested methods and tools that practitioners rely on in real-world deployments. This book offers a clear, unified introduction to the methods that make this possible. Built on decades of teaching at Stanford, the text develops the core elements of modern autonomy stacks within a single conceptual framework, bridging classical robotics and modern physical AI. Every major topic is paired with hands-on Jupyter notebooks and implementation-driven exercises, so readers build practical intuition alongside theoretical understanding. The result is a principled, accessible, and deployment-aware foundation for anyone seeking to design, analyze, or contribute to the next generation of autonomous systems. This is a comprehensive resource for students, engineers, and researchers entering one of today's fastest-growing fields.
comment: 531 pages. Pre-publication version of a book forthcoming from Cambridge University Press, posted with the permission of the publisher
♻ ★ On the Gradient Domination of the LQG Problem
We consider solutions to the linear quadratic Gaussian (LQG) regulator problem via policy gradient (PG) methods. Although PG methods have demonstrated strong theoretical guarantees in solving the linear quadratic regulator (LQR) problem, despite its nonconvex landscape, their theoretical understanding in the LQG setting remains limited. Notably, the LQG problem lacks gradient dominance in the classical parameterization, i.e., with a dynamic controller, which hinders global convergence guarantees. In this work, we study a PG method for the LQG problem by adopting an alternative parameterization of the set of stabilizing controllers and employing a lifting argument. We refer to this parameterization as a history representation of the control input as it is parameterized by past input and output data from the previous p time-steps. Under this parameterization, the infinite-history LQG optimization reduces to an equivalent static state-feedback LQR problem, allowing us to recover gradient dominance and smoothness. For finite history length p, we establish model-based and model-free convergence guarantees up to a geometrically decaying history-truncation error.
comment: Refined theoretical results and improved analysis in the revised version
♻ ★ Bayesian Ambiguity Contraction-based Adaptive Robust Markov Decision Processes for Adversarial Surveillance Missions
Collaborative Combat Aircraft (CCAs) are envisioned to enable autonomous Intelligence, Surveillance, and Reconnaissance (ISR) missions in contested environments, where adversaries may act strategically to deceive or evade detection. These missions pose challenges due to model uncertainty and the need for safe, real-time decision-making. Robust Markov Decision Processes (RMDPs) provide worst-case guarantees but are limited by static ambiguity sets that capture initial uncertainty without adapting to new observations. This paper presents an adaptive RMDP framework tailored to ISR missions with CCAs. We introduce a mission-specific formulation in which the CCA alternates between movement and sensing states. Adversarial uncertainty is represented by finite threat prototypes defining observation, exposure, and state-action transition models. Our approach incrementally refines policies by eliminating inconsistent threat models, allowing the CCA to move from worst-case protection toward more efficient mission policies while maintaining robustness. We establish fixed point stability, agreement with nominal planning as credible sets identify the true node-specific threat prototypes, and a conservative expected value guarantee while those prototypes remain credible. Experiments under Gaussian and non-Gaussian threat models across diverse network topologies show higher mission rewards and fewer exposure events compared to nominal and static robust planners.
♻ ★ Real-Time Disturbance-Aware Koopman Model Predictive Control for Quadrotors in Spatially Varying Winds
Reliable quadrotor flight in disturbed environments requires predictive models that remain accurate as unmodeled aerodynamic effects change during deployment. We introduce a disturbance-aware Koopman Control Family (DA-KCF) framework that preserves an offline-identified nominal model while learning only a disturbance-induced correction online. A reference-dependent KCF is first learned from simulation data and calibrated using limited hardware data. During deployment, the nominal KCF remains fixed while a lightweight residual ensemble learns a low-dimensional disturbance correction from causal state-command-state transitions. The Adaptive DA-KCF model is incorporated into MPC, where disturbance corrections are evaluated along the prediction horizon to optimize position and yaw references sent to the onboard stabilizing controller. Hardware experiments on a nano-quadrotor under uniform wind and a spatially varying wake demonstrate that online adaptation improves multi-step prediction and trajectory tracking as disturbance-induced model mismatch increases, while satisfying real-time control requirements.
♻ ★ Initialization-Free LEO Doppler Positioning via Semidefinite Relaxation: A Cold-Start Approach
Low Earth Orbit (LEO) satellite Doppler measurements offer an alternative source of positioning information for Internet of Things devices when Global Navigation Satellite System signals are unavailable. Conventional local optimisation methods depend on an accurate initial position and can fail during a complete cold start. We propose an initialization-free LEO Doppler positioning framework based on semidefinite relaxation. The method reformulates the nonlinear positioning problem into a semidefinite programme to obtain a receiver position estimate without prior location knowledge. We analyse relaxation tightness under noiseless conditions and derive sufficient noise bounds under parameter disturbances. Simulations and field experiments using opportunistic Iridium-NEXT signals evaluate initialization sensitivity, positioning accuracy and computational cost. In the field experiment, the proposed method achieves a three-dimensional positioning error of 140 m without an initial position, whereas local solvers converge to poor solutions under large initial offsets. Using the semidefinite estimate to initialise Gauss-Newton or Dog-Leg reduces the positioning error to 130 m. The framework provides a cold-start initialization stage for subsequent local refinement in opportunistic LEO positioning. The implementation and datasets are publicly available to support reproducibility and further research.
comment: Revised version
Robotics
★ Joints, Camera, Action! Co-Generating 3D Motion and Video for Arbitrary Rigs
Video generation models can now depict remarkably realistic motion. Yet it is unclear how to use this motion beyond pixels: to animate a specific 3D asset and replay the animation in its 3D scene. For instance, a video model can show a monkey jumping over a crossbar, but it cannot return that jump as replayable 3D motion placed in the 3D scene. Here we present a model that directly co-generates the video and 3D motion for any supplied rig, along with a camera that registers this motion to the scene. Prior video-motion models either handle only human skeletons, often adding billions of parameters, or triangulate 3D motion from generated 2D pose maps. We instead keep 3D motion in a separate model that supports arbitrary skeletons, and connect it to a pretrained video model through a small set of cross-attention layers. This separation lets the motion model first learn how skeletons move from motion-only data, which is easier to obtain and augment than paired video. Additionally, our model learns to locate each joint in the video frames, which lets us recover the camera and place the 3D motion in the scene. To train across many skeletons, we use a generative video editor to place animated 3D assets in scenes suited to their actions, curating the edits so their known motion remains valid. Together with human and robot video, this yields nearly 200K training pairs. Our evaluations demonstrate that our model animates diverse rigs from text more faithfully than video-first pipelines, and its motion responds to the surrounding scene. It also turns video into rig motion, rig motion into video, and enables keyframe-based control of the 3D motion. Videos and additional results are available on our project page: https://research.nvidia.com/labs/par/projects/jca/.
★ Learning Adaptable Dexterous Skills via Hybrid Dexterous Mimicking
Dexterous multi-finger hands hold promise to unlock diverse skills like functional grasps, in-hand reorientation, and dynamic handover that the human-designed world assumes, yet existing learning recipes force a trade-off. Goal-reaching reinforcement learning generalizes to arbitrary object poses but, lacking any human prior, converges to unnatural and unreliable finger strategies; motion tracking reproduces human-like behavior from a single clip but cannot deviate from it. We present AdaDex, a goal-conditioned RL framework that turns a single hand-object trajectory of a few seconds---from mocap data or HOI reconstruction from in-the-wild RGB(D) video---into an adaptable dexterous skill. At the core of AdaDex is Hybrid Dexterous Mimicking: a shared policy is trained on two populations of massively parallel environments: motion tracking environments trained with dense motion tracking objective, which instill the human prior from human motion; and generalization environments with noised initial states and goals and a sparse object-success reward, which force the skill to work far off the reference. From seconds of human motion per skill, the resulting policies can learn diverse adaptable dexterous skills, such as functional grasping, object reposing, in-hand reorientation with finger gaiting, and dynamic behaviors like throw-and-catch. Our method outperforms baselines in terms of generalization and functionality, and we demonstrate transfer across simulators and to the real world. Project page: https://yuxuank.com/AdaDex/.
comment: Project page: https://yuxuank.com/AdaDex/
★ UniWALM: Unified World Action Language Model for Embodied Robot Control
Motivated by the human ability to integrate experience across language, perception, and action, we introduce UniWALM, a unified world-action-language model that learns video, action, and language in a single shared representation, where each modality is both a conditioning signal and a supervision target. We ask: can jointly generating language enrich this shared representation, improving manipulation and supporting memory-dependent behavior? Unlike prior approaches that attach a reasoning or vision-language model to a video-action backbone and run them as distinct stages, UniWALM unifies all three modalities inside a single mixture-of-transformers backbone whose per-modality diffusion heads co-denoise synchronously, decoding subtask and textual memory in the same pass as action with no separate planning stage; the generated language is re-consumed on the next step, closing a language-as-memory loop without an external planner or memory bank. To keep the action stream reliable under this richer objective, we further introduce Flow Matching with Correction (FMC), a training objective that teaches recovery from perturbed action trajectories at no inference-time cost. UniWALM consistently improves over same-backbone baselines (e.g., FastWAM) on compositional pick-and-place (MolmoSpaces) and across the memory-dependent subset of RMBench, where it reaches 34.5% without any pretrained language or reasoning model and exceeds Mem-0's keyframe-equipped setup (28.5%) using only a single anchor frame. On a real-world Find-the-Item task, generating textual memory lifts success from 11.1% to 61.1%, while state-of-the-art VLAs stay at or below chance (<=33.3%).
comment: Preprint: 25 pages
★ Industrial ReMEmbR: Scaling Spatiotemporal Memory to Robot Fleets
Warehouse robot fleets continuously observe operational conditions, but preserving these observations as reusable semantic memory incurs substantial computation, communication, and storage costs. For example, explaining recurring aisle blockages can require reviewing video from multiple robots across several shifts. We present IndustrialRemembr, a shared spatiotemporal memory system that extends NVIDIA ReMEmbR to robot fleets and reduces server-side computation by generating scene descriptions only on demand. On each robot, an NVIDIA Jetson encodes video clips into retrieval embeddings with Cosmos-Embed1, vision tokens with Cosmos3-Edge, and reconstruction latents with the Wan2.2 VAE, without uploading raw camera frames. The encoding models are re-used to detect people in image patches, zeroing them out to support PII abstraction for GDPR-compliant deployments. We split Cosmos3-Edge at its vision-language boundary, placing the vision encoder on the robot and the language decoder on an on-premises server. A Nemotron agent retrieves memories by content, position, and time and queries the decoder using cached vision tokens. This design eliminates caption generation at write time, while cached-token decoding reproduces the unsplit model's output in all 450 tested decodes.
★ Planning under Noise with Neural Operators
Planning under uncertain control execution noise requires reliable feedback policies across diverse environments. We introduce Planning Under Noise (PUN), a neural operator that learns noise-aware value fields for planning in unseen geometries. Building on an established screened-Poisson formulation of stochastic optimal control, we address two challenges: specifying boundary conditions that support navigation across varying geometries and efficiently computing fields with reliable gradients. Zero-Dirichlet obstacle boundaries terminate sampled paths at walls, weakening the goal signal along long routes and making it harder to estimate reliably. We instead impose reflecting Neumann conditions on obstacles and a Dirichlet condition at the goal, allowing sampled paths to continue exploring free space until they reach the target. To solve the resulting PDE without a configuration-space mesh, we extend a Monte Carlo (MC) solver, Walk on Stars (WoSt), to higher dimensions. However, local MC fluctuations can reverse estimated gradient directions and cause planning failures, making direct planning require costly sampling. PUN amortizes this computation by learning geometry-to-field mappings from noisy, partially converged Monte Carlo labels, avoiding expensive convergence for every training example. At inference, test-time scaling uses bounded local search guided by the predicted global field. We evaluate PUN on planning tasks across dimensions. Across four noise levels, PUN matches or outperforms all compared planners, achieving 95-100% success on the 4D replanning benchmark while maintaining the lowest runtime. At the highest tested noise level in 4D, PUN achieves 10.7-60.6% relative gains in success rate over other methods while running 3.29-32.15 times faster.
★ Physical Canvas: A Vision Tactile World Model For Robot Policy Evaluation And Learning
Evaluating and post-training robot policies through physical interaction is costly and difficult to scale, motivating action-conditioned world models as interactive simulators. However, visually plausible predictions can misrepresent contact dynamics, providing unreliable feedback for policy evaluation and learning. We introduce PhysCanvas, a state-consistent visuo-tactile world model for contact-rich robot manipulation. PhysCanvas unifies multi-view RGB, tactile images, marker displacement, task state, and actions within a unified vision space. All modalities share a video VAE and pretrained generative backbone, allowing optical touch to reuse visual and temporal priors. Motion images derived through forward kinematics and camera projection provide spatially aligned action conditioning, while an initial RGB anchor and updated reference observations guide sequential generation. Our consistency-aware post-training uses the frozen Scene Reward Model, Modality Reward Model, and State Reward Model to supervise scene preservation, visual-tactile compatibility, and agreement with real task outcomes. The resulting world model supports closed-loop policy rollouts for evaluation and provides imagined interactions and rewards for GRPO policy updates. Across four real-world manipulation tasks, when policies are rolled out in the world model, State Reward Model supervision increases the arithmetic mean of task-wise Pearson correlations between world-model and real-world success rates from 0.602 to 0.885. Policy post-training in the reward-guided world model further increases $π_{0.5}$'s mean real-world success rate from a 59.1% SFT baseline to 70.8%. These results demonstrate the value of unified multimodal modeling and consistency-aware post-training for robot policy evaluation and learning. Project page: https://zebinx.github.io/PhysCanvas_HomePage/.
★ AgenticRoboRL: Bringing Coding-Agent Post-Training to Agentic Robotics
Agentic robotics systems combine planners with robot harnesses to perform complex manipulation tasks. Recent work improves these systems through better harnesses and external memory, but relies on powerful closed-source models as planners and does not improve the planners' own capabilities. In contrast, coding agents have improved substantially by post-training the model itself. We therefore investigate whether post-training can also improve robot planners. However, applying it across robot harnesses requires accommodating different execution mechanisms, including VLA skill invocation and control-code generation. To this end, we introduce AgenticRoboRL, a framework for post-training robot planners across different harnesses. The training recipe combines supervised fine-tuning with multi-turn reinforcement learning. To accommodate different execution mechanisms, harness-specific adapters connect recorded planner interactions with task outcomes, enabling a common RL training pipeline while keeping execution components fixed. On LIBERO-PRO, AgenticRoboRL raises the success rate of a 9B-parameter planner from 18.1% to 71.0% without external memory, approaching the 72.1% achieved by Codex (GPT-5.5) with external memory. These results demonstrate post-training as an effective approach for improving robot planners. We will release the framework and trained planner checkpoints.
★ SceneRig: An Agentic System for Simulation-Ready 3D Scene Reconstruction from Single Images
Simulation is a powerful tool for robotics, but building diverse, simulation-ready scenes at scale remains difficult. Reconstructing scenes from single images is a promising alternative, but visual similarity alone is insufficient: objects must also be recovered, accurately placed, and physically grounded. Existing methods that rely on post-hoc physics settling often misplace objects, producing scenes that look correct but behave poorly under manipulation. We present SceneRig, an agentic system that turns a single image into an editable, simulation-ready scene. SceneRig combines geometric, visual, and physical tools to reconstruct a faithful 3D scene. An agent then performs edit-and-validate loops inside a physics simulator to achieve physical stability. Across diverse inputs, SceneRig improves instance segmentation, object-centric appearance, and 3D placement while keeping the reconstructed scenes physically stable. Furthermore, SceneRig is effective for downstream robotics: its scenes predict real-robot policy outcomes more accurately than the baseline method and more reliably support replay of recorded manipulations without trajectory adaptation.
comment: Accepted at CoRL 2026 Agentic Robotics Workshop. Project page: https://red-fairy.github.io/SceneRig/
★ MAPLE: Mondrian-Anchored Prior for Learning-free Indoor Exploration
Robots exploring unknown indoor environments must decide where to move while much of the environment remains unseen. Predicting the occupancy of unseen space beyond the current sensor range can improve these decisions, but recent approaches commonly rely on learned models that require offline training and repeated inference during exploration. We introduce the Mondrian-Anchored Prior for Learning-free Indoor Exploration (MAPLE), a training-free predictive approach that adapts the Mondrian process into an online generative prior conditioned on observed geometric cues in the current partial map. MAPLE generates multiple plausible map hypotheses and evaluates visibility separately within each to estimate frontier gain, preserving differences in predicted visibility across hypotheses. Across KTH floor-plans and real-SLAM environments at multiple sensor ranges, MAPLE consistently improves over non-predictive exploration and remains competitive with learned predictive approaches. Its lightweight procedural construction enables predictive exploration with limited computational resources. Experimental results further show that more accurate map prediction does not necessarily produce better exploration, highlighting the importance of how predictions are represented and used in exploration decisions.
★ CoGaussian: Open-Vocabulary Gaussian-based Collaborative 3D Scene Understanding
Collaborative perception enables agents to retrieve the scene semantics and geometry beyond the field of view of any individual agent. This broader scene understanding enhances transportation safety and improves efficiency in multi-robot systems. However, existing collaborative semantic occupancy methods rely on dense 3D voxel annotations and are limited to recognizing objects within a predefined set of semantic classes. To overcome these limitations, we present CoGaussian, an open-vocabulary Gaussian-based collaborative semantic occupancy prediction framework. CoGaussian represents a scene as a set of foundation model-aligned sparse 3D Gaussians, which serve as a compact representation for communicating and fusing geometric and semantic information across multiple agents. On the Semantic-OPV2V benchmark, our method outperforms existing open-vocabulary single-agent approaches, highlighting the effectiveness of open-vocabulary collaborative scene understanding.
★ VLM-Guided Contact Retargeting and Generative Partner Modeling for Humanoid-Human Interaction
Humanoid-human interaction requires robots to operate under severe partial observability: during close physical contact, the human partner is occluded, truncated, or outside the robot's field of view, causing pose estimation to fail and often necessitating external tracking or privileged human state. At the same time, heavy mutual occlusions introduce errors into the human-human motion-capture data used for learning. We present REACT, a framework for learning deployable humanoid-human interaction policies from human-human demonstrations. REACT treats semantic contact structure as the key supervisory signal connecting demonstration processing, interaction learning, and onboard control. First, a VLM-guided contact modeling and retargeting pipeline identifies semantically meaningful contacts and transfers them onto humanoid-human geometry while preserving interaction structure. Using these annotations, privileged interaction policies learn physically grounded behaviors through interaction-aware contact rewards and are then distilled into deployable egocentric depth policies trained entirely from onboard sensing using photorealistic human depth rendering. To address severe partial observability, we introduce generative partner-state modeling objectives that predict partner motion from occluded egocentric views. We evaluate REACT in simulation and on a real Unitree G1 humanoid across five contact-rich interactions. Semantic contact modeling substantially improves preservation of demonstrated interactions, while generative partner-state modeling improves contact recall on four of five interactions, most under heavy occlusion. On real hardware, REACT performs all five interactions using onboard sensing alone.
comment: CoRL2026, project page: https://yutoshibata07.github.io/REACT/
★ SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model
Tracking large human-motion datasets has become the dominant route to general, expressive humanoid whole-body control, but scaling across many motions is expensive. Current pipelines use model-free reinforcement learning, either merging per-motion specialists through lossy distillation, or brute-forcing a single policy with cluster-scale compute. SuperMimic amortizes learning across all motions with one differentiable world model of their shared dynamics, fit from logged transitions of a standard rigid-body simulator. The world model and the tracking policy train together in one loop, and backpropagation-through-time over short imagined rollouts gives the policy analytic gradients. Fast imagined rollouts and analytic gradients make policy learning efficient. In 3 hours on a single GPU, SuperMimic surpasses the return of a state-of-the-art motion tracker baseline trained for 7 days on 128 GPUs. It reduces GPU-hours by more than $1000\times$ and fails on only 25 of 129,514 benchmark motions, versus 179 failures for the baseline. After 9.8 hours on two GPUs, SuperMimic achieves zero failures. From 1 to 264 hours of motion, the world model's prediction error stays stable, and GPU-hours to a target return grow slowly. Our policy transfers to a physical Unitree G1 across diverse and acrobatic motions. SuperMimic removes compute as the bottleneck to scaling humanoid whole-body control, making it practical to measure motion-tracking scaling laws.
comment: Accepted at CoRL 2026 (Oral). Project website: https://supermimic.github.io
★ CLOSER: Evidence-Aware Active Visual Verification for Embodied Object-Goal Navigation
Object-goal navigation (ObjectNav) requires an embodied agent to find an instance of a specified category in an unseen environment and stop near it. The agent typically relies on an object detector but must judge online from a limited egocentric video stream, sometimes a single frame, whether a detected candidate is the target. Detectors often confidently mistake similar objects, such as a sofa for a bed, and stopping at a false positive fails the episode. Prevailing systems also approach candidates greedily, leaving them occluded or partially observed. We present CLOSER, an evidence-aware active verification framework that treats stopping as closed-loop evidence acquisition, coupling vision-language model (VLM) verification with active perception. CLOSER fine-tunes an open-source VLM to label each view of a candidate tracked across frames as target, non-target, or uncertain, selects complementary views with an embedding-based diversity gate, and fuses their per-view probabilities with a lightweight network. If the fused judgment remains uncertain, a diffusion policy drives the agent toward the candidate using only the tracked mask and depth until the decision becomes decisive. CLOSER runs locally and plugs into existing ObjectNav systems without retraining them. On Gibson and Habitat-Matterport 3D, it raises average success rate by 3.6 points and success weighted by path length by 2.0 points, showing that verifying evidence before committing improves embodied navigation.
comment: 13 pages, 8 figures, 10 tables
★ Fast Motion Retargeting Pipeline for Scalable Humanoid Robot Learning
While large-scale human motion capture datasets provide a rich source of demonstrations for humanoid robot learning, retargeting the diverse motions to different humanoid robots remains a nontrivial challenge. Existing kinodynamic trajectory optimization pipelines are computationally expensive and often require manual adjustments per motion. We present FRDV (Fast Retargeting with Dynamics Validation), a scalable framework consisting of rapid constrained kinematic retargeting and an additional dynamics validation pipeline. Its kinematic stage integrates a custom Sequential Quadratic Programming solver, Real-Time Iteration, and parallel chunk-based processing, achieving up to 9,156 frames per second, approximately up to 136 times faster than GMR and ProtoMotions, while robot-specific constraints are incorporated into the optimization to reduce common retargeting artifacts, such as ground penetration and self-collision. For motions requiring additional verification, a contact implicit iLQR controller is utilized to test the retargeted reference under full rigid body dynamics. This dynamics-based validation identifies dynamically infeasible motions and refines contact-related artifacts such as foot floating and foot skating. We evaluate retargeting quality by training motion tracking reinforcement learning policies using several state-of-the-art pipelines and comparing their tracking fidelity and success rates. Furthermore, sim-to-real transfer is demonstrated on a Unitree G1 humanoid. Videos and code are available at: https://daroslab.github.io/frdv/.
★ MARE: An Open-Source Lunar Rover Simulator with Shadow Ground Truth for Perception and Autonomy
Lunar surface exploration increasingly relies on autonomous rovers that must perceive and traverse unstructured terrain under extreme illumination conditions. Near the lunar poles, low sun elevations produce long, high-contrast shadows that challenge stereo vision, terrain segmentation, and localization. Developing perception and navigation algorithms for these conditions requires large amounts of sensor data with accurate ground truth, which field campaigns and analog facilities can only provide at high cost and with limited variety. This paper presents MARE (Modular Autonomy and Rendering Environment), an open-source simulation suite for lunar rover perception and autonomy built on Unreal Engine 5 and ROS 2. MARE consists of three integrated components. Terrain Studio generates cratered and rocky lunar Digital Elevation Maps (DEMs) to build simulation scenes. A dedicated Unreal Engine project hosts two physics-based rovers with plugins for stereo, lidar, event, and Single-Photon Avalanche Diode (SPAD) cameras, streaming time-stamped sensor data and pixel-accurate ground truth. Lastly, a Guidance, Navigation, and Control (GNC) stack provides web-based mission control. Validation shows a mean Intersection over Union (IoU) of 0.79 between predicted and rendered shadows at low sun elevations, and stereo depth analysis verifies decreased pixel coverage in shadowed regions relative to sunlit terrain.
comment: 7 pages, 12 figures, 2 tables. Code: https://github.com/raulcastar/MARE-Lunar-Simulator. Video: https://youtu.be/EI4HgbMFbBE
★ GPT-6-Astra Understands Touch and Outperform Vision--Tactile--Language--Action Models
General-purpose multimodal models have shown promise as direct robot action policies, yet whether they grasp the physics underlying real-world interactions and can apply that understanding to guide action remains underexplored. We investigate this question using contact-rich manipulation as a representative testbed, examining tactile understanding as a concrete instance of this broader capability. Contact-rich manipulation requires interpreting tactile feedback and adjusting actions after obstruction. Existing learned tactile policies acquire these capabilities through specialized pretraining and per-task optimization, with substantial data and training costs. We present GPT-as-Tactile-Policy, which summarizes expert demonstrations into task guidance and tactile references for contact assessment and corrective action. GPT-6-Astra combines this context with multimodal observations and interaction history to generate control commands and adjust actions through contact feedback. Without additional robotics-specific pretraining or per-task policy fine-tuning, our method achieves 93.8\% mean success across five UniVTAC tasks. Overall performance is comparable to state-of-the-art policies, with higher success on selected tasks. On Insert Hole, tactile feedback improves success from 31\% to 99\%. Execution analysis further demonstrates contact-guided probing and correction, supporting manipulation progress through explicit expert experience and online feedback.
comment: https://gpt-as-tactile-policy.github.io/
★ Physical AI for Intralogistics: Benchmarking the NVIDIA Sim-to-Real Ecosystem for Cosmos-Enhanced COMPASS Navigation on Industrial Mobile Robots
Classical robot navigation stacks are reliable but often require re-engineering for each new robot type and site. Learning-based navigation shifts this effort from hand-tuned parameters to data generation and training, where a policy adapts to each new embodiment or facility by fine-tuning in simulation. Whether policies adapted this way stay reliable on real industrial robots has seen little systematic testing. We study this for NVIDIA's X-Mobility and COMPASS workflow for training learned navigation policies on Idealworks robots. In simulation, we extend the COMPASS benchmark with two embodiments, the differential-drive iw.hub and an omnidirectional Humanoid Mobile Platform (HMP), and with a digital twin of the Idealworks R&D center. For the real world, we fine-tune X-Mobility on data from this digital twin, with and without sim-to-real variants generated by NVIDIA Cosmos, train COMPASS on top of it from scratch in the digital twin, deploy the resulting policies on the iw.hub's onboard Jetson AGX Xavier, and benchmark them in the Idealworks R&D center against COMPASS trained only on NVIDIA's environments and the existing Idealworks navigation stack on task success, efficiency, safety, smoothness and onboard compute. Running every static scenario both in a digital-twin replica and in the real facility measures the sim-to-real gap and how far Cosmos-augmented data closes it. Training in the digital twin with Cosmos sim-to-real augmentation raises the real-world success rate of learned navigation from 65.3% to 91.7% across four scenarios and nearly halves its weighted travel time compared with simulation-only training, bringing it within 8.3 points of the production navigation stack, fully onboard; with an integrated global planner, it also reaches far, occluded goals in every trial.
★ Plausible Future, Wrong Choice: Steering World-Action Models toward Intended Future
World-Action Models (WAMs) have emerged as a promising paradigm for embodied generalist policies by jointly modeling future world evolution and robot actions across diverse manipulation tasks. However, representing diverse futures is only the first step: a capable WAM must also generate futures that match the requirements of the current task and execution progress. In long-horizon manipulation where similar observation-action segments occur across different tasks and stages, existing approaches may generate locally plausible yet misdirected rollouts to drift toward wrong task stage, i.e., a phenomenon we term Stage Confusion. To address this problem, we propose TaskWAM, a closed-loop, stage-aware framework that represents fine-grained task requirements with Mode Tokens and tracks execution progress with State Tokens, forming a structured task state to steer world--action generation toward intended future. We further introduce StageBench, a controlled multi-stage manipulation benchmark with parameterized stage variations and automated evaluation.Extensive experiments on both simulation benchmarks and real-world tasks demonstrate that TaskWAM effectively reduces confusion and achieves competitive performance with state-of-the-art methods. Moreover, TaskWAM provides strong plug-and-play transfer across heterogeneous WAM architectures and robot embodiments, including single-arm robots, dual-arm grippers, dexterous hands, and humanoid robots.
★ CAPSafe: A Benchmark for Contextual Action and Plan Safety Reasoning in Embodied Agents
Reasoning about safety is essential for embodied agents powered by large language models (LLMs). We introduce the Contextual Action and Plan Safety benchmark (CAPSafe), designed to evaluate safety reasoning at two levels: (1) an action-level dataset that tests whether a single action is safe given its environmental context, and (2) a plan-level dataset that evaluates whether agents can generate safe long-horizon plans for complex hazardous household tasks. CAPSafe models safety conditions where hazards are situational (inferred from object properties), multiple hazards can co-occur within the same environment, and each is annotated with a relative severity level for fine-grained analysis. Evaluating nine closed and open-source LLMs reveals a substantial gap between hazard recognition and safe long-horizon planning. Several models achieve high accuracy in recognizing unsafe actions (exceeding 90% in the best cases), yet safe long-horizon planning remains substantially harder. Gemini-3-pro achieves the highest safe success rate (78.19%), while other models remain below 65%. These results highlight that robust safety reasoning over long action sequences remains an open challenge.
★ SPAR: Search-trained Push Action Ranker for Efficient Navigation Among Movable Obstacles
Navigation Among Movable Obstacles (NAMO) requires a robot to rearrange objects that block access to its goal. For a pushing robot, progress may depend on a setup push that enables a later opening without itself clearing a passage. The number of candidate sequences can grow exponentially with sequence length, making physics-based search costly. We introduce the Search-trained Push Action Ranker (SPAR), which learns to prioritize push primitives from simulated search outcomes. Searches over sequences of up to two pushes per object provide supervision that distinguishes immediate openings from setup pushes. A transformer trained with regression and ranking losses scores primitives using contact tokens conditioned on scene geometry and reachability regions. The same model guides simulator evaluations in SPAR Search and selects actions for direct execution in SPAR Policy. Both modes solve successive keyhole problems that open access to neighboring free-space regions within a NAMO planner. Across successful runs on $300$ simulated two-push keyhole problems, SPAR Search reduces median simulator calls by $7.0\times$, leading to a $3.8\times$ reduction in planning time relative to Random Search. Without retraining, the same ranker guides search on $600$ keyhole problems which have no solution within two pushes. At a depth limit of five pushes, SPAR Search achieves $50.7\%$ success within $100$ simulator calls, compared with $20.2\%$ for Random Search. These gains carry over to NAMO planning: on hard simulated scenes, SPAR Search achieves $47\%$ success within $100$ simulator calls, compared with $21\%$ for Random Search. In real-robot experiments, SPAR Policy succeeds in $37/40$ trials without a physics-based digital twin, compared with $33/40$ for SPAR Search and $27/40$ for Random Search.
★ Reinforced Iterative Learning Control for elastic joint arms: Guaranteed cross-trajectory generalization under model uncertainties
Trajectory tracking for compliant robots is challenging due to nonlinear dynamics and modeling uncertainties, often requiring learning-based methods to avoid explicit parameter identification. Iterative Learning Control (ILC) ensures theoretically high precision and preserves the passive compliance of the robot. However, traditional ILC requires a separate learning phase for each new trajectory. Conversely, Reinforcement Learning (RL) generalizes to new trajectories, but practically suffers from high sample complexity and limited tracking precision. This work proposes Reinforced Iterative Learning Control (RILC) that combines the strengths of both approaches to achieve fast adaptation and guaranteed tracking convergence across diverse trajectories. By establishing a concurrent coupling between the two controllers, RILC overcomes the high initial errors and slow adaptation of traditional ILC, enabling high-performance tracking from the very first trials. In this architecture, ILC provides high-quality samples, while RL contributes as a generalizing best-effort policy across iterations. Additionally, a theoretical proof is presented to show that the RL component preserves ILC stability and, under specific conditions, can also reduce the error bound asymptotically. The efficacy of this framework is demonstrated through simulation and experiments on an underactuated 4-degree-of-freedom serial elastic robot using different trajectories and modeling mismatches.
comment: Accepted for publication in Control Engineering Practice, 2026. 16 pages, 10 figures
★ Distributed Platoon Control for Safe and Smooth Crossing at an Unsignalized Intersection-Part I: Stability Analysis
We propose a distributed platoon controller that guarantees safe and smooth crossing of multi-vehicle systems at an unsignalized intersection. To support the stability analysis, we consider a multi-vehicle platoon maneuvering along a closed path with a single unsignalized intersection. Unlike traditional bi-level decoupled designs on open paths, which primarily focus on intersection control while neglecting its coupling with platoon convergence, and thus hindering rigorous convergence analysis, the closed-path setting allows us to analyze the asymptotic behavior of the platoon. The local exponential convergence analysis of the switched closed-loop system, induced by repeated vehicle switching at the intersection, is presented. Under some mild conditions on the controller gains, we show that safe intersection crossing is ensured during the transient phase and smooth crossing is achieved in steady state, where all vehicles travel at a common constant speed while avoiding collisions at the intersection. Finally, the effectiveness of the proposed distributed controller is demonstrated in a high-fidelity CARLA simulator using multiple commercial vehicle models.
★ Large Multimodal Models for Direct Robot Control: GPT-6 as a Beginning
Recent advances in visuospatial understanding and agentic reasoning enable direct robotic manipulation with general-purpose multimodal models, but their embodied capabilities remain poorly understood. This paper systematically evaluate GPT-6 Astra for closed-loop robotic manipulation using a lightweight agent execution framework. We first probe three fundamental embodied capabilities-depth perception, embodied in-context learning, and tactile understanding-and demonstrate that the model can exploit depth information, acquire simple manipulation strategies from contextual demonstrations, and use tactile feedback to correct its actions. We then evaluate GPT-6 Astra across five established manipulation benchmarks along six capability dimensions. Despite competitive performance on several benchmarks-26.92\% success on RoboDojo and 76.0\% on RoboTwin 2.0-our analysis reveals a pronounced gap between semantic reasoning and physical execution, particularly in consequence-aware action planning, precision, and complex motion coordination. To further examine these limitations, we introduce LIBERO-GeoMotion, a new benchmark of geometry-constrained tasks that require precise pose alignment, coordinated translation and rotation, and anticipation of physical action consequences. GPT-6 Astra achieves only 11.1\% zero-shot success, exposing the difficulty of translating spatial understanding into physically feasible motion. Real-world experiments corroborate these findings: zero-shot execution is feasible for simple tasks, yet persistent deficits remain in motion feasibility, precise control, and physical safety, while substantial inference latency further hinders practical deployment. Overall, our results characterize the emerging embodied capabilities of general-purpose multimodal models and delineate the key challenges toward safe, efficient, and reliable robotic manipulation.
★ Safe Multi-Robot Collaboration With Uniform Exclusion Using Trust
Robot swarms coordinate through directed, time-varying local communication links, some of which may be compromised to allow the transmission of arbitrary malicious data. We provide an algorithm that acts as a wrapper around robot communication---when a robot seeks to update its state using data from some incoming links, our wrapper provides a subset of trusted links whose data can be safely incorporated. It selects these links by testing a stream of trust values---data collected from an anomaly detector on each robot---using Safe Anytime-Valid Inference. We require only that the detector be better than chance. This wrapper ensures safe and performant autonomy by providing two powerful statistical guarantees: it uniformly excludes all compromised links for all time with high user-chosen probability, but nevertheless quickly learns to trust legitimate links. Both guarantees are vital to secure generic tasks, but no previous work provides them simultaneously without restrictive assumptions on connectivity structure or the number of compromised links that are not guaranteed in practice. We study our Safe Wrapper's performance in hardware for two contrasting tasks---neural implicit mapping and heading consensus---deploying on 3 LoCoBots and 8 GoPiGos, respectively. We provide simulations using trust values sampled from hardware to validate that our approach scales well to swarms of hundreds of robots.
comment: 44 pages (30 main text + 14 supplementary); 10 figures, 6 tables
★ Perceives but Doesn't Brake: A Perception-Action Binding Gap in Reasoning Driving VLAs
Reasoning-augmented vision-language-action (VLA) driving models can perceive and reason about safety-critical hazards yet fail to translate that evidence into safe actions. We characterize this failure as a perception-action binding gap: hazard information is available inside the model but exerts insufficient control over trajectory generation. Using matched simulated counterfactuals and logged driving replay, we show that in-path hazards are actually encoded in the visual representation, but longitudinal plans are dominated by ego-speed history and respond only weakly to hazard proximity or the model's own reasoning trace. Further analysis shows that the trajectory generator is capable of braking but sometimes fails to do so due to the perception-action binding gap. Without retraining or modifying any weights, we inject a hazard-conditioned energy into the frozen flow-matching sampler of Alpamayo-1.5, restoring the missing coupling at generation time. With exact obstacle geometry, guidance cuts footprint intrusion from 58.1% to 14.5% and recovers safe braking whenever sufficient stopping distance remains, eliminating collisions entirely in our braking-relevant closed-loop battery. It also reverses the diagnosed dependence of the plan, from tracking ego speed to responding to obstacle range. These results separate hazard perception from hazard control in driving VLAs, show that unsafe behavior can arise even when the model has both the evidence and the ability to act on it, and propose a simple fix for bridging this binding gap.
★ Reasoning as a Distribution: Generative Plan Latents for Vision-Language-Action Models
Vision-Language-Action (VLA) policies increasingly reason before acting, but existing reasoning models compress this intermediate decision into a single representation. When several plans are valid, this commitment hides alternatives that could otherwise be selected or revised. We introduce the Generative Plan Latent (GPL), which represents the reasoning state as a low-dimensional conditional distribution over plans, such as which object, grasp, or route to use. This makes the reasoning state addressable: without retraining the VLA, plans can be sampled (coverage), selected on request (steering), and reweighted by deployment outcomes (outcome adaptation). We test all three on SimplerEnv manipulation with $π_0$, and extend the analysis to RoboCasa, LIBERO, and a maze domain. Steering the plan raises the fraction of executions that follow the request from 0.48 to 0.94, while the same guidance on the native action distribution shows no reliable gain. Finally, we show that an explicit plan is most useful when it carries decision information not already determined by the observation.
★ From Predictive Geometry to Action: Learning Complementary Predictive Representations for Embodied Manipulation
Effective manipulation requires a policy to capture not only about the current scene geometry, but also about how that geometry will evolve under interaction. Existing approaches either augment policies with geometric information from current observations or introduce future prediction as auxiliary supervision, leaving open how predictive geometry should be represented for action generation. Therefore, we propose a two-stage framework, which learns complementary predictive representations from multi-view video clips. During pretraining, a frozen 3D foundation model provides dense geometric features that are compressed into predictive geometric representations and optimized to preserve scene structure and its temporal evolution. Action supervision is then applied to geometric transitions to extract a complementary motor representation that emphasizes action-relevant transitions. During downstream post-training, predictive geometry is transferred through geometry queries, while the motor representation guides intermediate features along the action-generation pathway. Future observations are used only as privileged supervision during training and are not required at inference time. Across LIBERO, LIBERO-Plus, SimplerEnv, and real-world bimanual and single-arm tasks, our approach consistently improves downstream policies, including an absolute gain of 10.4 percentage points on the WidowX evaluation. We further demonstrate transfer across multiple VLA architectures and WAMs, with improvements in both future-world and action prediction. The code will be available at https://github.com/xuxiaoxxxx/CPR.
★ Design and implementation of thermal control strategy for micro-size lunar exploration rover hakuto
The research focuses on the thermal interface design of the HAKUTO Rover Flight Model (FM) and its evaluation using the final thermal vacuum experiment. Initially, an analytical thermal model was developed, dividing the rover into six distinct radiating surfaces. The surface temperature of the rover was then determined based on its orientation relative to the solar radiative flux, as well as other thermal factors such as lunar radiation, solar albedo, and conduction from the lunar regolith. By considering the impact of these heat sources, suitable mounting locations for the avionics and the angle of the solar panels were determined to optimize the rover's thermal performance and power generation while ensuring the rover's mass remained under 4 kg. After the preliminary thermal analysis, numerical computational simulations were conducted to further elaborate on the thermal response of the final HAKUTO Rover FM design, which was then verified through a thermal balance test in a vacuum environment.
comment: 69th International Astronautical Congress (IAC), Bremen, Germany, 1-5 October 2018
★ De-Diffusing WAMs: Action-Sufficient Direct Prediction for Robot Control
Cascade world-action models (WAMs) use predicted visual futures to condition robot actions, but inherit costly iterative inference from diffusion-pretrained experts. We ask what a single-step predicted future must preserve to support accurate single-step action prediction. In the evaluated WAM, resampling video noise has little effect on downstream actions, yet naively truncating both streams to one step substantially reduces task success. Video-space supervision only partially closes this gap, and a local sensitivity analysis shows that vision prediction errors have highly unequal consequences for action accuracy. These findings motivate DirectWAM, which converts pretrained video and action experts into single-step predictors without stochastic sampling through action-sufficient distillation, while retaining the explicit future-to-action cascade. After learning a one-step action model, we train the future predictor by decoding actions from its own predicted futures and backpropagating demonstration action error through the expert, with an optional future-latent reconstruction loss. The resulting objective evaluates the student-generated future through its downstream control consequences rather than relying on representation matching alone. DirectWAM achieves 97.1% average success across four LIBERO suites and 85.4% on RoboTwin 2.0, outperforming Flash-WAM at the same one-evaluation-per-stream budget. This advantage has also extended to three real-world tasks that we designed. Compared with the full iterative teacher, it reduces inference latency by $28.0\times$.
★ Interruption-Aware Recurrent Reinforcement Learning for UAS Navigation Under Temporary Overrides
Safe autonomous navigation of UASs requires tactical deconfliction to address conflicts that strategic coordination cannot prevent. This task is challenging under partial observability, when sensor or traffic data are missing, delayed, or degraded. Geometry-based methods can offer separation assurances but face limitations in dense encounters. Recurrent reinforcement learning has therefore been proposed to learn navigation policies that integrate observation-action histories to optimise long-term safety-efficiency trade-offs. However, operational safeguards may require a remote pilot, onboard detect-and-avoid system, or runtime assurance mechanism to temporarily override the learned policy. A research gap remains in understanding how these interruptions affect recurrent policies when the agent regains authority. Benchmark evidence shows that interruptions make task variables less decodable from the recurrent hidden state and increase post-interruption collision risk. The findings suggest that altered recurrent context prolongs post-handback degradation through an action-observation feedback loop, in which changed actions alter the flight path and subsequent recurrent updates. To address this vulnerability, we propose three interruption-aware architectures that condition recurrent processing on interruption context through separate feature streams, dedicated interruption memory, or gated environmental features. Across simulations, all three architectures preserve more decodable task information than a standard recurrent policy. Moreover, the strongest architecture reduces post-interruption collisions by 74% and interruption-induced mission failures by 69% relative to the recurrent baseline, without degrading interruption-free performance. These results show that safe interruption does not guarantee safe handback, and support interruption-aware recurrent design for resilient UAS autonomy.
comment: This work has been submitted to IEEE for possible publication
★ FIT-MAC: A Closed-Loop Multi-Agent Framework for Robust Vision-Language-Action Models
Vision-language-action (VLA) models have achieved strong performance on standard robotic manipulation benchmarks. However, their reliability can degrade substantially when task conditions deviate from the nominal setting. Perturbations in object positions and task instructions may lead to complete task failure. Existing VLA policies generally lack an explicit runtime mechanism for distinguishing these failure modes and selecting an appropriate recovery strategy. To systematically mitigate this issue, we propose FIT-MAC, a failure-aware, closed-loop multi-agent control framework for improving the robustness of VLA policies at inference time. FIT-MAC augments a VLA policy with an inference-time control layer. First, the Planner Agent parses the language instruction into a structured task specification that captures the manipulation goal and spatial constraints, and the Grounder Agent resolves each instruction-level reference to a visual target. Next, the Verifier Agent tracks phase-specific execution progress, distinguishes target-binding errors from local execution failures, and determines whether intervention is warranted. The Correction Agent then routes permitted interventions to target rebinding or bounded grasping or placement recovery, depending on the failure type. When no intervention is required, the pretrained VLA remains the default action generator. Extensive experiments on LIBERO-PRO demonstrate that FIT-MAC improves robustness across multiple VLA backbones, consistently increasing success rates and achieving state-of-the-art performance under diverse perturbations.
★ Temporal Difference Learning of Probabilistic Reach-Avoid Value Functions ICRA 2027
Reach-avoid sets, defining the states from which a task can be completed while avoiding unsafe regions, are fundamental tools for deploying robots in environments subject to disturbances and uncertainty, as well as for runtime task-feasibility monitoring, allowing prediction of whether a state is safe and will lead to successful task completion. Classical Hamilton-Jacobi Reachability (HJR) and its recent neural extensions (e.g., Lipschitz Continuous Reachability Learning, DeepReach) treat disturbances adversarially, computing the reach-avoid set under worst-case disturbances from a bounded set. This guarantees safety but can yield overly conservative sets when disturbances are stochastic rather than adversarial, and additionally requires the disturbance set to be known or specified in advance. We propose Probabilistic reach-avoid Value Function (PVF), a self-supervised method, applicable to black-box systems, for computing reach-avoid sets as the states with a certain reach-avoid probability under stochastic, potentially unbounded disturbances. We evaluate our approach against HJR-based baselines on systems of increasing complexity, up to a Hopper locomotion task, and show that our probabilistic sets are less conservative and more accurate than adversarial HJR sets when disturbances are non-adversarial.
comment: Submitted to ICRA 2027
★ PACT-VLA: Speculative Action-Chunk Verification with Pre-Commit Observations
Vision-language-action (VLA) policies are commonly deployed with asynchronous inference in real-world settings, where the next action chunk is generated while the robot is still executing the previous one. By the time the new chunk is committed, the robot and scene have changed, so the chunk is conditioned on a stale observation and task success drops sharply as this inference delay grows. Generating multiple candidate chunks could let newer observations inform the final choice, but existing multi-candidate methods either rank candidates with the stale inference-start observation or repeatedly switch between candidates during execution, which can make the executed action sequence discontinuous. Our key insight is that inference delay can be used as decision time. While candidate generation is still underway, candidate selection can reuse the VLA itself to choose the complete action chunk that best fits a later observation. We propose PACT-VLA, a training-free framework that generates candidates from the inference-start observation and selects one using this pre-commit observation. Its scoring function, FlowRes, measures the residual between each candidate's implied flow direction and the velocity predicted by the frozen policy under the pre-commit observation, requiring no separate verifier. A pipelined implementation overlaps observation processing with candidate generation and batches scoring, so selection adds no control-step delay on the evaluated hardware. At a four-step delay, PACT-VLA improves average success by 15.0 percentage points over RTC on Kinetix and by 5.2 points over single-candidate asynchronous execution with GR00T N1.7 on LIBERO, and it further improves existing delay-robust methods. On three real-robot tasks, it raises average success from 60% to 82% over naive asynchronous execution with the same frozen $π_{0.5}$ checkpoint.
★ Knowing When to Stop Thinking: A Dissection of Expected Free Energy as a Test-Time Compute Controller for Latent World-Model Planning
A planner using a learned world model can improve its predicted outcome while choosing a worse action. We study when expected free energy (EFE) can help decide how much computation to spend on a plan. An EFE-inspired objective combines goal distance with disagreement among stochastic predictions. We test candidate scoring, stopping, and online predictor updates separately. Stopping when elite cost stalls reduces refinement on tasks whose success plateaus with depth, with mean success close to fixed-depth references. At the larger tested weights, rewarding disagreement lowers success with a frozen model. Penalizing it helps at some task and weight settings but harms others. When training data omit a region needed to reach the goal, online updates recover part of the resulting loss. The tested cost-stall rule can fail when further optimization exploits model errors, because predicted cost keeps falling after task success peaks. Cost progress can guide computation on a plateau, but protecting performance after a peak requires information about the consequences of the chosen actions.
★ LocoPilot: Single-Operator Bilateral Teleoperation for Loco-Manipulation Policy Learning
Learning loco-manipulation policies requires demonstrations that capture coordinated base and arm motion on the target embodiment. Robot-free interfaces allow demonstrations to be collected independently of the target robot, increasing flexibility but without directly capturing platform-specific commands, proprioception, and body-mounted observations. On-robot teleoperation allows to capture these signals directly, but requires an interface capable of commanding the complete system concurrently. This paper presents LocoPilot, a single-operator bilateral teleoperation and data-collection framework implemented on a 12-DoF Unitree Go2 quadruped equipped with a 6-DoF AgileX Piper L manipulator, an Intel RealSense D435 wrist camera, and an Intel RealSense D405 head camera. A second 6-DoF Piper L arm, used as the leader and two handheld controllers provide simultaneous control of the base, body posture, manipulator, and gripper. The follower arm interaction forces are estimated without a dedicated force/torque sensor and rendered to the operator through a passivity-constrained haptic channel. A ROS~2 pipeline records synchronized multimodal observations together with a unified 11D action shared by teleoperation and policy deployment. Compared to prior work, LocoPilot uniquely unifies robot-in-the-loop legged data collection, force feedback without motion-capture instrumentation, and the same action interface at deployment. We evaluate the interface across multiple loco-manipulation tasks and validate the complete collection-to-deployment pipeline by fine-tuning MolmoAct2 \cite{MOLMO} on pick-and-place demonstrations and deploying on the physical robot without action retargeting or additional inverse kinematics. Code available at: https://github.com/iit-DLSLab/LocoPilot
★ Learning Tactile-Aware Hand Representations from Human Hand-Object Interaction Data
Tactile representation learning for dexterous manipulation often requires substantial robot-side data collection, limiting its scalability across different hands and sensors. We explore whether large-scale human hand-object interaction (HOI) data can provide a useful prior for robotic tactile learning. To this end, we propose HuTact, a framework that pretrains tactile-aware hand representations from human HOI data without requiring real tactile signals during pretraining. HuTact transfers the interaction structure learned from human hand motions to robotic tactile observations and can be further adapted with a small amount of unlabeled robot interaction data. We evaluate HuTact on tactile perception and manipulation tasks, including grasp prediction, slip detection, object classification, and tactile-conditioned policy learning. Across these tasks, human HOI pretraining improves data efficiency and provides a strong initialization for downstream learning. Notably, with HOI pretraining followed by midtraining on unlabeled robot play data, HuTact achieves 88.26% Macro-F1 on grasp prediction using only 25\% of the labeled downstream data, comparable to training from scratch on the full labeled dataset (88.23%). These results suggest that human interaction data can serve as a scalable complementary source for robotic tactile representation learning.
comment: 9 pages, 4 figures
★ EventSync: Event-Driven Memory Updates for History-Dependent Manipulation
History-dependent robotic manipulation requires task context and interaction history, as the current observation alone is insufficient to determine the next action. Hierarchical frameworks use task memory for high-level subgoal reasoning and low-level policies for execution. However, memory updates require accurate completion assessment: similar observations may correspond to different execution phases, and prematurely recording completion can mislead subsequent subgoal reasoning. We present EventSync, a hierarchical framework that coordinates task reasoning and action execution through event-driven memory updates. A vision-language model generates subgoals from task plans and interaction history, while a vision-language-action model executes them. A lightweight Event Monitor combines visual observations with end-effector trajectories to determine whether the current subgoal is complete. The framework records the current subgoal in its completed-subgoal history and triggers subsequent reasoning only after obtaining completion evidence. With visual history in the executor, EventSync achieves 64.5% overall success on RoboMME, exceeding the best baseline reported in the benchmark by 20.0 percentage points. Further analysis shows that this update mechanism helps build a reliable completed-subgoal history. Experiments on four real-robot tasks with different memory requirements validate the framework's effectiveness in real-world robotic manipulation.
★ What Novice Operators See Changes What the Policy Learns: A User Study of Keypoint Hand Visual Representation for Teleoperated Demonstration
Imitation learning increasingly relies on novice operators, whose demonstrations fall short of experts' in number and inter-trial consistency. Operators act on what they are shown, yet how the rendering of the robot affects their demonstrations, and the policies trained on them, has not been examined. We study this question with PointHand, a teleoperation interface that renders a multi-fingered robot hand as sparse keypoints at its fingertips and joints, the same keypoints a policy can observe. In a simulated mug-in-box task, 18 novices and 4 experts collected 2,200 trials under three renderings: the real shape of the hand, all keypoints emphasized, or the thumb and index finger alone emphasized. The novices shown the thumb and index finger emphasized collected 441 successful demonstrations from 600 trials, against 320 for those shown the real shape. We then trained diffusion policies on these demonstrations, replacing the scene point cloud and proprioceptive state they observe with the hand keypoints and two object keypoints. Within 400 steps these demonstrations went from the weakest of the three novice conditions under the point-cloud policy (46.3%) to the strongest under the keypoint policy (57.3%), a gain of +11.0 percentage points in all five seeds. Those collected with all keypoints emphasized changed by at most +0.7 percentage points, and in no more than three seeds. The value of a demonstration therefore depends on the rendering shown during collection and on what the policy observes together. A shared representation can pay off on both sides, in the number of successful demonstrations and in the success rate of the policy trained on them. Project page: https://omron-sinicx.github.io/point-hand/
comment: 9 pages, 8 figures, 2 tables
★ High-Precision RCM Motion for Ophthalmic Robotics via Hierarchical QP Control and Authority-Guided Recovery Planning
General-purpose manipulators can position ophthalmic instruments under a software Remote Centre of Motion (RCM). However, at some reachable configurations that are nonsingular for the tip task alone, the RCM and tip-tracking objectives conflict and the servo oscillates. We formalise this failure as a constraint-induced task singularity: the RCM-projected tip Jacobian loses rank while the unprojected tip Jacobian stays well conditioned. Thus, the effective tip authority collapses although the target remains reachable. We present a high-precision RCM servo architecture with built-in singularity recovery: for each commanded target, an authority-guided planner selects a well-conditioned arm configuration that reaches it and plans an RCM-consistent route, executed by a fixed hierarchical-QP controller. On a 200-scenario kinematic benchmark, the complete nominal-and-recovery protocol reaches 128/128 near-singular targets and 71/72 near-joint-limit targets (100% and 99%, respectively), with the software-RCM residual below 0.142 mm. On a physical 6-DOF arm it executes across four RCM entry points, with a worst-case kinematically reconstructed RCM residual of 178 μm and a median tip-regulation error of 32μm, within the 200μm control requirement adopted for the ophthalmic task.
comment: 8 pages
★ Enumeration-Guided Training of Neural Hamilton-Jacobi Safety Certificates
Hamilton-Jacobi (HJ) reachability analysis offers formal safety guarantees for control systems, but training accurate neural certificates remains difficult, particularly when unsafe regions are hard to characterize. We propose a verification-guided training framework that addresses this by using probabilistic enumeration of safety constraints to detect forward-invariance violations, i.e., states where the learned certificate incorrectly classifies unsafe behavior. This produces a compact, targeted set of counterexamples that more precisely captures the failure boundary than inputs generated by standard adversarial attack methods. Crucially, using this representation to fine-tune a pretrained HJ value function yields larger certified regions and reduces certification time across all tested benchmarks. We further introduce a shielding mechanism based on the refined certificates to enforce safety constraints at deployment time. We evaluate the proposed safety filter in a realistic, physics-based Isaac Sim environment and through real-world autonomous racing experiments, achieving zero safety constraint violations compared to a nominal controller and a backup policy based on Linear Quadratic Regulation (LQR). These results demonstrate the effectiveness of the proposed shielding mechanism in ensuring safe operation under high-speed, safety-critical conditions.
★ Frontier LLMs Enter the World of Robotics: A Cross-Model Evaluation on RoboColiseum
Recent studies have explored the potential of multimodal large language models for robot control, yet the models' embodied capabilities remain insufficiently characterized across models, tasks, and execution settings. We conduct a multidimensional comparative evaluation on RoboColiseum across four tracks: Instruction Following, Spatial Reasoning, General Manipulation, and Robustness. We implement Agent2Act, a control harness that integrates GPT-6 Astra, GPT-6.1 Sol, Claude Opus 5.5, and Gemini 4 Argon into the evaluation environment through shared observation, action execution, and feedback interfaces. Under common task settings and scoring criteria, all four models exhibit a consistent capability profile: high instruction-following scores, with three approaching perfect performance; strong spatial reasoning; and general manipulation scores below those of the other three tracks. Within the evaluated configurations, Astra leads instruction following and robustness, while Opus leads spatial reasoning, general manipulation, and overall performance. These results reveal complementary strengths across models and a persistent gap between task understanding and reliable physical execution. Component comparisons on selected tasks further show that harness design affects task success, planning cost, and execution time. Additionally, case studies with Astra examine whole-body coordination, manipulation across robot embodiments, precision manipulation, dexterous-hand manipulation, and real-robot tasks, providing further insight into the model's capabilities and limitations across embodied tasks.
★ RoboCasa-Shift: Stage-Aware Robustness Evaluation of Vision-Language-Action Policies in Long-Horizon Household Manipulation
Vision-language-action (VLA) robustness is commonly summarized by aggregate task success, which obscures how individual executions change under distribution shifts and where failures emerge in long-horizon tasks. We introduce \textbf{RoboCasa-Shift}, a diagnostic robustness benchmark built on RoboCasa365 that combines controlled single-factor shifts, matched clean-perturbed rollouts in which each pair shares the same underlying settings and differs only in the target shift, and simulator-grounded stage-aware diagnostics. We evaluate GR00T N1.5 on four long-horizon tasks that pass a predefined clean-performance gate under four perturbation factors and three calibrated severity levels, including conditions that drive the evaluated policy to complete task failure, totaling 1,040 formal rollouts. Robustness responses vary substantially across factors and tasks, and increasing severity does not always produce monotonic degradation. For example, one camera condition yields $Δ\mathrm{SR}=0$ despite 8 of 80 matched episodes changing outcome. Stage-aware analysis further shows that the same end-to-end degradation can correspond to different execution-depth profiles. Camera $L_2$ and Gaussian blur $L_1$ both yield $Δ\mathrm{SR}=0.188$, yet their first-stage failure rates are 19.0\% and 36.4\%, respectively. Our findings suggest that VLA robustness should be viewed as a multi-dimensional property of execution rather than a single end-to-end performance measure, motivating evaluation protocols that jointly capture outcome degradation, paired outcome sensitivity, and failure progression. The code is publicly available at https://github.com/HaochengWang23/RoboCasa-Shift.
★ Agentic discovery of interpretable car-following laws from multi-source trajectory data
Car-following (CF) models is a fundamental component of microscopic traffic simulation and underpin the analysis of collective traffic dynamics, while its modeling becomes more challenging due to diverse human driving behaviors and the increasing presence of automated vehicle. Analytical CF models offer interpretable representations of longitudinal vehicle interactions, but their prescribed functional structures may limit the representation of heterogeneous driving behavior. This study proposes an agent-centered, physics-aware framework for the empirical discovery of CF laws from multi-source trajectory data. The framework iteratively combines agent-guided hypothesis generation, informed by traffic flow knowledge and state-dependent reconstruction errors, with evolutionary symbolic search and physics-aware multi-objective evaluation. Using NGSIM US-101, highD, and exiD, the framework discovers the Adaptive Equilibrium Response Model (AERM), which couples state-dependent relative-speed response with spacing-dependent equilibrium-speed restoration. AERM admits analytical equilibrium speed-spacing and flow-density relationships; its stability conditions are derived for the zero-delay formulation, and platoon simulations examine disturbance propagation. Evaluated against seven classical CF models on Shanghai naturalistic driving data, NGSIM I-80, and Waymo trajectories, AERM attains the lowest spacing and speed reconstruction errors under trajectory-specific calibration, reducing spacing RMSE by 9.11%, 7.59%, and 21.73% relative to the strongest baselines, and remains competitive under dataset-level shared parameters. These results demonstrate the fitting flexibility and analytical tractability of the discovered CF law, providing an interpretable basis for microscopic traffic simulation.
★ Multi-View Latent Action Pretraining for Cross-Embodiment Robot Learning from Videos
Action-free videos are an appealing substitute for costly teleoperated data. Latent action models (LAMs) turn videos into a supervision signal by inferring abstract actions between frames. Existing LAMs observe a single camera view, which resolves 6D robot actions only ambiguously missing out-of-plane movements and potential occlusions. We propose a LAM that ingests synchronized multi-camera views using cross-attention to resolve depth and occlusions before compressing them into latent actions. The latent actions are fed into a policy in which a latent expert and an action expert are coupled by sequential flow matching around a frozen VLM backbone. We further evaluate in strict cross-embodiment settings where task-specific demonstrations come from other embodiments. We show that dual-view LAM pretraining improves the decodability of ground-truth 6D actions by 17% compared to single-view, while scaling single-view data by two orders of magnitude is not sufficient to fully recover out-of-plane motions. The downstream policy improves success rate by +13.5% compared to +6.2% for single-view LAM. A single cross-embodiment is sufficient for skill transfer, and adding a second viewpoint outperforms adding up to four further embodiments. The results transfer to hardware where demonstrations come from a human hand improving success rate by +20% over action-only policies in 850 rollouts. We show that video and robot data are complementary, and that benefits of learning from video are task-dependent. Website: https://learnsyslab.github.io/multiview_latent_action_pretraining/
comment: Website: https://learnsyslab.github.io/multiview_latent_action_pretraining/
★ RoboJudge: Multimodal Language Models as Judges for Embodied Video Generation
Video generation models are promising foundations for embodied world models, but only if their generated rollouts are physically valid and faithful to task instructions. Multimodal large language models (MLLMs) are increasingly used to judge these properties, yet their agreement with human judgments on embodied videos has not been systematically evaluated. To address this gap, we introduce RoboJudge-Bench, a benchmark for evaluating MLLMs as judges of embodied video generation. The benchmark measures two fundamental dimensions, Physical Adherence and Instruction Alignment, across 800 human-annotated videos from 8 source corpora and 10 video generators, spanning grippers, human hands, and dexterous hands. Experiments reveal uneven alignment between existing judges and human ratings: general-purpose MLLMs show lower agreement on physical adherence, while specialized video judges designed for general-domain physical reasoning do not consistently transfer to embodied scenarios. Motivated by these findings, we construct RoboJudge-Train, comprising 12K videos annotated with 25K human-verified scores and model-assisted rationales for Physical Adherence and Instruction Alignment. We fine-tune a 9B MLLM on this dataset with supervised learning followed by reinforcement learning, yielding RoboJudge-9B. Among the model judges evaluated on RoboJudge-Bench, RoboJudge-9B achieves the highest overall agreement, reaching a Pearson correlation of 0.719 with a statistically significant 0.089 gain over Gemini-3.7-Flash, the best-performing proprietary baseline in our evaluation. Training on RoboJudge-Train also yields gains on external benchmarks, including VideoPhy2 and Physion-Eval. We further use RoboJudge-9B as a downstream evaluator to benchmark video generation models according to Physical Adherence and Instruction Alignment.
comment: Project page: https://spherelab.ai/RoboJudge
★ Give the Model a Finger: Developmentally Aligned Evaluation of Counting in Vision-Language Models NeurIPS 2026
Vision-language models (VLMs) are exact on small sets and drift into estimation on larger ones, a profile reminiscent of children before they master counting. Children cross the barrier with epistemic actions: pointing at each item while reciting number words, moving counted items aside, partitioning the array. We give VLMs the same actions as dumb image tools with no visual intelligence of their own in a training-free agent loop. Our central claim is about measurement, not intervention: a model that counts by marking leaves a normally hidden serial procedure on the image, and the developmental principles of counting turn those marks into an interpretable audit of serial visual reasoning: every tag is classified against point-level ground truth as on-target, duplicate, phantom, or omitted, the answer is checked against the number of tags, and order irrelevance is tested by forcing opposite traversal orders. Across 9 models, 682 images, and 7 harness conditions we find that (i) the exact-counting range is a model-specific signature spanning nearly two orders of magnitude (R80 from 3 to 150 dots); (ii) as an intervention the finger mostly fails: tags help 4 of 9 models on small-count photographs, hurt imprecise pointers, and rarely beat a verbal routine or a blind point list; what the loop buys is an auditable trace, not accuracy; (iii) as measurement it succeeds: a tag-perturbation probe shows that precise pointers read their answers off the tags, pointing precision tracks which models benefit, dense scenes fail by omission, and models rarely revise after seeing their own tags; and (iv) the capacities dissociate: models follow an instructed traversal order yet often change their answer when it reverses, and children's regular-over-scattered advantage appears only under the routine and survives lattices that defeat a rows-times-columns shortcut.
comment: NeurIPS 2026 Workshop BabyVLM
★ AWLS: Advantage-Weighted Latent Steering for One-Step Generative Robotic Manipulation Policies
Human-collected real-world robotic manipulation demonstrations often contain mixed-quality trajectories, including hesitant, indirect, or inefficient action segments. While diffusion- and flow-based policies can model such multimodal data, standard behavior cloning (BC) assigns equal likelihood to all demonstrated actions and may therefore reproduce suboptimal modes. Offline RL value estimates offer a natural signal for biasing policies toward higher-value behavioral modes, but directly fine-tuning the generative policy or using critic gradients for policy steering can amplify critic errors and induce prior drift. We propose Advantage-Weighted Latent Steering (AWLS), an offline post-training method for improving frozen one-step generative manipulation policies without updating their decoders. AWLS learns a residual noise shaper, an observation-conditioned residual correction to the policy's latent noise, and trains it with an advantage-weighted action reconstruction objective. The advantages are detached, scale-normalized, and used only as supervised weights on demonstrated actions, avoiding critic-gradient policy updates while preserving fast single-step inference. We further introduce a rollout-free decoder-sensitivity criterion for selecting pretrained checkpoints that remain responsive to latent-noise perturbations. Experiments on simulated and real-world robotic manipulation tasks with clean and mixed-quality demonstrations show that AWLS improves over the frozen base policy while limiting drift from the pretrained behavior prior. Project page: https://awls-corl2026.github.io/
comment: Accepted at CoRL 2026
★ Prescribed Performance Guided Safe Reinforcement Learning for Roundabout Navigation of Autonomous Vehicles
Connected and Autonomous Vehicles (CAVs) face significant challenges in roundabout navigation due to dense traffic interactions and unpredictable agent behaviors, necessitating high levels of adaptability and safety. While the combination of Reinforcement Learning (RL) and Model Predictive Control (MPC) offers a promising approach, existing frameworks suffer from a structural mismatch where pointwise RL decisions fail to align with MPC's trajectory level optimization, and often lack rigorous safety guarantees. This work proposes RLTrajGen-PPMPC-CBF, a hierarchical framework for safe, smooth and comfort roundabout navigation, which regulates the coupling between RL-based planning and MPC-based control. We employ Prescribed Performance Constraints (PPC) to enforce a gradual, stable convergence between reference trajectories and control outputs, ensuring MPC tracking performance remains consistent with intended behavior from RL. Additionally, a Control Barrier Function (CBF) layer provides real-time safety corrections that are fed back into the RL process. Simulation results in the MetaDrive environment demonstrate that our method achieves a 6.25\% increase in average success rate and a 65.87\% reduction in Motion Sickness Dose Value (MSDV) compared to the optimal baseline, meanwhile accelerating training convergence.
comment: 8 pages, 10 figures
★ When Does Experience Memory Help Closed-Loop LLM Driving Agents? A Controlled Evaluation with Resampled Warmup Pools, Token-Matched Fillers, and Audited Baselines ACML 2026
When does experience memory help a closed-loop LLM driving agent? Published gains cannot say, because the two factors we find to matter most, which warmup experiences the pool contains and what any injected tokens do to the prompt, are controlled in none of them. We answer with a controlled evaluation on one audited HighwayEnv testbed (1Hz, gpt-4o-mini for the decisive contrasts). We treat the warmup pool as a resampled random factor (ten disjoint pools) and add token-matched irrelevant-filler controls, oracle-certified diagnostics, caution-pricing metrics, and quantified audits of the baseline ports. The probe suite spans flat-text retrieval (a DiLu port), a single-frame grid port (RESPOND-inspired), our episodic temporal-graph memory, a retrieval-free consolidated rulebook, and a flat K-step observation-history baseline. Four findings follow, each a property of this testbed. (1) Headline memory gains belong to the warmup draw. The reported pool is the best of ten for both retrieval memories, which beat No-Memory in only one pool of ten (sign test p=0.011), and even the rulebook's apparent pool-immunity collapsed on five fresh draws. (2) No retrieval design tested exceeds token-matched filler. The collision-only merge ceiling is reached by a single 15-token caution instruction, and metrics that price caution erase the rulebook's lead. (3) Representational sophistication is nowhere robustly operative. (4) Baseline port quality silently inverts conclusions. On this testbed, a single-pool comparison without filler controls would have reported memory gains that these controls reverse. We release the testbed, audited ports, seeds, and caches.
comment: ACML 2026
★ PanoGraphGS: Unifying Panoptic Segmentation and Hierarchical 3D Scene Graphs with Gaussian Splatting
3D scene graphs support language-guided navigation by organizing physical entities and their containment and spatial relations into hierarchical maps. Building such structured maps requires a scene representation that consistently distinguishes semantic regions and object instances across viewpoints. To this end, we present PanoGraphGS, a unified framework that integrates panoptic 3D Gaussian representation learning with hierarchical 3D scene graph construction. The framework learns semantic and instance features on shared Gaussian geometry. Prototype-based semantic learning and association-based instance learning produce consistent panoptic predictions across viewpoints. The learned representation supports novel-view panoptic segmentation and direct Gaussian-level panoptic decoding. A semantic-aware scene graph module further recovers a floor-room-stuff-object hierarchy with containment and spatial relations. We further release NaviGS, a real-world dataset for panoramic scene understanding. The dataset provides 3DGS, mesh, RGB, semantic, and depth data. For 3D scene graph identification, PanoGraphGS outperforms the strongest evaluated baseline in object-node accuracy by more than 50\% points on average across Replica and NaviGS. PanoGraphGS also achieves state-of-the-art performance among the evaluated methods in 2D panoptic segmentation. These results demonstrate the effectiveness of PanoGraphGS in both hierarchical 3D scene understanding and 2D panoptic segmentation. The project website is available at: https://github.com/gaojh0333/PanoGraphGS.
comment: 9 pages, 5 figures
★ Adaptive Traversal Tubes for Posture-Aware Navigation of Bipedal Humanoids
Bipedal humanoids can change their effective footprint by adjusting body yaw, enabling both narrow-passage traversal and socially aware yielding. This paper presents a posture-aware navigation framework centered on \emph{Adaptive Traversal Tubes} (ATT), \emph{Interval Velocity--Heading} (IVH) control, and a sidestep-enhanced AMP locomotion prior. ATT represents a behavior-feasible corridor width field along a reference path, unifying geometric clearance and social comfort into a continuous signal. IVH maps this width to a continuous frontal-to-sidestep posture schedule and omnidirectional velocity commands. A velocity-conditioned AMP policy, trained with GVHMR-retargeted frontal/sidestep motions and coupled \((v_x,v_y)\) tracking rewards, is designed to improve transition stability over a generic locomotion controller while remaining interchangeable with the default onboard policy. This preprint presents the full method; a planned evaluation protocol on a Unitree G1 is outlined, with quantitative results to follow. Demo videos of MuJoCo simulation and real-robot deployment are available at: github.com/lborogzj997/Humanoid-Navigation-in-Crowds-and-Narrow-Spaces.
★ CovNav: Conformal Prediction-Guided Adaptive Inference for Object Goal Navigation
In object goal navigation (ObjectNav), an embodied agent must find a target object (e.g., "find a bed") in a previously unseen environment, relying on spatial commonsense such as object co-occurrence. Large Language Models (LLMs) encode this commonsense, but deploying them on robots exposes a trade-off between reasoning quality and efficiency: cloud-based models incur latency and privacy risks, whereas small on-device LLMs are fast but prone to hallucinated spatial judgments. Hybrid reasoning LLMs offer both a fast non-thinking mode and a slow thinking mode, which raises a key question for embodied agents: when is slow thinking worth its cost? We propose CovNav, a training-free framework that answers this question with Conformal Prediction (CP), turning dual-process (System 1 / System 2) inference into an uncertainty-aware decision. Much like recent decision models such as Jev, CovNav casts each spatial query as a structured binary decision rather than free-form generation: a local LLM in fast mode judges whether the target is likely near an observed object, CP converts its token probabilities into a prediction set, and only ambiguous queries are escalated to slow chain-of-thought reasoning. The object-pair formulation keeps calibration tractable and supports open-set objects through off-the-shelf detectors. Experiments on Gibson and HM3D show that CovNav surpasses the strongest zero-shot baseline by 0.8 and 1.7 percentage points in success rate on HM3D v1 and v2, while invoking slow reasoning for only 12.4% of queries and retaining most of the gain of always-on thinking at a fraction of its latency. Real-robot experiments on a TurtleBot 4 further validate the approach. CovNav shows how calibrated uncertainty can decide when an embodied agent should think slowly, offering a principled form of adaptive test-time reasoning for LLM-based robots.
comment: 13 pages, 6 figures, 7 tables
★ PLaCA: Learning Patch-Level Future Latent Change for Action Generation
Vision-Language-Action (VLA) models generate robot actions from visual observations and language instructions, but effective manipulation also requires anticipating how the environment will change. Existing world-action models explore future image prediction and latent dynamics modeling. However, generating complete future images can be computationally expensive, while highly compressed latent representations may obscure local changes. To address these challenges, we propose PLaCA, a world-to-action framework built around future latent change. PLaCA directly predicts spatially structured patch-level future latent change from the current multimodal input, preserving local information about expected environmental changes. We introduce patch-level change-aware supervision to emphasize regions undergoing larger changes and a future-to-action interface that explicitly incorporates predicted future latent change into action generation. Experiments on LIBERO-Long show that PLaCA achieves a 79.0% success rate, improving upon the Action-only baseline by 5.0 percentage points. On CALVIN, PLaCA improves the average sequence length from 2.784 to 3.564. These improvements are achieved while updating only 2.95% of the model parameters during the second training stage, demonstrating the effectiveness of future latent change modeling for downstream robotic manipulation.
★ Minimum-Time Escape from the Conic Field of View of a Reeds-Shepp Pursuer: Classification and Game Value for a Slower or Equally Fast Evader
We study minimum-time escape of an omnidirectional evader from the forward conic field of view of a Reeds-Shepp pursuer with minimum turning radius $r$. For speed ratio $ν$ and cone half-angle $0<α<π/2$, we prove a global escape/holding classification: straight reversal holds the whole cone for $ν\le\sinα$, escape is confined to a barrier-bounded dart for $\sinα<ν\le1$, and escape is possible from every state for $ν>1$. For the slower or equally fast evader, $\sinα<ν\le1$, we construct the solution with Isaacs' characteristics and verify it in feedback form, using a Filippov pursuer relay and a one-sided evader branch. The finite escape time is the first root of one scalar equation, and the holding region is bounded by the nonusable parts of the rays and a barrier that closes with a convex corner on the front axis. Sample-and-hold counter-controls show that this escape time is the value of the game in nonanticipative strategies at every state except the corner. There the two strategy values differ: the pursuer's is infinite, whereas an evader that chooses which barrier branch to cross from the initial germ of the pursuer's control escapes before the barrier time $τ_I$, which is the evader's exact strategy value. An independent level-set solver, refined on fixed comparison points, agrees with the closed-form escape time and reproduces the holding region at the tested parameters.
comment: 30 pages, 5 figures, 2 tables
★ A Head Camera Is (Almost) All You Need: An Electronics-Free UMI Glove Labelled and Observed by One Moving Camera
Handheld demonstration devices decouple data collection from the robot, but conventional designs burden the hand with a wrist camera, cables and a pose tracker (SLAM, an IMU, a VR controller, or a fixed external camera that the collector's body occludes), and burden the pipeline with as many data streams to timestamp, synchronise and cross-calibrate. We show that a single egocentric camera (USD 122) worn on the collector's head replaces the wrist camera, the pose tracker and their separate streams with one. It reads printed fiducials on an electronics-free glove (USD 22) against paper tiles taped around the workspace, giving the 6-DoF gripper pose and the jaw width from the same frames the policy will see; at deployment the same camera rides a low-cost (USD 120) arm beside the robot that mimics the collector's head from the robot's joint state. Against the robot's encoders the labels are accurate to 6.5 mm and 1.6 degrees median with 93-100% frame coverage, and a policy trained on them instead of encoders pays 3% in validation loss. Adding 194 glove demonstrations to 48 teleoperated episodes lifts rollout success from 10/25 to 17/25, level with 172 teleoperated episodes alone (18/25), and the full co-trained head-camera policy succeeds in 20/25 against 15/25 for the wrist-camera recipe, whose policies drop the object when the goal leaves their view, and 12/25 for a fixed camera.
★ JARVIS: From Visual Manuals to Autonomous Robotic Assembly
Visual manuals communicate procedural knowledge through stepwise diagrams, but translating these instructions into reliable long-horizon robotic execution remains challenging. We introduce JARVIS, a method-agnostic benchmark for autonomous visual-manual-guided execution without additional language instructions. JARVIS comprises 236,679 LEGO assembly tasks across 8,987 scenes, with a carefully curated JARVIS-v1 suite of 50 tasks for interactive robotic evaluation. Tasks span varying initial structural complexities and assembly horizons, requiring agents to interpret manuals, ground actions in an evolving workspace, and execute the intended procedure. A structure-aware evaluation protocol captures geometric accuracy, structural relations, construction order, partial progress, and completion. We further introduce JARVIS-agentic, a tool-grounded embodied agent that composes robot primitives into executable programs and uses execution feedback for verification and recovery, without task-specific action trajectories or model parameter updates. Across nine multimodal and reasoning models, task-oriented primitives increase the average assembly score. Performance nevertheless declines sharply with assembly horizon, and execution errors remain difficult to correct. Prior-task experience improves initial performance, while repeated reflection provides inconsistent benefits, highlighting persistent limitations in connecting visual procedural reasoning with reliable physical execution.
★ EchoNav: Evidential BEV Completion from Historical Observations for Visual Navigation
Visual navigation in cluttered environments requires robots to recover previously observed structure and infer obstacles and free space that remain occluded. We present a framework that combines temporal aggregation and occupancy completion in an evidential bird's-eye-view (BEV) representation predicted from short RGB sequences. A decoder adapts frozen VGGT-Omega features and combines historical residual proposals using cellwise weights and a learned no-update option. The BEV predictor produces a local map in the latest ego frame without depth or camera-pose inputs or explicit geometric warping. The evidential model treats observed free space separately and predicts occupancy with associated confidence across the remaining supported region. We train the predictor on simulated RGB sequences paired with complete and visibility-masked occupancy annotations. Our experiments evaluate navigation in simulation and on a wheeled robot, with additional demonstrations on quadruped and humanoid robots. In simulated sequential two-goal navigation, our system achieves 89.6% success, outperforming the strongest evaluated baseline by 32.5 percentage points. Across 60 wheeled-robot trials in 20 scenes, it achieves 73.3% success, compared with 33.3% for the strongest evaluated baseline.
comment: 8 pages, 4 figures
★ Embodied Task Agents Can Perform Fine Manipulation through Tactile In-Context Learning
Can embodied task agents use tactile experience to perform fine manipulation without task-specific model training? We investigate this capability through Tactile-ETA, a system that incorporates tactile experience into an embodied task agent through in-context learning. Successful expert demonstrations show how contact changes during manipulation. We pair each recorded robot movement with its camera views and bilateral tactile observations. We keep tactile observations at the start and end of each movement and add two neighboring time points when the tactile images change markedly in between. The agent combines these demonstrations with current vision, tactile observations, and proprioception to select actions through general-purpose robot tools, without updating model parameters. Across four simulated manipulation tasks, adding tactile observations to the same two expert demonstrations increases average success from 15.75% to 24.00%, with current sensing and execution interfaces held fixed. Further experiments show that performance depends on the sensory content and representation of demonstrations, while comparing agent models on Lift Bottle reveals substantial differences under the same system configuration. On the real-robot tasks, demonstrations with aligned visual and tactile observations increase the task success rate from 0% to 60%. These findings demonstrate the feasibility of using tactile experience in context for fine manipulation and suggest a path toward extending embodied task agents to additional sensory inputs.
comment: 24 pages. Project page: https://no-518.github.io/tactile-eta/
★ Sensitivity-Orthogonal Residual Learning for Adaptive Sideslip and Cornering Stiffness Estimation
Online adaptation of the cornering stiffness keeps a model-based sideslip estimator valid as tires change, but competes with learned model correction for the same filter innovation. A corrector with enough parameters can absorb the innovation pattern that a stiffness change produces, and adaptation is suppressed while the fit improves. We propagate the stiffness sensitivities of the predicted measurements through the vehicle states and construct a corrector orthogonal to them. A null-space parameterization enforces the constraint without a penalty coefficient and preserves the static least-squares stiffness estimate on each identification segment. In full-vehicle simulation, the stiffness response of the unconstrained corrector declines as its parameter count grows, whereas the orthogonal corrector responds at least as fully as the uncorrected filter at all sizes and lowers sideslip error by 12.7% to 22.0%. In a simulated drive with a stiffness step, the orthogonal corrector adapts as quickly as the uncorrected filter and lowers sideslip error during and after the transient. On public real-vehicle data, the 430-parameter unconstrained corrector loses most of its response to deliberate offsets of the filter's nominal stiffness, whereas the orthogonal corrector of the same size retains most of it, with a sideslip error slightly above the uncorrected filter's.
comment: 25 pages, 15 figures, 5 tables. Submitted to Vehicle System Dynamics
★ Velocity Optimization via Thermal Imaging Enhances Autonomous Tumor Resection
Objective: To demonstrate and evaluate THERMO, a thermal-velocity optimization algorithm for autonomous electrosurgery, in ex vivo porcine tissue using the UR10e-based ASTR system. Methods: Thermography for Electrosurgical Rate Modulation via Optimization (ThERMO), a closed-loop algorithm that uses real-time thermal imaging data to minimize thermal injury while ensuring accurate cut margins, was tested on ex vivo porcine tissue. Histological analysis was performed to detect thermal injury extent at a cellular level, and 3D photogrammetric measurement was used to verify margin accuracy. ThERMO's performance was compared against two previously used fixed velocity methods. Results: ThERMO outperformed both fixed velocity comparison methods without requiring data-driven training or iterative tuning, reducing thermal injury by up to 52%, improving cut accuracy by up to 59%, and maintaining task completion speed. Conclusions: Closed-loop thermal feedback via ThERMO improves autonomous robotic electrosurgery by simultaneously reducing thermal injury and improving margin accuracy, reinforcing its value over fixed velocity approaches for tumor resection applications. Significance: This work advances autonomous robotic tumor resection by demonstrating that real-time closed-loop thermal feedback control can substantially reduce collateral tissue damage and improve surgical margin accuracy.
comment: submitted to IEEE Access
★ SynCalib: Breaking Dataset Shortcuts in LiDAR-Camera Calibration via Extrinsic-Controllable Synthesis
Online LiDAR-camera calibration remains a fundamental yet challenging task in the field of robotics. Recently, deep learning-based data-driven approaches have emerged, offering high-precision, targetless calibration capabilities. However, existing methods often suffer from a critical dataset shortcut phenomenon due to the lack of extrinsic diversity in their training data. Networks tend to overfit to fixed nominal extrinsic configurations, leading to severe performance degradation and poor generalization in real-world deployments. To address this limitation, we propose a highly generalizable data generation framework tailored for training LiDAR-camera calibration networks. Our framework leverages existing omnidirectional image datasets annotated with depth ground truth to synthesize multi-modal calibration data under rigorous extrinsic constraints. Extensive experiments demonstrate that the data generated by our pipeline significantly enhances the generalization and robustness of typical learning-based calibration networks. Furthermore, we conduct comprehensive real-world evaluations across various camera-LiDAR configurations, validating the zero-shot transfer capability of models trained exclusively on our synthetic data. To facilitate future research and community use, the code for data generation, the pre-trained models, and online benchmark datasets will be made publicly available.
comment: 8 pages, 6 figures
★ OmniVLoc: Topo-metric Visual Localization for Omnidirectional Cameras
Omnidirectional cameras provide surround-view information, significantly overcoming the field-of-view constraints for traditional visual localization systems. However, existing localization systems fail to fully exploit the characteristics of omnidirectional cameras, struggling to deliver accurate and robust localization performance amidst scene content variations and illumination-induced appearance changes. In this work, we propose OmniVLoc, a surround-view localization system based on a topo-metric framework. By integrating state-of-the-art global place recognition and monocular depth estimation methods, we introduce a multi-constrained visual localization paradigm. By solving a scale-aware pose graph and performing precise spherical bundle adjustment, we accurately recover the metric-level pose for each query frame.Furthermore, a hierarchical verification mechanism is designed to filter out erroneous outputs across different stages, effectively bolstering the overall system robustness. Extensive experiments on both public and our custom datasets validate the superior accuracy and reliability of OmniVLoc. To benefit the community, our source code and dataset will be made publicly available upon publication.
comment: 8 pages, 7 figures
★ Robot-Task Suitability in Heterogeneous Multi-Robot Task Allocation IROS 2026
Heterogeneous multi-robot task allocation depends on estimates of how suitable individual robots are for candidate tasks, yet suitability can be defined in substantially different ways. We compare ten deterministic methods and two LLM-based variants spanning rule-based, similarity-based, requirement-oriented, fuzzy, and semantic approaches. Using common robot and task profiles, we compare how the methods score and rank candidate robots, identify feasibility, and select a top robot, while examining how these outcomes change with team composition, scale, spatial context, capability uncertainty, and task representation. The methods frequently disagree despite operating on identical scenarios. Generic similarity measures can violate properties expected under a one-sided requirement-satisfaction interpretation of suitability, while LLM-based scoring introduces additional stochastic and linguistic sensitivity. These results show that robot--task suitability should be treated as an explicit modeling choice whose assumptions, robustness, and decision-level consequences should be considered before allocation.
comment: Accepted at Workshop on Foundation Models for Multi-Robot Systems (FM-MRS), IROS 2026
★ Evaluating Social Reasoning in Embodied Vision-Language Models IROS 2026
Foundation models now serve as reasoning layers for robot teams, but it remains unclear whether embodiment-specialized vision-language models (VLMs) preserve the social-reasoning capabilities of general-purpose VLMs. We evaluate Gemini Robotics-ER on SHREC, a benchmark of real human-robot social interactions, and compare it with published general-purpose VLM baselines. Robotics-ER outperforms GPT-4o-mini on error/competence detection but performs worse on detailed social-attribute judgments; adding a visual frame helps one task while hurting the other. These results suggest embodiment-specialized models may perform differently on social judgments that can be read more directly from observable interaction cues versus those requiring more inference about a person's internal state. Because the comparison spans model families, these differences cannot be attributed to embodiment specialization alone. This matters for multi-robot systems, where a shared foundation model could cause systematic social-perception errors to influence multiple coordination decisions. This work presents an initial evaluation, with broader cross-context and video-based analyses left for future work.
comment: Accepted at Workshop on Foundation Models for Multi-Robot Systems (FM-MRS), IROS 2026
★ HATCH: Hardware-Attested Task Chains for Verifiable Manipulation IROS 2026
Capability claims in robot manipulation are unverifiable, and existing leaderboards are each closed, simulated, or gameable. We propose HATCH, a robot escape room: an openly specified, physically replicable, sensor-scored, and unrehearsable benchmark. Tasks span five skill categories at three difficulty levels and are chained so that the sensor confirming one task's completion unlocks the next, turning a room into a single long-horizon evaluation. Success is confirmed by instrumentation rather than human judgment, scoring is embodiment-agnostic, and object placements are randomized by a run-day seed. Entrants submit attested self-runs (uncut video, sensor logs, and the seed), with occasional organized events for calibration. Task specifications, build guide, bill of materials, and scoring rubrics are released under an open license.
comment: 2 pages, 1 table. Accepted at the 5th Workshop on Open Robotics Software, IROS 2026, Pittsburgh, PA, USA. Benchmark repository: https://github.com/shizacharania/hatch-benchmark
★ Towards Autonomous Bricklaying Using Boston Dynamics Spot
Mobile legged robots offer unique advantages for on-site construction, including the ability to traverse unstructured terrain, climb stairs, and operate independently between spatially separated work zones. This paper presents a proof-of-concept system for autonomous brick stacking using a Boston Dynamics Spot quadrupedal robot equipped with a manipulator arm. The system operates between a supply area and an assembly area, using AprilTag fiducials for localization and autowalk missions for navigation. Target brick positions and orientations are specified in CAD software and exported directly for robotic execution. We evaluate placement accuracy through ten repeated assemblies of a curved four-layer wall comprising 14 wooden bricks. Results show mean translation errors below 10 mm and mean yaw errors below 1.5 degrees, while the overall RMSE across all runs is 27.72 mm (translation) and 2.06 degrees (rotation). The system was also tested outdoors with stairs between the two areas. The workflow is modular and easily extendable to different wall geometries.
comment: 8 pages. Published in the Proceedings of the 43rd International Symposium on Automation and Robotics in Construction (ISARC 2026), Singapore
★ Real-Time Disturbance-Aware Koopman Model Predictive Control for Quadrotors in Spatially Varying Winds
Reliable quadrotor flight in disturbed environments requires predictive models that remain accurate as unmodeled aerodynamic effects change during deployment. We introduce a disturbance-aware Koopman Control Family (DA-KCF) framework that preserves an offline-identified nominal model while learning only a disturbance-induced correction online. A reference-dependent KCF is first learned from simulation data and calibrated using limited hardware data. During deployment, the nominal KCF remains fixed while a lightweight residual ensemble learns a low-dimensional disturbance correction from causal state-command-state transitions. The Adaptive DA-KCF model is incorporated into MPC, where disturbance corrections are evaluated along the prediction horizon to optimize position and yaw references sent to the onboard stabilizing controller. Hardware experiments on a nano-quadrotor under uniform wind and a spatially varying wake demonstrate that online adaptation improves multi-step prediction and trajectory tracking as disturbance-induced model mismatch increases, while satisfying real-time control requirements.
★ RoPT: Robust 6DoF Object Pose Tracking in Video
Accurate 6-DoF object pose tracking is essential for robotic interaction, yet conventional monocular pipelines frequently suffer from catastrophic divergence under rapid object motion. We present RoPT, a robust tracking framework engineered to overcome this limitation by unifying visual perception, temporal kinematics, and global re-localization into a tightly coupled closed loop. Unlike existing open-loop trackers, RoPT delivers three critical capabilities that fundamentally alter system resilience. First, the visual perception component tracks object masks to robustly handle partial and severe occlusions. Second, a kinematic factor graph regularizes frame-to-frame updates, forecasts forward pose predictions, and drives a parallel multi-hypothesis verification scheme to bypass local minima. Finally, an active re-localization module transforms the pipeline into a closed-loop system, making it failure-aware and capable of deterministic self-recovery. We demonstrate the effectiveness of our framework across challenging benchmarks alongside real-world robotic demonstrations, achieving significantly higher tracking survivability under rapid motion.
★ SLoSa: Small, Low-Power, and Safe Foundation Models for Robotic Task Planning
Safe robotic task planning requires planners that can reliably achieve task goals while satisfying safety constraints. Classical planners search over explicit symbolic task representations and can provide formal guarantees with respect to encoded safety constraints, but require rigid input formats. Their computational cost also increases rapidly with the search space. In contrast, large language models (LLMs) offer flexible task interfaces and strong generalization capabilities, but they are large, power-hungry, and pre-trained with little robot-specific safety data and knowledge. Thus, they can be difficult to deploy locally on robotic systems with limited resources. To address these limitations, we present SLoSa, a neuro-symbolic compression and fine-tuning framework for training foundation-model-based robotic task planners with Small size, Low power demand, and Safety awareness. The framework first constructs a safety-aware dataset using classical planners, an advanced teacher LLM, and formal verifiers, followed by two training stages: supervised fine-tuning and reinforcement learning (RL) fine-tuning guided by a verification-based reward machine to enforce safety alignment. We evaluate SLoSa across four Planning Domain Definition Language (PDDL) domains, two household simulation environments, local edge platforms, and a desktop robotic arm. Across the PDDL tasks, verifier-guided post-training raises the success rate of compact 4B-14B planners to 70.5-76.0%, while INT4 deployment on a Jetson Orin requires 17.85-26.41 W of average power. Results in household simulators and on the physical robot further demonstrate the applicability of SLoSa across diverse robotic task settings.
★ GGraph: A GPU-Accelerated Library for Graph-Based Robot Deployment and Learning
Autonomous mobile robots require spatial representations that capture geometry and semantics while staying sparse enough for planning in large, unknown environments. Graphs provide a natural representation for this purpose and underpin both classical planning methods and a growing class of reinforcement-learning (RL) policies. However, RL training needs hundreds to thousands of parallel environments, each constructing and updating its own graph. Existing graph libraries are typically designed for either single-robot deployment or fixed graphs, and CPU-based graph construction limits scalable training to only a few dozen environments. We introduce GGraph, an open-source GPU-accelerated graph library for both robot deployment and large-scale learning. GGraph supports real robots through ROS 2 interfaces and vectorized GPU-based policy training, with built-in support for tasks including 3-D navigation, exploration, and loop-closure integration. For single-robot operation, GGraph updates graphs in milliseconds. At scale, it enables graph-based exploration policies to train across hundreds of parallel robots, reducing wall-clock training time by one order of magnitude. Across simulation and real-world experiments, these results establish GGraph as a unified graph infrastructure for both large-scale robot learning and deployment.
comment: 9 pages, 11 figures
★ HyperDrone: Simulation-to-Reality Transfer at Millions of Frames per Second
We present HyperDrone, an extremely fast quadrotor simulator focused on high-performance batch rendering that is more than 20x faster than existing state-of-the-art batch quadrotor simulators. HyperDrone combines highly optimized multirotor dynamics with HyperLux, a new ray-tracing engine that supports Nvidia OptiX, Vulkan, Apple Metal and WebGPU backends that takes full advantage of modern ray-tracing hardware, capable of running both components entirely on the GPU. HyperLux renders RGB, depth, semantic/instance segmentation, surface normals and optical flow observations at configurable levels of fidelity, including reflections, realistic lighting, motion blur, anti-aliasing, and Physically Based Rendering (PBR). Even in the highest-fidelity settings, HyperLux achieves millions of Frames Per Second (FPS) on consumer-grade GPUs while rendering realistic 3D scenes. We demonstrate the merits of HyperDrone by training a vision-based yaw-control policy purely in simulation and transferring it to a real small-scale quadrotor (Crazyflie) with a camera, running completely onboard.
comment: Accepted at Conference on Robot Learning (CoRL 2026)
★ COWBOY: A Scalable Sim-to-Real Framework for Learning Contextual Whole-Body Manipulation
Mobile manipulators must adapt not only to objects, but also to the spatial context in which they appear: where to position the base, how to manipulate, and where to look while navigating surrounding scene geometry. To address this, we present COWBOY, a scalable sim-to-real framework for learning such contextual whole-body manipulation. On whole-body dexterous grasping, COWBOY transfers zero-shot to hundreds of objects and diverse in-the-wild scenes, spanning cluttered tabletops, shelves, drawers, and other unstructured settings. Rather than learning scene adaptation directly with RL in simulation, a vectorized environment-aware whole-body controller (WBC) handles reaching in thousands of scenes by coordinating base-arm motion, collision avoidance, and active vision. Then, local RL experts are trained in local scenes for object interaction and can be reused across thousands of full-scene layouts. We distill these composed WBC-RL teachers into a single vision policy that controls the mobile base, arm, dexterous hand, and actuated camera from point-cloud observations. The resulting policy exhibits contextual whole-body behavior, automatically adapting base placement, camera viewpoint, arm trajectory, and grasp strategy to observed scene and object geometry. Videos are available at https://cowboy-manip.com
comment: Website at https://cowboy-manip.com
★ One Policy, Many Styles: Where to Aggregate Preferences for Steerable Humanoid Control?
Modern robotic agents are increasingly expected to support steerable control, where a single policy can adjust its behavioral style at runtime under changing user preferences. Preference-conditioned reinforcement learning provides this interface, but conditioning alone does not ensure that a requested behavior change is strong, does not leak into unrelated behaviors, and does not compromise task performance. Our evaluation combines Preference Reward Responsiveness (PRR) with coverage and utility metrics. We use these measures to compare reward-level, advantage-level, loss-level, and PCGrad-based aggregation across two humanoid-control frameworks. No aggregation level performs best across all environments and criteria. Experiments show that aggregating objectives early produces a wider range of behavioral responses but greater unintended changes in other objectives, while aggregating them later improves control precision at some cost to responsiveness.
★ Navigation Explainability for Object Avoidance with LLMs for Assistive Robots
Blind and visually impaired individuals face uncertainty when navigating unfamiliar areas. Whether a cane or a guide dog is used for navigation, this may prove to be challenging. To mitigate this challenge, we present the Context-Aware Navigation Explanation framework that explains guide dog movements for obstacle avoidance and navigation using a large language model. This system integrates perceptual context with planned future trajectories to generate natural-language explanations of robot movement. We deploy the system on a Unitree Go2 platform in indoor navigation scenarios and evaluate how different explanation context configurations affect user trust and explanation satisfaction. Results show a strong positive relationship between explanation satisfaction and trust, and suggest that combining obstacle information with future trajectory leads to the most effective explanations. These findings highlight the potential of context-aware language explanations to improve transparency and trust in assistive robot navigation.
★ Learning Dynamic Games for Cooperative Iterative Tasks
We consider cooperative and iterative tasks involving multiple agents whose interactions are modeled as a dynamic game with shared constraints. We propose a general learning framework applicable to nonlinear and nonconvex dynamic games that uses previously executed feasible trajectories to construct a sampled joint safe set and player-specific cost-to-go functions. At each time step, the agents jointly select a previously sampled terminal state and a Generalized Nash equilibrium used to reach it. A new solution is adopted only if it satisfies a cooperative acceptance rule agreed upon by the players. The framework accommodates individual non-degradation requirements and selection criteria such as weighted costs and Nash bargaining. We show that the proposed framework guarantees recursive feasibility and characterize conditions under which each player's cost is non-increasing across iterations.
★ Flexible Language-Guided Representation Editing for Adaptive Robot Behavior
Trajectory demonstrations show robots how to behave but not why, leaving them to guess which features of the task actually matter. Incorrect guesses produce misaligned internal representations and lead to incorrect robot behaviors. Because task features are often semantically defined, we argue that natural language is better suited for specifying features rather than relying on demonstrations. Although existing representation editing methods have proposed using language to specify new features, only allowing additive edits produces feature sets that are often over-specified or too narrowly scoped, especially in sequential editing tasks. We propose the Flexible Feature Editing Pipeline (FLEX), which uses language models to iteratively refine a robot's internal representation of task features from natural language corrections via three complementary operations: Add missing features, Generalize existing specific features, and Mask irrelevant features. We evaluate FLEX on 2D navigation scenarios, 7DoF Franka tabletop manipulation tasks, and a real-world study using naturalistic human language corrections and real-robot demonstrations. Comparisons against single-operation and unscaffolded baselines show that all three operations are necessary in interactive scenarios, and that FLEX achieves consistently higher normalized reward compared to other feature editing approaches while maintaining compact, transferable representations. Project available at https://mit-clear-lab.github.io/FLEX/
comment: Accepted to the 10th Conference on Robot Learning (CoRL 2026), Austin, TX. 25 pages, 7 figures, 2 tables
★ inr2action: Encoding Robot Episodes as Neural Implicit Representations for Recall and Action
Long-horizon robotic operation requires memory that scales with essential information rather than execution time. We present a parameter-space memory framework where each episode is stored as a separable object. A neural implicit representation is fitted to episode video from a shared, meta-learned initialization; the resulting weight delta constitutes the memory. Crucially, this delta remains a fixed size regardless of whether the episode spans a hundred or a hundred thousand frames. A learned projector maps the weight delta into token embeddings, conditioning a language model for question answering and a decoder for joint command regression, without frame retrieval at query time. On RoboMME, the language model exceeds chance across all question types, and the decoder achieves a mean $R^2$ of 0.51. Replacing the delta with matched noise drops performance by 11.4 points, while random-origin weights are non-decodable, proving the meta-learned initialization is essential for embedding episode content. By representing individual experiences as fixed-size weight deltas in a common parameter space, this approach decouples memory cost from time horizon and provides a basis for scalable streaming memory.
★ Real2Sem2Real: 3D Semantic Augmentations for Robot Learning from a Single Human RGB Demonstration
Human video scales easily and does not need robot hardware, but existing methods require depth sensing, overlay the robot onto 2D video, or generate data only for the demonstrated objects. We present Real2Sem2Real (R2Sem2R), a real-to-semantic-to-real pipeline that turns one monocular RGB video of a human demonstration into thousands of robot demonstrations across the semantic object instances of a category. R2Sem2R tracks the SE(3) pose of each object with DINOv3-distilled Gaussian splats, outperforming existing methods and FoundationPose with estimated metric depth. R2Sem2R then aligns digital cousins, each generated from one or a few photographs, to the demonstrated objects with a semantic alignment. This alignment matches whole objects and the functional regions of target objects, such as a coffee machine's tray. R2Sem2R maps the human hand to a feasible robot grasp that extends to the cousins. Robot demonstrations are generated either in IsaacSim, for photorealistic rendering, or MuJoCo, for contact physics, and R2Sem2R can generate over 2,000 episodes in about two hours. On two real robots and four tasks, we train a vision-language-action policy on data from one human video per task, and policies trained on our data match or exceed policies trained on teleoperation across diverse object instances. The project page is available at https://real2sem2real.github.io/.
comment: Matthew Strong and Ankush K. Dhawan contributed equally. Project page: https://real2sem2real.github.io/
★ T-WLA: Bridging Slow Vision and Fast Touch
In contact-rich manipulation, contact conditions can change before a visual policy updates its action plan. Recomputing the plan through the visual policy adds latency, while tactile feedback alone does not specify how the ongoing motion should be revised for the task. We introduce T-WLA, a world-language-action architecture that uses shared visual memory to guide both action generation and tactile correction. At each visual update, the visual branch integrates the current image, task instruction, and retrieved history into a visual memory that conditions base-action generation. The same memory is cached for a lightweight tactile editor, which combines it with the active action chunk and incoming touch to predict residual corrections to unexecuted actions without rerunning the visual branch. Future-image and future-pressure prediction provide training-only supervision, encouraging the visual memory and tactile features to capture interaction dynamics relevant to action generation and correction. We evaluate T-WLA on tactile-conditioned hand-motion prediction using EgoTouch and Tachin, and on four real-world dexterous manipulation tasks. T-WLA reduces finger-joint MPJPE by 30% relative to the strongest reported baseline and achieves an average success rate of approximately 50.0% across real-robot manipulation tasks. Videos and code are available at https://anonymous-submission-20.github.io/T-WLA.github.io/.
★ VeriCalib: Verifiable Target-Based LiDAR-Camera Calibration
Target-based extrinsic calibration estimates the rigid transform between sensors from a few observations of a known target, and its result is commonly assessed by the root-mean-square error of the fitted residuals. However, this scalar error is computed on the same observations and therefore cannot indicate whether to accept the result, re-extract, recollect or add observations. This paper presents VeriCalib, which formulates extrinsic calibration as a verifiable estimation problem and reports a calibration state. The key insight is a two-level extrinsic compensation mechanism, in which the extrinsic compensates the rigid part of each observation error at the residual level, and rotation and translation compensate each other at the parameter level. Consequently, the fitted residual can reveal neither an inconsistency between the observations and the model nor whether the estimated extrinsic is reliable in every direction. Accordingly, we assess the calibration through a structured covariance that jointly propagates the uncertainty of the camera and LiDAR board features with a known-geometry prior, and estimate its scale as a variance component from the residuals. On this covariance, we decouple rotation and translation to resolve the precision along each direction and guide data collection toward the weakest one. Since this precision presumes consistent observations, we further develop within-view and cross-view checks that examine the internal residual and the rigid discrepancy between observations, respectively, with thresholds that hold their significance level under an unknown noise level. We also extend the minimal detectable bias to the extrinsic to bound the smallest detectable fault along each direction. These criteria form a calibration state that indicates whether to accept the result or how to collect the data...
comment: 35 pages, 20 figures, 13 tables
★ Learning-Enhanced Observer for Sensorless Task-Null-Space Compliant Control
This paper presents a novel framework integrating a robust wrench observer with a hybrid pose and force controller for physical Human-Robot Interaction (pHRI). In pHRI tasks, robots are expected to perform task-space motions robustly while remaining compliance with human disturbance, which proposes distinct control objectives. Most existing approaches focus mainly on either robustness or compliance and rely on force sensors for real-time tracking. To address these limitations, we propose a novel framework with the following key contributions. Firstly, a neural network (NN) is trained to improve the observer performance by compensating for system friction and unmodeled dynamics. Secondly, a task-null-space decomposition is applied to separate the joint velocity and reduce the effect of the human disturbance on the observer. Finally, a hybrid pose and force controller is introduced to achieve the end-effector pose and force tracking. Experimental results demonstrate that the proposed framework provides robust and accurate wrench estimation while maintaining reliable pose and force tracking performance. The proposed approach offers an effective solution and framework for pHRI without the need of force/torque sensor measurements which can be applied to many robotic manipulator platforms which do not have force/torque sensors.
★ Human-Robot Interaction via Meta-Learning Based Mode Recognition and Adaptive Control
In human-robot interaction (HRI) tasks, humans may interact with the robot in either an active guiding or passive following state, resulting in distinct control objectives for each interaction mode. Most existing approaches rely on predefined switching thresholds to distinguish interaction modes, but their performance can be sensitive to measurement noise. To address this limitation, we propose a novel framework with the following key contributions. First, a neural network (NN) trained via meta-learning is developed to recognize human intention without manually defined thresholds. By successfully incorporating force-dependent parameters from the controller, the recognition rates are improved from 90.1\% to 99.1\% for the passive mode and from 79.8\% to 97.1\% for the active mode. Second, an adaptive meta-learning controller is introduced to compensate for human disturbances and enable online adaptation to varying interaction forces. Third, a multiple-trajectory meta-learning algorithm is designed to jointly learn an environment-basis representation and force-dependent parameters, allowing the controller to deal with unmodeled dynamics efficiently. Experimental results demonstrate that the proposed framework achieves robust and accurate control performance, while providing reliable mode recognition for safe and adaptive physical HRI.
★ Warping the Workspace: Expanding Visual Access with Adjustable Reach Mapping for Humanoid Teleoperation
First-person humanoid teleoperation ties access to side and rear targets to operator rotation. We present coupled visual--motor remapping, pairing a panoramic azimuth warp with its inverse upper-limb yaw transform. A working prototype implements Global Zoom (GZ) and Focus+Context (F+C). In a within-subject study (N=8); 72 trials), GZ had the largest completion count (23/24; Fixed: 20/24; F+C: 21/24), whereas F+C was preferred by 6/8 participants. Therefore, completion and preference favored different mappings in this sample.
♻ ★ TAPDreamer: Transferable Adversarial Patches for World Action Models
World models learn to predict how their environment will evolve, making them an important foundation for general-purpose robotic control. Yet world action models depend on camera inputs whose manipulation can corrupt the visual representations used across tasks and action policies. Existing attacks on these models optimize against the victim's actions or predicted futures and therefore require access to target-model outputs. In this paper, we propose an attack, TAPDreamer, against world action models that instead uses a public encoder alone to construct a fixed local perturbation that transfers across tasks and action architectures. TAPDreamer requires no target-policy queries. Our key insight is that interactions between patch-induced changes in attention weights and value vectors broadcast a nearly identical representation shift far beyond the patch footprint, and this shift remains stable across task observations. Guided by this insight, TAPDreamer uses six frames from one source task to maximize the global L1 distance between clean and patched encoder representations. In closed-loop evaluation, one frozen patch per benchmark, covering about 6.5% of the input, reduces FastWAM's success rate from 97.7% to 0.0% across 40 LIBERO tasks and from 90.86% to 0.0% across 50 RoboTwin tasks; matched random patches retain 81.5% and 79.2% success. The same patches reduce success to 1.45% and 1.00% on two DreamWAM configurations and to 10.60% on Motus. These results show that protecting downstream action generation alone is insufficient: defenses for world action models must also secure shared visual encoders against persistent local perturbations.
comment: Project Page: https://tapdreamer.github.io
♻ ★ Physical Words: A Vocabulary for Structuring Latent Reasoning in Robot Manipulation
Multimodal reasoning provides robotic foundation models with predictive context needed for reliable action generation. However, language-based reasoning captures task semantics but struggles to express fine-grained physical dynamics, while latent reasoning models future state evolution but offers limited structure for organizing these dynamics into reusable reasoning primitives. To bridge this gap, we introduce PhysWord, a reasoning-before-acting foundation model built on Physical Words, which use atomic action verbs as shared semantic anchors for continuous representations of physical dynamics. Specifically, PhysWord predicts upcoming atomic action words and directly aligns their contextual hidden states with latent targets derived from the corresponding future observations. Building on these primitives, we construct a temporally structured PhysWord space with visual and optional tactile variants of each action word. Autoregressively predicting Physical Words across future timesteps captures transitions between execution stages, providing structured multimodal context for action generation. PhysWord achieves state-of-the-art success rates of 90.3\% on visuotactile UniVTAC and 99.6\% on vision-based LIBERO-Long, and outperforms competing methods on real-world tasks using bimanual grippers and dexterous hands. Further evaluations demonstrate that our structured latent reasoning framework generalizes to unseen scenarios and enables long-horizon manipulation.
♻ ★ Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models NeurIPS 2026
A fine-tuned Vision-Language-Action (VLA) policy can easily place the alphabet soup in the basket on command, just as it saw during fine-tuning. However, if the basket is shifted five centimeters to the left, it drops the soup off the table. We developed an interpretability pipeline to investigate what such a policy is computing. Injecting task A's activations into the action expert while the policy looks at task B's scene makes pi0.5 execute task A's motor trajectory in 99.6\% of episodes ($n{=}1{,}968$). The injected program is bound to absolute workspace coordinates and ignores the visible scene, which provides a mechanistic account of the perturbation brittleness that concurrent benchmarks report. The binding holds on a physical UR10e: directed replay redirects the commanded task in 26/26 rollouts in one direction and 18/26 in the other, with no-injection and random-direction controls at the floor, and once the source object is moved 10--15\,cm the arm grasps and releases at the recorded coordinates in 50/50 trials, picking up the wrong block when one sits there and closing on empty air when none does. Applying the same interventions to six VLAs on LIBERO, MetaWorld, SimplerEnv, and ALOHA over 420{,}000+ rollouts yields three further findings: every architecture encodes the instruction, and four of the five weight it below vision whenever the scene alone identifies the goal; under the standard replacement write-back the SAE pooling that preserves behavior is architecture-specific (pi0.5 per-token, SmolVLA indifferent), and goal selection and motor generation separate into VLM and expert pathways wherever the two are distinguishable. We release \textbf{Action Atlas} (https://action-atlas.com), an interactive platform covering all 520+ trained SAEs, 79 identified concepts, and the scored UR10e rollouts. Code: https://github.com/CWRU-AISM/action-atlas
comment: NeurIPS 2026 Poster, ICLR Multimodal Intelligence Workshop Oral, 44 pages, 14 figures
♻ ★ Gibson Env: Real-World Perception for Embodied Agents CVPR 2018
Developing visual perception models for active agents and sensorimotor control are cumbersome to be done in the physical world, as existing algorithms are too slow to efficiently learn in real-time and robots are fragile and costly. This has given rise to learning-in-simulation which consequently casts a question on whether the results transfer to real-world. In this paper, we are concerned with the problem of developing real-world perception for active agents, propose Gibson Virtual Environment for this purpose, and showcase sample perceptual tasks learned therein. Gibson is based on virtualizing real spaces, rather than using artificially designed ones, and currently includes over 1400 floor spaces from 572 full buildings. The main characteristics of Gibson are: I. being from the real-world and reflecting its semantic complexity, II. having an internal synthesis mechanism, "Goggles", enabling deploying the trained models in real-world without needing further domain adaptation, III. embodiment of agents and making them subject to constraints of physics and space.
comment: Access the code, dataset, and project website at http://gibsonenv.stanford.edu/ . CVPR 2018 (spotlight)
♻ ★ EMMA: Enhancing Robotic Control Robustness through a Generative Visual Model
The generalization of vision-language-action (VLA) models heavily relies on diverse training data. However, acquiring large-scale robot manipulation data across varied object appearances is costly and labor-intensive. To address this, we introduce Embodied Manipulation Media Adaptation (EMMA), a framework for augmenting VLA policies that combines a generative data engine with an effective training pipeline. We introduce DreamTransfer, a diffusion Transformer-based architecture for generating multi-view consistent and geometrically plausible embodied manipulation videos. It supports text-driven video editing on foreground, background and lighting while preserving 3D structure and geometric fidelity. We also utilize a hybrid training set of real and generated data and propose AdaMix to enhance the training process. AdaMix is a training strategy that adaptively weights samples according to policy performance to emphasize challenging samples. Evaluations show DreamTransfer outperforms previous video generation models in multi-view consistency, geometric fidelity, and text-conditioning precision. In real-world robotic tasks with zero-shot visual settings, our framework achieves a relative performance increase of over 92% compared to training with real data alone, and improves by an additional 17% with AdaMix.
♻ ★ Taskonomy: Disentangling Task Transfer Learning CVPR 2018
Do visual tasks have a relationship, or are they unrelated? For instance, could having surface normals simplify estimating the depth of an image? Intuition answers these questions positively, implying existence of a structure among visual tasks. Knowing this structure has notable values; it is the concept underlying transfer learning and provides a principled way for identifying redundancies across tasks, e.g., to seamlessly reuse supervision among related tasks or solve many tasks in one system without piling up the complexity. We proposes a fully computational approach for modeling the structure of space of visual tasks. This is done via finding (first and higher-order) transfer learning dependencies across a dictionary of twenty six 2D, 2.5D, 3D, and semantic tasks in a latent space. The product is a computational taxonomic map for task transfer learning. We study the consequences of this structure, e.g. nontrivial emerged relationships, and exploit them to reduce the demand for labeled data. For example, we show that the total number of labeled datapoints needed for solving a set of 10 tasks can be reduced by roughly 2/3 (compared to training independently) while keeping the performance nearly the same. We provide a set of tools for computing and probing this taxonomical structure including a solver that users can employ to devise efficient supervision policies for their use cases.
comment: CVPR 2018 (best paper). See project website and live demos at http://taskonomy.stanford.edu/
♻ ★ Visual-Inertial Odometry for Ground Robots with Dynamic Camera-IMU Extrinsic Calibration
Visual-inertial odometry systems are often constrained by the assumption that sensors are rigidly mounted, which limits their adaptability in practical scenarios involving structural motion. To address this limitation, this paper investigates time-varying camera-IMU extrinsic calibration. First, a dynamic camera-IMU extrinsic calibration method based on the Denavit-Hartenberg convention is proposed by incorporating the robot kinematic model. Second, a two-step initialization strategy is developed to efficiently estimate the initial joint angles, providing a reliable starting point for subsequent extrinsic parameter optimization and enabling online estimation of timevarying camera-IMU extrinsic parameters. Simulation experiments demonstrate that the proposed framework can effectively accommodate variations in camera-IMU extrinsic parameters while maintaining competitive localization accuracy.
♻ ★ Comfortably Stuck: Evaluation Pitfalls in Bench2Drive's Closed-Loop Protocol
Closed-loop evaluation is widely regarded as a more faithful way to assess end-to-end (E2E) driving models than open-loop evaluation, and Bench2Drive, built on the CARLA simulator, has become the dominant benchmark for it. We show that its evaluation protocol itself contains pitfalls that can misrepresent the models scored on it. Bench2Drive's comfort metric, adapted from nuPlan, has several implementation errors and a segmentation scheme that rewards standing still, so it mostly measures how long a vehicle stays stationary, correlating with stationary time at r = 0.97 across eight models. We propose a corrected, distance-based metric that cuts that correlation to r = 0.06. Under it, the strong driving-score/comfort tradeoff the original metric reports is no longer detectable once models are compared at matched driving speed, which suggests that comfort trades against how fast a model drives rather than against how well it drives. We also uncover two pitfalls in how the harness supplies navigation signals: the default controller can replace a model's own steering with an angle toward the route target point, and the target point supplied at evaluation time can differ substantially from what a model saw in training. We demonstrate both on UniAD and VAD, where their effects differ sharply, and argue that future closed-loop evaluations should check for them. We will release all fixes as standalone code, so corrected comfort scores can be obtained from logged trajectories without re-running closed-loop simulation.
♻ ★ KITE: Decoupling Kinematics and Interaction for Zero-Shot Cross-Embodiment Manipulation
Generalizing manipulation policies across robot embodiments remains difficult because standard policies tie the interactions needed for a task to the actions of a particular robot. We study zero-shot cross-embodiment manipulation, where a policy trained on source embodiments must be deployed on a structurally different target embodiment without additional task demonstrations. We introduce Kinematic Interaction Transfer across Embodiments (KITE), which decouples manipulation into embodiment-agnostic interaction prediction and embodiment-specific kinematic decoding, connected through a learned representation of interaction intent based on contact patterns. An interaction policy is trained to predict the intent using demonstration data only collected on a source robot, and shared across morphologically different target robots. A kinematic decoder realizes these intents by learning from synthetic data generated from each embodiment's kinematic model. Through such decoupling, KITE can robustly transfer learned skills to a new embodiment given only its kinematic model, without recollecting any task-specific demonstration data. We evaluate KITE on four manipulation tasks in simulation and the real world spanning transfer between parallel grippers, dexterous hands, and composite embodiments. KITE successfully achieves zero-shot transfer to structurally different target embodiments, and consistently outperforms state-of-the-art baselines. Project website: https://kite-manip.github.io/.
comment: 13 pages, including appendix. Project website: https://kite-manip.github.io/
♻ ★ Kamino: GPU-based Simulation of Rigid Multi-Body Physics with Kinematic Loops
We present Kamino, a GPU-based physics solver for massively parallel simulations of heterogeneous highly-coupled mechanical systems. Implemented in Python using NVIDIA Warp and integrated into the Newton framework, it enables the application of data-driven methods, such as large-scale reinforcement learning, to complex robotic systems that exhibit strongly coupled kinematic and dynamic constraints such as kinematic loops. The latter are often circumvented by approximating the system's topology as a kinematic tree, and incorporating explicit loop-closure constraints or so-called mimic joints. Kamino aims at alleviating this burden by natively supporting these types of couplings. Moreover, Kamino supports heterogeneous worlds, allowing for batched simulation of structurally diverse robots on a single GPU. At its core lies a state-of-the-art constrained optimization algorithm based on ADMM, which computes constraint forces by solving the constrained rigid multi-body forward dynamics transcribed as a nonlinear complementarity problem. This leads to high-fidelity simulations that resolve contact dynamics without resorting to approximate models that simplify or convexify the problem. We demonstrate RL policy training on a biped with six nested kinematic loops, generating a feasible walking policy while simulating $4{,}096$ parallel environments on a single GPU.
♻ ★ HexaShell: A Thin Harness for Exploring the Limits of VLM Agents as Robot Coding Agents
Agents built on vision language models (VLMs) now compete with learned policies on robot manipulation leaderboards. We ask how far a coding agent can go when it controls a robot instead of a computer, and where it stops. Our system, HexaShell, gives the agent a thin harness: one command line that turns its commands into robot motion and makes no decisions of its own. On the robot, the agent keeps doing what it does best: it reads output, writes code and runs tests. Two agents of the same model share the work. An acting agent runs each episode. After a failure, an optimizer agent reads the full record of the episode, with every command, image and message, finds the cause, and writes or fixes small programs called tools, each with its own tests. What the agents learn stays as code, and no demonstrations or training are needed. At run time, the tools use only what a real robot also has: RGB images, camera calibration and the state of the arms. They find objects by triangulation between camera views. Because these inputs are the same in simulation and on hardware, the tools move to a real robot with a different arm after only a small change for the arm. We build the tools on ten layouts of one seed per task with Codex and GPT-6-Astra and freeze them; the eight Open tasks share one set of tools. In RoboDojo's official evaluation on a hidden seed, 50 episodes for each of the 42 tasks, HexaShell reaches 46.9% success and a progress score of 53.2, the state of the art on the official RoboDojo leaderboard. The limit is contact, not reasoning: memory tasks reach 81.0%, but precision tasks only 32.0%. We open source the tools evolved so far at https://github.com/HexaFuture/HexaShell.
comment: Code: https://github.com/HexaFuture/HexaShell
♻ ★ CondenseVLA: Learnable History Condensation for Efficient Multi-Frame VLA NeurIPS 2026
Robot manipulation often depends on short-term cues - such as brief contacts, temporary occlusions, and intermediate object states - that are ambiguous in a single frame and can only be resolved by integrating observations over time. However, injecting historical information solely by enlarging the history window is computationally costly and frequently yields limited gains due to substantial redundancy and task-agnostic, inflexible sampling of observations. We propose CondenseVLA, a multi-frame VLA policy with a learnable decoder-side history condensation module for using short-term visual context in action decoding. Given a sequence of cached per-frame features from the VLM backbone, CondenseVLA distills observation history into a fixed set of slot tokens using learnable queries and multi-head cross-attention. Simple auxiliary regularization penalizes overlap among slot attention maps and decorrelates slot embeddings, encouraging complementary attention and reducing slot redundancy. Across the evaluated tasks, CondenseVLA improves Simpler-WidowX success from 58.7% to 69.8% (+11.1 percentage points), achieves 97.2% on LIBERO, and reaches 52.8% on real-world tasks. It achieves 21.6% success on memory-dependent RMBench tasks. These results support selective condensation of bounded recent visual evidence for action decoding, while harder multi-observation tasks expose the limits of a finite history window. Project page: https://github.com/FeiyangHong/CondenseVLA
comment: Accepted at NeurIPS 2026. 39 pages, including supplementary material and the NeurIPS paper checklist. Project page: https://github.com/FeiyangHong/CondenseVLA
♻ ★ Test-Time Trajectory Optimization for Autonomous Driving
End-to-end planners for autonomous driving typically generate a set of candidate trajectories, score each one, and return the highest-scoring candidate. However, the scorer is applied only after the proposals are generated and cannot influence the set of trajectories: a weak set of candidates limits planning performance regardless of the scorer's quality. We instead treat the scorer as a learned trajectory-level reward function and search for trajectories that maximize it. Our method, TOAD, runs the Cross-Entropy Method at test time, warm-started from the planner's proposals. It requires no retraining and is plug-and-play for existing planners. Across seven base planners, TOAD improves results on NAVSIM-v1 (94.7 PDMS), NAVSIM-v2 (56.3 EPDMS), and the closed-loop HUGSIM benchmark. The code is made publicly available via the project page: https://valeoai.github.io/TOAD/.
♻ ★ NestedVLA: Learning to Consolidate and Generate Skills for Vision-Language-Action Models NeurIPS 2026
Vision-language-action (VLA) models can learn a wide range of robotic skills, but combining separately trained skills remains challenging. The original training data may be unavailable for joint retraining, while directly merging model weights can cause interference. We propose NestedVLA, which consolidates independently fine-tuned VLA policies without access to their original training data. Instead of merging all updates into one checkpoint, a hypernetwork embedded in the VLA generates action-expert weight updates for a selected task. Each task's compressed vision-language model update is stored separately, and both updates are added to a frozen base VLA to instantiate the corresponding policy. We evaluate NestedVLA by consolidating three policies fine-tuned with reinforcement learning on MetaWorld, ManiSkill, and CALVIN. It achieves 66.7% average success, improving on the strongest model-merging baseline by 41.1 percentage points. It performs within 1 percentage point of per-task fine-tuning on ManiSkill and substantially narrows the gap on CALVIN. In this three-policy setting, total storage is 1.5x that of a single full checkpoint. Code is available at https://github.com/WanyuGroup/NeurIPS2026-NestedVLA.
comment: Camera-ready version accepted at NeurIPS 2026. Updated to the final conference format and included the NeurIPS checklist. 30 pages, 3 figures. Code: https://github.com/WanyuGroup/NeurIPS2026-NestedVLA
♻ ★ From Instrument-Mounted Demonstrations to In-Vivo Execution: Learning Bimanual Laparoscopic Appendectomy Without Robot-Collected Demonstrations ICRA 2027
Most minimally invasive surgery is still performed with hand-held laparoscopic instruments, and the surgeon's instrument kinematics are lost when the operation ends; only the endoscope video is kept. This paper presents an end-to-end pipeline that captures this motion in the operating room and uses it to train a surgical robot policy, validated on live animals. We introduce a surgical instrument-state logger that mounts on the shaft of a standard laparoscopic instrument and recovers its pose and jaw state from an inertial sensor, a time-of-flight sensor and a Hall sensor, with no external camera or tracker. A data pipeline measures the latency of every sensor channel against a robot ground truth and aligns the channels before forming observation-action pairs. On these demonstrations we train a diffusion policy with a fine-tuned DINOv3 backbone, selecting its design by closed-loop rollouts in a physics simulator reconstructed from depth maps of an ex-vivo rabbit appendix. The policy is then retrained on 849 in-vivo demonstrations from four live rabbits and deployed on four additional live rabbits with electrosurgery armed. With the surgeon selecting the surgical phase, the policy completed the appendectomy in three of the four animals. The results show that demonstrations recorded from a surgeon's own instruments are sufficient to train, select and deploy a bimanual surgical policy in vivo. The robot serves only as the timing reference for sensor calibration and as the executor, and collects no demonstrations. Both demonstration corpora are released to support future surgical robot learning research.
comment: Submitted to IEEE ICRA 2027. Project page & Dataset: https://rosota-research.github.io/instrument-mounted-demonstrations-in-vivo/
♻ ★ DEFLECT: Temporal Counterfactual Preference Learning for Delay-Robust Asynchronous VLAs
Vision-Language-Action (VLA) policies increasingly rely on asynchronous inference to hide model latency behind ongoing robot motion. During inference, the robot continues executing its current action chunk while the next chunk is generated from the observation available at inference start. By the time the new chunk is executed, the robot or scene may have changed, so an action that fits the prediction-time state can be misaligned with the execution-time state. Training-time methods modify the policy input or supervised target based on the expected execution delay, but do not directly supervise how actions should change as the scene evolves between prediction and execution. Runtime repair methods improve chunk continuity or modify actions during deployment, but can still leave actions aligned with the prediction-time state rather than the execution-time state. Our key insight is that execution-time observations are unavailable when the online prediction is made but are available in recorded trajectories during offline training, making the prediction-execution mismatch observable during training. We propose DEFLECT, an offline post-training framework that turns this temporal contrast into preference supervision for a policy acting from stale inputs. Across Kinetix, LIBERO, and three real-robot tasks, DEFLECT improves delay robustness across the evaluated VLA policies and settings. At the longest supported delay, it improves success by 4.2 percentage points over VLASH on Kinetix and by 22.7 points over naive asynchronous execution with GR00T N1.7 on LIBERO.
♻ ★ ForeTac: Predictive Contact Guidance for Generative Robot Policies
In contact-rich manipulation, actions that appear equally plausible from vision can produce different contact outcomes, while reactive control based on current tactile feedback may respond too late to prevent failure. We present ForeTac, a predictive contact-guidance framework organized as a represent-predict-evaluate-guide loop. ForeTac encodes the current tactile state, predicts future tactile latent trajectories conditioned on candidate action sequences, and evaluates each predicted trajectory with a contact-quality objective learned from successful and failed physical interactions. At inference, gradients of this objective are backpropagated through the tactile predictor to guide action generation before execution while keeping all model parameters frozen. We evaluate ForeTac on four real-robot tasks (planar and curved-surface wiping, card swiping, and fragile-object grasping) with diffusion and flow-matching policies. Compared with vision-only Diffusion Policy, ForeTac improves the average task success rate (SR) from 36.3% to 71.3% and the average contact success rate (CSR) from 20.0% to 66.3% across these tasks, and yields gains in both metrics with a flow-matching policy. Component analyses support the learned tactile representation, action-conditioned foresight, and generation-time gradient guidance, while predicted tactile latent trajectories preserve contact-quality ordering. Project page: https://foretac.github.io/.
♻ ★ Learning Reachability-Based Safety Representations for Real-Time Robot Navigation in Unknown and Dynamic Environments
Safe robot navigation in unknown, dynamic environments requires more than geometric collision checking: a robot still far from an obstacle may be unable to avoid it because of its speed, orientation, or the obstacle's motion. Hamilton--Jacobi (HJ) reachability analysis captures this in a value function whose sign separates states that admit a safe continuation from those where collision is unavoidable, but computing it online for every newly observed environment is too expensive. This dissertation trains neural implicit representations offline on reachability solutions to give optimization-based controllers this information in real time. Since the value function never exceeds the constraint, it equals a directly computed constraint, such as a signed distance, minus a nonnegative dynamics correction. Learning only this correction keeps represented obstacles outside the predicted safe set despite approximation errors. The first two of five methods use the learned function as a terminal constraint in model predictive control, for static local environments and predicted obstacle motion. The next two use it as a barrier candidate in control barrier function safety filters, for static environments and, through pairwise barriers in relative coordinates, for a varying number of moving obstacles. Simulations and hardware experiments on ground and aerial robots show better collision avoidance than geometric and hand-designed baselines, especially for short horizons and moving obstacles. The fifth method replaces the grid-based offline solver, which limits the first four to low-dimensional models. For control-affine dynamics with polytopic inputs, the HJ Hamiltonian is maximized at a vertex, so a parallelizable tree search over vertex inputs can generate training labels. With a loss from the HJ variational inequality, these labels yield barrier candidates for systems with up to 13 states.
♻ ★ Action-Aligned Latent Planning for Versatile Humanoid Loco-Manipulation
Learning a single perceptive policy for diverse humanoid loco-manipulation requires unifying high-level task decisions without losing the state-dependent physical responses that make each interaction succeed. This makes the representation between high-level planning and whole-body control critical: kinematic or contact targets provide compact abstractions but do not directly encode interaction-specific control responses, while directly predicting low-level actions requires the planner to model high-frequency closed-loop feedback. We propose FIBER, a framework that uses the executable latent space of a frozen behavioral foundation model (BFM) as a shared planning representation for heterogeneous interaction skills. Our latent policy projection (LPP) maps independently acquired task-specific policies into this representation by identifying latent commands whose induced actions match their local control responses, enabling lower-rate planning while retaining the BFM's high-frequency state feedback. After projection, we aggregate closed-loop latent rollouts from all skills and train a single perceptive flow planner entirely offline. We further introduce AH-BFM, an adaptive-horizon BFM trained with multi-gamma forward-backward (FB) learning to support both precise control and temporally extended whole-body behavior. We evaluate FIBER on box pickup, door opening, and object grasping in simulation and on a real Unitree G1 humanoid. Across 128 diverse variations per task in simulation, FIBER achieves 42.0% mean success, more than tripling projected latent replay (11.6%) and substantially outperforming kinematic planning (3.5%). Real-world experiments further demonstrate sim-to-real transfer, skill chaining, and conditional steering through the same unified planner.
comment: Project page: https://fiberaa.github.io/
♻ ★ WorldScape Policy 2.0: Empowering Steerable World Action Modeling with Reasoning-Augmented Memory and In-Context Learning
World Action Models (WAMs) offer a promising paradigm for robotic manipulation by jointly modeling visual state transitions and robot actions. However, existing WAMs are constrained by limited temporal context, coarse episode-level language supervision, and predominantly text-only conditioning, which hinder task-progress tracking and fine-grained language-video-action grounding while limiting visual-context reasoning and cross-embodiment transfer. In this paper, we introduce WorldScape Policy 2.0, a controllable WAM with reasoning-augmented long short-term memory. Its causal short-term visual memory supplies recent observations as DiT prefill to preserve local interaction dynamics, while its long short-term event memory organizes historical VLM outputs into global-history, local-active, and event-boundary representations for progress-aware retrieval. The retrieved history augments perception and autoregressively generated planning tokens, yielding an implicit subgoal condition for autonomous planning; semantic forcing further transfers event-level instruction semantics into this latent planning pathway. To establish fine-grained multimodal controllability, we construct ManipEvent-5M, an event-grounded embodied pretraining dataset containing nearly 5 million event segments with aligned action trajectories, episode-level task instructions, segment-level subtask captions, goal images, and video demonstrations. These designs provide a unified interface for autonomous planning from high-level instructions and controllable execution from fine-grained text, goal-image, or video-context prompts. Experiments in both simulation and real-world platforms demonstrate superior capabilities in long-horizon autonomous planning, fine-grained instruction following and in-context adaptation.
♻ ★ ZeroDex: Zero-Shot Long-Horizon Dexterous Manipulation via Multi-View 3D-Grounded VLM Reasoning
We present ZeroDex, a zero-shot framework for long-horizon dexterous manipulation that grounds language instructions into executable 3D task plans from calibrated multi-view RGB images. Our framework builds on the grounding ability that vision-language models (VLMs) already exhibit in 2D and lifts their predictions into 3D through multi-view fusion, which extends VLM grounding beyond image-space keypoints to the metric 3D quantities that manipulation actually requires: contact points, placement targets, and tool-motion anchors. Because execution is driven by these 3D groundings rather than a learned policy, the system preserves the VLM's open-world generalization, without collecting demonstrations or post-training for a specific task or embodiment. The VLM decomposes a complex instruction into a sequence of atomic tasks, each executed by a primitive from a small reusable library that covers manipulation tasks including pick-and-place and tool-use: for tool-use, we retrieve an object-centric atomic action corresponding to the inferred skill category and align its stored 6D tool trajectory to the grounded keypoints; for dexterous execution, we expand the lifted grasp keypoint into a task-conditioned grasp affordance region and generate feasible grasp-motion pairs with an arm-hand motion generator. Real-world experiments show improved 3D grounding accuracy and higher task success than zero-shot visual-prompting and constraint-based baselines, both of which ground from a single RGB-D view, and our demonstration-free system outperforms VLA policies fine-tuned on 100 task demonstrations. We further demonstrate long-horizon manipulation, where execution verification and replanning recover from intermediate failures, and show that the system generalizes to unseen objects and tool-use tasks in novel scenes.
♻ ★ Exploring Nonlinear Body Oscillations for Natural Quadruped Gaits
Animals' body morphology shapes the gait patterns they can perform, where mechanical resonance reduces the need for active control. By tuning posture and muscle stiffness, they leverage their embodied intelligence to achieve effective gaits for different speeds. In contrast, most quadruped robots are not specifically designed to exploit mechanical resonance due to the complexity of nonlinear dynamics and require dedicated locomotion controllers. To provide an alternative, we present a proof of concept framework making the nonlinear dynamics of a robot predictable in the design process and show how this knowledge can be leveraged such that multi-gait locomotion can emerge from nonlinear resonances, shaped by gravity, inertia, and elasticity. We present the highly compliant quadruped robot eBert, on which we identify six nonlinear normal modes (NNMs) using our new theoretical tools and validate their existence in simulation and hardware. With black-box optimization to determine step length, simulations show how each NNM naturally develops into a distinct gait, manifesting different speeds, which also largely transfers to the robotic hardware. Our experiments show that eBert can exploit its mechanics to generate task-specific movements which may serve as foundation for designing a new generation of agile and efficient robots leveraging embodied intelligence.
♻ ★ OmniInteract: Towards Unified Whole-Body Interaction Control for Dual Humanoids
Physical interaction is a key capability for humanoid collaboration, requiring coordinated whole-body motion and precise contact. However, learning diverse dual-humanoid interactions faces two major challenges: (1) human-robot morphological differences disrupt contact interfaces and spatial relationships during retargeting, and (2) jointly learning motor skills and partner coordination makes it difficult to retain control fidelity as interaction diversity grows. In this paper, we propose OmniInteract to address these challenges through surface-aware modeling and scalable residual policy learning. First, we introduce Surface Contact-Aware Retargeting, which leverages dense human-robot surface correspondences to jointly retarget both humanoids, preserving explicit interaction contact interfaces and their near-field geometric context. Second, we propose a Scalable Flow-based Interaction Policy for partner-conditioned residual generation. It factorizes dual-humanoid control into a reusable motor foundation and interaction-specific adaptations: partner-aware experts learn corrections over a frozen whole-body motion prior, while conditional flow matching unifies them in a shared residual action space. This enables scalable skill consolidation while retaining pretrained motor capabilities. Extensive experiments demonstrate that OmniInteract improves retargeting fidelity and dual-humanoid coordination while better retaining interaction performance as skill diversity grows. Real-world deployment further demonstrates successful contact establishment, stable whole-body coordination, and recovery from external perturbations.
comment: 30 pages including appendix, 11 figures. Project page: https://calvinyhli.github.io/omniinteract.github.io/
♻ ★ FERPO: Forward Entropy-Regularized Policy Optimization
Several state-of-the-art methods for online reinforcement learning in continuous control improve policies using action gradients of a learned critic. However, critics are typically trained to predict returns, and accurate value predictions do not necessarily yield accurate action derivatives, potentially leading to unreliable policy updates. We propose Forward Entropy-Regularized Policy Optimization (FERPO), an on-policy maximum entropy reinforcement learning algorithm that performs policy improvement using critic values without differentiating the critic with respect to actions. FERPO derives an optimal target action distribution from a policy-improvement objective regularized by entropy and Kullback-Leibler (KL) divergence. We then fit the actor to this target by minimizing a forward-KL objective, estimated using self-normalized importance sampling (SNIS) with actions drawn from the rollout policy. By limiting the target distribution's deviation from the rollout policy, the KL regularization helps keep these importance weights well behaved. In contrast to reverse-KL objectives, which can favor a subset of the target distribution's modes, the forward-KL objective encourages coverage of multiple high-value modes and thereby promotes exploration. Experiments and ablations on MuJoCo Playground and ManiSkill show competitive performance and sample-efficiency gains. Computational benchmarks also demonstrate faster actor updates than Relative Entropy Pathwise Policy Optimization (REPPO).
comment: v2: Added citations and discussion of related work. Code: https://github.com/Atarilab/FERPO
♻ ★ Robotic Physical Comparison of Biologically Motivated and Counterfactual Sensory Arrangements using Biomimetic Skin
Human skin contains diverse mechanoreceptors, including Merkel cells and Pacinian corpuscles, each localized in specific regions. Although their spatial organization is known to be shaped by molecular signaling and neural induction during development, experimentally examining its functional significance under controlled conditions in living systems remains challenging. In particular, physically comparing alternative receptor arrangements that do not occur biologically is difficult. In this study, we developed a layered biomimetic robotic skin designed to mimic key structural features of human skin tissue by embedding receptor-analog sensors within compliant soft media. Using this platform, we implemented both biologically motivated and counterfactual sensory arrangements and quantitatively evaluated their performance across three passive tactile tasks: detection of weak contacts, a two-point discrimination-like assessment, and texture identification. Arrangements reflecting key features of the biologically motivated sensory organization exhibited higher performance than the alternative configurations examined. This study addresses a biological question concerning how spatial organization influences tactile function by presenting a biomimetic robotic approach that enables physical and empirical comparison of biologically motivated and counterfactual sensory arrangements that are difficult to investigate in living systems.
comment: Published in Bioinspiration & Biomimetics (IOP Publishing). Project page: https://poyotamu000.github.io/biomimetic-skin-sensory-arrangements/ Youtube: https://www.youtube.com/watch?v=25HPwOTodQ8
♻ ★ FPTAS for Polygonal Multi-Robot Motion Planning
We study multi-robot motion planning in a polygonal environment with polygonal robots. The setting of the problem consists of a polygonal domain $\mathcal{W}$ with $n$ vertices and $k$ robots $R_1,\dots,R_k$ translating in $\mathcal{W}$ each of which is represented by a polygon of constant complexity. Given a source configuration $\mathbf{s}$ and a target configuration $\mathbf{t}$ of the robots in $\mathcal{W}$, the goal of the problem is to find a collision-free motion plan from $\mathbf{s}$ to $\mathbf{t}$ that is shortest with respect to some cost function. We present the first fully polynomial-time approximation schemes (FPTAS) for motion planning with any fixed number $k$ of robots, under the min-sum objective, the min-max objective, and the min-makespan objective. Here, the min-sum variant aims to minimize the total traveling distance of all robots, the min-max variant aims to minimize the maximum traveling distance of a single robot, and the min-makespan variant aims to minimize the total time cost under the assumption that all robots move with a maximum speed $1$. The algorithm directly generalizes to the problem in any fixed dimension $d$, with a polytope environment and polytope robots. Our result settles an open question of Agarwal, Halperin, Sharir, Steiger [SODA'24] and Agarwal, Holmgren, Steiger [SODA'26], who gave an FPTAS for $k=2$ unit-square robots and a bicriteria-approximation algorithm for a fixed number $k$ of unit-square or unit-disk robots, respectively. Compared to the previous results, our algorithm is substantially simpler.
comment: To appear in SODA 2027
♻ ★ Learning Where it Matters: Autonomous Data Collection and Offline Reinforcement Learning for Local Contact-Rich Manipulation
Learned policies trained end-to-end on large datasets often remain brittle in high-precision contact-rich tasks. Further, they typically require large amounts of manually collected data and still struggle to generalize to scenes outside of the training scenarios. Classical motion planning, in turn, generalizes well but lacks the precision and contact awareness for tight-tolerance tasks. To reliably solve challenging contact-rich tasks, we propose to leverage robot data and learning where it matters in a modular approach. First, we introduce an autonomous data collection scheme for the real world that collects dense data locally in the contact-rich segment of the task, eliminating reliance on a teleoperator's skill. Second, we show that using offline RL, the collected data can be translated into policies with high success rates. We then integrate our local policies with traditional motion planning to solve long-horizon assembly tasks. Across four real-world tasks, using only 2-2.5 h of autonomous data collection, we achieve an average success rate of 96% for long-horizon tasks, compared to 55% for a planning-only baseline and 37% for the strongest end-to-end baseline. Our local policies also reduce applied forces by 49%. Notably, performance remains at 90% for scene-level out-of-distribution scenarios, while end-to-end approaches struggle. In total, we run over 1500 real-world evaluation rollouts. Our results pave the way for targeted data collection in contact-rich tasks and for high success rates in precision applications. Videos and qualitative results are available at https://learnsyslab.github.io/data_and_learning_where_it_matters/
comment: Project webpage: https://anonymous.4open.science/w/data_and_learning_where_it_matters/
♻ ★ InteractAgent: Agentic Human Motion Interaction with Memory-Augmented Reflection
Long-horizon human-scene interaction requires coordinated spatial reasoning, physical feasibility, and the ability to recover from compounding execution errors. Text-driven motion models are largely scene-agnostic and lack closed-loop correction, while geometry-conditioned approaches depend on paired scene-motion training or explicit 3D preprocessing that limits generalization to novel layouts and open-vocabulary goals. We present InteractAgent, an agentic multimodal LLM system for long-horizon human-scene interaction that plans, executes, observes, and reflects in a closed loop, using a single MLLM as both planner and evaluator and a frozen MotionLLM as the executor. InteractAgent integrates Online Observation for step-wise recovery decisions (continue/redo/replan) and Memory-Augmented Reflection for multi-round plan refinement with adaptive stopping. On UniHSI, InteractAgent improves task success and reduces penetration compared with training-free baselines. Code: https://github.com/AIGeeksGroup/InteractAgent.
♻ ★ VLA-Rolling: Fewer Denoising Steps, Better Control via Adaptive Action Reuse
Flow-matching vision-language-action (VLA) models generate an action chunk by iteratively denoising Gaussian noise, but typically execute only a short prefix before replanning with updated observations. Under this receding-horizon paradigm, the unexecuted remainder---the action tail---is discarded, and the next chunk is regenerated from fresh noise. We show that this discarded tail retains transferable denoising progress: action-tail initialization can match or outperform a native 10-step Gaussian baseline using fewer denoising steps. However, the optimal denoising budget varies substantially across samples, indicating that tail reuse should be adapted to the updated environment. Based on this observation, we introduce VLA-Rolling, a lightweight inference framework that reuses action tails through three stages: action-tail inheritance, per-action confidence estimation, and confidence-guided refinement through renoising and denoising budget allocation, while the underlying VLA remains frozen. Across $π_{0.5}$, SmolVLA, X-VLA, and MolmoAct2 on LIBERO, MetaWorld, LIBERO-Plus, and RoboTwin 2.0, VLA-Rolling reduces action-chunk inference cost in all 16 model--benchmark pairs, achieving up to a $1.48\times$ speedup, while improving success rate in 14 pairs by up to $15.08$ percentage points. These results show that action tails are not merely discarded predictions, but reusable intermediate states that enable more efficient and accurate VLA inference. These results demonstrate that the action tail can support the generation of better actions with fewer denoising steps. Our project is available at https://chenbao123ww.github.io/VLA-Rolling/.
♻ ★ Synergistic Action and Style Priors for Learning Legged Locomotion
Deep Reinforcement Learning (DRL) enables robots to learn complex behaviors through interaction with the environment. However, due to the unrestricted nature of the learning algorithms, the resulting solutions are often brittle and appear unnatural. This is especially true for learning direct joint-level torque control, as inductive biases are difficult to integrate into the learning process. We propose an inductive bias for learning locomotion that is especially useful for torque control: latent actions learned from a small dataset of expert demonstrations. This prior allows the policy to directly leverage knowledge contained in the expert's actions and facilitates more efficient exploration. We observe that the agent is not restricted to the reward levels of the demonstration, and performance in transfer tasks is improved significantly. Latent action priors combined with style rewards for imitation lead to a closer replication of the expert's behavior. Videos are available at https://sites.google.com/view/latent-action-priors.
comment: Humanoids 2026
♻ ★ PAKE: Learning Whole-Body Loco-Manipulation with Partial Kinematic Embeddings
Loco-manipulation has recently shown promising capabilities; however, achieving high-precision control, managing the high-dimensional action space induced by many degrees of freedom (DoFs), and fully exploiting the inherent redundancy of whole-body systems remain challenging. In this paper, we propose a novel whole-body control framework that effectively addresses these challenges by decomposing the complex loco-manipulation problem into partial reference motion generation and low-level imitation control. We introduce a new Kinematic Normalizing Flow (KNF) model, trained on a large-scale kinematic dataset, that generates diverse partial kinematic references. A high-level controller is then trained to navigate the KNF's latent space to exploit redundant solutions, while a low-level controller learns dynamically consistent motion execution. We validate our approach on the quadrupedal robot equipped with a six-DoF robotic arm. In simulation, PAKE outperforms the evaluated baselines in tracking accuracy and feasible workspace coverage on the 4,000-target benchmark. For hardware deployment, we evaluate the system over 24 episodes across 8 different mobile loco-manipulation tasks. The system achieves end-effector pose-tracking errors of 4.5 cm and 0.14 rad, while maintaining accurate locomotion tracking with linear and angular velocity errors of 0.1 m/s and 0.01 rad/s, respectively. Our method represents a practical and powerful solution for accurate and generalized whole-body loco-manipulation in high-DoF robotic systems, with promising potential for diverse downstream robotic tasks.
comment: IEEE ROBOTICS AND AUTOMATION LETTERS 2026
♻ ★ Policy Search in Code Space for Multi-Agent Pathfinding
Developing coordination policies for complex multi-agent systems relies on either labor-intensive algorithm design or opaque task-specific reinforcement learning. With recent advances in autonomous coding agents, we show that policy search directly in code space offers a compelling third path that combines automated optimization with explicit interpretability. We investigate a human-directed workflow in which coding agents iteratively propose, implement, evaluate, diagnose, and eventually refine algorithms guided by simulation feedback and explicit correctness verification. We conducted two case studies and found that this workflow robustly outperformed any manually engineered solutions. On a challenging map in the 2024 League of Robot Runners competition, a two-day search discovered a policy that exceeds the stronger locally reproduced competition entry by 35.0% on the densest instance and generalizes to unseen map instances. In simulated real-world warehouses with irregular lanes, continuous robot geometry, and asynchronous motion, a week-long search discovered an entirely new class of multi-agent pathfinding algorithm that outperforms a calibrated proprietary solution in throughput by 58.9%. Under stated assumptions, we further prove that the new algorithm is collision-free and completes, in finite time, every accepted goal that is not withdrawn or replaced. Together, the studies show that policy search in code space powered by state-of-the-art coding agents offers a highly competitive path to building real engineering solutions that are both effective and analytically tractable.
comment: 18 pages, 11 figures
♻ ★ RoboCap: A New Platform for Egocentric Robot Learning
Despite its promise for scaling robot learning, egocentric manipulation data is still scarce today. Collection at scale requires vertically integrating ergonomic hardware with centimeter-precise 3D algorithms, at a precision that has not been publicly demonstrated. To address this gap, we introduce RoboCap, a 250\,g six-camera dual-IMU hat designed for in-the-wild egocentric data capture, and the Grounded API, a suite of device-agnostic 3D algorithms tuned for RoboCap. In this report, we demonstrate how hardware, calibration, and 3D algorithms interact to achieve state-of-the-art performance on the public benchmarks: our SLAM across diverse settings and rigs, our depth estimation on egocentric settings, and our hand tracking when adapted to third-party devices.
♻ ★ Safe-Night VLA: Seeing the Unseen via Thermal-Perceptive Vision-Language-Action Models for Safety-Critical Manipulation IROS
Current Vision-Language-Action (VLA) models rely primarily on RGB perception, preventing them from capturing modalities such as thermal signals that are imperceptible to conventional visual sensors. Moreover, end-to-end generative policies lack explicit safety constraints, making them fragile when encountering obstacles and novel scenarios outside the training distribution. To address these limitations, we propose Safe-Night VLA, a multimodal manipulation framework that enables robots to see the unseen while enforcing rigorous safety constraints for thermal-aware manipulation in unstructured environments. Specifically, Safe-Night VLA integrates long-wave infrared thermal perception into a pre-trained vision-language backbone, enabling semantic reasoning grounded in thermodynamic properties. To ensure safe execution under out-of-distribution conditions, we incorporate a safety filter via control barrier functions, which provide deterministic workspace constraint enforcement during policy execution. We validate our framework through real-world experiments on a Franka manipulator, introducing a novel evaluation paradigm featuring temperature-conditioned manipulation, subsurface target localization, and reflection disambiguation, while maintaining constrained execution at inference time. Results demonstrate that Safe-Night VLA outperforms RGB-only baselines and provide empirical evidence that foundation models can effectively leverage non-visible physical modalities for robust manipulation.
comment: Accepted to the 2026 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
♻ ★ FallGown: A Soft Pneumatic Wearable Device for Fall Prevention
Concerns about falling and potential resulting injuries prevent many older adults from participating in forms of health-improving exercise such as walking, dance, and tai chi. Existing passive devices like walkers require use of the arms for support, and active devices like exoskeletons and robotic frames are often rigid, bulky, and stigmatized, which limits their use. The FallGown is a soft wearable support structure consisting of pressurized pneumatic pouches within a fabric shell; it can support the full weight of a user's body and aims to prevent falls. It is portable (6.9 kg and 0.051 m3 when stored), minimally burdensome (7.24-20.8 N drag force depending on floor type and whether an optional rolling base frame is used), and convenient (less than 2.5 minutes on average to don and less than 0.5 minutes on average to doff). The optional rolling base lowers friction with the floor, but reduces portability and skirt deformation (which can be useful to fit through doors and other tight spaces). Fall prevention testing with a single user revealed a tradeoff between device size and stability against tipping; a skirt with 12 pneumatic pouches and a floor diameter of 1.4 meters with a base frame prevented all falls except in one case with a mechanical failure. On-board pressure sensors can sense the user's weight distribution, providing an opportunity for monitored exercise and adaptive support.
♻ ★ AFFORD2ACT: How Far Can Sparse Keypoints Go for Robot Manipulation?
Vision-based manipulation policies are commonly learned from rich visual observations, typically using dense visual features extracted from full resolution images. It remains unclear how much of this visual information is really needed for effective policy learning. In this paper, we investigate the opposite end of the spectrum: how effective can manipulation policies be if they use as input only a compact visual representation. Sparse 2D keypoints provide a deliberately compact representation, giving the policy spatial information while removing direct access to appearance, texture, and background. We design AFFORD2ACT (Affordance-Guided Keypoints to Act) to study how much manipulation capability can be retained under this representation constraint. An off-the-shelf language-conditioned affordance model identifies the task-relevant region, from which up to 15 object keypoints are selected, together with 4 end-effector keypoints. The policy input is therefore reduced to only 19 2D keypoints (38 coordinates). A lightweight gating further decides which keypoints to pay attention to during execution. Across six real-robot manipulation tasks, this compact state outperforms the evaluated RGB, RGB-D, and dense-affordance behavior cloning baselines, degrades less as the number of demonstrations is reduced, and achieves 82% average success in held-out settings. Replacing learned gating with uniform weights reduces success from 92% to 52%. At the same time, the representation depends on reliable affordance localization and keypoint tracking, and remains vulnerable to incorrect masks, occlusion, tracking drift, and distractors.
♻ ★ Vector-Quantized World Model: Latent Feedback Motion Planning for Robot Manipulation
Visual world models have shown strong potential for robot planning, yet long-horizon manipulation remains challenging due to compounding prediction errors, large search complexity, and latent plans that the low-level controller cannot reliably reach. In this work, we introduce Vector-Quantized World Model (VQ-WM), a latent feedback motion planning framework that integrates discrete long-horizon search with continuous visual feedback control: a discrete latent predictor models transitions between vector-quantized latent states, allowing the planner to look far ahead and find a shortest path to the goal through discrete search and a continuous learned controller sequentially reaches the planned states through visual feedback, correcting deviations to mitigate compounding errors. Whereas most vector quantization uses Euclidean distance, we use a novel estimator of the first-passage-time as a quasi-metric, which aligns the planning abstraction with the controller's capabilities. We evaluate on the physical robot manipulation of the Tower of Hanoi puzzle: whereas existing state-of-the-art foundation models (VLAs, WAMs) only succeed when the solution trace in their training dataset, our method achieves 100\% success rate with only short-play data, demonstrate its ability to compose long-horizon solutions beyond the short demonstrated trajectories and execute them reliably. Our solution also outperforms strongest world-models by 13.3--85.3 percentage points on simulation benchmarks spanning manipulation and locomotion, demonstrating consistent progress and new capabilities.
♻ ★ Robot Learning with Visual Predicted Force
Force-aware manipulation typically relies on specialized force or tactile sensors. We show that force-aware manipulation can instead be achieved through visual force prediction from the deformation of a compliant Fin Ray gripper. Our approach trains two models. First, we train a visual force estimator on calibration data and use it to annotate task demonstrations with force estimates. Second, we train an action--force proposal policy on these force-augmented demonstrations to jointly generate candidate robot actions and their associated forces. At test time, we sample candidate actions and the forces they are expected to produce, then execute the action whose predicted force is closest to a target from the demonstrations. We evaluate our approach on berry picking, empty-can grasping, in-hand reorientation, and plug insertion. Our results show that visual force prediction can guide inference-time action selection for contact-rich manipulation without requiring force or tactile sensors at deployment.
comment: 9 pages, 8 figures. Project Website: https://visual-force.github.io/
♻ ★ PolyUMI: Accessible Visual-Tactile-Audio Data Collection for Object Inference and Manipulation
Humans typically rely on vision, touch, hearing, and proprioception to perceive contact and adapt their actions during manipulation. Providing robots with comparable responsiveness therefore requires hardware that can retain and use these complementary sensory signals. Most imitation-learning systems, however, observe demonstrations primarily through vision and proprioception, limiting access to contact information that is difficult to infer visually. We present PolyUMI, an open-source platform for scalable visual--tactile--audio demonstration collection and robot deployment. Its lightweight, wireless handheld gripper records synchronized wrist-camera, optical tactile, contact-audio, and proprioceptive observations without requiring a tethered workstation. The same sensing finger can be transferred to the robot end effector, preserving the sensing geometry between demonstration collection and policy execution. To effectively use these heterogeneous observations, we further introduce VisTA, a token-level multimodal policy that integrates information across sensors and time to predict contact-aware robot actions. Experiments spanning object inference, slip control, and contact-rich manipulation show that touch and audio reveal task-relevant information beyond vision and that VisTA is competitive with or outperforms existing multimodal policies. Together, PolyUMI and VisTA provide an accessible pipeline for collecting multimodal demonstrations and learning policies that perceive physical interaction beyond vision. Project Page: https://polyumi-vista.github.io
comment: 9 pages, 10 figures, preprint
♻ ★ Stochastic Neural Control Barrier Functions
Control Barrier Functions (CBFs) are utilized to ensure the safety of control systems. CBFs act as safety filters in order to provide safety guarantees without compromising system performance. These safety guarantees rely on the construction of valid CBFs. Due to their complexity, CBFs can be represented by neural networks, known as neural CBFs (NCBFs). Existing works on the verification of the NCBF focus on the synthesis and verification of NCBFs in deterministic settings, leaving the stochastic NCBFs (SNCBFs) less studied. In this work, we propose a verifiably safe synthesis for SNCBFs. We consider the cases of smooth SNCBFs with twice-differentiable activation functions and SNCBFs that utilize the Rectified Linear Unit or ReLU activation function. We propose a verification-free synthesis framework for smooth SNCBFs and a verification-in-the-loop synthesis framework for both smooth and ReLU SNCBFs. and we validate our frameworks in three cases, namely, the inverted pendulum, Darboux, and the unicycle model.
♻ ★ $μ_0$: A Scalable 3D Interaction-Trace World Model
World models that capture how actions induce physical change enable scalable robot learning without reliance on embodiment-specific action labels. Pixel-space video models provide broad visual priors but expend model capacity on dense appearance reconstruction, while direct action models require embodiment-specific labels that hinder scalability. We present $μ_0$, a scalable world model based on 3D traces. Rather than predicting dense pixels or directly modeling actions, $μ_0$ forecasts smooth 3D trajectories for salient interaction points such as objects, tools, hands, and contact regions, yielding a compact, embodiment-agnostic motion interface. To enable training from diverse video sources, our TraceExtract system automatically extracts 3D supervision by selecting keypoints, constructing globally aligned traces, and associating motion segments with hierarchical language captions. This TraceExtract supervision pretrains $μ_0$ by combining a pretrained vision-language backbone with a modular trace expert, which represents each query via B-spline control points and predicts future traces. Experiments show that $μ_0$ outperforms baselines in both 2D and 3D trace prediction, including trace prediction models and tokenized VLM methods. Because $μ_0$ is frozen and reusable, it can be paired with action experts for downstream robot embodiments. Despite action-free pretraining, the resulting trace-conditioned policies achieve performance competitive with VLA models pretrained with action supervision, such as $π_0$. These results establish 3D traces as a scalable and transferable representation for cross-embodiment manipulation.
comment: Comments: Accepted to CoRL 2026. Project Page: https://mu0-wm.github.io/
♻ ★ AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems
We explore how scalable robot data can address real-world challenges for generalized robotic manipulation. Introducing AgiBot World, a large-scale platform comprising over 1 million trajectories across 217 tasks in five deployment scenarios, we achieve an order-of-magnitude increase in data scale compared to existing datasets. Accelerated by a standardized collection pipeline with human-in-the-loop verification, AgiBot World guarantees high-quality and diverse data distribution. It is extensible from grippers to dexterous hands and visuo-tactile sensors for fine-grained skill acquisition. Building on top of data, we introduce Genie Operator-1 (GO-1), a novel generalist policy that leverages latent action representations to maximize data utilization, demonstrating predictable performance scaling with increased data volume. Policies pre-trained on our dataset achieve an average performance improvement of 30% over those trained on Open X-Embodiment, both in in-domain and out-of-distribution scenarios. GO-1 exhibits exceptional capability in real-world dexterous and long-horizon tasks, achieving over 60% success rate on complex tasks and outperforming prior RDT approach by 32%. By open-sourcing the dataset, tools, and models, we aim to democratize access to large-scale, high-quality robot data, advancing the pursuit of scalable and general-purpose intelligence.
comment: Project website: https://agibot-world.com/. Github repo: https://github.com/OpenDriveLab/AgiBot-World. The author list is ordered alphabetically by surname, with detailed contributions provided in the appendix
♻ ★ ASPIRE: Agentic /Skills Discovery for Robotics SP
Traditional robot programming is notoriously challenging: it requires orchestrating multimodal perception, managing physical contact dynamics, and handling diverse environment configurations and execution failures. We introduce ASPIRE (Agentic Skill Programming through Iterative Robot Exploration), a continual learning system that autonomously writes and refines robot control programs in a code-as-policy paradigm while compounding experience into a reusable skill library. ASPIRE operates in an open-ended learning loop, consisting of three components: (1) a closed-loop robot execution engine that exposes fine-grained multimodal traces, enabling the agent to autonomously diagnose failures, synthesize repairs, and validate outcomes; (2) a continually expanding skill library that distills validated fixes into reusable, transferable robotic knowledge; and (3) an evolutionary search procedure that generates diverse programs, systematically debugging them to explore beyond single-trajectory refinement. ASPIRE surpasses prior methods by up to 77% on manipulation tasks under perturbation, 72% on Robosuite's bimanual handover task, and up to 32% on long-horizon household tasks. The accumulated skill library further enables zero-shot generalization: on representative unseen long-horizon tasks, ASPIRE achieves 31% success, outperforming the 4% success rate of prior methods. Performance scales with the capabilities of frontier models. With Claude Opus 5.5, ASPIRE nearly saturates these benchmarks, achieving 99% average success across all 80 LIBERO-Pro tasks and 100% success on representative Robosuite and BEHAVIOR-1K tasks. Beyond simulation, skills discovered in simulation reduce real-world robot programming effort across embodiments and robot APIs, while skills acquired through online interaction with physical robots enable ASPIRE to solve a Rubik's Cube with progressively improving efficiency.
comment: 41 pages, 19 figures, 26 tables. Project page: https://research.nvidia.com/labs/gear/aspire/ Code: https://github.com/NVlabs/ASPIRE
Multiagent Systems
★ GBClaw: An agentic AI framework for atomistic grain boundary research and mechanistic discovery
Agentic artificial intelligence (AI) systems hold immense potential for automating complex workflows in materials discovery and design, but extending them to domain-intensive mechanistic investigations remains challenging. Here, we present GBClaw, an agentic AI framework for investigating the structure and deformation of general grain boundaries (GBs). Although general GBs constitute the vast majority of real-world interfaces in polycrystalline materials, they remain understudied because their structural complexity and interfacial mismatch complicate experimental and modeling research. GBClaw integrates large language models (LLMs) into a modular architecture comprising an orchestrator, specialized agents, domain-specific tools, simulation interfaces, and hierarchical memory. Through dynamic replanning and iterative workflow refinement, the framework effectively connects systematic simulations with targeted mechanistic analysis. Applied to 14 general asymmetric tilt GBs in nickel, GBClaw characterized interfacial defect structures and identified diverse deformation modes across these incommensurate boundaries. Comparing boundary sliding with and without secondary GB misfit dislocations revealed substantial differences in shear strength, underscoring the critical role of defect hierarchies in sliding resistance. Furthermore, researcher-guided follow-up calculations traced an anomalously high shear resistance to metastable boundary structures. This work demonstrates that combining agentic AI with domain expertise bridges high-throughput computational screening with deep physical understanding of complex material interfaces.
★ Proof-Carrying Contracts: Commitments AI Agents Cannot Break
AI agents increasingly act on behalf of people or other agents, purchasing services, exchanging information and delegating work. What an agent must respect is written in instructions, policies or agreements, but text alone cannot make it comply. It may make mistakes, be redirected by what it reads, or pursue different interests. For an economy of such agents to function, parties need agreements they can rely on without inspecting or trusting one another's agents. We introduce Proof-Carrying Contracts (PC2), a framework for making agreed constraints enforceable throughout an agent's interactions with a system. Before a requested operation can run, the agent must supply a machine-checkable proof that it preserves the agreed constraints in every case allowed by its model. An execution system called the Airlock checks the proof and controls access to the operation. The parties specify the conditions to preserve, while the agent supplies the reasoning needed to justify its decisions and remains free to adapt its strategy. This avoids having to design a separate permission policy and prove that every decision it permits is safe. We prove that the constraints hold at every instant during execution, even for an adversarial agent, and identify when this implies safety in the world. Reusable capability modules provide tools and guarantees from which contracts can be constructed. PC2 provides a foundation for an agent economy whose agreements protect both those delegating work and those carrying it out.
comment: 70 pages, 8 Figures,
★ Claims of emergence in AI should be grounded in information decomposition NeurIPS
When a language model develops capabilities not traceable to its individual components, when a multi-agent AI system outperforms each agent on its own, or when a human-AI pair succeeds where either fails in isolation, AI researchers describe it as a case of ``synergy'' or ``emergence''. However, these are usually informal assessments not supported by a theoretical framework, making these claims impossible to compare or even meaningfully dispute. Here we argue that \emph{information decomposition can serve as a unifying formal framework to operationalize assessments about synergy in AI systems}. Information decomposition provides a way to distinguish qualitatively different \emph{types} of information, including unique information available from individual components, redundant information shared across multiple components, and synergistic information accessible only from parts taken together. This gives claims of emergence a precise, falsifiable criterion: a system's behavior is emergent only if it carries synergistic information about task-relevant outcomes. We apply this lens to attention heads, autonomous agents, and human-AI teams, showing how it organizes existing knowledge across scales and substrates, while also generating cross-level conjectures that siloed fields would miss.
comment: NeurIPS Spotlight Paper
★ Toolflation: Seller Agents Distort Tool Markets Through Spontaneous Generative Engine Optimization
Agents are starting to sell to other agents. Seller agents write the listings of tools, services and products, and buyer agents read these listings when they choose what to use. We show that seller agents told only to be selected more often practice generative engine optimization on their own, and that this distorts the market for sellers and for buyers. We build tool markets from real data tools on Amazon and Instagram and run every tool on every question, so we know which tools can answer each question. Under all five production models we test, sellers rewrite their descriptions to claim more and more fields, including fields that their tools return for fewer than half of the targets. We call this process toolflation. Nothing in the market limits it, and how far it goes depends on the model, as the number of claims that contradict the sellers' own evidence differs by an order of magnitude between models. Toolflation shifts the buyers' choices between sellers, and the most capable tool loses share in seven of the ten combinations of seller model and platform. Under four models, the choices stay with capable tools and buyer agents keep their task success. Under the fifth, sellers copy one another until the descriptions no longer tell the tools apart. Buyer agents then pick tools by their position in the list, tools that answer almost nothing win a quarter to two fifths of the choices, and task success falls from 89% to about 30% on Amazon, close to random choice, and from 77% to 40% on Instagram. The sellers did only what their vendors asked, and the cost fell on buyers they never dealt with. We therefore argue that honesty toward third parties under competition is part of safety alignment, and that models should be tested as sellers before they are deployed as sellers.
comment: 24 pages, 8 figures, 9 tables. Code: https://github.com/lilv98/Toolflation. Data: https://huggingface.co/datasets/lilvjosephtang/Toolflation
★ Local Work, Global Progress: A Convergence Theory of AI Agent Collaboration
Teams of large language model (LLM) agents are increasingly applied to complex tasks, but their collaboration protocols are mostly designed empirically, without convergence guarantees. We formalize collaborative problem solving as block optimization: each agent improves one part of a shared solution, guided by its local objective plus a correction for how its changes affect the other parts. A coordinator checks every proposal against the global objective and merges proposals that do not interfere with each other. We prove linear convergence up to an error floor that vanishes when the corrections are exact. Interfering proposals can also be merged once their combination has been checked; the rate then depends on how well the proposals actually combine, not on worst-case interactions. With GPT-OSS-120B agents on robot routing, a three-agent team finds collision-free plans in every run, better than those of a single agent that controls all robots. On a larger routing task, a six-agent team completes its search about twice as fast as the single agent (in median). Merging the best checked combination of proposals, rather than only proposals that do not interfere, cuts the team's search time by about 30%. With constants measured on the runs, our convergence bound holds in every round and follows the observed progress. We also test the theory on query planning in SQLite and on a synthetic task.
comment: 59 pages, 15 figures, 28 tables. Code: https://github.com/vectozavr/local-work-global-progress
★ Safe Multi-Robot Collaboration With Uniform Exclusion Using Trust
Robot swarms coordinate through directed, time-varying local communication links, some of which may be compromised to allow the transmission of arbitrary malicious data. We provide an algorithm that acts as a wrapper around robot communication---when a robot seeks to update its state using data from some incoming links, our wrapper provides a subset of trusted links whose data can be safely incorporated. It selects these links by testing a stream of trust values---data collected from an anomaly detector on each robot---using Safe Anytime-Valid Inference. We require only that the detector be better than chance. This wrapper ensures safe and performant autonomy by providing two powerful statistical guarantees: it uniformly excludes all compromised links for all time with high user-chosen probability, but nevertheless quickly learns to trust legitimate links. Both guarantees are vital to secure generic tasks, but no previous work provides them simultaneously without restrictive assumptions on connectivity structure or the number of compromised links that are not guaranteed in practice. We study our Safe Wrapper's performance in hardware for two contrasting tasks---neural implicit mapping and heading consensus---deploying on 3 LoCoBots and 8 GoPiGos, respectively. We provide simulations using trust values sampled from hardware to validate that our approach scales well to swarms of hundreds of robots.
comment: 44 pages (30 main text + 14 supplementary); 10 figures, 6 tables
★ S$^2$-MAS: A Society-Scale Multi-Agent Simulation Framework with 500 Million Household Persona Agents
Simulating economic responses at society scale requires agents whose decisions reflect household circumstances and the effects of earlier actions. We introduce S$^2$-MAS, a society-scale multi-agent simulation framework built on 500 million synthetic household personas grounded in real-survey data. Its Genese engine separates decisions from state updates, while reusable financial scenarios and application ledgers track consumption, cash, debt and payments. FinL-mlx, a compact decision model trained with SFT + RLCD on 2.93 million examples, achieves 74.55% accuracy and 38.37% question-macro F1 in consumer financial-choice scenarios, outperforming Jev 1.13 and GLM-5.3-Flash on both measures. We evaluate eight financial task families and simulate 12 million distinct household agents in paired purchase-funding conditions. Jointly raising purchase goals and approved credit limits increases consumption, with responses differing across age and income groups. Education, employment and household-size differences persist after age--income standardization and mainly reflect transitions from deferral to purchase. Asset and debt groups differ in cash drawdown, repayment and closing debt. A 24-month panel varies purchase goals under a common credit product, with repayment and arrears updating subsequent offers. S$^2$-MAS links heterogeneous household decisions to evolving consumption, cash and repayment paths, supporting economic experiments across household groups, population scales and time horizons.
comment: 58 pages, 13 main figures, 7 supplementary figures
★ Does Multi-Agent Collaboration Pay When Agents Serve Users? Testing Agent Scaling in Dual-Control Settings
Multi-agent systems (MAS) built from language models are deployed on the premise that collaboration beats a single-agent system (SAS), yet when it pays is unclear. The most systematic study to date gives a capability-saturation rule: coordination helps while the SAS is weak and hurts once it is strong, with the crossing near 45% single-agent accuracy. That rule was derived on autonomous benchmarks, where no user takes part. We test it where agents now work: in conversation with a user under a written policy, on $τ^2$-bench, the dual-control benchmark. On its airline, retail and telecom domains we compare the stock SAS with a centralized MAS built from the same model for six models from three families, over about 7,950 simulated conversations, with one completion-token ledger per episode per system. The study was pre-registered with a public change log; the MAS is reported as built, with one validity bar unmet (its planner never saw the tool schemas). Under a fixed simulated user no cell shows a MAS benefit: the MAS is worse in 12 of the 13 valid cells and tied in the thirteenth, and the fitted gain is negative across the observed capability range (weighted mean -28 points, 95% CI [-44, -12]). On the two cells where the rule predicts a benefit, the MAS loses by 13 and 22 points, and without a budget ceiling it at best draws level at 5.5 times the SAS's token spend. The one gain appears on the cell nearest 45%, under a second simulator. Whether the gain declines with capability is unresolved (slope -0.41, 95% CI [-1.51, +0.68]), because model-by-domain effects dominate. The MAS does not detectably reduce wrongful database writes; it converts other failures into budget exhaustion and inaction. For agents that serve people, the SAS is the default; a multi-agent design must show compute-matched evidence from the setting itself. We release all trajectories, ledgers, prompts and code.
★ Complete Street Design informed by Multi-Agent Simulation
Objectives: Complete Street design is currently guided largely by qualitative principles despite the increasing complexity of multimodal urban traffic. This paper develops a microscopic simulation framework for quantitatively evaluating alternative street designs by explicitly representing heterogeneous road users, including Autonomous Vehicles (AVs), Connected Vehicles (CVs), Human-Driven Vehicles (HDVs), pick-up and drop-off (PUDO) services, and micromobility users. The framework jointly evaluates mobility and safety within a unified testbed. Methods:The framework enriches microscopic simulation with behavioral models for heterogeneous road users. It explicitly captures lateral movements, parking maneuvers, pedestrian crossings and jaywalking, and shared-lane bicycle and scooter interactions. Safety is quantified using a conflict severity measure grounded in collision physics, while candidate street layouts are optimized using a genetic algorithm. Findings: Applied to a segment of Belmont Avenue in Chicago, Illinois, the framework shows that dedicated bicycle lanes reduce conflict severity at the expense of vehicular throughput. Increasing AV penetration mitigates congestion by reducing traffic shockwaves while improving interactions with pedestrians relative to CVs and HDVs. The optimization framework efficiently identifies Pareto-efficient street layouts that balance mobility and safety. Novelty: The proposed framework integrates heterogeneous behavioral modeling, physics-based conflict assessment, stochastic performance evaluation, and design optimization within a single quantitative framework for Complete Street design. Practical Applications: The framework provides transportation planners with a predictive virtual testbed for evaluating street designs and quantifying the tradeoff between mobility and safety before implementation.
comment: Under review for publication in Transportation Research Record (TRR)
★ Robot-Task Suitability in Heterogeneous Multi-Robot Task Allocation IROS 2026
Heterogeneous multi-robot task allocation depends on estimates of how suitable individual robots are for candidate tasks, yet suitability can be defined in substantially different ways. We compare ten deterministic methods and two LLM-based variants spanning rule-based, similarity-based, requirement-oriented, fuzzy, and semantic approaches. Using common robot and task profiles, we compare how the methods score and rank candidate robots, identify feasibility, and select a top robot, while examining how these outcomes change with team composition, scale, spatial context, capability uncertainty, and task representation. The methods frequently disagree despite operating on identical scenarios. Generic similarity measures can violate properties expected under a one-sided requirement-satisfaction interpretation of suitability, while LLM-based scoring introduces additional stochastic and linguistic sensitivity. These results show that robot--task suitability should be treated as an explicit modeling choice whose assumptions, robustness, and decision-level consequences should be considered before allocation.
comment: Accepted at Workshop on Foundation Models for Multi-Robot Systems (FM-MRS), IROS 2026
★ The Butterfly Effect in LLM Social Simulations: From Architecture Sensitivity to Robustness Audits
A butterfly effect occurs when a local perturbation propagates through a system and produces divergence in its collective outcome. In LLM social simulations, this creates a particular challenge for scientific inference: sensitivity to implementation choices can enter agent interactions and alter the collective outcomes on which scientific claims are based. We study this problem through a repeated Prisoner's Dilemma and a social-media echo chamber, spanning representation- and design-level perturbations across four LLM families. We find that controlled perturbations can substantially alter collective outcomes, with sensitivity varying across both perturbations and models. In the Prisoner's Dilemma, changing only the representation of a matched persona shifts two-agent cooperation by 75.2 percentage points for GPT-5.2, while game framing and memory representation produce smaller or less consistent effects. Multi-agent interaction further reshapes representation sensitivity, producing amplification or attenuation depending on the model and perturbation. In the echo-chamber simulation, persona representation changes interaction targeting and is accompanied by divergence in collective interaction structure, extending representation sensitivity from agent behavior to network-level outcomes. These results show that robustness cannot be inferred from a single simulation configuration and that local sensitivity can change as it propagates through interaction. We introduce TRAILS (Taxonomy for Robustness Audits In LLM Simulations) and the Social Simulation Audit Card as a framework for systematically auditing these dependencies and calibrating the scope of scientific claims to the robustness evidence that supports them.
★ Transformers as Generic Deep Neural Networks: Exact Compilation, the Cost of Removing Activation-Activation Products, and Multi-Agent Attention Clustering
Transformers are deployed as a special model class. Neural processing units (NPU) add dedicated operators for attention and normalization, and quantization relies on Transformer-specific algorithms. We show the computation does not force this. Every Transformer block, with attention, LayerNorm and a GELU MLP, compiles exactly into a generic deep neural network built from fixed affine maps, routing and the square, sigmoid and inverse square root, plus GELU or its sigmoid approximation. Without activation-activation products, every transform-wise compilation adds at least $N^2$ scalar nonlinear evaluations per head for dense scores. Keeping that one operation type removes this cost; its elementwise form, present on every surveyed NPU, suffices with the sigmoid and inverse square root. For top-$k$ attention with exchangeable tail scores, we prove that exact or conditional-mean tail mass gives an unbiased, minimum-mean-square-error estimate of the attention distribution, while tail drop is conditionally biased. When queries or agents cluster, centroids choose the keys first, reducing the number of computed scores from $N^2$ to $Nk$ per head. On DeiT-Tiny, MambaIRv2-Light and Pythia-70M the compiled models match their references with all keys, standard post-training quantization applies, and the exact tail-mass estimate beats tail dropping with half as many keys on DeiT-Tiny and, for $k\le8$, on Pythia-70M.
comment: 36 pages, 6 tables and 2 figures in the main text, 8 supplementary tables. Code: https://github.com/iceslice0/transformer-to-dnn
★ JaxMARL-BC: Multi-Agent Reinforcement Learning for Macroeconomic Modelling, in Seconds
Multi-agent reinforcement learning (MARL) offers a principled way of populating simulated economies with many heterogeneous decision makers. Its adoption, pioneered in finance, has nonetheless been slow in macroeconomics because of two main limitations: the lack of a shared frame of reference connecting MARL systems to dynamic stochastic general equilibrium (DSGE) models the profession already trusts, and the very long training times. The first limitation was recently addressed by the Multi-Agent Reinforcement Learning Business Cycle (MARL-BC), which places deep RL households in a real business cycle economy and reduces to classic DSGE models in specific limits. Here we address the second limitation. We present JaxMARL-BC, a JAX-accelerated simulator that trains MARL-BC economies in seconds (between 75x and 724x faster than the reference implementation), and scales to 200,000 independent learning households. We verify its correctness against the known DSGE limits, characterise its scaling through a simple cost law, and demonstrate its value by a proof-of-concept calibration of an extended model with heterogeneous returns on capital, in a two-hour experiment that would previously have taken over a month. We release JaxMARL-BC as open source.
comment: 9 pages, 7 figures
★ Fig: Automated Multiagent Behavior Elicitor
Many social evaluations for multiagent systems study behaviors as they emerge from environments instead of systematically intervening in the environment to elicit such behaviors. We introduce Fig (Finding Interactions in Games), an agentic system that performs targeted elicitation interventions on game environments. It takes a natural language behavior description, decomposes it into sub-behaviors, generates new social games or selects existing ones, and executes single or combined interventions to the games' engine, opponents, instructions, and context. To evaluate model behavior, it combines built-in mechanical checks and synthetic rubric judgments. We validate the interventions, generated games, and evaluation mechanisms via human review. Models attempt the target behavior in 45.2% of episodes on average for valid scenarios generated by Fig for six test behaviors -- cooperation, exploitation, deception, covert coordination, betrayal, and sabotage. For game-behavior pairs with available controls, Fig's interventions increase attempt rates by 10.9% on average compared to matched control games. Fig shows that multiagent environments can serve as programmable test beds for intentional, controllable, and reproducible behavior evaluation. Code is available at https://github.com/fleet-ai/fig.
★ To Manipulate or Not: Opinion Dynamics in LLM Interactions
Large language models increasingly take part in extended conversations where they can both shape and be shaped by another party's views, yet most persuasion research measures only one side of that exchange. We introduce a controlled LLM-LLM debate framework in which two language-model roles repeatedly discuss a contested topic (climate-change attribution): one is given an assigned stance and, in some conditions, an explicit instruction to persuade, while the other receives no assigned position. Across four ordered model combinations involving GPT-6 Luna and Gemini 2.5 Flash Lite, we track both each role's explicitly reported position and a separate measure of its turn-by-turn behavior inside the conversation. Opinion dynamics differ substantially across model combinations: same-model interactions narrow the opinion gap on average, clearly for only one of the two models, while cross-model interactions widen it. Explicit reports and in-dialogue behavior capture distinct aspects of opinion dynamics, and a blanket instruction to persuade does not reliably increase influence. These findings suggest that conversational influence depends not only on assigned roles and initial positions but also on the participating models, with implications for how opinion change is measured in extended LLM interactions.
♻ ★ GenoMAS: Guided Multi-Agent Programming for Scientific Discovery from Genomic Data EMNLP 2026
Gene expression analysis holds the key to many biomedical discoveries, yet extracting insights from raw transcriptomic data remains challenging due to the complexity of multiple large, semi-structured files and the need for extensive domain expertise. Current approaches either rely on inflexible workflows that fail in edge cases or fully autonomous agents that lack the precision required for rigorous scientific inquiry. GenoMAS addresses this by presenting a team of LLM-based agents that integrates structured workflows with autonomous adaptability, orchestrating diverse specialized agents through typed message-passing protocols. At its core lies a guided-planning framework where programming agents decompose task guidelines into Action Units and dynamically choose to advance, revise, bypass, or backtrack, maintaining logical coherence while adapting to the idiosyncrasies of genomic data. On the GenoTEX benchmark, GenoMAS achieves 89.13% CSC for data preprocessing and 60.48% $F_1$ for identifying trait-associated genes, surpassing prior art by 10.61% and 16.85%. Beyond transcriptomics, it outperforms strong agent baselines on ScienceAgentBench and DiscoveryBench, two independent bioinformatics benchmarks. Qualitative analysis shows that GenoMAS identifies gene-phenotype associations that are biologically plausible and corroborated by the literature.
comment: Accepted to EMNLP 2026 Findings. Project page at https://liu-hy.github.io/GenoMAS/ and code at https://github.com/Liu-Hy/GenoMAS
♻ ★ ROAR: Benchmarking LMM Agents under Real-Time Constraints in Multiplayer Action-Shooter Games NeurIPS 2026
Large multimodal models (LMMs) are increasingly deployed as agents that perceive, reason, and act in interactive multi-agent environments. Yet most existing benchmarks for LMM-based agents in multi-agent settings remain turn-based: the environment and other agents pause while an agent performs inference. Such protocols obscure a fundamental challenge of real-time constraints, where the world continues to evolve and delayed decisions may become outdated before execution. Motivated by this limitation, we introduce ROAR, a multiplayer action-shooter game environment for evaluating LMM-based agents in multi-agent settings under real-time constraints. ROAR covers cooperative, competitive, mixed cooperative-competitive, and basic single-agent scenarios. We further propose a Multi-Agent Asynchronous Gym interface that decouples environment evolution from agent reasoning by running the environment and each agent in separate threads, enabling the environment to advance at fixed time intervals without waiting for LMM inference and allowing agents to reason independently without blocking one another. ROAR also features convenient deployment, multimodal observations, and customizable levels. ROAR offers a practical testbed for advancing LMM-based agents beyond static, turn-based evaluation toward real-time multi-agent decision-making.
comment: NeurIPS 2026
♻ ★ OpenAgenet / OAN Yellow Paper: Technical Architecture for Trust-Governed Resource Identity and Discovery
This yellow paper describes the technical architecture of OpenAgenet / OAN. OAN is a protocol-neutral trust layer for open Agent interconnection and discoverable AI resource products. It specifies the role architecture, did:oan identity objects, registration workflow, governance-backed Root lifecycle enforcement, Root-verified package model, authorization-aware Discovery, Root-issued infrastructure authorization VCs, signed trusted invocation, Root's trust, data-distribution, and semantic-governance hub responsibilities, verification requirements, state transitions, security properties, implementation boundaries, and deployment considerations. The design is intended to support heterogeneous Agent frameworks and interaction protocols, including MCP, A2A, ANP-like systems, domain-specific Agent protocols, Skills, MCP Servers, and Tool/API resources. OAN does not define the entire business conversation among Agents or the native protocol of every resource; it defines how resource identities become admissible, discoverable, verifiable, and safe to approach before protocol-specific interaction begins, and how defined profiles carry that identity into protected protocol interaction.
♻ ★ OpenAgenet / OAN White Paper: Open Infrastructure for Trusted Agent Interconnection
OpenAgenet, abbreviated as OAN, is an open infrastructure project for trusted Agent interconnection. It addresses a problem that becomes visible when Agents move from isolated applications into open, multi-operator networks: before an Agent can safely discover, select, and invoke another Agent, it needs a way to verify identity provenance, governance state, discovery authorization, freshness, and pre-connection trust evidence. OAN is designed as a protocol-neutral trust layer. It does not replace Agent interaction protocols, tool protocols, model orchestration frameworks, or application-level workflows. Instead, it provides routing-aware did:oan-based resource identity, local OAN Identity material for signing and authentication, governance-backed admission, Registrar-assisted onboarding, Root-verified package publication, authorization-aware Discovery, Root-issued infrastructure authorization VCs, controller-aware authorization, and profile-defined signed trusted invocation. The architectural center of OAN is the combination of federated governance, resource identity, trusted Discovery, and the Root Node's three hub responsibilities. Root acts as a trust authorization hub for accepted identity state and evidence, a data-distribution hub for Root-verified packages, and a semantic-governance hub for the tag tree shared by registration and Discovery. This white paper explains the motivation, architecture, roles, governance model, relationship with MCP, A2A, and ANP, deployment patterns, cooperation model, on-chain governance layer, prototype status, public pilot network, standardization surfaces, performance profile, and roadmap of OAN.
♻ ★ From Debate to Decision: Conformal Social Choice for Safe Multi-Agent Deliberation
Multi-agent debate improves LLM reasoning, yet agreement is not evidence of correctness. When agents converge on a wrong answer through social reinforcement, consensus stopping commits that error to automated action with no recourse. We introduce Conformal Social Choice, a post-hoc layer that converts debate outputs into calibrated act-versus-escalate decisions. Verbalized probability distributions from heterogeneous agents are aggregated via a linear opinion pool and calibrated with split conformal prediction, yielding fixed-round marginal coverage: at any pre-specified round, the correct answer is included with probability ${\geq}\,1{-}α$, without assumptions on individual model calibration. A hierarchical action policy maps singletons to autonomous action and larger sets to human escalation. Across eight MMLU-Pro domains and two additional benchmarks, fixed-round coverage stays close to target. The key finding is not that debate becomes more accurate, but that the conformal layer makes its failures actionable: at the final round, 81.9\% of wrong-consensus cases are flagged by non-singleton sets at $α{=}0.05$. Separately, among cases that reach a singleton before the final round, accuracy reaches 90.0--96.8\% on MMLU-Pro (up to 22.1pp above consensus stopping): a selection effect, not a reasoning improvement. Simple probability- and entropy-threshold rules attain comparable interception; conformal's distinct value is a finite-sample fixed-round coverage guarantee together with explicit candidate sets for escalation. Adaptive first-singleton stopping is evaluated empirically and not covered by this guarantee.
♻ ★ ECHO: Learning Epistemically Adaptive Language Agents with Turn-Level Credit
Effective multi-turn agents must decide what information to seek, how to use new evidence, and when they are certain enough to act. We introduce Epistemic Decision Processes (EDPs), a belief-state formulation of multi-turn information seeking in which actions produce external observations that update the agent's posterior over a latent task variable. EDPs make epistemic adaptivity explicit: good policies choose actions that are useful under the current belief, not merely those that correlate with eventual success. We then introduce ECHO (Epistemic Credit for History-Conditioned Optimization), a practical clipped policy-gradient objective that assigns turn-level credit using posterior-sensitive rewards. In the Clue Selector Game, a novel controlled evidence-seeking benchmark, and in CRAFT, a challenging multiagent coordination game, we show that ECHO substantially improves resolution, information gain, and efficiency over trajectory-level GRPO, and matches or exceeds frontier baselines on epistemic metrics such as grounding, recovery, and calibration.
♻ ★ How Far Can an Agent Push? Trust-Safety Margins in Live Language-Based Economic Games
Game theory identifies when a strategic action should be accepted, but not how long a live, language-based counterpart will continue to accept it. We entered the GLEE competition in its final week with one protected agent, Milton, and four instrument agents that ran deterministic, per-game randomized trials of policy changes; Milton received changes only through pre-specified gates or registered, rollback-guarded production rules. In negotiation, learned pricing improves adjusted percentile by 0.071 (n=21,168); in bargaining, a continuation model conditioned on the rejected offer did not improve on a recency-weighted incumbent and was retired. In persuasion, a preregistered five-dose experiment (7,391 assigned games, blinded until a fixed stop) shows that recommending every unit raises sales by 0.194 of the maximum (95% CI [0.147, 0.241]) where the receiver's slack makes obedience rational (52% of the Bayesian benchmark), lowers them where it does not (-0.067), and produces a larger early-to-late purchase decline than honest recommendations (-0.123). A rule fixed before unblinding now routes Milton's dose per cell. Milton stood 11th of 460 agents in bargaining at the fixed stop; we also report the close, where the final day's pool shift moved all five agents' bargaining ratings together while their play was unchanged, and three dispatch defects were found by checking delivered behavior against the design.
♻ ★ FDE-Bench: Evaluating End-to-End Delivery from Underspecified Real-World Business Requests
Real-world business requests often leave key requirements unstated because frontline practitioners struggle to articulate rules they know from experience. Forward-deployed engineers (FDEs) must elicit these requirements through dialogue and iteratively optimize solutions until they meet customer acceptance criteria. Existing optimization benchmarks typically start from explicit objectives, while clarification benchmarks often introduce information gaps into existing tasks. We introduce FDE-Bench, the first benchmark built from real customer projects to evaluate agents on end-to-end delivery tasks that require interactive requirement clarification and iterative optimization. Its 49 cases span combinatorial optimization and machine learning. Clarification targets are distilled from discussions between frontline practitioners and FDEs. Agents elicit missing requirements from a simulated customer, and case-specific evaluators assess final artifacts against acceptance criteria using customer-accepted FDE solutions as quality references. Across 18 agent configurations, the highest mean delivery score is only 0.5510, compared with an FDE reference of 1.0. The highest delivery success rate (DSR), defined as the fraction of planned runs reaching this reference, is only 6.12%. These results show that reliable FDE-level end-to-end delivery remains out of reach for the evaluated agents on underspecified real-world business tasks. The benchmark project page is available at https://baidubce.github.io/FM-Agent/FDE-Bench/.
♻ ★ Non-Stationarity Breaks Permutation Surrogates in Multi-Agent Reinforcement Learning: Diagnosis and Remedies
Reporting guidance for information-theoretic measures is rarely tested against ground truth. We test one guardrail in two multi-agent reinforcement learning games, a social dilemma and a coordination race, where directed influence between selected agent pairs is zero by construction, over 100 seeds. Omitting one precondition, exclusion of the non-stationary training transient, gives false-positive rates of 100.00% and 99.95%: agents annealing exploration independently, in runs that never met, are flagged as influencing one another. Excluding the transient reaches 3.05% in the social dilemma but 11.8% in the coordination game, so the non-stationarity must be treated and exclusion is neither the only way nor sufficient. What we recommend instead changes the null model. Series, statistic and estimand are left untouched: permuting the source within blocks of training time reaches 5.25% and 5.50%, the only one of the three at or below the size of the test in both games. The construction is not new, being a restricted permutation local to blocks of training time; the measurement is. A third null control, four learners in the social dilemma, defeats all three, and there conditioning costs three times what ours does, 15.20 points against 4.55. Titrating injected links separates it from conditioning on identical pairs, 82.5% against 58.0%, while the ablated test reports a positive with or without a link, so its apparent sensitivity is uninformative. Five block counts spanning 16 to 256 land in the nominal region. Code and data are released.
comment: 33 pages, 4 figures. Substantially revised: rebuilt around a controlled experiment with ground-truth null and positive controls. Code and data at https://doi.org/10.5281/zenodo.22658530 and https://github.com/dentros/te-nonstationarity
♻ ★ Model Capacity Frontier: A Development Framework for LLM-Supported Social Simulation
Large language model (LLM) agents are increasingly incorporated into agent-based models (ABMs) on the premise that richer, context-sensitive human behavior can improve empirical fidelity. However, LLM-supported ABMs introduce an open-ended design space in which subtle modeling and configuration choices, such as prompt design and environment coupling, can substantially alter outcomes. Developing LLM-supported ABMs to simulate human collective behavior therefore requires navigating configuration sensitivity while identifying the limitations of the current model architecture. In this paper, we introduce the Model Capacity Frontier (MCF), a framework that characterizes the empirical performance attainable by a specified architecture across its admissible configurations. In multi-objective settings, this capacity is represented by an estimated Pareto frontier obtained through systematic configuration exploration. We demonstrate MCF in crowd simulation of university commencement events with approximately 12,000 agents, using empirical movement data on temporal occupancy, spatial distribution, and flow direction. We first characterize the capacity of a conventional crowd model and then evaluate successive LLM-supported architectural extensions. In the process, MCF reveals which dimensions of attainable performance expand and which remain limited, providing empirical targets for refinement and re-evaluation. Initial LLM integration improves flow direction but degrades temporal occupancy and spatial distribution; subsequent architectural refinement of the LLM layer improves temporal occupancy and recovers spatial performance, while retaining much of the earlier gain in flow direction. Our case study shows how MCF guides the systematic architectural development of LLM-supported ABMs in configuration-sensitive, open-ended design spaces.
♻ ★ ASPIRE: Agentic /Skills Discovery for Robotics SP
Traditional robot programming is notoriously challenging: it requires orchestrating multimodal perception, managing physical contact dynamics, and handling diverse environment configurations and execution failures. We introduce ASPIRE (Agentic Skill Programming through Iterative Robot Exploration), a continual learning system that autonomously writes and refines robot control programs in a code-as-policy paradigm while compounding experience into a reusable skill library. ASPIRE operates in an open-ended learning loop, consisting of three components: (1) a closed-loop robot execution engine that exposes fine-grained multimodal traces, enabling the agent to autonomously diagnose failures, synthesize repairs, and validate outcomes; (2) a continually expanding skill library that distills validated fixes into reusable, transferable robotic knowledge; and (3) an evolutionary search procedure that generates diverse programs, systematically debugging them to explore beyond single-trajectory refinement. ASPIRE surpasses prior methods by up to 77% on manipulation tasks under perturbation, 72% on Robosuite's bimanual handover task, and up to 32% on long-horizon household tasks. The accumulated skill library further enables zero-shot generalization: on representative unseen long-horizon tasks, ASPIRE achieves 31% success, outperforming the 4% success rate of prior methods. Performance scales with the capabilities of frontier models. With Claude Opus 5.5, ASPIRE nearly saturates these benchmarks, achieving 99% average success across all 80 LIBERO-Pro tasks and 100% success on representative Robosuite and BEHAVIOR-1K tasks. Beyond simulation, skills discovered in simulation reduce real-world robot programming effort across embodiments and robot APIs, while skills acquired through online interaction with physical robots enable ASPIRE to solve a Rubik's Cube with progressively improving efficiency.
comment: 41 pages, 19 figures, 26 tables. Project page: https://research.nvidia.com/labs/gear/aspire/ Code: https://github.com/NVlabs/ASPIRE
Systems and Control (EESS)
★ Adaptive Multi-Token Prediction: How far should we go?
Multi-token prediction (MTP) can accelerate language-model decoding by drafting several tokens and verifying them in a single target-model pass. However, the best speculation horizon changes with context: longer drafts offer greater amortization when reliable, but can waste compute on suffixes discarded after a rejection. Unlike autoregressive drafting, where a stopping rule can decide token by token, MTP must fix the horizon before drafting, as all heads fire in the same forward pass. We therefore formulate horizon selection as an online control problem, using verification feedback to learn how far the draft can be trusted in each context, and introduce Adaptive Multi-Token Prediction (AMTP): a lightweight controller that requires no retraining and leaves the exact verifier unchanged. To guide horizon selection, we characterize the expected number of tokens committed at each horizon via acceptance survival, the probability that a draft prefix is fully accepted. Before each round, AMTP predicts how far the draft is likely to survive and picks the horizon with the highest expected tokens per second, learning freely from every verification pass. Theoretically, we show that any online learner's prediction error translates into throughput regret against the best horizon for each context, and in turn into a bound on wall-clock completion time. With online gradient descent, this regret is sublinear. Across five draft-target pairings built on Llama-3.1-8B-Instruct with Medusa heads, Meta's MTP models, and Qwen3-4B-Inst-2507-MTP, evaluated on AIME 2025 and GSM8K, AMTP decodes up to 4.1x faster than standard decoding and matches or outperforms the fixed window in 8 of 10 settings, with throughput gains of up to 10%.
comment: 25 pages, 5 figures, 9 tables
★ A Comparative Study of LED Lighting Control Strategies for Energy-Efficient Greenhouse Operations
Greenhouse supplemental lighting is essential for meeting crop Daily Light Integral (DLI) requirement. However, conventional always-on and rule-based strategies are inherently energy-inefficient since they ignore future sunlight availability. This paper reports the results of deploying various supplemental lighting control strategies, namely, always-on, always-off, heuristic, optimal prediction-based, and two peak-shift variants, simultaneously in a single greenhouse at the Hampton Roads AREC of Virginia Tech, to ensure that every treatment experiences identical conditions. %The shared infrastructure is built on an MQTT-based IoT platform with Photosynthetic Photon Flux Density (PPFD) sensing and PWM-driven LED control; The prediction-based and peak-shift strategies optimize LED schedules using a Markov chain-based solar forecast. Experimental results over two replicates %(20~February--15~April~2026, 72~plants per treatment) show that the prediction-based strategy meets the desired DLI target at 65\% cost reduction relative to the always-on baseline while achieving the highest dry-mass yield among the proposed adaptive strategies. The heuristic strategy incurs only a 6\% cost premium over prediction-based control, offering a competitive forecast-free alternative. The peak-shift variants achieve the lowest active-strategy costs by shifting LED demand into off-peak tariff windows. A yield-per-dollar efficiency metric is finally introduced to jointly capture energy cost and crop output showing that the prediction-based strategy ranks highest at 18.976~g/\$, more than double the always-on value of 9.391~g/\$. These findings can provide greenhouse operators with an evidence-based framework for selecting a lighting control approach that balances energy cost, crop quality, and system complexity.
comment: Accepted at Modeling, Estimation and Control Conference (MECC 2026)
★ Proof-Carrying Contracts: Commitments AI Agents Cannot Break
AI agents increasingly act on behalf of people or other agents, purchasing services, exchanging information and delegating work. What an agent must respect is written in instructions, policies or agreements, but text alone cannot make it comply. It may make mistakes, be redirected by what it reads, or pursue different interests. For an economy of such agents to function, parties need agreements they can rely on without inspecting or trusting one another's agents. We introduce Proof-Carrying Contracts (PC2), a framework for making agreed constraints enforceable throughout an agent's interactions with a system. Before a requested operation can run, the agent must supply a machine-checkable proof that it preserves the agreed constraints in every case allowed by its model. An execution system called the Airlock checks the proof and controls access to the operation. The parties specify the conditions to preserve, while the agent supplies the reasoning needed to justify its decisions and remains free to adapt its strategy. This avoids having to design a separate permission policy and prove that every decision it permits is safe. We prove that the constraints hold at every instant during execution, even for an adversarial agent, and identify when this implies safety in the world. Reusable capability modules provide tools and guarantees from which contracts can be constructed. PC2 provides a foundation for an agent economy whose agreements protect both those delegating work and those carrying it out.
comment: 70 pages, 8 Figures,
★ Attracting Rotations Obstruct Almost-Global Consensus on Spheres over Strongly Connected Graphs
We study discrete-time consensus on the Euclidean unit sphere, where weighted combinations of neighboring states are normalized to unit length. For symmetric weights on connected undirected graphs and ambient dimension $d\ge3$, earlier results establish instability of non-consensus fixed points under suitable weight conditions. For directed graphs on the unit circle, earlier results show that fixed points other than consensus and antipodal configurations occur only for a measure-zero set of weights. We show that these conclusions do not extend directly to general strongly connected directed graphs in higher dimensions. First, we construct full-rank non-consensus fixed points whose tangent eigenvalues all equal one, showing that strict linear instability need not hold. In a separate construction, for every $d\ge3$, we present a sparse strongly connected graph admitting full-rank fixed points on an open set of its permitted weights. Our main result establishes robust counterexamples to almost-global consensus: attracting rotating motions exist for $d=2$ with every $n\ge3$ and for $d=3$ with every $n\ge6$. These motions are locally asymptotically stable modulo common rotations and have asymptotic phase. They persist on open sets of strictly positive weight matrices. For each such matrix, an open set of initial configurations approaches neither consensus nor any fixed configuration. Thus the absence of full-rank non-consensus fixed points does not ensure almost-global convergence to consensus.
comment: 21 pages including appendices, 2 figures
★ Latency-Neutral Passive Cyber Monitoring for Grid-Edge Operational Technology Networks
Deterministic industrial networks often present an operational trade-off between maintaining microsecond timing of primary control loops and capturing security telemetry. This work investigates the engineering feasibility of the Operational Technology Supervisor (OTSV), a three-tier passive monitoring framework designed to resolve this visibility challenge within timing-sensitive grid-edge infrastructure. By utilizing a Layer-3-unconfigured passive ingestion mode on bare-metal hardware via a physical switched port analyzer (SPAN) loop, the deep packet inspection engine is isolated from the production network, implementing a software-enforced unidirectional export policy. Performance validation conducted over a continuous audit comprising 32,927 active-monitoring round-trip time (RTT) samples yielded stable latency distributions across a laboratory power-monitoring loop running Modbus TCP and IEC 61850 MMS protocols. The measured latency delta between the unmonitored baseline and the active monitoring state was less than 0.12 milliseconds, supporting the feasibility of continuous high-fidelity security observation alongside stringent grid-edge metering timing bounds without measurably degrading the monitored reporting path.
comment: 5 pages, 4 figures, 2 tables. Accepted for presentation at the IEEE PES Grid Edge Technologies Conference and Exposition 2027
★ Distributed Platoon Control for Safe and Smooth Crossing at an Unsignalized Intersection-Part I: Stability Analysis
We propose a distributed platoon controller that guarantees safe and smooth crossing of multi-vehicle systems at an unsignalized intersection. To support the stability analysis, we consider a multi-vehicle platoon maneuvering along a closed path with a single unsignalized intersection. Unlike traditional bi-level decoupled designs on open paths, which primarily focus on intersection control while neglecting its coupling with platoon convergence, and thus hindering rigorous convergence analysis, the closed-path setting allows us to analyze the asymptotic behavior of the platoon. The local exponential convergence analysis of the switched closed-loop system, induced by repeated vehicle switching at the intersection, is presented. Under some mild conditions on the controller gains, we show that safe intersection crossing is ensured during the transient phase and smooth crossing is achieved in steady state, where all vehicles travel at a common constant speed while avoiding collisions at the intersection. Finally, the effectiveness of the proposed distributed controller is demonstrated in a high-fidelity CARLA simulator using multiple commercial vehicle models.
★ Conformal Koopman Certificates for Nonlinear Multi-Agent Consensus on Time-Evolving Graphs
Robot fleets must often agree on a shared value before they act, over wireless networks that change as the robots move. Their supervisor needs a deadline: the number of communication rounds after which the fleet has agreed with a stated probability. Today it comes from simulation plus a safety margin, with an open probability of being met. We certify it. Our method, conformal Koopman certification, rests on one observation: averaged over the random network changes, the expected behaviour of the fleet evolves linearly, for any nonlinearity of the agents. The decay rate of this linear evolution gives the shape of the deadline. One remaining constant is calibrated from 200 short test runs by conformal prediction, a statistical method that turns any predictor into a guarantee that holds with a user-chosen probability. The certified deadline holds for every agreement tolerance at once and for any error in the estimated rate. The same linear model predicts the value the fleet will agree on and certifies how much corruption of the exchanged messages the collective decision absorbs. For 60 simulated robots the certified deadline was met in all 500 test runs against a 90\% target and exceeded the slowest run by 16\%. The classical worst-case bound exceeds that run by more than 160 orders of magnitude, a linearised deadline is missed in up to 26\% of runs, and direct conformal calibration costs 80 times more simulation per tolerance.
★ Norm-optimal model order reduction in reproducing kernel Hilbert spaces
We consider the problem of model order reduction (MOR) in reproducing kernel Hilbert spaces (RKHSs). We prove generalizations of well-known necessary interpolatory conditions for norm-optimal reduced-order models, originally stated for rational functions in $H_2$ Hardy-Hilbert spaces. We construct an alternative to the iterative rational Krylov (IRKA) algorithm for RKHSs with differentiable kernels to obtain norm-optimal reduced-order models. Then, we introduce generalized Malmquist-Takenaka (GMT) functions that can be used to define orthonormal bases in an arbitrary RKHS. Using GMT functions, we propose unitary transformations between separable RKHSs. This allows us to extend our optimality conditions and subsequent algorithms to MOR problems posed in RKHSs with non-differentiable kernels. The potential and versatility of the proposed MOR framework is demonstrated in several numerical experiments. We perform norm-optimal MOR on various machine learning models such as support vector machines and deep ReLU networks. In addition, we show how the proposed MOR framework can be used to solve seemingly unrelated problems. In particular, we provide an example for fitting high degree polynomials to data and also obtain a FEM-like method for solving partial differential equations (PDEs) with an optimal kernel basis.
comment: 92 pages, 7 figures
★ Controllability Allocation Scores for Linear Infinite-Dimensional Systems
We define finite-time controllability allocation scores (CASs) as optimal solutions to two convex optimization problems that allocate a fixed actuation budget among prescribed input functions of linear infinite-dimensional continuum systems with selected target outputs. The objectives are expressed using controllability Gramians for the selected outputs and correspond to maximizing the volume of output changes reachable with unit control energy or minimizing the mean energy required for unit output changes. To compute these CASs, we discretize the system in space, construct the resulting Gramians, and solve the resulting allocation problems by a domain-aware projected gradient descent method. We study the behavior of CASs when the actuator-to-target responses of the spatially discretized systems converge to those of the continuum system as the mesh is refined. If the continuum system is target controllable, we show that the spatially discretized systems are also target controllable on sufficiently fine meshes and that the optimal values of the approximate problems converge to the optimal value for the continuum system. If the allocation problem for the continuum system has a unique CAS, we show that the CASs obtained from the spatially discretized problems converge to it. We also derive quantitative bounds on the errors between these CASs when the Gramian map for the continuum system distinguishes allocations. A two-dimensional advection-diffusion example illustrates how transport dynamics affect target controllability and CASs. Spatial refinement reveals apparent controllability caused by numerical diffusion and supports the theoretical convergence results.
comment: 16 pages, 4 figures
★ From Market Signals to Maintenance Decisions: Grid-Informed Maintenance Optimization in Offshore Wind Farms
The reliable and cost-effective integration of large-scale offshore wind farms into power systems is largely influenced by maintenance operations. To date, power producers primarily rely on farm-level information (e.g., health signals, power generation, weather conditions) for maintenance planning of wind turbine fleets. We propose to consider additional grid-level information in maintenance operations, such as generation curtailment and market pricing. Periods of high generation curtailment or negative electricity pricing can be viewed as opportunities for preventive maintenance, since assets are fully or partially shut down due to grid-imposed constraints. We first analytically establish that access to market information has non-negative economic value for maintenance scheduling under a stylized setting, providing a theoretical foundation for treating market and grid information as a maintenance opportunity. We then formulate a grid-informed opportunistic maintenance scheduling model as a receding-horizon mixed integer linear program that jointly leverages both farm- and grid-level signals under realistic conditions to determine a maintenance plan for offshore wind turbines. Considering a 600-MW offshore wind farm in the United States, our proposed method reduces annual maintenance costs by up to 10%, while enhancing asset availability and resource utilization, relative to advanced maintenance models that solely rely on farm-level information.
★ Control of Isolated Four-Quadrant Single-Stage Three-/Single-Phase X-Rectifier for EV On-Board Chargers
The single-stage isolated eXtended-functionality rectifier (X-Rectifier) electric vehicle (EV) on-board charger (OBC) employs 650V GaN monolithic bidirectional power transistors (M-BDS) in the input stage, provides sinusoidal input currents and four-quadrant operation in three-phase and split/single phase grids without power derating, operates from asymmetrical three-phase mains, provides reactive power if needed, and supports extremely asymmetric phase loading in standalone vehicle-to-load (V2L) mode. This paper presents a new improved phase-modular control method for the X-Rectifier that enables tracking of arbitrary and independent phase current references regardless of the instantaneous grid voltage (i.e., enables four-quadrant operation). The method supports grid-tied (i.e., grid-to-vehicle, G2V, or vehicle-to grid, V2G) and grid-forming (i.e., V2L) operation. Further, aiming towards a practical realization of the X-Rectifier, the paper addresses implementation aspects of the M-BDS half-bridges, like commutation logic around the grid-voltage zero crossing, fault handling, and soft-switching behaviour. Finally, a realized 5.5 kVA hardware demonstrator of the X-Rectifier is used to validate the proposed control method experimentally for all operating modes, including grid-tied three- and single-phase four-quadrant operation (G2V, V2G, and full reactive power modes). Further, robust V2L operation is verified with real-world loads, i.e., in three-phase mode with asymmetrical loading of the phases including a woodworking power tool, and in single-phase mode with a high-power vacuum cleaner with high inrush current and hence momentary overload of the X-Rectifier demonstrator.
comment: Submitted to an IEEE journal
★ Temporal Difference Learning of Probabilistic Reach-Avoid Value Functions ICRA 2027
Reach-avoid sets, defining the states from which a task can be completed while avoiding unsafe regions, are fundamental tools for deploying robots in environments subject to disturbances and uncertainty, as well as for runtime task-feasibility monitoring, allowing prediction of whether a state is safe and will lead to successful task completion. Classical Hamilton-Jacobi Reachability (HJR) and its recent neural extensions (e.g., Lipschitz Continuous Reachability Learning, DeepReach) treat disturbances adversarially, computing the reach-avoid set under worst-case disturbances from a bounded set. This guarantees safety but can yield overly conservative sets when disturbances are stochastic rather than adversarial, and additionally requires the disturbance set to be known or specified in advance. We propose Probabilistic reach-avoid Value Function (PVF), a self-supervised method, applicable to black-box systems, for computing reach-avoid sets as the states with a certain reach-avoid probability under stochastic, potentially unbounded disturbances. We evaluate our approach against HJR-based baselines on systems of increasing complexity, up to a Hopper locomotion task, and show that our probabilistic sets are less conservative and more accurate than adversarial HJR sets when disturbances are non-adversarial.
comment: Submitted to ICRA 2027
★ Observer-Based Control for Multi-Agent Systems under Communication-Task Graph Mismatch and Directed Switching Topologies
This paper addresses decentralized estimation and control of heterogeneous multi-agent systems communicating over directed switching topologies. The objective is to satisfy cooperative tasks that require coordination among agents according to a time-invariant task graph. To address the potential mismatch between the communication and task graphs, each agent is equipped with a k-hop Prescribed Performance State Observer (k-hop PPSO) to estimate the states of agents up to k communication hops away using only information exchanged with its 1-hop neighbors. A novel prescribed performance funnel construction is developed to account for the switching communication topology, and a sufficient lower bound on the dwell-time is derived to guarantee the prescribed performance specifications under topology variations. Further analysis shows that, if the desired control objective is achieved with exact knowledge of the required nonlocal states and the resulting closed-loop system is uniformly set-ISS with respect to the state estimation error, then replacing these states with their estimates guarantees convergence to a neighborhood of the desired set. The size of this neighborhood can be made arbitrarily small by appropriately selecting the observer performance bounds. A simulation study demonstrates task satisfaction under directed switching topologies.
★ Speculative Imbalance Positions by Non-Physical Traders: Impacts on Electricity System Balancing in Great Britain
The electricity system in Great Britain (GB) allows participants to deliberately produce or consume a different volume to their traded position and has incentivised this practice with a single imbalance price since 2015. In particular, non-physical traders (NPTs) often take an open position to imbalance without any intention to produce or consume power. However, new government proposals could restrict this practice due to concerns over supply-demand balancing feasibility and costs. This article presents the first empirical analysis of speculative imbalance positioning (SIP) by NPTs in the GB electricity system through its effects on imbalance volumes, balancing costs, the imbalance price, and associated settlement costs over the period 2021-2024. Our research uses party-level imbalance data in each settlement period to run counterfactual simulations of the electricity balancing mechanism without the aggregate NPT imbalance. Our results indicate that SIP by NPTs reduced the total net balancing volume required by 3.1-3.9 TWh (16-20%), reduced system imbalance standard deviation by 48-65 MWh (13-17%), reduced balancing costs by £347-396m (5.9-6.7%), and reduced supplier imbalance costs by £193-325m (65-75%) over the analysis period. These results suggest that consumer bills would have been higher without SIP by NPTs, since the extra costs would likely have been passed on to consumers through levies, although the potential for effects in the wholesale (primarily late-stage intraday) markets complicates this picture.
★ Hierarchical Bayesian modeling of agrivoltaic dairy-calves under missing information
Livestock 5.0 provides a framework to optimize animal welfare and productivity amidst escalating climate change and current pressures to sustainably increase production through limited expansion of agricultural lands, while equilibrating the food-water-energy nexus. Directly tackling the food-water-energy nexus, agrivoltaics emerges as a promising solution for dual-land use for food and renewable energy production. In this paper, we evaluated the effectiveness of agrivoltaics for dairy-calves in a commercial farm through a joint hierarchical Bayesian model, developed to account for non-linear and time-varying relationships, pedigree structure, and missing information. Uncertainty from missing information was propagated to the posterior estimates. While weight at birth and calf sex were not associated with serum Brix, we found that serum Brix was positively associated with colostrum Brix, which allowed to estimate the probabilities of the serum Brix quality (poor, fair, good, or excellent) given colostrum Brix. Poor serum Brix quality was positively associated with disease event rate, with similar rates for the other quality classes. Serum Brix quality was not associated with calf weight, indicating that farm management overcame the negative effects of low serum Brix quality. Previous number of diseases was positively associated with disease event rate, while weight at birth was negatively associated with disease event rate. Agrivoltaics were not associated with disease event rate, indicating that agrivoltaics can be implemented as a safe, dual-use system for increased calf comfort an welfare.
★ Execution-Time Validation of Context-Carrying Action Intents in Industrial Cyber-Physical Systems
In industrial cells, commands from learned policies, operators and schedulers reach the controller after inference, confirmation and transport, and the plant may have changed meanwhile, for instance by an operator pause. Such a command is released, or takes effect, when it is no longer correct. We call this the execution gap and ask which component should make the last check, and against which version of which state. The Action Intent Schema (AIS) is a contract in which a producer states how long its decision stays valid and what it assumes. The runtime XAIR rechecks each intent before release, can commit its authorization atomically in the context store, and can release it as a version-conditional command, which the controller executes only if its version of the state is unchanged. In controlled ablations with declared ground truth on emulated and physical cells, OpenPLC and ARM and RISC-V boards, a check on a replica always left an unprotected interval: a pause written from a few to over 100ms after submission escaped it. Only a check in the scan of a software controller, either a controller-local interlock or the version-conditional command, closed the interval for the state that controller owns; only the version-conditional command also refused commands after a pause that had been undone. A global version revoked up to every valid command under unrelated updates. Versions scoped to what the intent reads, or to the truth of its predicates, did not; with continuous signals on the variables the intent reads, only the latter avoided false revocations, though it burdens the store as predicates accumulate. On one embedded-board core, XAIR validated 148-168 intents/s.
comment: 46 pages, 6 figures, 11 tables
★ Neural Closed-Loop Identification of Nonlinear Systems with Stability and Generalization Guarantees
Neural networks are expressive models of nonlinear dynamics, but identifying such systems from closed-loop data raises two difficulties: the feedback loop correlates inputs with measurement noise, biasing standard open-loop estimators, and the identified model must be stabilized by the data-collection controller, so that its closed-loop behavior matches the true plant's. We derive a complete parameterization of all nonlinear systems stabilized by a given controller, generalizing the classical dual-Youla parameterization to the nonlinear setting. Building on this characterization, we propose Internal Controller Identification (ICI), a computationally tractable framework for identifying potentially unstable systems while ensuring closed-loop stability by construction; in the linear case, our indirect method reduces to an open-loop identification problem, for which standard consistency results apply. The framework accommodates contextual information, letting a single certified model identify a family of systems, and supports a PAC-Bayesian training objective that, when data are scarce and prior knowledge is available, bounds the identified operator's expected prediction loss. Experiments show that ICI identifies open-loop unstable plants where uncertified baselines diverge, outperforms direct methods under weak excitation and feedback-correlated noise, generalizes zero-shot to environments unseen during training, and, with limited data, avoids the overfitting of empirical risk minimization while providing a non-vacuous generalization bound.
comment: 16 pages, 11 figures. Preprint submitted to Automatica
★ Explicit Convex Inner Approximation of TSO-DSO Flexibility Regions via Exact Second-order Cone Relaxation in Radial Networks
Accurate representations of the flexibility in distribution grids via tractable algorithms are essential for TSO-DSO coordination. However, flexibility region (FR) estimation based on nonlinear optimal power flow (OPF) generally lacks feasibility guarantees, while conventional convex relaxations may suffer from inexactness. This paper proposes a framework for constructing explicit inner approximations of active-reactive power FRs in radial networks based on the augmented relaxed OPF (AR-OPF). First, less-conservative sufficient conditions for the AR-OPF exactness are derived and extended to generalized objectives. Furthermore, an interior-feasibility analysis is established, guaranteeing that the entire region enclosed by the traced boundary is AC feasible under the proposed sufficient conditions. Building on these theoretical results, two adaptive algorithms are developed to construct explicit inner approximations and numerically convex representations of FRs. Numerical evaluations on different benchmark networks demonstrate consistently small relaxation residuals and high flexibility area retention in most cases, showing that the proposed methods provide a grounded approach to construct operationally admissible FRs for TSO-DSO coordination.
comment: 10 pages, 4 figures
★ Make the Adversary Spend More Energy: Generative Deception against Language-Model Attackers in Critical Infrastructure
Language-model attackers consume computational resources as they explore and interact with a target, creating an opportunity for cyber deception to impose costs. Whether such engagements favor the defender requires measuring both sides. We evaluate an autonomous language-model attacker against a decoy that uses a defending language model to analyze attacker commands and select defensive responses. Across 330 sessions, eight model pairs, two attacker profiles and three counter-injection levels, both sides run on identical GPUs and their energy is measured above idle. With runtime thinking disabled, the attacker-to-defender energy ratio has a 95% interval above one in 41 of 48 configurations under an allocation excluding attacker diagnostic calls, and in all 48 when those calls are included. Token ratios remain within approximately a factor of two in configurations without generation-cap truncation, showing that token counts can obscure substantial energy asymmetry. Counter-injection increases attacker tokens by approximately 9% per level without a detectable pooled change in energy consumption. Enabling thinking on both sides increases defender energy by 8--17 times and reverses the exchange for one model pair in the original campaign. We use these findings to motivate a proposed SCADA deception architecture with private physical state per identified adversary and deterministic protocol and process emulation.
★ Anytime-Valid Runtime Monitoring of Learned Barrier Certificates AAAI 2027
Learned barrier certificates provide a principled mechanism for establishing safety of stochastic systems, but their validity may deteriorate during deployment when the underlying dynamics differ from those used for training. We propose an anytime-valid runtime monitor that audits a learned barrier certificate using observed closed-loop trajectories. The monitor is intended to detect loss of statistical support for the deployed barrier condition; it does not by itself guarantee detection before unsafe entry or safety of a subsequent intervention. Specifically, the conditional barrier-drift condition is formulated as an episode-wise sequential hypothesis test, and observed barrier increments are transformed into a nonnegative test supermartingale, yielding finite-sample false-alarm control under arbitrary stopping times. Inspired by finite-horizon safety verification, we further introduce a deterministic temporal shaping of the monitoring boundary, making the monitor more conservative early in a prescribed deployment horizon while preserving anytime-validity. The monitor is designed to detect statistical evidence that the safety certificate is no longer consistent with the observed dynamics, rather than to predict unsafe entry. To assess the monitor empirically, we evaluate empirical alarm behavior under valid and invalid certificates. Code and data are available at https://github.com/FormalCert-AI/runtime-barrier-monitor.
comment: 17 pages, 7 figures. An earlier version of this work was submitted to AAAI 2027 on 21 July 2026 (submission no. 28239). The core monitoring framework and main theoretical results were already present in that submission
★ Adaptive Traversal Tubes for Posture-Aware Navigation of Bipedal Humanoids
Bipedal humanoids can change their effective footprint by adjusting body yaw, enabling both narrow-passage traversal and socially aware yielding. This paper presents a posture-aware navigation framework centered on \emph{Adaptive Traversal Tubes} (ATT), \emph{Interval Velocity--Heading} (IVH) control, and a sidestep-enhanced AMP locomotion prior. ATT represents a behavior-feasible corridor width field along a reference path, unifying geometric clearance and social comfort into a continuous signal. IVH maps this width to a continuous frontal-to-sidestep posture schedule and omnidirectional velocity commands. A velocity-conditioned AMP policy, trained with GVHMR-retargeted frontal/sidestep motions and coupled \((v_x,v_y)\) tracking rewards, is designed to improve transition stability over a generic locomotion controller while remaining interchangeable with the default onboard policy. This preprint presents the full method; a planned evaluation protocol on a Unitree G1 is outlined, with quantitative results to follow. Demo videos of MuJoCo simulation and real-robot deployment are available at: github.com/lborogzj997/Humanoid-Navigation-in-Crowds-and-Narrow-Spaces.
★ Data-Driven Minimum Energy Control for Stochastic Linear Systems with Finite Sample Guarantees
The open-loop minimum-energy control (MEC) problem asks for control inputs that minimize the control energy while ensuring the terminal state reaches a desired value. In this paper, we study the open-loop MEC problem for stochastic linear time-invariant systems. We consider a variety of stochastic constraints that enforce the terminal error to be \emph{small}, and show that Stochastic MEC (SMEC) problem can be formulated as a QCQP. We obtain the optimal solution for SMEC and show that it generalizes the well-known Deterministic MEC problem by introducing novel stochastic variants of controllability and Gramian matrices and show that there exists a fundamental lower bound on the optimal control energy which cannot be overcome by increasing the time horizon of the problem. We then extend the formulation to systems with unknown dynamics using maximum-likelihood estimation and conformal calibration, and compare these approaches with direct sample-based estimation that do not incorporate structural information of SMEC problem into account. Further, with the help of bias-variance analysis, we show when incorporating structural information proves to be better than direct sample based approach. Finally, we establish finite-sample high-probability upper bounds on optimal-cost sub-optimality and original constraint-violation probability under the data-driven controller, quantifying their dependence on the number of experiments, number of samples per experiment, system dimension, input sequence dimension, and noise-to-signal ratio. Using simulations, we validate the theoretical results numerically.
★ Sensitivity-Orthogonal Residual Learning for Adaptive Sideslip and Cornering Stiffness Estimation
Online adaptation of the cornering stiffness keeps a model-based sideslip estimator valid as tires change, but competes with learned model correction for the same filter innovation. A corrector with enough parameters can absorb the innovation pattern that a stiffness change produces, and adaptation is suppressed while the fit improves. We propagate the stiffness sensitivities of the predicted measurements through the vehicle states and construct a corrector orthogonal to them. A null-space parameterization enforces the constraint without a penalty coefficient and preserves the static least-squares stiffness estimate on each identification segment. In full-vehicle simulation, the stiffness response of the unconstrained corrector declines as its parameter count grows, whereas the orthogonal corrector responds at least as fully as the uncorrected filter at all sizes and lowers sideslip error by 12.7% to 22.0%. In a simulated drive with a stiffness step, the orthogonal corrector adapts as quickly as the uncorrected filter and lowers sideslip error during and after the transient. On public real-vehicle data, the 430-parameter unconstrained corrector loses most of its response to deliberate offsets of the filter's nominal stiffness, whereas the orthogonal corrector of the same size retains most of it, with a sideslip error slightly above the uncorrected filter's.
comment: 25 pages, 15 figures, 5 tables. Submitted to Vehicle System Dynamics
★ Multiplexed Label-Free Cytokine Sensing Using a 128x128 CMOS ISFET Array with Flexdym Microfluidic Partitioning
Cytokine-mediated inflammation disorders such as sepsis and cytokine release syndrome demand bedside multiplex panels that resolve pro- and anti-inflammatory dynamics in minutes, yet label-free FET cytokine sensors remain limited to few devices, single analytes, and PDMS packaging that sorbs small molecules and leaches oligomers. We present a CMOS platform pairing a 128 x 128 ion-sensitive field-effect transistor (ISFET) array (SkyWater 130 nm, SiN gate) with a FlexDym 4 x 1 micro-well overlay that partitions the membrane into four chemically independent regions (approximately 2,176 pixels each, square-root-of-N averaging). Two wells carry capture antibodies (anti-IL-6, anti-IL-10), one is a non-specific reference, and one tests specificity; the transduced signal is defined as the differential current relative to the reference signal (Delta I_D(W_n) - Delta I_D(W_3)). Single-analyte dose-response measurements spanning 100 fM to 10 nM successfully resolve both IL-6 and IL-10 above the non-specific detection floor, including at healthy-serum baseline concentrations. The array shows a per-pixel pH sensitivity of 1.13 microamperes per pH unit (CV approximately 14%). Across two chips, device-to-device variation is dominated by transconductance spread (g_m CV approximately 62%); a one-time per-device g_m calibration reduces analyte-measurement variance to approximately 16%. We further validate the surface functionalization, characterize device-to-device variation, and demonstrate the FlexDym integration workflow. Simultaneous multiplexed cytokine measurements resolve IL-6 (W1) and IL-10 (W2) against the chemistry-only reference (W3). At the same time, cross-specificity characterization is assessed by applying IL-6 to an anti-IL-10 well (W4), establishing a scalable label-free ISFET architecture for multiplexed cytokine sensing.
★ Real-Time Disturbance-Aware Koopman Model Predictive Control for Quadrotors in Spatially Varying Winds
Reliable quadrotor flight in disturbed environments requires predictive models that remain accurate as unmodeled aerodynamic effects change during deployment. We introduce a disturbance-aware Koopman Control Family (DA-KCF) framework that preserves an offline-identified nominal model while learning only a disturbance-induced correction online. A reference-dependent KCF is first learned from simulation data and calibrated using limited hardware data. During deployment, the nominal KCF remains fixed while a lightweight residual ensemble learns a low-dimensional disturbance correction from causal state-command-state transitions. The Adaptive DA-KCF model is incorporated into MPC, where disturbance corrections are evaluated along the prediction horizon to optimize position and yaw references sent to the onboard stabilizing controller. Hardware experiments on a nano-quadrotor under uniform wind and a spatially varying wake demonstrate that online adaptation improves multi-step prediction and trajectory tracking as disturbance-induced model mismatch increases, while satisfying real-time control requirements.
★ HyperDrone: Simulation-to-Reality Transfer at Millions of Frames per Second
We present HyperDrone, an extremely fast quadrotor simulator focused on high-performance batch rendering that is more than 20x faster than existing state-of-the-art batch quadrotor simulators. HyperDrone combines highly optimized multirotor dynamics with HyperLux, a new ray-tracing engine that supports Nvidia OptiX, Vulkan, Apple Metal and WebGPU backends that takes full advantage of modern ray-tracing hardware, capable of running both components entirely on the GPU. HyperLux renders RGB, depth, semantic/instance segmentation, surface normals and optical flow observations at configurable levels of fidelity, including reflections, realistic lighting, motion blur, anti-aliasing, and Physically Based Rendering (PBR). Even in the highest-fidelity settings, HyperLux achieves millions of Frames Per Second (FPS) on consumer-grade GPUs while rendering realistic 3D scenes. We demonstrate the merits of HyperDrone by training a vision-based yaw-control policy purely in simulation and transferring it to a real small-scale quadrotor (Crazyflie) with a camera, running completely onboard.
comment: Accepted at Conference on Robot Learning (CoRL 2026)
★ Experimental Characterization of SMD Inductors for Cryogenic Power Conversion at 4 K
With the increasing complexity and scalability demands of cryogenic electronic systems, such as quantum computing platforms, conversion of power within the cryogenic environment has become a primary concern. Magnetic inductors are among the key components affecting converter performance, as their electrical and magnetic properties can change significantly at low temperatures. In this paper, five commercially available Vishay SMD power inductors are experimentally characterized down to 4 K: DC resistance is measured continuously during cooldown, while frequency-dependent AC inductance and resistance are measured up to 2 MHz at both room temperature and 4 K. The results show substantial DC resistance reductions of 25x-58x at 4 K, while the inductance degrades by only about 5-5.5%. Although cryogenic operation slightly increases the AC resistance (due to the intensified skin effect) and raises core losses, converter-level analysis shows an inductor-only efficiency improvement of approximately 5.3-6.9 percentage points at 1 A, and up to 22.8 points at 3 A. The measured frequency-dependent impedance of each inductor is imported into Cadence and a complete single-phase 24 V-to-1 V buck converter with GaN power devices is simulated at 1, 2, and 3 A at 4 K. The full-converter efficiency at 4 K is shown to improve by 3.0-5.2 percentage points at 1 A and by up to 22.5 points at 3 A for the AC-characterized devices. These findings indicate that commercially available off-the-shelf power inductors remain operationally viable under cryogenic conditions and can simultaneously increase inductor efficiency and reduce the overall power loss in cryogenic power conversion systems.
★ Funnel-Precompensated Disturbance Observer for Uncertain Minimum-Phase SISO Linear Systems of Arbitrary Relative Degree
Disturbance observers (DOBs) recover nominal behavior, but fast filters risk peaking. We propose a funnel-precompensated DOB for uncertain minimum-phase single-input single-output linear systems of known relative degree. A stable-factor residual yields a minimum-phase channel. At relative degree two, gain conditions require positive plant high-frequency gains and one positive common design gain; for higher degrees, the common-design result also uses a plant-gain interval and mismatch condition. Global boundedness, practical nominal-output recovery, and open-funnel initialization are established.
comment: 8 pages, 1 figure, 2 tables
★ Spatial-Frequency Control Authority of Grid-Forming Inverters over Voltage Waveforms in Distribution Feeders
This paper develops a spatial-frequency framework to quantify the impact of distributed energy resources with grid-forming inverters (GFMs) on voltage waveforms at different locations along a feeder. An analytical authority function is formulated to characterize how controllability varies with both electrical distance from the point of common coupling (PCC) and frequency. It is shown that the GFM control authority generally decays with both distance and frequency, reflecting the limited ability of inverter actions to influence voltage waveforms at distant points. To better capture controllability near resonance, a relative dominance metric is introduced to compare the influence of inverter-induced and grid-induced voltages at a given location and frequency. These results highlight that waveform controllability is fundamentally a spatial-frequency phenomenon along the feeder. Simulation studies validate the analytical findings by demonstrating both the spatial decay of control effectiveness along the feeder and the frequency-dependent trends in control authority in a realistic electromagnetic transient setting. The proposed framework provides insights into the placement and coordination of GFMs to enable effective waveform-level voltage control across electrical distance along the feeder.
comment: Accepted at the 2027 IEEE PES Grid Edge Technologies Conference & Exposition, Salt Lake City, Utah, April 19-22, 2027. 5 pages
★ Brittle by Design? Multi-Scenario Reinforcement Learning for Robust Mission Coordination in Systems of Systems
As systems engineering (SE) shifts from monolithic systems to Systems of Systems, Mission Engineering (ME) has emerged as a new line of thinking in SE. Mission environments are uncertain and dynamic, and outcomes depend on how mission assets interact with them, rendering static architectures brittle and calling for flexible yet analytically rigorous ME approaches. We propose an intelligent mission coordination methodology integrating high-fidelity digital mission models with Reinforcement Learning (RL) for adaptive task allocation. Using a Digital Engineering (DE) infrastructure comprising a digital mission model and agent-based simulation, we formulate mission tactics management as a Markov Decision Process and train an RL coordinator with Proximal Policy Optimization to map mission states to tactical actions from realized outcomes. An aerial firefighting mission with three scenarios demonstrates the approach. First, all trained coordinators significantly outperform random and heuristic baselines. Second, single-scenario coordinators can be brittle: the coordinator trained on the more forgiving scenario degrades to random tactics performance in more demanding scenarios, whereas the demanding-scenario coordinator retains most of its advantage when transferred. Third, within the same training budget, a coordinator trained on both scenarios concurrently matches the scenario-specific coordinators in their own scenarios and significantly outperforms every other policy. Sensitivity analyses on ignition location, weather, scenario, and training curricula show that the variability represented during training shapes post-deployment performance. This proof of concept shows DE-enabled mission simulations combined with RL offer a reusable methodology for improving ME practice, which can be extended to fleet design and selection from a mission-first perspective.
★ Hybrid PPO-Based Multi-Objective Optimization for UAV-mounted STAR-RIS with Energy Harvesting
This paper investigates a multi-objective optimiza tion framework for UAV-mounted reconfigurable intelligent sur face (STAR-RIS)-assisted Sixth Generation (6G)-enabled Internet of Things (IoT) systems. The considered framework jointly supports communication enhancement and radio-frequency (RF) energy harvesting (EH) through adaptive STAR-RIS transmission and reflection control. A weighted multi-objective optimization problem is formulated to jointly optimize the base station (BS) beamforming vectors, STAR-RIS transmission and reflection coefficients, RF power-splitting factor, and UAV trajectory under quality-of-service (QoS), power, and mobility constraints. Due to the strong coupling among the communications, EH, and UAV mobility variables, the resulting problem is highly non convex and difficult to solve using conventional optimization methods. To address this challenge, a hybrid proximal policy optimization (H-PPO) framework is proposed. The developed hybrid actor-critic architecture combines a categorical policy for discrete UAV movement decisions with Beta-distributed policies for continuous communication-control variables. In addition, a two-stage reward mechanism is introduced to enforce QoS feasibility while balancing system sum rate, energy efficiency, and harvested energy. Simulation results demonstrate that the proposed H-PPO framework achieves faster convergence and more stable training behavior than proximal policy optimization (PPO) and twin delayed deep deterministic policy gradient (TD3) baselines. In particular, H-PPO achieves approximately 66% and 60% faster convergence than PPO and TD3, respectively, while maintaining more stable training behavior. Furthermore, the proposed framework effectively balances communication performance and EH under different QoS requirements, STAR RIS sizes, transmit power levels, and user densities.
★ Robust Procurement Planning for Community Choice Aggregators: Managing Market Price Uncertainty
Community Choice Aggregators (CCAs) play an important role in the energy transition by procuring renewable electricity for local communities while balancing affordability, resource adequacy, and clean-energy requirements. In California, these long-term procurement decisions are coordinated through integrated resource planning (IRP). However, conventional IRP frameworks typically represent wholesale electricity prices deterministically, even though price uncertainty can expose CCAs to substantial financial risk by affecting the cost of market imports and, particularly, the value of export opportunities. This paper develops a robust optimization framework for CCA resource procurement that hedges against uncertainty in wholesale import and export prices. We extend the open-source GenX capacity expansion model with a budgeted robust optimization formulation that protects procurement decisions against adverse price realizations while remaining computationally tractable for large-scale, multistage planning studies. Using data from a California CCA, we show that market price uncertainty can materially change optimal procurement strategies. Rather than aggressively expanding renewable capacity based on expected export revenues, robust portfolios reduce dependence on uncertain market revenues, leading to more conservative renewable investments and less reliance on market exports. We demonstrate that moderate uncertainty budgets provide most of the risk-reduction benefits of robust planning at limited additional cost, whereas higher robustness levels yield diminishing benefits as portfolio conservatism increases. These findings provide guidance for selecting uncertainty budgets in IRP studies and demonstrate how robust optimization can support affordable and resilient procurement decisions as CCAs contribute to the transition toward low-carbon electricity systems.
★ Reinforcement learning for hardware acceleration and system-level optimization on FPGA and integrated-circuit platforms: A survey
Reinforcement learning (RL) enables adaptive and autonomous decision-making in embedded, edge, and heterogeneous computing systems. Field-programmable gate arrays (FPGAs) and integrated circuits (ICs) offer low latency, energy efficiency, and architectural flexibility for RL deployment, while introducing challenges in resource constraints, implementation complexity, and hardware-algorithm co-design. This review surveys more than 100 studies on RL implementations across FPGA and IC platforms, emphasizing architectures, implementation strategies, performance evaluation, and applications. The literature is organized by RL family, including model-based Markov decision processes, classical value-based methods, deep RL, actor-critic, policy-gradient, and multi-agent approaches. We further classify studies by abstraction level, hardware realization strategy, and application domain, examining how RL algorithms map onto reconfigurable and fixed hardware. Unlike surveys restricted to particular algorithms or platforms, this work provides a unified analysis of FPGA and IC implementations, covering acceleration, reconfigurable architectures, energy-aware control, and system-level optimization. Comparative assessments consider resource utilization, latency, throughput, power consumption, and reported performance gains. Finally, we identify recurring architectural patterns, practical limitations, open challenges, and research directions for efficient RL-driven hardware systems. The survey offers a consolidated reference for researchers developing and evaluating RL solutions on specialized hardware.
♻ ★ Estimating Density Functions for Probabilistic Power Flow Using Invertible Neural Networks
Probabilistic power flow (PPF) is essential for quantifying operational uncertainty in modern power systems with high penetrations of renewable generation and flexible loads. Conventional PPF methods primarily rely on Monte Carlo (MC)based power flow (PF) simulations or simplified approximations of voltage probability density functions. Although MC methods provide high accuracy, they incur substantial computational and data-storage costs, whereas simplified approximations, though lightweight, often sacrifice accuracy due to the reliance of certain assumption. In this paper, we propose a novel PPF density approximation framework that leverages the expressive power of neural networks (NNs) and can, in principle, approximate complex voltage distributions directly without restrictive assumptions. The core idea is to learn an explicit mapping between stochastic power injections and system voltages using invertible neural networks (INNs), and then to apply the change-of-variables theorem to obtain the voltage densities. Extensive numerical studies demonstrate that the proposed framework achieves stateof-the-art performance both as an accurate PF solver and as an efficient PPF density estimator
♻ ★ Implicit Predecessor-Based Region of Attraction Estimation and Robust Invariance Analysis for a Two-Wheeled Inverted Pendulum
Estimating the region of attraction (RoA) of nonlinear systems is fundamental for assessing closed-loop stability and ensuring safe operation. While Lyapunov-based approaches provide certified stability guarantees, they often yield conservative inner approximations of the RoA. This paper combines a certified Lyapunov-based positively invariant set with a predecessor-based implicit representation to compute a significantly less conservative inner approximation of the RoA while preserving formal stability guarantees. In addition, the robust positive invariance of the initial certified Lyapunov-based invariant set is analyzed under bounded additive input disturbances, providing formal robustness guarantees. The proposed methodology is demonstrated on a nonlinear two-wheeled inverted pendulum stabilized by a saturated linear quadratic regulator. The resulting RoA approximation is compared with the initial Lyapunov-certified invariant set and validated through Monte Carlo simulations and hardware experiments, showing a substantially enlarged certified operating region that matches the empirical closed-loop behavior. These results demonstrate the practical applicability of combining certified Lyapunov analysis with predecessor-based set propagation for RoA approximation and robustness assessment of nonlinear systems.
comment: Accepted for presentation at ICINCO 2026
♻ ★ Learning Reachability-Based Safety Representations for Real-Time Robot Navigation in Unknown and Dynamic Environments
Safe robot navigation in unknown, dynamic environments requires more than geometric collision checking: a robot still far from an obstacle may be unable to avoid it because of its speed, orientation, or the obstacle's motion. Hamilton--Jacobi (HJ) reachability analysis captures this in a value function whose sign separates states that admit a safe continuation from those where collision is unavoidable, but computing it online for every newly observed environment is too expensive. This dissertation trains neural implicit representations offline on reachability solutions to give optimization-based controllers this information in real time. Since the value function never exceeds the constraint, it equals a directly computed constraint, such as a signed distance, minus a nonnegative dynamics correction. Learning only this correction keeps represented obstacles outside the predicted safe set despite approximation errors. The first two of five methods use the learned function as a terminal constraint in model predictive control, for static local environments and predicted obstacle motion. The next two use it as a barrier candidate in control barrier function safety filters, for static environments and, through pairwise barriers in relative coordinates, for a varying number of moving obstacles. Simulations and hardware experiments on ground and aerial robots show better collision avoidance than geometric and hand-designed baselines, especially for short horizons and moving obstacles. The fifth method replaces the grid-based offline solver, which limits the first four to low-dimensional models. For control-affine dynamics with polytopic inputs, the HJ Hamiltonian is maximized at a vertex, so a parallelizable tree search over vertex inputs can generate training labels. With a loss from the HJ variational inequality, these labels yield barrier candidates for systems with up to 13 states.
♻ ★ General-Sum Markov Games with Borel Spaces: Existence of Approximate Nash Equilibria via Finite Model Approximations under Setwise or Weak Continuity
Establishing the existence of exact or near Markov or stationary perfect Nash equilibria in nonzero-sum Markov games over Borel spaces is a challenging problem with limited positive results. Motivated by problems in multi-agent and Bayesian learning, this paper demonstrates the existence of approximate Markov and stationary Nash equilibria for such games under relatively mild regularity conditions; notably under setwise continuity. Our approach is constructive: For both compact and non-compact state spaces, we approximate the Borel model with finite state-action models and show that their equilibria correspond to \(ε\)-equilibria for the original game. Compared with previous results in the literature, which we comprehensively review, we provide more general conditions, along with explicit approximation models whose equilibria are $ε$-equilibria for the original model. For completeness, we also study the approximation of zero-sum Markov games and Markov teams to highlight the key differences between zero-sum and nonzero-sum settings. In particular, while for zero-sum and team games, joint weak (Feller) continuity of the transition kernel is sufficient (as the value function is continuous), this is not the case for general nonzero-sum games.
♻ ★ NANO: A Unified Optimization Framework for Nonlinear Gaussian Filtering
Conventional Gaussian filtering typically approximates nonlinear models or their statistical moments and retains the Kalman update structure. The resulting approximation errors can affect both state estimates and their uncertainty. This paper presents the Natural Gradient Gaussian Approximation (NANO) framework for nonlinear filtering. Both prediction and measurement update are formulated as optimization problems over Gaussian distributions, with the entire Gaussian distribution serving as the optimization variable. The framework integrates observation losses, distributional discrepancies, natural-gradient optimization, and finite-step update rules, providing a unified formulation for constructing different filters. Natural-gradient iterations account for the geometry of the Gaussian family and jointly update the distribution parameters. The framework incorporates the existing NANO-P method using the Kullback-Leibler divergence and introduces NANO-W using the Wasserstein distance. NANO-P maintains covariance positive definiteness through a positive-semidefinite curvature approximation, while NANO-W uses a covariance transport update that accommodates indefinite curvature. Their update rules and sufficient conditions for covariance positive definiteness are established within the common framework. Through these two realizations, the proposed framework achieves the best estimation accuracy among the evaluated methods on each of six benchmarks. Runtime evaluations further assess its computational feasibility for practical applications.
comment: This manuscript was posted prematurely, before all coauthors had approved its public release. We therefore request its withdrawal
♻ ★ AgentStripes: An Evidence-Graded Maturity Model for AI Agents
Benchmarks report what an Artificial Intelligence (AI) agent can do on a task suite, but not whether one released version has been engineered with the controls that make it reliable, safe and observable, or how strongly that is shown. This paper proposes AgentStripes, an evidence-graded maturity model for AI agents. It defines levels (0 to 7 stripes), process areas (12 areas holding 40 capabilities), three gates (basic, safety, proof) that hold an agent at a level however high its score, and evidence grades E0 to E5. Each applicable capability receives a rating from 0 to 3 and a separate evidence level; these are combined into a credit, aggregated into a severity-weighted score and converted into stripes that the gates limit. Instruments gather evidence, language-model judges interpret it under explicit criteria, and the arithmetic is deterministic. The model is compared with standards at topic level without a claim of conformance, and an unexecuted validation protocol is specified. An empirical evaluation (exploratory, non-confirmatory) of 30 public Python agent repositories, with static evidence only, is a demonstration and not a validation: static evidence caps every final stripe at 2, 26 repositories end at one stripe and 4 at two, and this distinction is unreliable, since re-judging changed the final stripe of 4 of 22 repositories (engine, dossier and panel changed), whereas none of 8 panel-only repeats, all at stripe 1, changed. The parameters are expert conventions without outcome based calibration, and the stripes have not been shown to predict reliability or safety.
comment: 32 pages, 7 figures, 4 tables, 116 references; supplementary information (15 pages, 8 figures, 21 tables) included in the same PDF. Code and data available on request
♻ ★ Anywhere, Any-Stymie: Remote Activation of Trojan Malware on LiDAR with Modulated Signals
LiDAR sensors are widely deployed in autonomous systems for safety-critical 3D perception. We identify a previously unexplored attack surface in which dormant malware, embedded in the LiDAR sensing pipeline, remains inactive during normal operation and is triggered externally after deployment, without access to sensor hardware or networking at attack time. This malware was developed in a closed research test environment with vendor technical support, rather than by exploiting an inherent production supply-chain vulnerability. To trigger the malware, we repurpose the sensor's own native dual-return signal processing as a covert channel for a modulated optical signal delivered into the environment. Once triggered, the malware performs false data injection and suppression on static and mobile platforms. Unlike optical spoofing, where the transmitted signal is itself the attack, here it carries only a small key that the sensor decodes with existing signal processing. We establish feasibility with static triggering at 300~ft and drive-by runs reaching 35 mph, and verify that injected artifacts are taken up as detections by a fine-tuned 3D object detector. On a deployed tactical autonomous vehicle running an unmodified production autonomy stack, we demonstrate both fail-safe and fail-dangerous impact: emergency stop, mission abort, and collision. These results motivate stronger integrity guarantees across LiDAR development and deployment pipelines.
♻ ★ Geometric Transition of Latent Dynamics in Intermittent Chaos: SVD-based Effective Dimension for Detection and Proactive Control
This paper investigates the geometric detection of latent dynamic transitions (where the latent space is defined as a finite-dimensional state space reconstructed via a delay-embedding Hankel matrix) in mutually coupled phase-locked loops (PLLs) exhibiting on-off intermittent chaos. We propose a data-driven approach to identify the transition of the effective dimension in latent dynamics as a precursor to physical bursting events. By applying Singular Value Decomposition (SVD) to a Hankel matrix constructed from high-resolution oversampled data, we extract the effective dimension that reflects the dynamical shift of intrinsic variables - from a synchronized two-dimensional manifold to a higher-dimensional chaotic state - prior to the actual burst. This geometric indicator enables a proactive, event-driven intervention using Pyragas-type delayed feedback control, which is activated only when the detected dimension exceeds a critical threshold. Simulation results demonstrate that this geometric precursor effectively compensates for control latencies and substantially reduces accumulated control energy compared to conventional continuous feedback. The findings establish a methodological strategy for stabilizing infinite-dimensional delayed systems by exploiting early warning signals embedded in dimensional transitions of latent dynamics.
comment: 13 pages, 7 figures. Accepted for publication in Chaos: An Interdisciplinary Journal of Nonlinear Science
♻ ★ Evaluating Grid Resilience in the Era of Ever-Increasing Data Centers
The rapid growth of artificial intelligence workloads is increasing the scale and concentration of data center demand, creating new concerns for power system resilience under disruptive events. This paper extends a validated multi-time-step DC optimal power flow framework to evaluate the impact of aggregated data center demand on contingency-induced unserved energy. Using an IEEE 30-bus system with flexible resources, we replace a conventional load at a contingency-exposed bus with an energy-matched constant data center load and examine two capacity-growth levels under generator derating, transmission line derating, and coupled derating. The results show that data center capacity growth substantially increases both system-level and data-center-bus unserved energy under transmission-constrained contingencies. Under coupled derating, the high-growth case increases total unserved energy from 3.203 MWh in the energy-matched case to 22.891 MWh. A supplementary energy-matched coincident-demand case further increases total unserved energy by 34.4%, indicating that temporally concentrated data center demand can amplify resilience impacts even without increasing total energy consumption.
comment: 8 pages, 2 figures, accepted for Dynamic Data Driven Applications Systems (DDDAS) 6th International Conference
♻ ★ Distributed Mixed-Integer Quadratic Programming for Mixed-Traffic Intersection Control
In this paper, we develop a distributed solution framework for coordinating traffic light systems and connected automated vehicles (CAVs) in mixed-traffic intersections. We formulate a mixed-integer quadratic program (MIQP) aimed at optimizing the coordination of traffic light systems and CAVs, thereby fully capitalizing on the advantages of CAV integration under conditions of high penetration rates. To effectively solve the challenging multi-agent MIQP approximately, we employ proximal ADMM for solving the convex relaxation of the MIQP while systematically tightening the constraint coefficients using a finite-grid tightening rule to uphold integrality requirements. \hilitediff{The obtained relaxed solutions are corrected using a confidence-guided binary recovery step, which fixes high-confidence binary variables and solves a substantially reduced MIQP over the remaining uncertain binaries to improve the feasibility of the final mixed-integer solution.} The performance of our control framework and the efficacy of the distributed algorithm are rigorously validated through a series of simulations conducted across varying penetration rates and traffic volumes.
comment: final version to Automatica
♻ ★ Analytical Prediction of Equilibrium Existence in Current-Limited Grid-Forming Inverters
As converter-interfaced units become dominant in power systems, understanding nonlinear stability phenomena involving discrete switching events becomes essential for secure operation. The limited overcurrent capability of grid-forming (GFM) inverters makes current limiting essential during large disturbances. Activation of a circular current limiter (CCL) can cause the operating equilibrium to disappear under certain circumstances. This paper develops an analytical framework to predict the grid-voltage boundaries for CCL activation and determine whether the operating equilibrium persists or disappears after activation. For CCL-based GFM inverters with frozen anti-windup, closed-form expressions for the activation boundaries are derived while accounting for the filter capacitor. A criterion is established to determine whether a saturated (i.e., current-limited) stable equilibrium point (satSEP) persists following limiter activation. A continuation method is also developed to trace equilibria across normal-control and current-limited operation. Power-angle analysis and dynamic-model continuation in single-inverter and modified 9-bus systems validate these predictions. Time-domain simulations further confirm the results. The results show that CCL activation can either cause immediate loss of the stable equilibrium or allow stable current-limited operation to continue until the equilibrium is lost as operating conditions change further.
comment: Submitted to IEEE Transactions on Power Systems
♻ ★ Stochastic Neural Control Barrier Functions
Control Barrier Functions (CBFs) are utilized to ensure the safety of control systems. CBFs act as safety filters in order to provide safety guarantees without compromising system performance. These safety guarantees rely on the construction of valid CBFs. Due to their complexity, CBFs can be represented by neural networks, known as neural CBFs (NCBFs). Existing works on the verification of the NCBF focus on the synthesis and verification of NCBFs in deterministic settings, leaving the stochastic NCBFs (SNCBFs) less studied. In this work, we propose a verifiably safe synthesis for SNCBFs. We consider the cases of smooth SNCBFs with twice-differentiable activation functions and SNCBFs that utilize the Rectified Linear Unit or ReLU activation function. We propose a verification-free synthesis framework for smooth SNCBFs and a verification-in-the-loop synthesis framework for both smooth and ReLU SNCBFs. and we validate our frameworks in three cases, namely, the inverted pendulum, Darboux, and the unicycle model.
♻ ★ Fast Relax-and-Round Unit Commitment with Economic Horizons
The US energy system is under increasing pressure to serve expanding data centers loads and to accommodate a larger number of generating units with varying technologies and ownership structures. Therefore, developing new unit commitment (UC) methods remains a priority for reliable and affordable grid operations. We expand our novel computational method for UC to include ramping constraints and long-horizon planning, and provide a theoretical bound on its accuracy. We solve problems with thousands of generators at 5-minute market intervals. We show that our method can solve UC problems with over 20,000 generators in approximately 10 seconds on commodity hardware and that an increased planning horizon leads to sizable op- erational cost savings. This is twice as fast as the leading commercial solver, Gurobi. The gap increases when adding more constraints. Our method can be implemented using existing continuous optimization solvers and adapted for different applications. We prove a bound on the error of these solvers and show that it vanishes (in relative terms) as the problem becomes larger. Our novel algorithm could allow an operator to make horizon-aware economic decisions for large systems.
comment: 19 pages, 5 figures
Robotics
★ Ropedia W0: Scaling Loco-Manipulation World Models across Embodiments with Human Experience
World models enable embodied agents to anticipate action consequences, yet jointly modeling locomotion and manipulation across diverse embodiments remains challenging due to heterogeneous action spaces and limited multimodal supervision. We present Ropedia W0, the first egocentric loco-manipulation world foundation model spanning humans and robots with diverse embodiments. We curate Xp-Data, comprising 10k hours of egocentric human data and 3k hours of robot data with unified multimodal annotations and systematic quality control. A unified kinematic representation encodes locomotion through camera poses and manipulation through projected skeletons, with robot skeletons combining embodiment-specific forward kinematics with extensive manual keypoint annotation and correction. We also introduce Xp-Teleop, a long-horizon loco-manipulation benchmark based on high-quality dexterous humanoid teleoperation data. Evaluations demonstrate state-of-the-art performance across five cross-embodiment benchmarks (DreamDojo, $Ψ_0$, $ω$-HOME, Xp-Teleop, and AgiBot) and on held-out human egocentric video, from zero-shot inference to few-shot and full-data post-training. Scaling human pre-training data improves performance on both human and robot evaluations. After distillation, our model supports real-time, minute-long rollouts, enabling live VR interaction. Applications in policy evaluation, model-based planning, and world-action modeling further demonstrate the model's utility for embodied decision-making.
comment: Project page: https://ropedia.github.io/ropedia-w0/ ; Code: https://github.com/Ropedia/ropedia-w0
★ Physical Words: A Vocabulary for Structuring Latent Reasoning in Robot Manipulation
Multimodal reasoning provides robotic foundation models with predictive context needed for reliable action generation. However, language-based reasoning captures task semantics but struggles to express fine-grained physical dynamics, while latent reasoning models future state evolution but offers limited structure for organizing these dynamics into reusable reasoning primitives. To bridge this gap, we introduce PhysWord, a reasoning-before-acting foundation model built on Physical Words, which use atomic action verbs as shared semantic anchors for continuous representations of physical dynamics. Specifically, PhysWord predicts upcoming atomic action words and directly aligns their contextual hidden states with latent targets derived from the corresponding future observations. Building on these primitives, we construct a temporally structured PhysWord space with visual and optional tactile variants of each action word. Autoregressively predicting Physical Words across future timesteps captures transitions between execution stages, providing structured multimodal context for action generation. PhysWord achieves state-of-the-art success rates of 90.3\% on visuotactile UniVTAC and 99.6\% on vision-based LIBERO-Long, and outperforms competing methods on real-world tasks using bimanual grippers and dexterous hands. Further evaluations demonstrate that our structured latent reasoning framework generalizes to unseen scenarios and enables long-horizon manipulation.
★ CoopUQ: Robust Multi-Robot Social Navigation via Uncertainty-Aware Cooperative Forecasting
Reinforcement learning enables mobile robots to make fast and scalable navigation decisions in constrained human-shared environments. However, directly extending single-robot approaches to multi-robot settings often fails to ensure safety and efficiency, especially under occlusions and out-of-distribution (OOD) scenarios, where individual robots may overlook potential risks or make inaccurate predictions. Leveraging cooperation effectively for both prediction and navigation remains a major challenge. We propose CoopUQ, a multi-robot social navigation framework that bridges uncertainty-aware cooperative occupancy forecasting and robust decision making. CoopUQ predicts future occupancy grid maps through cross-robot spatiotemporal feature fusion and further leverages forecasts and their uncertainty to guide policy via constrained multi-agent reinforcement learning, where the cost penalizes navigation through regions predicted to be occupied or highly uncertain. This design enables the policy to adapt its risk sensitivity under distribution shifts. Experimental results across multiple Unity3D-based simulation environments with both in-distribution and OOD settings, including diverse layouts, pedestrian densities, and rushing behavior, demonstrate improved prediction performance and safety over representative baselines. Real-robot experiments further demonstrate its robustness across various scenes. Our code and videos are available at https://coopuq.github.io/.
comment: Conference on Robot Learning (CoRL 2026); Project Website: https://coopuq.github.io/
★ Where is Physical RSI capability from? An Observational Scaling Law of LLM-as-policy
In this paper, we systematically investigate the mechanisms, scaling behavior, and generalization of physical Recursive self-improvement. We introduce PhysRSI-eval to quantify physical RSI through a five-axis metric: First-Attempt Performance, Efficiency, Success, Budget and Improvement from failure. Different VLMs are evaluated on 18 manipulation tasks under a fixed interaction budget. The analysis of experimental results motivates the Physical Barrel Hypothesis: different capabilities may constrain different parts of improvement. The five-axis metrics also distinguish systems with close public benchmark averages and show that high scores need not come with early or efficient gains. We study observational scaling from two perspectives: predicting the relative performance of VLM agents from their public capabilities, and forecasting later progress from early physical feedback. Together, these analyses ask how quickly and how far an agent can improve. Moreover, we conduct workspace interventions and then analyze what improvement leaves behind. Removing history reduces late-round success, frozen workspaces remain useful on unseen layouts, and three cross-task cases achieve success earlier. The results suggest that scene-specific adaptation and reusable experience can coexist.
comment: Tech Report
★ GPT-6 Astra Is a Native Visuo-Tactile Policy for Contact-Rich Manipulation
Recent advances in multimodal models have enabled GPT-6 Astra to perform robotic manipulation through in-context learning. Yet contact-rich control and tactile perception are often regarded as strengths of specialist policies, leaving open whether Astra can directly understand and use tactile observations. We provide, to our knowledge, the first demonstration that Astra directly interprets vision-based tactile images and uses their contact cues for manipulation. Tactile RGB images enter through its existing visual interface, enabling few-shot closed-loop control on nineteen simulated UniVTAC and OpenViTac tasks with 210 sampled scene seeds per setting. With visuo-tactile input, Astra achieves mean success rates of 93.8% and 81.8%, exceeding the strongest reported specialist baselines in our comparison by 22.9 and 13.1 percentage points, respectively. Vision-only performance also leads these baselines, reaching 76.3% and 80.6%. Trajectory analysis reveals active probing, cautious grasping, contact-guided alignment, and recovery, alongside difficulties with tangential loading and corrective execution. Our findings challenge the exclusivity of specialist policies in contact-rich manipulation and show that fine-grained vision can partially compensate for missing touch. They also identify vision-based tactile sensing as an interface for scaling contact understanding with foundation models and reveal signs of benchmark saturation. These results motivate joint visual-tactile pretraining, harder contact-rich evaluations, and physical-robot validation.
comment: 22 pages, 19 figures, 5 tables. Project page: https://tactalent.github.io/
★ PICA-WAM: Visual Perturbation Invariance via Cross-Appearance Self-Alignment in World Action Models
World Action Models (WAMs) jointly predict future video and generate actions, yet remain sensitive to task-irrelevant visual perturbations, i.e., changes in visual appearance that should not affect robot behavior. We identify a gap in conventional WAM training: video prediction encourages representations to preserve appearance information, while action generation does not explicitly enforce invariance to task-irrelevant appearance. Data augmentation exposes WAMs to such perturbations but does not require perturbed and nominal observations to produce consistent action representations, and alignment with external pretrained features does not guarantee the desired invariance. To address this gap, we propose PICA-WAM, a self-supervised training framework that casts perturbation invariance as an auxiliary cross-appearance self-alignment task for WAMs. An exponential-moving-average (EMA) teacher processes the nominal observation to provide a deeper-layer target, while the online WAM processes a perturbed observation and aligns its shallower-layer action representation to this target through a lightweight projection head. Across four strong WAMs evaluated on RoboTwin 2.0, LIBERO-Plus, EBench, and a real robot, PICA-WAM consistently improves robustness under visual perturbations, preserves nominal performance in simulation, and raises the real robot success rate. PICA-WAM further reduces the sensitivity of video features to task-irrelevant appearance and its propagation into actions, as shown by paired layer-wise diagnostics. Code: https://github.com/YudeZou/PICA-WAM-release
★ Bio-inspired Thrust-Assisted Trotting on a legged-Aerial Robot via Real-Time QP-MPC IROS 2026
This paper presents the hardware implementation of a quadratic program-based controller on Harpy: thruster-assisted bipedal robot to perform thruster-assisted trotting. This is inspired by wing-assisted locomotion observed in birds such as the chukar partridge, which employ thrust vectoring to manipulate the ground reaction force. The proposed controller formulates the thruster-assisted spring-loaded inverted pendulum dynamics as a condensed QP-based model predictive control problem, enabling simultaneous regulation of ground contact force and active thrust vectoring at 500 Hz. A decoupled control architecture is used, where the robot's sagittal dynamics are controlled using QP-MPC while the frontal dynamics is regulated using a cascade orientation stabilization controller. Two experiments were conducted with different forward reference velocities of 0 m/s and 0.1 m/s. While the instantaneous velocity oscillates, the mean velocities can track their respective reference. Additionally, joint tracking error remains within 1 deg while tracking the desired leg length given by the leg trajectory controller and desired swing trajectory given by Raibert's heuristic. The QP-MPC problem is consistently solved with 0.18ms, which is well within the 2ms computation budget while satisfying the GRF friction cone constraint.
comment: To be published in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2026)
★ HandsOn: A Large-Scale Simulation Benchmark for Generalizable Dexterous Manipulation
General-purpose manipulation requires reliable behavior across embodiments, objects, and task demands. We introduce HandsOn, a composable simulation platform and benchmark for cross-embodiment dexterous manipulation. It supports five robotic arms, six dexterous hands, and approximately 200 rigid, articulated, and deformable assets in randomized scenes. The benchmark pairs 18,000 curated human demonstrations with 66 human-verified tasks: Core-50 supports demonstration-based learning and execution tests, while Transfer-16 evaluates asset and relation transfer, skill composition, and deformable manipulation without target-task training demonstrations. Qualitative real-robot rollouts illustrate sim-to-real transfer. Across diffusion, vision-language-action, world-action, and prompted VLM-based methods, the best trained multi-task policy achieves 29.1% Core-50 success. Without benchmark finetuning, GPT-6 using move_eef for palm-and-finger control achieves 5.2% on Core-50 and 1.9% on Transfer-16 over 660 trials. Trained-policy stage diagnostics expose failures in contact establishment, action coordination, and maintaining achieved goals, with additional sequencing errors in composed tasks. HandsOn provides a shared basis for investigating manipulation generalization and the limits of reliable physical interaction.
comment: 56 pages, including appendices
★ DexOAK: Object-Aware Kinematic Retargeting
We introduce DexOAK, a learning-based object-aware kinematic retargeting framework for generating contact-consistent robot hand references from human motion-capture data. DexOAK first minimizes an interaction-graph Laplacian objective over a joint hand-object graph, so the robot reproduces the demonstrated object-relative interaction geometry rather than the human hand pose alone. These per-frame solutions form seed retargeted references, from which a feed-forward network conditioned on human keypoints and object point clouds learns the retargeting function. Object-aware finetuning then refines the network under contact, non-penetration, and self-collision losses that position the fingers on the demonstrated contacts and reduce artifacts destabilizing physics-based refinement. Once trained, the network transfers to unseen data by inference alone, without per-frame optimization. DexOAK references improve dynamic retargeting over the state-of-the-art across bimanual tasks with rigid and articulated objects. On the Inspire hand, success rises from 39.5% to 48.4% on OakInk-v2, and ADD-AUC rises from 62.7% to 77.9% on ARCTIC. Ablations trace the gains to object-aware kinematic retargeting and the amortized network refined with contact losses. Trained on OakInk-v2, it transfers to ARCTIC zero-shot and nears in-domain performance after fine-tuning. On hardware, 46/56 open-loop replays succeed across five tasks, enabling dexterous robot data generation from human data. Visualizations and interactive results are at: https://fspinola.github.io/dexoak/.
comment: 11 pages, 4 figures, 11 tables. Project page: https://fspinola.github.io/dexoak/
★ Tactile In-Hand Object Rotation: A Comparative Study of Hand Morphology in Simulation and Hardware
Continuous in-hand rotation requires coordinated contact maintenance and renewal, linking manipulation performance to both hand geometry and tactile feedback. We investigate these relationships through rotational progress, finger participation, and fingertip loading on two versions of the low-cost LEAP Hand with anthropomorphic and symmetric finger arrangements. Both hands are integrated with TacTip vision-based tactile sensors and use a shared reinforcement learning framework. Simulation evaluates full tactile, binary touch, and proprioception-only observations on 12 unseen objects; hardware experiments evaluate full tactile policies on ten objects under multiple hand orientations. In simulation, full tactile observations increase mean normalized rotation per episode over proprioception alone by 94.8% for the anthropomorphic hand and 66.9% for the symmetric hand, with longer mean episode durations and lower mean total fingertip loads. Hardware contact measurements reveal more differentiated finger participation in the anthropomorphic hand, while cumulative loading is more evenly distributed across the symmetric hand's fingers. Overall, the symmetric hand achieves a 32.0% higher mean rotation count and a 25.8% higher aggregate rotation rate, together with longer manipulation duration. Additional hardware force recordings show a 19.9% lower mean total fingertip load for this layout. These findings relate finger arrangement to contact organization and load sharing, and show that richer tactile information supports greater task progress at lower mean loading in both simulated embodiments.
★ Simulation Priors for Rapid Adaptation in Model-Based RL for Autonomous Racing
Reinforcement learning enables learning high-performance control policies, but its reliance on extensive interaction limits deployment on physical systems where data collection is costly, and failures are unsafe. We introduce Prior-Informed Adaptation (PIA), an offline-to-online model-based RL framework that uses structured simulation priors for rapid adaptation. PIA pretrains a world model on heterogeneous low- and high-fidelity simulated trajectories, then adapts it with limited target-domain interaction using uncertainty-regularized planning. Rather than requiring an accurate simulator for policy transfer, PIA treats simulated trajectories as biased priors whose residual mismatch is corrected during online adaptation. We evaluate PIA in autonomous racing using high-fidelity simulation and on a real full-size autonomous race car. In simulation, PIA adapts faster across track and surface shifts than strong baselines, achieving lower lap times and fewer failures. On hardware, PIA achieves reliable few-shot adaptation with less than one hour of on-vehicle interaction.
★ Robot Learning with Vision-Language-Action Models for Agricultural Manipulation
Robotic manipulation in agricultural environments remains challenging due to variations in plant geometry, occlusion, and camera viewpoints. Pretrained vision-language-action (VLA) and generalist robot policies offer a promising approach to addressing these challenges by learning manipulation behaviors from multimodal observations and language instructions. However, adapting these policies to agricultural tasks requires domain-specific demonstrations, which are costly and time-consuming to collect on physical robots. In this work, we develop a simulation-to-real robot-learning framework for agricultural reaching and grasping that integrates task-specific demonstration collection and pretrained policy adaptation. A MuJoCo-based agricultural environment is developed with diverse tomato plant structures to support demonstration collection and controlled evaluation. Expert demonstrations are collected through joystick teleoperation, with a low-level controller used to execute robot actions. Two pretrained generalist robot policies, Octo and $π_{0.5}$, are adapted using simulated demonstrations, followed by further fine-tuning on a physical Franka Research 3 manipulator using real-robot demonstrations. The adapted policies are evaluated under in-distribution, viewpoint/spatial-shift, and out-of-distribution conditions in simulation, followed by physical-robot experiments. The influence of proprioceptive information on policy performance is also investigated. Results show that $π_{0.5}$ achieves higher task-performance scores than Octo and exhibits less degradation under distribution shifts. These findings demonstrate the potential of integrating pretrained generalist policies with task-specific demonstration learning for agricultural manipulation.
★ Foundation Model Driven Optical Flow NeurIPS 2026
We investigate how frozen pretrained visual and depth representations support dense optical flow when coupled with a supervised matching network. Building on TTSFlow, this paper studies Dense Prediction Transformer (DPT) feature decoding and one coarse-to-fine refinement stage. Our method fuses features from frozen DINOv2 and Depth Anything V2 models and estimates optical flow through a trainable transformer-based matching pipeline. After optical-flow training on FlyingChairs and FlyingThings3D, it achieves 2.37 endpoint error (EPE) on the Sintel training split, Final pass, compared with 2.52 for the FlowSeek (T) configuration reported here. Our RoCo-Spring 2026 entry, RoCo-29.VSAI, obtains a robustness-balanced score of 1.1038, Spring EPE of 1.411, and RobustSpring proxy EPE of 4.739, with a lower balanced score than seven of the twelve listed reference baselines. These results characterize the complete pipeline, which reuses frozen foundation backbones and performs one refinement stage. Code is available at: \url{https://github.com/Visual-and-Spatial-AI-Lab/FFlow}
comment: 19-pages, 5 figures, 16 tables. Accepted for publication at 40th Conference on Neural Information Processing Systems (NeurIPS 2026). Workshop: RoCo-Spring: The Robust Correspondence Challenge
★ Robotic Physical Comparison of Biologically Motivated and Counterfactual Sensory Arrangements using Biomimetic Skin
Human skin contains diverse mechanoreceptors, including Merkel cells and Pacinian corpuscles, each localized in specific regions. Although their spatial organization is known to be shaped by molecular signaling and neural induction during development, experimentally examining its functional significance under controlled conditions in living systems remains challenging. In particular, physically comparing alternative receptor arrangements that do not occur biologically is difficult. In this study, we developed a layered biomimetic robotic skin designed to mimic key structural features of human skin tissue by embedding receptor-analog sensors within compliant soft media. Using this platform, we implemented both biologically motivated and counterfactual sensory arrangements and quantitatively evaluated their performance across three passive tactile tasks: detection of weak contacts, a two-point discrimination-like assessment, and texture identification. Arrangements reflecting key features of the biologically motivated sensory organization exhibited higher performance than the alternative configurations examined. This study addresses a biological question concerning how spatial organization influences tactile function by presenting a biomimetic robotic approach that enables physical and empirical comparison of biologically motivated and counterfactual sensory arrangements that are difficult to investigate in living systems.
comment: Published in Bioinspiration & Biomimetics (IOP Publishing). Project page: https://poyotamu000.github.io/biomimetic-skin-sensory-arrangements/
★ RoboValue: A Fine-Grained Sim-and-Real Benchmark for Unified Evaluation of Robotic Value Models
Robotic value models assess task execution from visual observations and instructions, providing feedback for policy learning and execution monitoring. However, existing evaluations cover only selected aspects of execution understanding, leaving the reliability of value feedback insufficiently tested. We introduce RoboValue, a fine-grained sim-and-real benchmark for unified evaluation of robotic value models. Its framework evaluates four complementary capability dimensions through shared interfaces: task-state understanding, temporal progress monitoring, failure and recovery reasoning, and value consistency. Diagnostic trajectories and instruction variations test whether value judgments accurately reflect task requirements and changes throughout execution. RoboValue-Dataset covers 15 simulation and 20 real-world manipulation tasks, providing expert demonstrations for training and annotated trajectories for fine-grained evaluation. We evaluate representative robotic value models in zero-shot and one-shot settings under standard conditions and separate shifts in robot embodiment and environment. Our results reveal errors in fine-grained instruction grounding, history-dependent progress tracking, and recovery assessment, even in models with strong outcome and forward-progress scores. The same semantic subtask can also receive substantially different value gains across valid execution orders. These findings identify concrete targets for value supervision, while the dataset and public leaderboard support continued assessment of feedback reliability. The website is available at https://robovalue-benchmark.github.io/.
★ EgoRayFormer: Ray-Space Stereo Refinement for Egocentric 3D Human Pose Estimation
Head-mounted stereo fisheye cameras provide a compact view of the wearer's body, but strong distortion, partial visibility and a short baseline make 3D pose estimation difficult. Existing refinement decoders sample image features independently in each view, without requiring the samples to correspond to a common 3D hypothesis. We introduce EgoRayFormer, a causal pose estimator that couples stereo feature sampling and geometric refinement in ray space. For each joint, the decoder projects shared 3D hypotheses along and around its viewing ray into both images. A geometric branch then corrects the projected joint locations and intersects the resulting rays; a learned gate blends this geometric estimate with a learned pose update. This formulation introduces an explicit stereo constraint while retaining flexibility when the rays provide weak depth constraints. In local evaluations on the official device-relative test splits, EgoRayFormer achieves three-seed mean MPJPEs of 17.64 mm on UnrealEgo2 and 49.55 mm on UnrealEgo-RW, improving to 47.57 mm with UnrealEgo2 pre-training. In paired validation ablations, removing the geometric branch increases MPJPE in all three seeds at both input resolutions. With a common ResNet-18 encoder and input resolution, our system reduces RW test MPJPE by 9.09 mm relative to the device-relative EgoPoseFormer baseline. A separate protocol analysis quantifies the difference between device-relative and oracle-root evaluation.
★ CAOS-SLAM: Class-Agnostic Object-Aware Real-Time vSLAM for Dynamic Environments
Robust visual Simultaneous Localization and Mapping (vSLAM) in dynamic environments faces challenges due to the presence of moving objects that may introduce erroneous observations, corrupt the map reconstruction and overall degrade localization accuracy. Existing vSLAM approaches often rely on predefined semantic classes or costly pixel-wise classification, limiting their robustness or applicability in real-world scenarios. This paper presents CAOS-SLAM, a dynamic vSLAM framework that extends ORB-SLAM3 with a hybrid perception pipeline to detect, track, and filter both semantically known and previously unseen moving objects in real-time. On-demand panoptic segmentation is performed for object discovery, with candidate unknown objects instantiated through depth-based clustering of unclassified regions. Between panoptic updates, persistent object tracks are updated using lightweight detection when available or temporal mask propagation based on world-frame Kalman prediction, depth consistency, and appearance similarity. The estimated object motion is then used to reject dynamic features from the SLAM pipeline. Experimental results on multiple dynamic RGB-D datasets and real-world experiments demonstrate improved localization accuracy over state-of-the-art dynamic vSLAM approaches, achieving up to 59% lower localization error in scenes characterized by unknown dynamic objects and up to 88% lower error in heavily occluded scenes, while achieving a median processing rate of 30 FPS on a laptop. Furthermore, the tracking module provides estimates of object trajectories and velocities, supporting potential future applications in perception, navigation, and motion-planning tasks in dynamic environments. Code and dataset will be released upon acceptance.
★ VaDar: Implicit Goal Conditioning via Value Disaggregation
Long-horizon tasks are important for modern AI and robotics, yet pose a challenge for reinforcement learning. Historically, a popular approach has been to decompose a task into subgoals, and learn a subgoal-conditioned policy to execute each subtask in sequence. We introduce an alternative called Value Disaggregation (VaDar), which trains a single subgoal-independent policy to maximize a linear combination of per-subgoal critic values. Because subgoal information is needed only at training-time, VaDar removes the need for possibly costly or cumbersome subgoal-selection during policy deployment. Moreover, through careful experiments, we show that VaDar (1) is always competitive with, and frequently outperforms, natural baselines like goal-conditioning and (2) succeeds on tasks with non-sequential goal structure and multiple optimization objectives. In particular, VaDar is the first algorithm to solve challenging, long-horizon tasks in the RoboCasa suite, even when starting from a policy pretrained from behavior cloning with near-zero initial task success. Taken together, VaDar's successes suggest that subgoal information benefits actor-critic reinforcement learning by enhancing the efficacy of critic learning, whereas policy goal-conditioning is often unnecessary.
★ A Probabilistic, Rotation-Preserving Kernel Operator Surrogate for Continuum-Robot Design Spaces ICRA
We develop an uncertainty-aware, operator-learning-based surrogate for tendon-driven continuum robots (TDCRs) that generalizes across robot design and actuation while predicting the complete equilibrium configuration. Our central idea is a nonstationary Gaussian-process (GP) kernel whose metric is learned by a shallow neural network from mechanics-aware design coordinates. The network learns how similarity changes across design space, while the resulting kernel defines the design-to-pose operator, its posterior uncertainty, and a criterion for sequential simulator acquisition. We represent backbone position and rotational strain with spatial kernels and reconstruct orientation directly on SO(3), preserving valid material frames and tendon geometry. We use ASkotch with Nystrom preconditioning for scalable kernel operator solves and a separate Nystrom approximation for posterior covariance. Experiments show that kernel-coverage acquisition reduces mean trajectory error by approximately 15% relative to IID sampling, while posterior variance increases for increasingly unsupported designs. Compared with prior neural-operator TDCR models, the method is 30-50% more accurate using approximately 5x fewer training designs while additionally providing posterior uncertainty estimates.
comment: Submitted to the 2027 IEEE International Conference on Robotics and Automation (ICRA)
★ HAF-VLA: Hierarchical Action Flow Framework for Humanoid Vision-Language-Action Control
Humanoid robots hold great promise as general-purpose agents in human-centered environments, yet adapting generalist vision-language-action (VLA) policies to real-world humanoid loco-manipulation remains challenging. Unlike fixed-base or wheeled manipulators, humanoids must coordinate locomotion, posture, gaze, and dual-arm manipulation over a high-dimensional whole-body action space, where inaccurate base or waist motions can easily induce unstable and aggressive upper-body compensation. We propose HAF-VLA, a hierarchical action-flow framework for humanoid VLA control. Built upon a pretrained flow-matching VLA, HAF-VLA inherits the vision-language prefix encoder and action expert weights, and adapts them to humanoid whole-body loco-manipulation with learnable stage embeddings and cross-stage KV-cache conditioning. Instead of generating the full action chunk in a single pass, HAF-VLA progressively denoises a shared whole-body action space: first locomotion and head commands, then waist refinement, and finally full-body manipulation. To preserve structural dependencies across stages, we introduce a cross-stage KV cache conditioning mechanism, enabling subsequent action generation stages to refine and augment their outputs based on the action representations from preceding stages. Experimental results across six real-world tasks demonstrate that our framework significantly enhances the performance of existing models in humanoid loco-manipulation tasks, demonstrating an effective pathway for adapting strong mobile-manipulation VLAs to humanoid whole-body loco-manipulation.
★ Learning Wheel Motion Uncertainty for Degeneracy-Aware LiDAR-Inertial Odometry on Quadrupedal Wheel-Legged Robots
Wheel kinematics can support odometry for quadrupedal wheel-legged robots when LiDAR geometry provides weak motion constraints. However, slip and contact loss make rolling predictions unreliable, and contact state alone cannot determine their reliability across motion directions. We propose a tightly coupled LiDAR-inertial-kinematic odometry framework that retains analytical rolling predictions and learns per-wheel, direction-dependent process covariances from proprioceptive history. The estimator maintains moving wheel-center states, allowing the learned uncertainty to regulate the influence of rolling predictions through forward-kinematic corrections. Degeneracy-aware projection (DAP) complements this mechanism by suppressing LiDAR constraints along geometrically weak directions. The learned covariance network (CovNet) is trained through a differentiable filter using simulated velocity supervision and deployed on the real robot without fine-tuning. On the six room sequences with the highest degeneracy ratios in our indoor dataset, the method reduces mean per-sequence position root mean square error (RMSE) by 66.2% relative to FAST-LIO2 and by 31.0% relative to a fixed-covariance variant. In a severely degenerate corridor run, it achieves an endpoint position error of 0.198 m, compared with 7.531 m for FAST-LIO2. These results demonstrate robust odometry under challenging geometric and wheel-motion conditions. We plan to publicly release our code and multimodal dataset. Project website: https://weihairuo.github.io/wheel-motion-uncertainty/
★ ReInsVLN: Revisiting Historical Evidence for Training-Free Vision-and-Language Navigation
Vision-and-language navigation (VLN) relies on retaining and using relevant observations. Yet growing histories introduce redundancy, and retained information may remain underused. We introduce ReInsBench, a diagnostic benchmark that examines history utilization through controlled continuation trials. Its results show that history-review reinstruction can recover navigation failures using the existing task context. Motivated by this finding, we propose ReInsVLN, a training-free framework supporting memory construction and utilization in frozen navigators. Coverage- and Instruction-Aware Compression constructs compact visual context through temporal redundancy reduction, coverage-aware protection, and instruction-guided token selection. Forgetting-Inspired History Reminders guide navigators to revisit retained observations according to a predefined schedule, without additional model calls. On R2R Val-Unseen, ReInsVLN improves success rates by 8.48 and 3.37 percentage points over the native StreamVLN and InternVLA-N1, reaching 64.27% and 64.44%, respectively. Visual compression largely preserves navigation performance while reducing prefill latency and KV cache memory by up to 21.77% and 32.25%, respectively, on matched observation workloads. Real-world trials on humanoid and quadruped robots further support the applicability of ReInsVLN to sequential instruction following and ordinal navigation.
comment: 8 pages, 6 figures, 6 tables
★ Learning Geometry-Aware Planning Representations for Monocular Flight with Cross-Modal Supervision
Existing high-speed navigation systems for micro aerial vehicles (MAVs) commonly rely on depth cameras, stereo vision, or LiDAR to obtain explicit geometric information, while monocular cameras offer a lighter and lower-cost alternative. However, monocular flight remains challenging because RGB images offer no direct metric cues and visual appearance varies considerably across environments. We propose a geometry-aware monocular visual planning framework that predicts trajectories from two image frames, their relative pose, and the MAV state. Without explicitly reconstructing depth, the proposed method complements the downstream planning objective with feature-level geometric supervision by transferring the planning-relevant representation learned by a privileged depth expert to the RGB planner through cross-modal distillation. The framework first trains a depth-based geometric expert, then distills its planning features into a monocular branch using paired RGB-D observations, and finally adapts the planner to the downstream trajectory planning objective. This design encourages the policy to learn planning-relevant geometric features, enabling collision avoidance. We further use a 3D Gaussian Splatting (3DGS) simulator to render visual observations for data collection, paired RGB-D supervision, and simulation evaluation. The planner trained in simulation is then deployed in real-world environments without real-world fine-tuning, achieving obstacle avoidance at speeds of up to 6.5 m/s.
★ Learn the Task, Not the Schedule: Reducing Spurious Cross-Arm Coupling in Bimanual Imitation Learning
A bimanual demonstration specifies both what the robot should do and one particular schedule for when each arm should act. Standard imitation learning does not distinguish task-required cross-arm constraints from incidental scheduling choices, and may therefore exploit opposite-arm progress as a shortcut cue, creating spurious cross-arm coupling even where relative timing is irrelevant for the current task phase. We introduce Constraint-preserving Temporal Recombination (CTR), a training-time data intervention that preserves recorded arm-local sensorimotor content while retiming the corresponding segments to realize admissible schedules that preserve task-required cross-arm constraints. We use CTR to train Diffusion Policy, Action Chunking with Transformers, and $π_{0.5}$, and evaluate them on four simulated and four real-robot tasks. Across six real-robot bimanual progress-state initializations, CTR raises worst-state success from 56% for the strongest non-CTR schedule mixture to 76%. With $π_{0.5}$ on an Independent task, a 4 s pause transfers 2.56 s of delay to the opposite arm under the Concurrent schedule distribution, but only 0.25 s with CTR. On real robots, CTR violates task constraints in only 0--4% of trials, versus 36--98% for narrow-schedule baselines. Code will be released at https://github.com/MINT-SJTU/CTR.
★ SPAR: Structural Pruning and Action Recovery for Vision-Language-Action Models
The limited onboard computing resources of robots make the inference demands of vision-language-action (VLA) models costly to accommodate. Reducing these demands through aggressive compression, however, can degrade manipulation performance beyond what supervised fine-tuning alone recovers. We present SPAR (Structural Pruning and Action Recovery), which simultaneously prunes the hidden and feed-forward intermediate dimensions of the large language model (LLM) backbone through pruning-aware tuning, then applies reinforcement learning (RL) to recover policy performance. Applied to $π_{0.5}$, SPAR reduces the language backbone from 2.34B to 0.09B parameters and the full model from 3.37B to 1.12B, removing approximately 96% and 67% of their parameters, respectively. At this compression level, RL raises average success across the four LIBERO suites from 84.55% to 93.45%, an 8.90-percentage-point gain. The compressed policy achieves a $1.98\times$ full-policy inference speedup over the unpruned model. These results identify substantial intermediate-dimension redundancy and show that post-pruning RL can recover manipulation performance in compact policies.
★ ALIGN: Aligning Condition Estimation and Expression in Advantage-Conditioned Behavior Cloning
Human-collected robot manipulation data often mix successful behavior with retries, pauses, recoveries, and failures. Advantage-Conditioned Behavior Cloning (AC-BC) can exploit such mixed-quality data, but only if the conditioning signal is both accurately constructed from offline trajectories and effectively incorporated into the policy's action-vector-field prediction. Existing pipelines can weaken this signal by thresholding estimator-derived advantage scores into noisy conditions, and by routing conditions through prompts that weakly affect the action vector field. We introduce ALIGN, a condition-aligned AC-BC framework for vision-language-action policies. ALIGN directly predicts categorical pairwise conditions in the policy-facing label space and injects them into a DiT action expert through AdaRMS modulation fused with flow time. We evaluate ALIGN on deliberately mixed real-robot datasets for ARX X5 bowl stacking and Franka key hanging, each combining standard demonstrations with retry-heavy trajectories. ALIGN improves condition fidelity, strengthens conditional vector-field separability, and achieves 90.0% and 56.7% closed-loop success, compared with 56.7% and 33.3% for the strongest AC-BC baseline. These results indicate that aligning conditions from estimation to action prediction is an effective way to extract robust manipulation policies from data in which success, recovery and error coexist.
★ A Human-in-the-Loop Comparison of Proportional Control and Adaptive Gravity Compensation in Elbow EMG-Driven Exoskeletons
Electromyography (EMG)-driven control enables wearable robots to assist users based on their neuromuscular activity. Upper-limb exoskeletons typically rely on proportional myoelectric control (PMC), which amplifies user effort but may compromise stability at high assistance levels. EMG-driven adaptive gravity compensation (AGC) offers a promising alternative, counterbalancing gravitational torques while continuously adapting to neuromuscular activity. However, these approaches have not been systematically compared in a human-in-the-loop setting. This study evaluates five control algorithms: two PMC strategies at different assistance levels, a purely integral AGC (AGC-PI), a kinematics- and acceleration-aware model-based (MB) AGC (AGC-AW), and a novel MB AGC approach based on Recursive Least Squares (AGC-RLS) for online payload estimation. Fourteen healthy participants performed target-reaching tasks using an active elbow exoskeleton under 0 kg and 2 kg loading conditions. Performance metrics included muscular activity, movement accuracy, task efficiency, and subjective workload. PMC exhibited a trade-off between physical effort and task performance: low assistance levels required greater physical effort under load, whereas high gains impaired task execution. AGC-PI reduced agonist activity but induced antagonist activation. In contrast, both model-based AGC approaches effectively compensated for external payloads, reducing physical and cognitive demands while improving movement accuracy and completion time.
comment: 8 pages, 10 figures, 4 tables
★ RoboMMT: Learning Stable Tracking of Modulated Expressive and Interactive Robot Motion
Expressive robot motion can improve naturalness, effectiveness, and perceived likability in human-robot interaction. When deployed on dynamically unstable platforms, however, reactive execution of expressive body motion may bring the robot to its stability limits, especially when expressivity is decoupled from physical control. In this paper, we present RoboMMT (Robot Modulated Motion Tracking), a modular control architecture that couples parameterized expressive motion synthesis, physics-based character control, and reactive adaptation to spatial interaction goals. Our approach trains a reference-conditioned tracking policy across a broad domain of expressivity, enabling it to faithfully realize the commanded motions while maintaining dynamic stability. Goal reactivity is achieved by training lightweight, task-specific interaction policies that modulate the reference generator's spatial goal-attraction point, with the stabilizing tracking policy kept in the loop. Evaluations on a quadruped and two humanoid robots show that the learned tracker remains stable and accurate across the entire modulation domain, whereas trackers trained without this coverage lose balance, tracking accuracy, or both. We show that layering closed-loop goal adaptation over this foundation enables interactive facing and pointing behaviors, while maintaining stable execution and comparable reference-tracking fidelity. The robot learns to proactively redistribute its weight and take corrective steps. Zero-shot sim-to-real transfer is demonstrated through real-robot experiments on a quadruped under heavy disturbances.
★ ForagerSim: Evaluating Embodied Map-Based Reasoning in Large-Scale Dynamic Worlds
Animals build cognitive maps of vast environments, use them to perform map-based navigation, and might even be reusing the same circuitry used in spatial reasoning for creating and using maps of arbitrary task-spaces. In contrast, current embodied AI systems either rely on hand-engineered mapping pipelines, which break in the presence of unmodeled dynamics and are restricted to metric space, or on learned policies that rarely form map-like representations at all. Developing architectures that could learn to build and use map-like representations as flexibly as animals is, however, difficult because few embodied AI benchmarks actually require map-based reasoning. To address this, we present ForagerSim --- an interdisciplinary research platform and simulation environment focused on embodied tasks that require map-based reasoning, with scenarios ranging from mobile-robot search and rescue to animal foraging in both complex mazes and vast, potentially infinite open areas. ForagerSim is highly modular and allows modifying independent axes of difficulty related to perception (abstract 2D sensing to full RGB or LiDAR), scale (room-scale to multi-km scale), topology (open-fields to complex mazes), dynamics (smoke, debris, sensorimotor failures and others) and tasks (from simple 2D coverage to foraging-inspired well-alternation tasks). We compare reinforcement learning baselines with a simple engineered robotics stack and a human player on 8 example worlds which show how different difficulty axes affect existing methods. Our results show that, when sensing is simplified to 2D, learned policies can perform multi-object search and well-alternation tasks surprisingly well in simple topologies, but they fall short of human performance in larger or more complex-topology environments where map-based baselines perform well, suggesting that map-based reasoning is the bottleneck in these scenarios.
comment: Code and data: https://github.com/MrTomzor/foragersim
★ Cartender: Verifiable Language-Conditioned Driving by Learning the Drives Not Taken
Every driving scene is logged once, but driving is one-to-many: the scene determines which behaviors are feasible, while an instruction selects among them. A log records only the action taken, leaving no direct supervision for alternative feasible actions or instructions the scene cannot support. Consequently, a policy that ignores language and one that follows it without regard to scene constraints can fit the same data. We present Cartender, which learns from the drives not taken. Counterfactual Branching constructs matched and mismatched instruction-action pairs from real recordings without a simulator: a vision-language model proposes alternative futures and their selecting instructions, while unsupported instructions provide negative labels. A learned scene-language compatibility score regulates language conditioning in trajectory generation, and a reward-trained language residual re-ranks generated candidates over a scene-based score. The branches are used for training; instruction following is evaluated by swapping instructions under identical recorded inputs and measuring behavior rather than agreement with generated targets. On WOD-E2E, the best of four proposals achieves an RFS of 8.642 across 3,603 feasible instruction conditions. Across 8,974 admitted proposals, mean RFS is 8.203 versus 7.906 for matched logged trajectories, and 67% score higher than their logged counterpart. Compared with matched-only imitation using the same backbone, Cartender improves same-scene paired instruction success from 23.67% to 57.00% and RFS from 6.32 to 7.10; its gate reduces infeasible-instruction execution from 53.36% to 14.18%.
★ Robotic Photography System based on Vision-Language-Action Model
Modern robotic photography requires understanding photographic intent, planning camera motions, and producing aesthetically pleasing images in complex environments. We present RoboPhoto-VLA, a unified Vision-Language-Action framework that jointly models perception, reasoning, and camera control for autonomous robotic photography. RoboPhoto-VLA incorporates PhotoCoT for structured action planning and Aesthetic Direct Preference Optimization (ADPO) to align camera behaviors with human aesthetic preferences. To facilitate training and evaluation, we construct OpenPhotography, a robotic photography dataset with language-guided demonstrations. Extensive real-world experiments demonstrate that RoboPhoto-VLA produces photographs better aligned with human aesthetic preferences while generalizing effectively to unseen subjects, viewpoints, poses, and complex photographic instructions.
comment: VCIP2026
★ AnchorFlow-VIO: Fixed-Template Monocular Visual-Inertial Odometry on LaMAria
We present AnchorFlow-VIO, an open-source Basalt-derived monocular visual-inertial odometry system, and evaluate a uniform configuration on LaMAria. The system uses one grayscale camera and one IMU, combining factory IMU correction, a fixed timing adjustment, windowed initialization and fixed-template patch tracking. Server evaluation of all 63 test recordings yields control-point scores of 47.2, 37.1, 24.8, 35.1 and 15.6 for Short, Medium, Long, Low light and Moving platform. Test pose recalls at 5 m are 84.6%, 69.1%, 53.9% and 62.9% for the four categories reporting dense recall. On ten training recordings, mean control-point score is 41.38 and all 265,694 image timestamps receive poses, including 204 filled poses. One training platform recording diverges and scores zero. A paired four-record comparison favors fixed over rolling templates on three recordings, while reversing that ranking on the fourth. Increasing feature density also yields mixed results. Ten fresh runs at prospectively fixed 30 s and 120 s offsets expose substantial start sensitivity: a low-light recording scores 6.14 at the earlier start and 42.00 at the later start. These results characterize the limits of the frozen system and show why complete output, feature counts and selected successful runs are insufficient evidence of robust localization. Source code, frozen configuration and reproducible submission tools are publicly available; training ablations are reported separately from the submitted test evaluation.
comment: 14 pages, 5 figures, 8 tables. Code: https://github.com/sudhirpratapyadav/anchorflow-vio
★ K-AoI: A Spatio-Temporal Synchronization Metric for Digital Twin-based Teleoperation in Dynamic Network Environments
Synchronization between a digital twin (DT) and its physical robot is critical for effective teleoperation, as mismatches directly degrade operator awareness and task performance. Existing synchronization metrics are predominantly time-based and rarely capture the spatial consequences of delayed information during motion. In this paper, we introduce Kinematic Age of Information (K-AoI), a spatio-temporal metric that weights AoI by instantaneous Cartesian velocity, translating temporal staleness into physically meaningful spatial divergence. We implement and evaluate K-AoI on a DT teleoperation testbed comprising a six-degrees-of-freedom (6-DOF) manipulator, a Unity-based twin, and controlled network impairments using tc netem across seven network conditions. Per-sample analysis shows that K-AoI exhibits rank-correlations with AoI at $ρ\leq 0.65$ across all networks, dropping to $ρ= 0.22$ under satellite connections. This shows that K-AoI is not a simple transformation of AoI, but instead captures velocity-dependent degradation effects that time-only metrics cannot resolve. We further analyze AoI tail behavior using Ultra-Low AoI (ULAoI) and find that the commonly assumed Generalized Pareto Distribution (GPD) model is rejected. This reflects the impact of deterministic forwarding and bounded jitter, which violate standard extreme-value assumptions. Overall, K-AoI provides a complementary and physically grounded synchronization metric, linking network delay to task-relevant spatial error in digital-twin teleoperation systems. The dataset and testbed software are publicly available at: https://github.com/luziyaotian/kaoi-testbed.
comment: 4 pages, 3 figures, 3 tables. Accepted as a short paper at IEEE Smart World Congress (SWC 2026)
★ PhantomTac: Hijacking Robotic Tactile Systems with Deploy-and-forget Tags CCS 2026
Tactile perception is crucial for precise robotic manipulation, providing an important foundation for dexterous, closed-loop force control. Among various tactile sensing techniques, magnetic tactile sensors stand out thanks to their fine-grained, three-dimensional force perception capability and strong robustness against visual occlusion. In this study, we discover a physical loophole called PhantomTac in magnetic tactile sensors. The attacker can alter the magnetic field, thus spoofing the force interpreted by magnetic tactile sensors. The attacker can stealthily deploy such an attack by simply attaching a thin (the thickness is around 0.5 mm), battery-free magnetic tag to the surface of an object. Hence, the attack tag can be easily concealed by a barcode sticker. Such an attack scheme is "deploy-and-forget" -- the attacker can deploy the tag and leave. When a robotic hand is picking up an object, it may severely damage the object. Specifically, spoofing the normal force can crush the object, whereas spoofing the shear force can lead to a loose grip, thus dropping the object. Compared with directly sabotaging the robotic hand, PhantomTac is much more concealed as the attacker does not need access to the device. We performed extensive reverse-engineering efforts to characterize the sensing mechanism of commercial-off-the-shelf tactile sensors and proposed a practical tag design. End-to-end evaluations conducted on real robots and robotic arms demonstrate that our attack can bypass existing anti-interference mechanisms and effectively hijack their closed-loop control processes, achieving an attack success rate exceeding 95% when the grasping-position offset is bounded within 8 mm. Finally, based on the limitations of existing anti-magnetic interference mechanisms, we propose a multimodal-consistency defense mechanism to mitigate such physical-layer security threats.
comment: Accepted at ACM CCS 2026. Project page: https://phantomtac.org
★ Data-Efficient Learning from Demonstration for Humanoid Robots via Feature-Level Perception and Prompt-Based Object Detection
Learning from Demonstration (LfD) allows robots to learn new skills from human examples; however, many vision-based methods require large datasets and resource-intensive end-to-end training. This paper introduces a data-efficient LfD framework for humanoid robots that learns manipulation skills from compact, task-specific features instead of raw images. The proposed approach integrates layered visual preprocessing, Kinect-based human joint estimation, prompt-based object detection, and depth information to extract human motion features and three-dimensional object locations from demonstration videos. These features are used to train a temporal deep learning model that predicts robot joint-angle trajectories for execution. The framework is implemented on a NAO humanoid robot, evaluated quantitatively in simulation, and demonstrated on physical hardware. In simulation, the robot achieves a 78% success rate on a pick-and-place task using only 16 demonstrations, which increases to 86% when trained with 100 demonstrations. The framework also achieves a 71% success rate on a pushing task using eight demonstrations, highlighting its effectiveness for data-efficient humanoid skill acquisition.
comment: 6 pages, 4 figures, 2 tables. Presented at IEEE ISAIA 2026
★ Compositional Humanoid Manipulation through Isomorphic Teleoperation and Independent Skill Routing
Humanoid manipulation requires reusable arm skills, yet demonstrating every possible skill combination is impractical. We present Humanoid Semantic Mixture-of-Experts Diffusion Policy (HS-MoDP), which composes separately demonstrated arm skills through independent semantic routing over a shared expert pool. This design preserves arm-specific skill control while sharing action-generation parameters across arms. A semantic scheduler integrates the learned policy with an existing locomotion controller through alternating manipulation and repositioning stages, extending the effective workspace and enabling skill reuse across location changes. To acquire demonstrations of whole-body contact manipulation, we develop G1Exo, an isomorphic exoskeleton that provides direct joint-level control of arm configurations. Experiments on a Unitree G1 show stronger skill composition than the baselines. G1Exo enables more reliable contact teleoperation than end-effector VR-IK, and its demonstrations improve contact-skill learning across four policy architectures. For HS-MoDP, switching from VR-IK to G1Exo demonstrations raises full cabinet workflow success from 35% to 65%. All project hardware designs and source code will be released as open source after the review process.
★ A GPU-Resident Pipeline for Multi-Robot 3D Inspection with Limited Perception
We present a complete, GPU-resident pipeline for multi-robot 3D inspection of underwater structures. Given a triangle mesh and a fleet of $K$ robots, each carrying a short-range, limited field-of-view camera, the pipeline computes 6-DOF trajectories that jointly cover a target fraction of the surface while minimizing a linearly interpolated makespan/total-cost objective. The joint problem is NP-hard, so we decompose it into five core stages - candidate viewpoint generation, visibility computation, set cover, min-max vehicle routing solved as a mixed integer program (MIP) with lifted Miller-Tucker-Zemlin subtour elimination, and priority-based multi-agent path finding with continuous-time inter-robot conflict checking - flanked by mesh loading and surface sampling as preprocessing and by trajectory smoothing as postprocessing. Our contributions are: an end-to-end GPU-accelerated integration of the five stages, with its kernel design documented; a GPU batch adaptation of Lien's $\varepsilon$-visibility with four custom CUDA kernels; a $β$-aware per-vehicle tour upper bound derived from the interpolated objective, supplementing the Desrochers-Laporte coefficient lift with forbidden-pair cuts; a 4D space-time extension of Zhou and Zeng's GPU parallel-frontier A*; and an experimental evaluation on 14 meshes (up to $7.2$M faces) that ablates the routing cuts and the parallel-frontier search, audits collision freedom and waypoint attainment \emph{after} trajectory smoothing, and measures the end-to-end gap against analytical and LP-relaxation lower bounds. Code and raw results are publicly available.
★ OmniInteract: Towards Unified Whole-Body Interaction Control for Dual Humanoids
Physical interaction is a key capability for humanoid collaboration, requiring coordinated whole-body motion and precise contact. However, learning diverse dual-humanoid interactions faces two major challenges: (1) human-robot morphological differences disrupt contact interfaces and spatial relationships during retargeting, and (2) jointly learning motor skills and partner coordination makes it difficult to retain control fidelity as interaction diversity grows. In this paper, we propose OmniInteract to address these challenges through surface-aware modeling and scalable residual policy learning. First, we introduce Surface Contact-Aware Retargeting, which leverages dense human-robot surface correspondences to jointly retarget both humanoids, preserving explicit interaction contact interfaces and their near-field geometric context. Second, we propose a Scalable Flow-based Interaction Policy for partner-conditioned residual generation. It factorizes dual-humanoid control into a reusable motor foundation and interaction-specific adaptations: partner-aware experts learn corrections over a frozen whole-body motion prior, while conditional flow matching unifies them in a shared residual action space. This enables scalable skill consolidation while retaining pretrained motor capabilities. Extensive experiments demonstrate that OmniInteract improves retargeting fidelity and dual-humanoid coordination while better retaining interaction performance as skill diversity grows. Real-world deployment further demonstrates successful contact establishment, stable whole-body coordination, and recovery from external perturbations.
comment: 30 pages including appendix, 11 figures. Project page: https://calvinyhli.github.io/omniinteract.github.io/
★ A Spiking Ring-Attractor Architecture for Representation and Control of Revolute Robot Joints
Robotic systems with revolute joints evolve on circular state spaces, yet most control architectures represent these states using Euclidean variables that ignore this underlying geometry. We propose a fully spiking neural control architecture in which ring attractors encode joint states, enforcing representations consistent with the topology of revolute systems. Homeostatic regulation computes goal-state mismatch directly over these distributed representations, and opponent gain-modulated populations convert this mismatch into controlled phase shifts of the activity bump, implementing set-point regulation as motion on the ring. We show that the bump dynamics follow an SO(2) geometry, and that gain asymmetries induce a signed phase velocity on the ring. To interface distributed activity with scalar regulation signals, we derive an analytic Fourier phase decoder and implement its signed components using push-pull spiking populations. For multi-joint systems, we demonstrate that local ring-based Fourier features compose into product representations that linearly decode SO(3) orientations through forward kinematics. Experiments in NEST validate the full SO(2) regulation mechanism and the spiking product-feature decoder for lift, pitch, and yaw. Robot experiments on a Tiago arm show that decoded ring dynamics generate joint velocity commands, producing directed motion with joint-dependent accuracy. Together, these results establish a neuromorphic control framework in which representation, regulation, and action are unified within a ring-attractor substrate aligned with the geometry of revolute systems.
comment: 23 pages, 10 figures
★ Self-Supervised Inertial Odometry
Learning inertial odometry for a new platform usually requires ground-truth trajectories. We show that an inertial odometry model can instead be trained from scratch by a myopic teacher that integrates the IMU over each short window independently, a scheme we call integration self-supervision.Interestingly, although supervised only by short-window velocity changes, the student learns to directly predict velocity, and does so more accurately than its integration teacher. Pose graph distillation then refines the model with sequence-consistent targets from a pose graph over many windows. Training on a new platform therefore no longer requires collecting ground-truth trajectories, but only calibrating the IMU and recording data during normal operation. Across six datasets spanning pedestrians, drones, quadrupeds, and a humanoid, the same recipe with no velocity or position labels trains models that come within $0.07$~m/s average velocity error of a same-architecture supervised baseline.
★ FR-VLA: Implicit Force Reasoning VLA for Contact-Rich Manipulation
Integrating force awareness into Vision-Language-Action (VLA) models involves not merely encoding wrench measurements as an additional modality, but determining when and how strongly physical interaction should influence robot action generation. Existing VLAs that use force typically inject it through a fixed pathway, without explicitly determining when it is relevant to the current action. This calls for an intermediate force relevance mechanism inside the VLA, in which visual-language context estimates the relevance of physical feedback to the current task before force enters the action pathway. FR-VLA addresses this gap with a Vision-Language Force-Reasoning Model that uses learnable force query tokens to derive a force admission signal grounded in visual-language context, and uses this signal to modulate an external tokenized force pathway for action generation. In real-world experiments on five contact-rich tasks across two robotic platforms, FR-VLA achieves the highest mean success rate over six evaluated stages, and trajectory visualizations show that its force admission varies with the interaction phase.
comment: Accepted to CoRL 2026
★ Reconstruction and Discrete-Event Simulation Assessment of an Analytical
This article presents an analytical-DES benchmark for intra-aisle robotic picking in automated warehousing. The investigated cycle combines horizontal shuttle travel, tote handling and item picking at storage locations. Simulation results are compared with the analytical formulation of Lerher et al. under the same rack geometry, shuttle-motion, handling and picking conditions, and the same definitions of cycle time and derived item throughput. Reproducing the published analytical tables requires the distance interpretation dES = 1.5 m and dTB = 0 m. The analytical-DES benchmark comprises 180 scenarios formed from three rack lengths, two paired motion profiles and n = 1 to 30 visited picking locations, with ten simulation replications per scenario. The six n = 1 cases are reported separately because no travel-between component is present. Across the remaining 174 scenarios, the mean absolute relative cycle-time discrepancy is 0.8655%, the median is 0.6758%, and the maximum is 7.0370%. The mean absolute error is 2.411 s and the RMSE is 2.744 s. The simulated scenario-mean cycle time is below the analytical value in all 174 principal scenarios. The results therefore show close numerical correspondence under the tested conditions, but not exact agreement, formal statistical equivalence, or validation against an operating warehouse.
★ SkateNav: Learning Energy-Efficient Navigation through Adaptive Skating on Passive-Wheel Quadrupedal Robots
Passive-wheel quadrupedal robots offer energy-efficient locomotion through leg-driven propulsion and momentum-driven gliding. However, navigation requires coordinating steering with intermittent propulsion and adapting push-glide timing to task demands. Velocity-tracking objectives may suppress the speed variations inherent in skating, discouraging efficient gliding. To address these challenges, we propose SkateNav, a unified skating-navigation framework based on reinforcement learning. A time-budgeted goal-reaching objective allows the policy to jointly learn steering, propulsion, and their temporal allocation. An adaptive skating cycle structures gliding, pushing, and recovery, while frequency modulation at every control step adjusts phase durations to navigation demands. Simulation experiments demonstrate higher average success rates and lower cost of transport than the evaluated planner-controller baselines, with ablations confirming the contributions of cycle structure and frequency adaptation. Real-world experiments demonstrate zero-shot transfer from simulation to a passive-wheel Mini Cheetah across cluttered indoor spaces, slopes, and paved outdoor surfaces. Supplementary videos are available on the project page: https://ktw1404.github.io/SkateNav/
comment: 8 pages, 12 figures, 3 tables. Project page: https://ktw1404.github.io/SkateNav/
★ PaceVLA: Adaptive Temporal Action Quantization in Vision-Language-Action Models
Fast and reliable manipulation is crucial for practical robot deployment. Although Vision-Language-Action (VLA) models have recently shown remarkable progress as generalist robot policies, their execution often remains temporally inefficient. A key source of this inefficiency is that policies learn fine-grained temporal patterns from teleoperated demonstrations, requiring many low-level action steps. Temporal action quantization offers a simple way to accelerate execution by aggregating consecutive low-level actions into temporally coarser actions. However, applying action quantization without considering when fine-grained control is required introduces a speed-accuracy trade-off: accelerated actions may efficiently reduce redundant steps in some phases, while discarding critical motion details in precision-sensitive interactions. To address this, we propose PaceVLA, a unified framework for adaptive temporal execution in VLAs that integrates two coupled components: (1) mode-specific action decoders built on a shared action expert, supporting multiple temporal granularities and prediction horizons, and (2) a quantizability-guided mode selection scheme which decides whether coarse execution is appropriate, and a context-conditioned router that selects an execution mode at each inference step. Experiments across three multi-task simulation benchmarks and six real-world tasks show that PaceVLA improves execution efficiency while maintaining task success rate across multiple embodiments, including long-horizon and dexterous manipulation.
comment: 25 pages, 7 figures, 10 tables. Project page: https://pacevla.github.io/
★ Policy Search in Code Space for Multi-Agent Pathfinding
Developing coordination policies for complex multi-agent systems relies on either labor-intensive algorithm design or opaque task-specific reinforcement learning. With recent advances in autonomous coding agents, we show that policy search directly in code space offers a compelling third path that combines automated optimization with explicit interpretability. We investigate a human-directed workflow in which coding agents iteratively propose, implement, evaluate, diagnose, and eventually refine algorithms guided by simulation feedback and explicit correctness verification. We conducted two case studies and found that this workflow robustly outperformed any manually engineered solutions. On a challenging map in the 2024 League of Robot Runners competition, a two-day search discovered a policy that exceeds the stronger locally reproduced competition entry by 35.0% on the densest instance and generalizes to unseen map instances. In simulated real-world warehouses with irregular lanes, continuous robot geometry, and asynchronous motion, a week-long search discovered an entirely new class of multi-agent pathfinding algorithm that outperforms a calibrated proprietary solution in throughput by 58.9%. Under stated assumptions, we further prove that the new algorithm is collision-free and completes, in finite time, every accepted goal that is not withdrawn or replaced. Together, the studies show that policy search in code space powered by state-of-the-art coding agents offers a highly competitive path to building real engineering solutions that are both effective and analytically tractable.
comment: 18 pages, 11 figures
★ CondenseVLA: Learnable History Condensation for Efficient Multi-Frame VLA NeurIPS 2026
Robot manipulation often depends on short-term cues - such as brief contacts, temporary occlusions, and intermediate object states - that are ambiguous in a single frame and can only be resolved by integrating observations over time. However, injecting historical information solely by enlarging the history window is computationally costly and frequently yields limited gains due to substantial redundancy and task-agnostic, inflexible sampling of observations. We propose CondenseVLA, a multi-frame VLA policy with a learnable decoder-side history condensation module for using short-term visual context in action decoding. Given a sequence of cached per-frame features from the VLM backbone, CondenseVLA distills observation history into a fixed set of slot tokens using learnable queries and multi-head cross-attention. Simple auxiliary regularization penalizes overlap among slot attention maps and decorrelates slot embeddings, encouraging complementary attention and reducing slot redundancy. Across the evaluated tasks, CondenseVLA improves Simpler-WidowX success from 58.7% to 69.8% (+11.1 percentage points), achieves 97.2% on LIBERO, and reaches 52.8% on real-world tasks. It achieves 21.6% success on memory-dependent RMBench tasks. These results support selective condensation of bounded recent visual evidence for action decoding, while harder multi-observation tasks expose the limits of a finite history window. Project page: https://github.com/FeiyangHong/CondenseVLA
comment: Accepted at NeurIPS 2026. 39 pages, including supplementary material and the NeurIPS paper checklist. Project page: https://github.com/FeiyangHong/CondenseVLA
★ Vision-Language-Delta-Action: Learning Compact Deltas as Actionable Transition Interfaces for Robotic Manipulation
Robotic manipulation is not merely observation-to-action generation, but the realization of task-specified state changes in the physical world. Existing vision-language-action (VLA) methods directly learn observation-to-action mappings, imposing a substantial learning burden due to the gap between semantic understanding and action execution. Although world-action models (WAMs) improve control through future prediction, modeling complete future states wastes capacity on static appearance and action-irrelevant changes. We propose Vision-Language-Delta-Action (VLDA), a paradigm that reformulates full future prediction as compact actionable transition modeling. Its core is a compact Delta variable that captures latent world-state changes, serving as an interface between vision-language understanding and action generation. VLDA learns Delta through a unified objective composed of the Local Transition Filter (LT-Filter) and Global Transition Anchor (GT-Anchor). LT-Filter captures task-relevant manipulation-centric changes while suppressing static reconstruction biases. GT-Anchor provides coarse future-level regularization to preserve the overall direction of state changes. Delta-Driven Action Refinement (DDAR) further grounds Delta in the action manifold and injects it as a structured seed to guide action denoising. We implement VLDA within LoongForge, an open-source training framework for large-scale foundation and embodied models, using its modular and distributed infrastructure for unified two-stage optimization. Experiments show that VLDA improves long-horizon consistency, robustness, and generalization, validating actionable Delta as an alternative to direct action prediction and full future-state modeling.
comment: 37 pages, including appendix. Code framework: https://github.com/baidu-baige/LoongForge/tree/master/cookbook/loongforge-vlda
★ AeroIntent: A Synchronized Multimodal Benchmark for UAV Mission Intent Recognition
Reliable recognition of mission-level UAV intent from multimodal observations is essential for low-altitude airspace surveillance and security. Existing multimodal UAV datasets primarily support detection, tracking, and localization, leaving limited support for evaluating mission-intent recognition across held-out trajectory programs and incomplete radar histories. We introduce AeroIntent, a simulation-based benchmark comprising 990 synchronized multimodal sequences generated by 22 trajectory programs across five mission intents. The benchmark provides radar, RGB, infrared, and depth observations, together with program-disjoint, radar-gap, and observation-prefix evaluation protocols. Building on this benchmark, we propose AeroMIR, a structured Multimodal Intent Reasoner that integrates radar trajectories, derived behavioral features, selected visual evidence, and task knowledge for candidate-based UAV intent recognition. Under a strict program-disjoint protocol evaluating unseen tactical realizations, AeroMIR achieves a mean accuracy of 71.56% and a mean Macro-F1 of 69.84% across three seeds, outperforming the strongest trajectory-only baseline by 7.79 Macro-F1 points and a conventional radar-RGB late-fusion baseline by 19.32 points. AeroMIR also demonstrates strong performance under controlled radar gaps and early intent recognition settings. AeroIntent and AeroMIR provide a reproducible foundation for multimodal UAV intent recognition. The benchmark is available at: https://anonymous.4open.science/r/AeroIntent
comment: 8 pages, 7 figures
★ OV-PEAR: Observation-Verified Pose-Scale Estimation and Reconstruction for Unknown Industrial Manipulation Objects
Fine industrial robotic manipulation requires 6D pose, scale, and an operation-ready 3D model. Existing industrial pose estimation relies on CAD models, object-specific training data, or stable workcells, limiting use with unseen objects; zero-shot methods suffer from single-view occlusion, generative bias, and scale uncertainty. We present OV-PEAR, an observation-verified framework for 3D reconstruction and pose-scale estimation of unknown industrial objects. Its three-stage object-centric pipeline: (1) segments RGB-D instances and generates object-level 3D hypotheses; (2) anchors them to the robot frame via appearance-aware view retrieval, explicit scale search, and visibility-sensitive registration; (3) back-projects multi-view RGB-D evidence into the object-canonical frame to verify voxel occupancy, suppress hallucinations, and produce reusable trusted object models. On BOP-Industrial, OV-PEAR achieves 0.657 average AR and improves pose and voxel-level geometric consistency in real scenes; real-robot connector assembly reaches 99.67% grasp and 84.00% assembly success. The model library and pose-scale log enable short-term object-level state maintenance at 0.02 ms/frame.
★ Long-Horizon Multi-Object Rearrangement via Distilled Tree Search over Continuous Actions
Despite recent advances in policy learning, long-horizon manipulation remains difficult because learned policies must avoid compounding errors while preserving future feasibility. While search-based planning can explicitly reason over such future consequences, it quickly becomes expensive in high-dimensional continuous action spaces. This raises a central question: how can long-horizon reasoning from search be transferred into policies in continuous action spaces? To this end, we propose DISCO: Distilled Tree Search over Continuous Actions. DISCO uses progressive-widening MCTS in physics simulation to generate long-horizon plans. During search, a learned policy proposes continuous actions, while a learned value function scores actions by their expected progress toward task completion. Using GPU-parallelized simulation, DISCO periodically distills MCTS-generated plans into the policy-value model, improving the action proposals and value estimates used in subsequent searches. Finally, the learned policy-value model can be directly deployed without the need for simulator-based search at test time. Across simulated and real-world rearrangement tasks, DISCO outperforms behavior cloning, reinforcement learning, and search-based baselines.
comment: Accepted at CoRL 2026 (Oral). Project page: https://distilled-search.github.io
★ TrajWAM: An Efficient Trajectory-Guided World Action Model for Dynamic Robotic Manipulation
Dynamic object manipulation requires robots to anticipate target motion while replanning fast enough to avoid executing outdated actions. Existing approaches either rely on fast reactive control without an explicit future target state or encode anticipated motion only implicitly. World Action Models (WAMs) offer a promising alternative by coupling future visual prediction with action generation, but scene-level video objectives may underrepresent target dynamics that are critical for manipulation, while iterative video-action denoising introduces substantial latency. We present TrajWAM, a trajectory-guided world action model for efficient dynamic robotic manipulation. Our key insight is to explicitly ground future visual imagination in predicted target-object motion and use the resulting future representation to condition action generation, while preserving this anticipation pathway after acceleration. Given a short history of object masks, a history-conditioned motion predictor forecasts future masks encoding the target object's trajectory. A lightweight Trajectory ControlNet then steers a pretrained video model toward trajectory-consistent futures. The resulting future representation conditions an action expert to predict executable action chunks. To enable one-step inference without breaking this interface, we introduce a decoupled distillation scheme that combines distribution matching for the video backbone, Trajectory ControlNet realignment, and action rollout distillation to support accurate one-step action prediction conditioned on the compressed future representation. Comprehensive evaluations in simulation and the real world show that TrajWAM achieves state-of-the-art performance on dynamic object manipulation tasks while enabling efficient one-step video-action inference. Project page: https://kevingeng3.github.io/TrajWAM/.
★ Learning from Fleet Experience: A Collective Learning Paradigm for Robots
Experience generated during deployment offers an opportunity for robots to improve policy reliability through learning from their own executions. However, learning through human corrections requires teleoperators to monitor execution and intervene when policies fail, a demand that is difficult to meet as robot fleets grow. We introduce collective experience learning (CEL), a learning paradigm in which deployed robots share their execution experience to improve a common policy. We realize CEL through failure-guided experience retrieval fine-tuning (FERT), which uses failure contexts from robot executions to retrieve relevant trajectory segments from the fleet's shared successful experience and uses their actions as supervision for policy updates, without collecting additional expert demonstrations or corrective actions. A fault-tolerant asynchronous training system (FATS) coordinates experience collection, policy optimization and policy distribution despite local interruptions within the robot fleet. Across five real-world manipulation tasks, the resulting policies achieve success rates of 93.3-99.2%, evaluated over 120 trials per task on robots excluded from training data collection. With collection time per robot held fixed, larger fleets provide more experience and FERT generally achieves higher success rates across the evaluated sizes. On multi-stage phone packaging, learning with 96 robots increases success from 79.2% to 98.3%. These results support collective experience as a practical basis for scalable robot learning and a path towards improving policy reliability during deployment without collecting additional expert corrective actions.
★ Think Densely, Act Sparsely: Latent Expert Cognitive Chains for Vision-Language-Action Autonomous Driving NeurIPS 2026
Autonomous driving fundamentally demands dense understanding of visual, semantic, and geometric information, yet existing Vision-Language-Action (VLA) models are typically trained with sparse supervision such as language instructions or trajectory signals. This direct mapping from dense visual observations to sparse representations forces the model to compress or even discard critical scene information essential for safe driving. We argue that this limitation does not stem from insufficient model capacity, but from the lack of explicit mechanisms that encourage the formation of dense world representations in the latent space. In this spirit, we propose a novel paradigm for driving intelligence, Think Densely, Act Sparsely, and instantiate it with an efficient and interpretable framework, LECDrive (Latent Expert Cognitive Chains for Vision-Language-Action autonomous Driving). The core idea is to mimic the hierarchical perception process of humans by embedding a progressive chain of latent expert cognition within the VLA model. Concretely, we inject a compact set of task-specific latent tokens as carriers, enabling the model to internalize dense knowledge from foundational vision models, including visual representations (DINOv3), semantic structures (SAM3), and spatial geometry (DepthAnything3), while maintaining efficient inference. This interdependent cognitive chain follows a curriculum learning principle, where supervision is progressively structured from representation to semantics and geometry. Extensive experiments demonstrate that LECDrive consistently improves the performance of base VLMs across multiple autonomous driving benchmarks, covering key tasks such as object perception, state prediction, and trajectory planning. These findings suggest that constructing dense world models through structured latent expert chains is a promising direction for VLA-based autonomous driving.
comment: NeurIPS 2026
★ Enhancing Lunar PNT Performance Through Surface Station Enabled Radiometric Observables
Future lunar surface missions require reliable positioning for autonomous rover operations. Lunar navigation satellite systems may provide limited vertical positioning accuracy due to satellite availability and observation geometry, particularly during early deployment. This study investigates a radiometric navigation architecture in which a lunar surface station (SS) improves rover positioning through double-differenced satellite observations and additional surface-to-surface ranging. Pseudorange and Doppler measurements from lunar navigation satellites are used for relative positioning between the SS and rover, while dual one-way ranging (DOWR) provides an additional range observable. The radiometric solution is integrated with rover dead-reckoning information. Monte Carlo simulations were conducted using five satellites representative of NASA's Lunar Communications Relay and Navigation Systems (LCRNS) constellation and a rover trajectory near Connecting Ridge at the lunar south pole. The results show that the SS improves horizontal and vertical relative positioning, with further gains from DOWR. In the simulated scenario, the mean 99.7% vertical positioning error decreased from 29.9 m without the SS to 5.5 m with double-differenced pseudorange and Doppler processing, and to 4.8 m with DOWR. An uncorrected DOWR hardware bias significantly degraded positioning, whereas the proposed bias-estimation method achieved a final estimation error of 0.058 +/- 0.057 m. The benefit of DOWR also depended on observation geometry. These results demonstrate the potential of a lunar surface station to improve vertical relative positioning through direct radiometric ranging, complementing its communication and navigation support roles.
★ SWAP: Test-Time Steerable Humanoid Whole-Body Collision Avoidance
Humanoid robots operating in human-centric environments must coordinate the entire body to traverse tight, cluttered spaces while maintaining balance. Existing methods often struggle to jointly consider dynamic stability and collision avoidance from high-dimensional perceptual inputs. Moreover, for humanoids, safe behavior is inherently multimodal: the same obstacle may admit multiple valid traversals, which is difficult to represent with a single deterministic policy. To address these challenges, we propose SWAP (Steerable Whole-body Avoidance Policy), a hierarchical framework that separates collision-aware decision making from stabilization. A high-level, perception-conditioned diffusion policy generates multimodal, collision-aware motion proposals, while a low-level whole-body RL controller tracks these proposals and maintains dynamic balance. To efficiently train the high-level policy, we obtain scalable supervision via hindsight labeling: we roll out the low-level policy under randomized commands, place obstacles only in the space that each executed motion did not sweep, so the motion is collision-free by construction, and re-render the perception the robot would have seen. At deployment, Monte Carlo sampling with simple cost signals steers the diffusion model toward safe, task-aligned behaviors without heavy reward tuning. Experiments in simulation and on a real Unitree G1 humanoid show reliable avoidance with static and dynamic obstacles. Test-time steering further enables customization, improving targeted rewards (e.g., speed, orientation, or posture) by 34% on average while maintaining a high collision avoidance rate. Together, these results support a practical solution to humanoid whole-body collision avoidance with test-time steerability. Project website: https://iscoyizj.github.io/SWAP-steerable-humanoid-oa-website/
comment: 20 pages, 11 figures, 4 tables. Project website: https://iscoyizj.github.io/SWAP-steerable-humanoid-oa-website/
★ Validation of Human Motor Development Scales with Humanoid Robots ICRA 2027
We investigate how humanoid robots can learn a series of motor skills defined by human infant motor development scales, and we evaluate which scales are comprehensive enough to serve as self-contained guides for developing gross motor skills up to standing and walking. We represent each motor development scale as a developmental graph that guides the training of a humanoid robot. We then apply this graph within a reinforcement learning framework so that the robot learns skills following the natural phases of human motor development described by the scale. Our work demonstrates that graphs derived from three out of four popular pediatric motor scales can successfully guide humanoid robots to develop these skills. The fourth scale misses some intermediate milestones that the other scales have, which causes some failures. After adding a missing milestone in its graph, it also enables the robot to develop those skills successfully.
comment: 8 pages, 7 figures. Submitted to ICRA 2027. Video available at https://youtu.be/SR3LUVhohV4
★ Progress-Guided Preference Generation for Aligning Generative Control Policies
Generative policies have achieved strong performance in imitation learning tasks, but their success is often constrained by the limited availability of expert demonstrations. Although preference-based fine-tuning can leverage suboptimal rollouts to improve performance, existing approaches heavily rely on expensive human annotations. In this work, we introduce Progress-Guided Preference Generation (PGPG), an unsupervised preference alignment framework that eliminates the need for human-in-the-loop supervision. Our method leverages ordered coverage alignment to measure sequential progress relative to a small set of expert demonstrations, enabling the automatic extraction of pseudo-preference labels from autonomously collected rollouts. Furthermore, we introduce key enhancements to improve the reliability of these progress-guided preferences, enabling effective fine-tuning of pretrained generative policies without requiring manually designed reward functions. Extensive experiments across a suite of challenging long-horizon manipulation tasks demonstrate that PGPG outperforms standard supervised fine-tuning baselines. Moreover, our conceptually simple framework is broadly applicable across multiple classes of generative policies, demonstrating strong versatility when integrated with both diffusion and flow-matching policies.
comment: CoRL 2026 (project site: https://pgpg2026.github.io/)
★ ReDOA: Real-World Distillation via Observation Alignment for Image-Based Manipulation Policy Learning
Data-driven learning for general-purpose robotic manipulation presents a fundamental trade-off between data-collection cost and adaptability to real-world environments. Imitation learning (IL) requires costly human demonstrations, whereas simulation-based learning is vulnerable to sim-to-real gaps in dynamics and sensor observations. We propose ReDOA (Real-World Distillation via Observation Alignment), a framework that decouples control learning from perception learning to learn image-based policies in the real world. A state-based teacher policy trained in simulation is executed on a real robot, and its control knowledge is distilled into an image-based student policy using the collected trajectories. Observation Alignment resolves observation mismatches between data collection and deployment. This framework collects successful, efficiently executed trajectories paired with real-world visual observations, without requiring demonstrations from human experts. In real-robot experiments on three non-prehensile manipulation tasks, including bimanual manipulation, our method achieves better overall performance when jointly considering task success and execution efficiency than both IL based on human demonstrations and policy distillation in simulation. https://isri-aist.github.io/redoa-project-page/
comment: 9 pages, 6 figures, 2 table
★ ScaleNav: Scalable Internet Video Pretraining for Language-Conditioned Local Navigation
We study language-conditioned local navigation, where a robot must navigate to a text-specified target visible in its initial observation and stop at the described location. End-to-end navigation policies require diverse training data to generalize across scenes, yet real-robot data is costly to collect and simulated data leaves a sim-to-real gap. Internet egocentric videos offer a scalable source of real-world navigation experience. Our key insight is to learn goal-directed visual dynamics from Internet egocentric videos, and use these dynamics to support robot action generation. To this end, we propose ScaleNav, a unified world-action framework with three-stage training. We first perform instruction-conditioned video pretraining on these videos to learn how observations evolve as the camera approaches instructed targets across diverse real-world scenes. Action alignment then uses action-labeled simulated data to train the model to predict executable action chunks. Joint fine-tuning subsequently optimizes all trainable components together for navigation. ScaleNav achieves success rates of 86.5% on a single-view adaptation of Short-Horizon OVON and 94.9% on our HabitatGS-LN benchmark, outperforming the evaluated baselines on both benchmarks. We further deploy and evaluate ScaleNav on a real quadruped robot, with results supporting the model's ability to generalize to real-world navigation. Code and videos are available at https://zju3dv.github.io/ScaleNav/
comment: 25 pages, 7 figures, 6 tables
★ Vehicle-State-Aware Confidence Estimation for Direct Wind Sensing on a Multirotor
Direct wind sensing from a multi-rotor UAV enables localized atmospheric measurements, but the reliability of the anemometer depends on vehicle motion. This work presents a vehicle-state-aware confidence method that uses horizontal and vertical vehicle speed to quantify the acquisition of each wind observation. The resulting confidence score is used to adapt measurement influence to a two-dimensional vector estimator. Experiments were conducted during horizontal translation, vertical motion, and held-out autonomous waypoint flight. The proposed two-feature formulation achieved an ROC-AUC of 0.995, with 98.84% of translation observations identified as unreliable. The confidence-aware adaptation reduced median estimator gain during translation from 0.207 to 0.064 while retaining median gain of 0.151 during hover. The results show that vehicle state provides useful proprioceptive context for regulating the influence of direct airborne wind observations.
comment: 8 pages, 4 figures, 2 tables
★ Revealing and Bridging the Physical Executability Gap in Vision-Language Navigation
Continuous vision-and-language navigation (VLN) is typically evaluated by goal-reaching success, assuming it reflects an agent's ability to execute language-grounded behavior. We show that this assumption can fail: permissive contact models in current simulators may mask physically infeasible actions, creating a physical executability gap between semantic competence and embodied realization. To bridge this gap, we introduce Visual-Semantic Traversability Alignment (VeSTA), a framework that aligns VLN policies with physically executable futures. We define visual-semantic traversability as the ability to predict instruction-conditioned trajectories that are semantically valid, geometrically feasible, and physically robust. VeSTA addresses this misalignment through two complementary mechanisms: improving executable support in trajectory generation and risk-aware selection among multimodal futures. Specifically, Trajectory Relative Group Policy Optimization uses strict-contact feedback to reshape the candidate distribution toward executable regions, while the Traversability Risk Critic learns a risk-adjusted scalar utility for candidate-wise future selection. Experiments on R2R and RxR reveal substantial dependence of existing VLN agents on simulator contact semantics. Under strict-contact evaluation, VeSTA substantially reduces collision-induced failures while preserving or improving goal-reaching performance, and controlled real-world robot experiments provide evidence that the learned preference transfers beyond simulation.
★ SpaRV: Sparse Radiance Voxels for Scalable Geometrically Consistent LiDAR-Visual Mapping and Rendering
Robotic simulation, navigation, and inspection need photorealistic and geometrically accurate maps. Radiance fields achieve photorealism, but building them from robot data raises three challenges. A robot's trajectory serves its task, not reconstruction, so images from few viewpoints leave the geometry under-constrained. Residual errors in camera poses and intrinsics blur the map. The map's GPU memory grows with scene size, which limits its level of detail in large scenes. We present \textbf{SpaRV}, a \textbf{Spa}rse \textbf{R}adiance \textbf{V}oxel map rendered by differentiable ray marching, which lets irregular LiDAR scans supervise it alongside images. LiDAR depth and occupancy anchor the under-constrained geometry, and normal and smoothness regularizers keep it smooth between LiDAR points. To correct camera errors, SpaRV alternates map training with photometric optimization of the camera poses and focal lengths. To bound GPU memory in training and rendering, SpaRV holds one submap at a time at full resolution and the rest at coarser octree levels. Experiments show that SpaRV keeps the geometry consistent away from the captured trajectory and sharpens textures by optimizing the camera parameters. On one 24\,GB GPU, SpaRV reconstructs a university campus captured in 63{,}582 images and simulates other cameras and LiDARs. Our code will be released at \url{https://github.com/hku-mars/SpaRV}.
★ InteractAgent: Agentic Human Motion Interaction with Memory-Augmented Reflection
Long-horizon human-scene interaction requires coordinated spatial reasoning, physical feasibility, and the ability to recover from compounding execution errors. Text-driven motion models are largely scene-agnostic and lack closed-loop correction, while geometry-conditioned approaches depend on paired scene-motion training or explicit 3D preprocessing that limits generalization to novel layouts and open-vocabulary goals. We present InteractAgent, an agentic multimodal LLM system for long-horizon human-scene interaction that plans, executes, observes, and reflects in a closed loop, using a single MLLM as both planner and evaluator and a frozen MotionLLM as the executor. InteractAgent integrates Online Observation for step-wise recovery decisions (continue/redo/replan) and Memory-Augmented Reflection for multi-round plan refinement with adaptive stopping. On UniHSI, InteractAgent improves task success and reduces penetration compared with training-free baselines. Code: https://github.com/AIGeeksGroup/InteractAgent.
★ TNVD-SC: Diagnosing Scan Context Failures in Forests with Terrain-Normalized Vertical Distributions
Long-term LiDAR place recognition in forests is difficult because repetitive vegetation and uneven terrain create severe structural aliasing. We diagnose Scan Context (SC) by separating candidate retrieval from fine matching: on the standardized TreeLoc++ Wild-Places protocol, a TNVD support key places a valid location in the top-25 set for approximately 99% of queries, while conventional SC matching on those candidates reaches only 61.84% mean R@1. With an identical raw-SC candidate set, terrain normalization improves max-height cells by only 1.21 points but vertical-distribution cells by 19.72 points. These observations motivate Terrain-Normalized Vertical Distribution Scan Context (TNVD-SC), which preserves the SC ring-sector topology while encoding terrain-relative vertical probability distributions. Under the standardized TreeLoc++ protocol, TNVD-SC reaches 91.83% mean R@1 and attains the highest R@1 mean and R@1 stability among the compared methods, while TreeLoc++ remains stronger in mean MR, MF1, and AUC. The unchanged method reaches 85.14% mean R@1 on the four-session CVTNet-style NCLT evaluation. A lossless count-coded implementation with exact hierarchical Jensen-Shannon search reduces descriptor payload by 3.86x and compute latency by 23.8% without changing the audited retrieval outputs.
comment: 8 pages, 5 figures
★ SteerAct: Unlocking Action Models through Few-Shot Temporal Steering
Improving pretrained robot action models at test time from only a few rollouts remains challenging. Activation steering offers a lightweight approach by modifying feature activations while keeping model weights fixed. However, effective steering requires identifying which features to modify and determining the timing and strength of modification. We introduce SteerAct, a few-shot temporal steering framework that uses a shared temporal descriptor of sparse-feature dynamics for offline feature selection and online control. A pretrained sparse autoencoder provides candidate features, and our temporal descriptor summarizes the distribution of local activation-order patterns for each feature, enabling comparisons across rollouts without requiring timestep alignment. Before steering begins (offline), we use successful and failed reference rollouts to select a single task-specific feature whose temporal descriptors best distinguish the two outcomes and construct its reference banks. During each new execution (online), we compare the feature's evolving temporal descriptor with these references to determine when to intervene and how strongly to increase or decrease its activation via a steering coefficient. Using only 24 outcome-labeled rollouts per task, SteerAct improves task success by approximately 10% relative to unsteered models in evaluations spanning $π_{0.5}$, Fast-WAM, RoboTwin 2.0, DexJoCo, and real-world manipulation, without updating action-model weights. Code and demos are available at: https://steeract.github.io/.
★ Where Do the Gaussians Go? Foreground Attraction for Camera-Based Weakly Supervised 3D Occupancy
Camera-based 3D semantic occupancy models can be trained without 3D labels by rendering their predictions into the images and supervising them with 2D pseudo labels. In sparse Gaussian representations trained this way, Gaussian positions are learned only through pixel-wise rendering losses, which we hypothesize give small foreground objects little weight. In trained GaussianFlowOcc models, the Gaussian density on foreground voxels is on average only 1.62 times that on background voxels. We propose a training-only foreground attraction loss that back-projects foreground pseudo-label pixels with pseudo depth and pulls nearby Gaussians toward these 3D points, favoring Gaussians that already predict the target class; the inference network is unchanged. On Occ3D-nuScenes, the loss raises this concentration to 2.40 over three seeds. Over three seeds, the mean foreground IoU increases from 6.29 to 6.96 and is higher for each seed, whereas the change in mean mIoU (13.53 to 13.82) is within the seed-to-seed variation of the baseline. Single-run ablations further indicate that pseudo-label targets outperform targets placed at ground-truth voxels and that acting on the Gaussian positions alone is not sufficient, and a single-seed test shows that the model degrades less when fewer Gaussians are kept at inference.
comment: 7 pages, 3 figures
★ Humanoid SRL: Augmenting Humanoids with Supernumerary Limbs for Coordinated Loco-Manipulation
Humanoid robots are envisioned to perform diverse human-level tasks through their anthropomorphic morphology. Their dual-arm configuration enables a broad range of everyday actions. Yet, as in humans, this morphology can constrain loco-manipulation when both arms are occupied or additional contacts are required beyond their reach. Supernumerary robotic limbs (SRLs) offer a way to extend these capabilities, but their integration requires the added limb to be coordinated with whole-body motion and balance. In this study, we develop and integrate a supernumerary arm into a humanoid platform to investigate how morphological augmentation reshapes its feasible action space. Through quantitative analysis and hardware experiments across two configurations, we show how the additional limb expands whole-body interaction capabilities by enabling manipulation and support to be coordinated beyond the constraints of the native dual-arm configuration. To control this augmented morphology, we leverage model-based trajectory optimization (TO) to generate dynamically feasible motions for goal-conditioned reinforcement learning (RL) controllers. The controllers are trained in simulation and transferred to hardware in a zero-shot manner, enabling coordinated whole-body skills for contact-rich tri-arm loco-manipulation. We further show vision-guided autonomous execution on selected tasks using diffusion policies trained from hardware demonstrations. Our findings suggest that humanoid morphology need not be restricted to human-like forms, but can be reimagined as an extensible design space for robotic augmentation.
★ Vector-Quantized World Model: Latent Feedback Motion Planning for Robot Manipulation
Visual world models have shown strong potential for robot planning, yet long-horizon manipulation remains challenging due to compounding prediction errors, large search complexity, and latent plans that the low-level controller cannot reliably reach. In this work, we introduce Vector-Quantized World Model (VQ-WM), a latent feedback motion planning framework that integrates discrete long-horizon search with continuous visual feedback control: a discrete latent predictor models transitions between vector-quantized latent states, allowing the planner to look far ahead and find a shortest path to the goal through discrete search and a continuous learned controller sequentially reaches the planned states through visual feedback, correcting deviations to mitigate compounding errors. Whereas most vector quantization uses Euclidean distance, we use a novel estimator of the first-passage-time as a quasi-metric, which aligns the planning abstraction with the controller's capabilities. We evaluate on the physical robot manipulation of the Tower of Hanoi puzzle: whereas existing state-of-the-art foundation models (VLAs, WAMs) only succeed when the solution trace in their training dataset, our method achieves 100\% success rate with only short-play data, demonstrate its ability to compose long-horizon solutions beyond the short demonstrated trajectories and execute them reliably. Our solution also outperforms strongest world-models by 13.3--85.3 percentage points on simulation benchmarks spanning manipulation and locomotion, demonstrating consistent progress and new capabilities.
★ Training Controllable World Models via Action Influence Maximization
Action-conditioned video generative models trained on successful, goal-directed demonstrations tend to ignore the commanded action, rendering a plausible expert-like future regardless of what is asked of them. We trace this failure to training data that lack sufficient action influence. We analyze this metric and prove that it upper bounds the action influence that any model trained on that data can express. Expert demonstrations, whose narrow action distributions let next states be predicted without attending to the action, set a low ceiling that no amount of model capacity alone can raise. Raising it requires collecting different data. We introduce Action Influence Maximization (AIM), an algorithm for actively collecting training data for improving dataset action influence. We compare the performance and data-efficiency of AIM against exploration heuristics and intrinsic-motivation methods on simulated navigation and manipulation tasks. In pixel-based manipulation, AIM is the only method that consistently improves controllability: 500 actively collected episodes added to 3k demonstrations match a world model trained on 300k expert episodes on end-effector controllability, an $85\times$ reduction in data.
comment: 31 pages, 13 figures, CoRL 2026
★ Refuses When Asked, Complies When Acting: A Runtime Safety Check for Robot Control Loops SP
Frontier models can control robots through agent harnesses that loop on one question: what motion comes next toward the user's goal? We show that this default loop carries out harmful instructions. With GPT-6 Astra controlling a low-cost SO-101 arm through the Inspect Robots harness on three household tasks whose hazard is common knowledge, such as "put the knife in the toy basket", the policy refused 0% of 15 trials, attempted the harm every time, and completed it in 27%. Its one exit, give_up, is meant for goals that cannot be achieved, and it never cited the hazard. We add an optional runtime safety check to the loop: before the first motion, the model is asked whether it would carry out the instruction, given the same camera frames. The check would have stopped all 15 hazardous instructions, including the 4 that completed the harm, and flagged none of the 15 harmless controls. Code and data: https://github.com/Arnav-Gr0ver/Refuses-When-Asked-Complies-When-Acting.
comment: 4 pages plus references. Submitted to the Science of Physical AI Safety (SPAIS) workshop at CoRL 2026
★ Determination of the Human-Robot Interaction Workspace for Robot Assisted Head Stabilization Control
Accurate head immobilization is critical for medical procedures such as neuroimaging and intracranial radiation therapy, where even minor motion can compromise precision. Traditional methods, such as rigid stereotactic frames and thermoplastic face masks, are invasive, uncomfortable for patients, and require a time-consuming setup. To overcome these limitations, a novel approach employs a 6-degree-of-freedom (6DoF) robotic head holder that is integrated with real-time optical motion tracking, enabling dynamic compensation for head movements without physical restraint. However, implementing this system requires a detailed biomechanical understanding of skull-cervical spine interactions to define safe and effective motion limits. In this study, we conduct a human volunteer investigation using a 6DoF Stewart platform robot to characterize biokinematic workspace of the head-neck system and establish foundational data for human-robot interaction in precision head stabilization.
★ Design and Control of a Multimodal Legged-Aerial Robot Through Separate Control Contraction Metrics
The integration of dynamic legged locomotion and aerial mobility offers versatility that is deterred by challenges in payload distribution, actuation, and the contrasting requirements of both domains. In this paper, we present the design and operation of a multimodal quadrupedal aerial-legged robot. The presence of legged aerial systems also allows for thruster-assisted legged locomotion, which we make possible with a unified control framework using Control Contraction Metrics (CCM). We search for separate domain-specific contraction metrics for walking and flight independently offline and unify them using an online energy-based Control Lyapunov Function Quadratic Program (CLF QP). This enables optimal, cross-domain control allocation, which we show using simulations. We also show experimental results that validate the mechanical viability of our hardware and the paradigm of thruster-assisted trotting. The stability of said experiments motivates the deployment of the control framework for multimodal operations.
comment: Presented at 2026 IEEE International Conference on Intelligent Robots and Systems, Pittsburgh PA
★ RadGeoLoc: Radar Geolocalization via Particle Filtering ICRA 2027
In real-time autonomous field applications, GNSS is often unreliable and inaccurate. A possible solution is localizing against a prebuilt map like a satellite image, as it is georeferenced and already surveyed. In this work, we localize a spinning radar measurement against overhead RGB imagery with a learned observation model. We use a radar representation pretrained by masked reconstruction and align it to features from a satellite image. A particle filter propagated by radar odometry proposes candidate poses, and our observation model provides a likelihood over these poses. We evaluate our method over 31 distinct routes, over 8 datasets spanning both urban and offroad conditions using only ground radar and a coarse local prior. We also examine the robustness over changing weather using repeated traversals over the same route over 13 months, in both urban and offroad conditions.
comment: Submitted to ICRA 2027
★ What Would Removing This Corruption Change? Failure Attribution for Vision-Language-Action Models
Vision-language-action (VLA) models are sensitive to visual corruptions, but knowing that a corruption is typically harmful does not establish whether removing it would benefit the current episode. Furthermore, when several corruptions co-occur, deciding what to correct requires understanding which ones contribute to failure risk and how much their removal would help. We define this problem as failure attribution. Detecting corruption presence alone cannot answer this question, as corruptions can be present but relatively harmless, while directly measuring removal benefits requires costly paired policy rollouts with and without the corruption present. To address this, we introduce a factorized risk model that learns separate representations of the scene and corruptions, allowing us to estimate removal benefits through latent intervention from the current image and task instruction, without paired clean images or additional counterfactual rollouts. We evaluate our approach on OpenVLA and $π_{0.5}$ in simulation and $π_{0.5}$-DROID on a Franka robot. Our method outperforms monolithic and vision-language-model (VLM) baselines, including achieving approximately 3x lower attribution error than frontier VLMs. We also demonstrate our method adapts to execution context and corruption intensity, and can be used for online failure monitoring on hardware.
comment: 8 pages, 7 figures
★ On Maximum Clique and Maximum Consensus Formulations in Point Cloud Registration
Maximum clique and maximum consensus are widely used formulations for robust point cloud registration in robotics. Maximum consensus seeks one transformation that aligns as many matches as possible, while maximum clique selects matches through pairwise distance tests. Maximum clique is often used as a substitute for or an approximation to maximum consensus, with empirical evidence suggesting a close relationship. However, how closely the two formulations agree is not fully understood. We study this relationship theoretically. We show that a maximum clique can have arbitrarily large alignment error and be arbitrarily larger than the maximum consensus set at any fixed positive tolerance, and derive geometric conditions that bound this error. When rotation is fixed, we derive sharp error bounds and tests that exactly characterize maximum consensus for the translation subproblem. Finally, we study group-$k$ consistency, which tests groups of $k$ matches, and determine the smallest group sizes that guarantee alignment error bounds independent of point cloud scale and geometry. These are $d+1$ for rotations and translations and $d+2$ when reflections are allowed, in dimensions $d\ge 2$. Interactive examples and proof verification using the Lean theorem prover are available at https://ariarobotics.github.io/mc-vs-mc-registration/.
comment: 8 pages, 7 figures. Interactive examples and Lean proof verification: https://ariarobotics.github.io/mc-vs-mc-registration/
★ XoNav: An Exoskeleton as a Personalized Haptic Interface for Assisted Navigation
Walking without vision requires timely, unambiguous guidance, yet many wearable aids encode directions as audio or vibrotactile signals that users must translate into movement. We present XoNav, a four-actuator hip exoskeleton that renders guidance cues directly at the hip joints. It renders an Advance cue and two-level Braking cues through sagittal-plane actuators and two-level Steering cues through frontal-plane actuators. A bilateral LSTM-based gait-state classifier uses two thigh-mounted IMUs to identify gait states, enabling haptic cues to be delivered in coordination with the wearer's gait while suppressing them during standing. The classifier generalized to unseen users with recalls of 95.2-97.9%, with mean detection delays of 38.6 and 63.1 ms for stance-to-swing and swing-to-stance transitions, respectively. Constrained human-in-the-loop Bayesian optimization personalized Braking for minimum resistance and Steering for maximum comfort while ensuring detectability and safety. All participants reliably distinguished between grades of their personalized cues and the firmest Braking cue remained below 53% of the safety limit. In blindfolded Wizard-of-Oz navigation trials along three paths, the participants kept 98.8% of the walked distance within a 0.75 m wide passage, with a 4.9% increase in mean progression speed per trial. A body-worn camera, RTAB-Map and Nav2 further enabled a feasibility study of autonomous navigation and detours around previously unmapped obstacles.
comment: A supplementary video illustrating the XoNav hardware, haptic cues, Wizard-of-Oz trials and autonomous navigation is available at https://doi.org/10.6084/m9.figshare.34701648
★ EmbodyHOI: Embodiment-Aware Generation of Hand-Object Interactions for Humanoid Dexterous Manipulation
Humanoid dexterous manipulation needs reference motions that are both semantically rich and kinematically feasible for the target robot. Text-conditioned hand-object interaction (HOI) generators supply diverse, language-specified demonstrations, but they synthesize motion in human (MANO) space. The embodiment gap is therefore deferred to post-hoc retargeting, where trajectories deployed on the robot can fail to maintain contact with objects and also degrade policy learning. Our key idea is that robot reachability should constrain the interaction trajectories while they are being generated. We present EmbodyHOI, which runs a differentiable inverse-kinematics (IK) solver for a specific humanoid embodiment inside the sampling loop of a frozen flow-matching HOI trajectory generator. We feed back gradients from the embodiment-aware IK solver into the HOI sampler and jointly optimize the position of the humanoid with respect to the object. Because guidance acts only at inference, one HOI trajectory generator trained on human hand poses transfers across humanoid embodiments without retraining. On ARCTIC trajectories with two contrasting humanoids (Unitree G1 and H1-2), our proposed guidance-based method cuts the retargeted wrist error from 8.9 cm to 0.6 cm while leaving text-to-HOI generation quality intact, and raises task success from 68% to 79% on 50 random G1 tasks and from 62% to 76% on a four-task H1-2 suite. Critically, cheaper alternatives for bridging the embodiment gap fail to beat EmbodyHOI. Post-hoc wrist refinement and compute-matched best-of-N selection both reach lower IK energy than our guidance yet train worse policies. For simulation results, see our website: https://embodyhoi.github.io/
★ Which Depth Estimator for Jetson AGX Orin? A 2026 Benchmark of Monocular and Stereo Methods
The Jetson AGX Orin is a common computer for robots and vehicles, yet measured accuracy, speed and energy of current depth estimators on it are hard to find. We benchmark nine model families as of October 2026: the monocular Depth Anything V2 and 3, Depth Pro, MoGe-2, Metric3D v2 and UniDepth V2, and the stereo methods libSGM, NVIDIA ESS and Fast-FoundationStereo. Every method runs on the Orin as a TensorRT FP16 engine or native CUDA code at up to three input sizes, up to the full 5.7 MP, and is scored for metric accuracy against Middlebury 2014 ground truth (non-occluded pixels of the 10 scenes whose disparity stays below 256 px), GPU time, energy and licence. Stereo is far more accurate. At 30 fps, NVIDIA's ESS reaches 0.70% AbsRel in 8.6 ms and libSGM 0.73% in 25 ms; with more compute, Fast-FoundationStereo reaches 0.33%. The best monocular models reach 12% with the true focal length (Depth Anything 3 Metric-L in 33 ms, Depth Pro in 803 ms), and 3.8-4.0% only with an oracle per-image scale (MoGe-2-L, UniDepth V2-B, Metric3D v2-L). Stereo error falls as the input resolution grows; no monocular model improves, and the metric scale of most drifts away from the truth outside their training resolution. At depth discontinuities, which AbsRel barely registers, the monocular models scatter 17-46% of the edge points between foreground and background, learned stereo 2-9% and libSGM at full resolution 1%. We give recommendations per frame-rate budget and release all code, per-scene results and engine build scripts.
comment: 18 pages, 5 figures, 8 tables. Code and per-scene results: https://github.com/nodarhub/orin-depth-benchmark
★ Learning to Be Different: Component Specialization for Compositional Diffusion Policies
Diffusion policies with linearly composed scores have emerged as a promising approach to multitask robotic learning by combining separate factors to enable modular, task-wise adaptation, compositional generalization, and interpretable functional decomposition. However, current training objectives do not encourage factor specialization, often resulting in uninterpretable denoisers whose composed prediction is an average over identical copies, undermining their architectural premise. To address this, we propose a drop-in score-decorrelation auxiliary loss, motivated by projective composition theory. Our approach encourages specialization by penalizing overlap directly between component noise predictions, with little architectural change or computational overhead. We demonstrate across multiple benchmarks that our decorrelation loss consistently reduces measurable double-counting between components, improves task decomposition, and produces behaviorally meaningful specialization without compromising multitask performance. Website: https://al3cl1.github.io/robotics-score-decorrelation.
★ Visual Task Modeling and Automated Planning for Humanoid Robots in Education of Individuals with Autism Spectrum Disorder
In robot-assisted education, a robot must understand how to complete a task in order to effectively guide a student through it. We present a novel method that enables a semi-autonomous, camera-equipped robot to recognise the task goal and plan actions to achieve it. Our work focuses on Shoebox Tasks, an activity commonly used in the education of individuals with Autism Spectrum Disorder (ASD). These tasks are designed so that the goal can be inferred directly from their visual structure. We have developed a solver that uses a vision-based Large Language Model to convert photos of a Shoebox Task into a Planning Domain Definition Language (PDDL) representation and generate action plans for solving it. We have also implemented a robotic interface that enables a Nao robot to autonomously execute various Shoebox Tasks and assist a student by suggesting next steps or signalling detected failures through action demonstrations, voice commands, or gestures. With an adapted robotic interface, the solver can be integrated into a variety of educational robots. No programming expertise is required from teachers, and the solver operates exclusively on captured images as input. Research in special educational needs indicates that students with ASD can benefit substantially from robot-assisted teaching, and our system aims to strengthen these learning opportunities while also fostering interest in robotics among individuals with ASD.
comment: 6 pages, presented at conference AICCONF 2026 in Prague, Czech Republic
★ Towards Interaction State Recognition in Sensorized Gecko-Inspired Adhesive Interfaces for Orbital Servicing
Reliable contact establishment remains a critical challenge for autonomous on-orbit servicing (OOS), particularly when interacting with non-cooperative targets. Gecko-inspired dry adhesives provide a lightweight, passive, and residue-free solution for robotic capture in space, yet current systems operate in open loop, without the ability to estimate their own contact or adhesion state during interaction. This paper presents a real-time interaction-awareness framework that enables sensorized gecko-inspired adhesive interfaces to estimate their contact state throughout the adhesion lifecycle using embedded capacitive sensing. Five discrete interaction states (idle, proximity, contact, engaged, and disengaged) are classified using experimentally collected and augmented sensor data, enabling a comparative evaluation of six lightweight machine learning models suitable for embedded deployment. Among the evaluated models, Random Forest achieved the highest offline classification accuracy of 95.65%. For online deployment, combining Random Forest with a rule-based state machine improved accuracy from 78.02% to 93.14%, achieving a macro F1-score of 0.9343 while maintaining a 53 ms inference cycle. These results demonstrate that incorporating physics-consistent state constraints significantly improves robustness during realtime inference, representing a step toward interaction-aware capture in future autonomous orbital servicing missions.
★ Physical AI: Concepts, Techniques, and Challenges
In recent years, physical artificial intelligence (physical AI) has experienced rapid development, capturing significant attention in academia and industry. However, the field has yet to reach consensus on its definition, technical landscape, and core challenges. To address this gap, this monograph seeks to define physical AI, present its technical landscape, and examine each key technical subfield in depth. In doing so, it establishes a unified framework for Physical AI: artificial systems that perceive, model, and act upon the physical world through continuous feedback under real-world constraints. Further, this monograph distinguishes physical AI from conventional digital AI, embodied AI, cyber-physical systems, and digital twins. It develops the foundations of physical AI across physics, control, cognition, learning, and reasoning. It reviews key methodologies, including multimodal perception, physical data and simulation, world models, embodied foundation models, motor-control systems, multi-agent coordination, and hardware-software co-design. Furthermore, it examines applications spanning robotics, manufacturing, infrastructure, scientific experimentation, and autonomous services. Particular attention is given to sim-to-real transfer, reliability, real-time execution, evaluation, safety, governance, and long-term deployment. By connecting perception, prediction, decision-making, action, and adaptation in a closed loop, this monograph identifies the technical challenges and research directions required to advance from narrow automation toward more generalizable physical AI and, ultimately, physical artificial general intelligence (Physical AGI).
comment: Monograph, 12 chapters, 388 pages, 64 figures
★ ACE-Brain-0.6: Scaling Spatial Grounding for Physical Foundation Models
General-purpose multimodal models, including GPT-6 Astra and Claude Opus 5.5, demonstrate broad capabilities in embodied manipulation. Our analysis indicates that this generality is accompanied by strong spatial perception, as Astra can determine where to interact with task-relevant entities amid visual distractors. Because spatial perception connects high-level reasoning to the physical scene, we investigate scaling visual grounding to strengthen the perceptual foundation of general embodied operation. We develop a unified data pipeline that combines robot manipulation recordings with complementary visual data from diverse sources. After automatically annotating the robot recordings, the pipeline converts the collected data into verifiable grounding supervision. Fine-tuning Qwen3.5-9B on approximately 1.5 million samples produced by this pipeline yields ACE-Brain-0.6. The model outperforms GPT-6 Astra on four of seven public grounding benchmarks, with an overall average score of 75.41 compared with Astra's 73.68. Under a fixed Qwen3.5-9B fine-tuning setup, aggregate grounding performance improves steadily as successive data releases expand the training collection to cover a broader range of grounding scenarios. These results show that scaling verifiable grounding supervision from diverse sources can strengthen the spatial perception of an open 9B model for general embodied operation.
★ Embodied Playground: A Long-Horizon Agentic Robotics Benchmark for Physical, Strategic, and Spatial Reasoning
Foundation models offer a promising basis for robotic agents, but long-horizon tasks require them to sustain coherent reasoning and reliable physical execution across interdependent stages. We introduce Embodied Playground, a benchmark for evaluating how foundation-model agents coordinate physical, strategic, and spatial reasoning throughout such tasks. Agents must track task progress, satisfy intermediate constraints, and revise their plans using execution feedback as earlier actions shape subsequent decisions. The playground comprises 410 task environments spanning basic manipulation, sequential puzzles, and multi-stage household activities, with variations in planning depth, geometric constraints, and physical execution demands. We curate EP-100, a representative selection of 100 tasks across five suites and 16 task types, together with the EP-Easy and EP-Hard difficulty splits and the compact EP-20 selection. A shared protocol specifies observation and tool access, resource budgets, repeated trials, and independent checks of goal completion, task-rule compliance, and physical validity. Embodied Playground provides a unified framework for studying whether foundation-model agents can sustain effective reasoning and action over extended task horizons. https://winniechen2002.github.io/Agent-bench-website/
comment: 38 pages, 2 figures
♻ ★ Sequential Planning via Anchored Robotic Keypoints
Changes in a robot's observations do not automatically imply changes in the task it is solving, but language-conditioned manipulation systems often fail to separate the two. If an object moves, for example, the locations at which actions should be executed may change while the underlying manipulation plan remains valid. We introduce Sequential Planning via Anchored Robotic Keypoints (SPARK), which separates a task's symbolic plan from its execution-time perceptual grounding, and we experimentally isolate whether a fixed symbolic plan can survive substantial changes to the scene through execution-time re-grounding. SPARK represents the task as a symbolic behavior tree whose object references are grounded to current 3D observations during execution, allowing the robot to update where actions are performed while preserving the plan. Runtime verification then enables local recovery and renewed grounding, only resorting to a new plan if necessary. To test the separation between plan and perceptual-grounding, we perform a set of manipulation experiments with a fixed plan while moving objects during execution. Under a 10 cm displacement, verification and adaptation maintain $66.7\%$ task success, compared with $2.2\%$ when disabled, demonstrating the importance of the underlying principle. Without task-specific model training and with an open-weight planner, SPARK also achieves $66.8\%$ mean success across six LIBERO-PRO settings, and $68\%$ across eleven physical task-embodiment settings on three robot platforms. Together, these results show that robustness does not require re-planning the task and for an important class of failures, it is sufficient to preserve the plan and update the system's perception of a changing world. Code and videos can be found at \url{https://cwru-aism.github.io/spark-page/}
comment: 29 pages, 7 figures
♻ ★ LLM-Flock: LLM-Driven Decentralized Multi-Robot Flocking via Influence-Based Consensus with Collision Avoidance
Large Language Models (LLMs) have shown strong capabilities in reasoning and planning, prompting researchers to explore their use as decentralized decision-makers for multi-robot formation control. However, directly applying LLMs to such tasks often leads to unstable behavior, in which robots may collapse to the centroid of their positions or diverge entirely, due to hallucinated reasoning and limited coordination awareness. We propose LLM-Flock, a decentralized multi-robot flocking framework that addresses these challenges through two key mechanisms: (1) an influence-based consensus protocol, where robots iteratively refine individually generated formation plans by adopting those of more connected neighbors, driving the team toward a unified plan without centralized control, and (2) a two-layer collision avoidance strategy that pairs LLM-based urgency reasoning with a programmatic collision checker and re-prompting mechanism. We evaluate LLM-Flock across five state-of-the-art LLMs, including both closed-source (GPT-5, Claude 4.5) and open-source (Llama3.1-405b, Qwen-Max, DeepSeek-R1) models, demonstrating notable improvements in formation stability and convergence over prior LLM-based methods. Real-world experiments with Crazyflie drones further validate the framework's applicability to physical multi-robot systems.
♻ ★ Has GPT-6-Astra Achieved Embodied Intelligence?
GPT-6-Astra demonstrates a level of competence in robotic manipulation and navigation that challenges the conventional paradigm of building specialized models for individual tasks and embodiments. In this report, we ask not only how capable GPT-6-Astra is, but what its capabilities imply for the field. First, we probe the limits of standalone GPT-6-Astra. We conduct a large-scale evaluation across 12 representative benchmarks spanning tabletop manipulation, humanoids, mobile platforms, and dexterous hands. Under a unified protocol that allows in-context learning (ICL) from training-set demonstrations and substantially increases the number of execution steps, GPT-6-Astra achieves unprecedented performance (Fig. 1). Future research may focus on improving 1) the generality of ICL 2) cost and execution time, 3) physical-state verification, 4) recovery from irreversible failures, and 5) long-horizon preservation of progress. Second, we study the agentic harness for embodied control. We identify that close-loop coordination between a high-level "System 2" reasoner (i.e., GPT-6-Astra) and a low-level "System 1" controller (e.g., π0.5) is costly but effective. Third, we examine GPT-6-Astra on real-world deployment. Our experiments on 8 robot configurations reveal substantial gaps: 1) robot demonstrations required for ICL are expensive to collect, and 2) it is implausible to trade longer execution for higher success rates with physical constraints. Fourth, we envision the next generation of embodied benchmarks. We develop a prototype that 1) derives tasks from real-world needs, 2) provides no in-domain demonstrations, and 3) judges agents on a world clock that keeps running while they deliberate. Taken together, our results suggest that GPT-6-Astra has largely crossed the threshold of embodied competence, but not yet that of practical embodied intelligence.
comment: Homepage: https://sensenova.ai/has-gpt-6-astra-achieved-embodied-intelligence. Authors are ordered alphabetically by their surnames
♻ ★ VQActFlow: Vector-Quantized Action Mode Steering for Multi-Task Robot Manipulation
Multi-task robot manipulation policies are challenging to learn from demonstration because traditionally a single network must select among qualitatively different action modes from a multimodal demonstration distribution, conditioned on language and visual context. A wrong mode selection means executing the wrong task or an action infeasible in the scene. Tokenizing continuous actions into a learned discrete codebook separates these modes at the representation level, offering structural advantages for multi-task learning. We propose VQActFlow, a multi-task manipulation policy that tokenizes action chunks and generates code sequences via Variational Flow Matching. VQActFlow maintains an explicit preference over action modes throughout generation. Inference-time guidance acts on this preference to steer mode commitment. We instantiate this with classifier-free guidance over language conditioning, which steers the policy toward the instructed action mode, and a learned codebook critic that supplies a complementary feasibility signal. We evaluate VQActFlow on three platforms: the LIBERO simulation benchmarks, a Unitree G1 humanoid performing whole-body pick-and-place, and an ALOHA-style bimanual platform performing contact-rich tasks. Across these benchmarks, VQActFlow outperforms both continuous and discrete baselines.
comment: 8 pages, 7 figures, 5 tables; Accepted at IEEE RA-L, 2026; Project website https://vqactflow.github.io/vqactflow/
♻ ★ Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models NeurIPS 2026
A fine-tuned Vision-Language-Action (VLA) policy can easily place the alphabet soup in the basket on command, just as it saw during fine-tuning. However, if the basket is shifted five centimeters to the left, it drops the soup off the table. We developed an interpretability pipeline to investigate what such a policy is computing. Injecting task A's activations into the action expert while the policy looks at task B's scene makes pi0.5 execute task A's motor trajectory in 99.6\% of episodes ($n{=}1{,}968$). The injected program is bound to absolute workspace coordinates and ignores the visible scene, which provides a mechanistic account of the perturbation brittleness that concurrent benchmarks report. The binding holds on a physical UR10e: directed replay redirects the commanded task in 26/26 rollouts in one direction and 18/26 in the other, with no-injection and random-direction controls at the floor, and once the source object is moved 10--15\,cm the arm grasps and releases at the recorded coordinates in 50/50 trials, picking up the wrong block when one sits there and closing on empty air when none does. Applying the same interventions to six VLAs on LIBERO, MetaWorld, SimplerEnv, and ALOHA over 420{,}000+ rollouts yields three further findings: every architecture encodes the instruction, and four of the five weight it below vision whenever the scene alone identifies the goal; under the standard replacement write-back the SAE pooling that preserves behavior is architecture-specific (pi0.5 per-token, X-VLA mean-pool, SmolVLA indifferent), and goal selection and motor generation separate into VLM and expert pathways wherever the two are distinguishable. We release \textbf{Action Atlas} (https://action-atlas.com), an interactive platform covering all 520+ trained SAEs, 79 identified concepts, and the scored UR10e rollouts. Code: https://github.com/CWRU-AISM/action-atlas
comment: NeurIPS 2026 Poster, ICLR Multimodal Intelligence Workshop Oral, 46 pages, 14 figures
♻ ★ GROW$^2$: Grounding Which and Where for Robot Tool Use
Can the robot use a plate to cut a cake if no knife is available? Tool use greatly expands robot capabilities, but to use tools creatively beyond their intended functions, the robot faces the challenge of $\textit{open-world affordance grounding}$: select an open-category object to act as a tool and localize its specific region of action. To this end, we introduce GROW$^2$ (GROunding Which and Where), which leverages object parts as a natural abstraction to split the grounding process hierarchically into semantic and geometric levels, thus bypassing the need for data-heavy, end-to-end training. Semantically, GROW$^2$ harnesses the commonsense reasoning of Vision-Language Models (VLMs) to parse a natural-language task instruction, select a suitable object as the tool, and identify task-relevant parts on the tool and the target object. Geometrically, vision foundation models then ground the selected parts into precise 3D regions from a single RGB-D image. Experiments on established benchmarks show that GROW$^2$ outperforms state-of-the-art baselines on affordance prediction benchmarks. Further, it achieves zero-shot generalization over open-category objects and outperforms baselines in both simulated and real-world robot tool use experiments.
comment: CoRL 2026
♻ ★ VLA-Pro: Cross-Task Procedural Memory Transfer for Vision-Language-Action Models NeurIPS 2026
Vision-Language-Action (VLA) models have shown strong potential for general-purpose robotic manipulation, yet they still struggle to generalize to unseen tasks that necessitate transferring relevant experience across objects, scenes, and action patterns. This paper proposes VLA-Pro, a plug-and-play framework designed to enhance cross-task generalization by storing task-relevant procedural memories at training time and transferring these memories during inference. Specifically, VLA-Pro stores task-specific LoRA adapters as parameterized procedural memories during training. At inference time, VLA-Pro retrieves relevant procedural memories based on the current multi-modal context and dynamically fuses these memories for generating the current action chunk. Experiments on RoboTwin, RLBench, and real-world manipulation tasks show that VLA-Pro consistently improves cross-task generalization across multiple backbones, achieving up to a 207% relative improvement in simulation and increasing real-world success rate from 5.8% to 65.0%. These results suggest that procedural memory retrieval and adaptation provide an effective mechanism for transferring manipulation experience to novel tasks while preserving modularity and execution stability.
comment: Accepted by NeurIPS 2026 (Poster)
♻ ★ MOSS: Multimodal Origami Swamp Robot System ICRA 2027
Robotic systems for environmental monitoring and ecological surveying in wetlands and swamps must operate across diverse terrestrial and aquatic conditions. Conventional fixed-geometry wheels cannot adapt their contact geometry to these varying conditions. This work presents MOSS (Multimodal Origami Swamp Robot System), a compact amphibious robotic platform with 3D-printed origami wheels. The compact crank mechanism switches the wheel between contracted and expanded configurations, changing its width, body diameter, and fin orientation simultaneously. The contracted configuration is better suited to efficient rolling on firmer ground, whereas the expanded configuration provides greater contact and terrain engagement on softer surfaces. MOSS was evaluated on soil, sand, mud with varying water content, and water, in addition to in-motion morphing and terrestrial-aquatic transition tests. The more energy-efficient wheel configuration depended on terrain. The contracted configuration achieved a lower cost of transport on firmer soil and mud, whereas the expanded configuration performed better on wetter mud, sand, and water, reducing the cost of transport by up to 29%. Expanded wheels also reduced slip in wet mud and reduced both slip and hub sinkage in sand. MOSS successfully completed configuration changes in $1.95 \pm 0.11$ s and demonstrated continuous locomotion across soil-mud-water transitions. Outdoor trials further demonstrated operation across natural deformable terrain and water. These results show that adaptive wheel geometry provides terrain-dependent locomotion benefits and enables a single compact platform to traverse substantially different terrestrial and aquatic conditions.
comment: 8 pages, 8 figures, submitted to ICRA 2027
♻ ★ UniW(A)M: A Unified Mixture-of-Experts Framework for World and World-Action Models
World models (WMs) and world-action models (WAMs) focus on future world prediction and executable action generation, respectively. Although these models learn shared physical dynamics and interaction semantics from embodied trajectories, they are typically trained in isolation, which restricts knowledge reuse between prediction and control. The central challenge is therefore to develop a unified learning framework that shares these dynamics while preserving the distinct modeling capabilities required by each paradigm. We propose UniW(A)M, a mixture-of-experts world-(action) model built upon diffusion Transformers that unifies action-conditioned world modeling and diverse WAM variants. Shared experts capture common dynamics, while atomic capability experts are reused across interfaces so that each interface keeps its own conditional pathway. At deployment, a plug-in adaptive interface router (PAIR) learns from a small subset of target-task chunk scenarios to select a suitable WAM interface for each execution chunk, balancing task success and execution efficiency while keeping the backbone frozen. We evaluate UniW(A)M on LIBERO, RoboTwin 2.0, and three real-world manipulation tasks, where it achieves 88.3% average real-world success versus 76.7% for the strongest baseline. On WorldArena, it leads action-conditioned world modeling on all three tasks. AC-WM co-training contributes to action generation on every action interface and PAIR raises OOD success from 63.3% to 76.7%.
comment: 34 pages, 19 figures, 15 tables
♻ ★ Move First, Commit Later: Selective LiDAR-to-BIM Global Initialization by Verifying Proposals Against Competing Explanations
Global LiDAR-to-BIM initialization must place a robot within an as-designed building model without a prior pose. In repetitive interiors, the principal failure mode is not low-confidence registration but confident aliasing: a submap can match several BIM regions with comparable scores, producing a high-scoring pose displaced by symmetry. We present Move First, Commit Later, a selective layer that treats a registration front-end as an evidence source and decides whether to commit. Candidates from multiple submaps are mapped to a common $\mathrm{SE}(2)$ anchor; a top-1 consensus $B_m$, invariant to non-champion multiplicity, aggregates cross-submap evidence; and topology serves only as a binary feasibility gate. The decision is typed--COMMIT, DEFER, or AMBIGUOUS($τ$), reporting the detected symmetry period--and reversible: symmetry-breaking motion upgrades AMBIGUOUS to COMMIT. On a real multi-room building and a controlled symmetric simulation, the layer commits correctly in every tested trajectory-scale configuration and otherwise abstains with a typed state, whereas forced-choice policies on the same front-end select wrong rooms in most cases. Committed anchors are within 0.02--0.36 m of an independent laser-tracker position reference and within $1.9^\circ$ of a BIM-registration orientation proxy. The evaluation covers one building and one front-end; the layer is designed to be front-end modular.
comment: 8 pages, 5 figures, 2 tables. Substantially revised version with an updated method and expanded evaluation
♻ ★ VLX-Go: Robust Embodied Tracking through Multi-Stage Learning and On-Policy Refinement
Embodied visual tracking requires a robot to identify a designated target from visual observations and control its own motion to follow that target through an environment. Following an isolated person is a comparatively simple demonstration of embodied visual tracking, but following the intended person among distractors remains challenging. Existing methods exhibit a substantial drop in success from single-target to distractor tracking, motivating a focus on the shared spaces that person-following robots must navigate. We present VLX-GO, a vision-language waypoint policy built around the hypothesis that visual and language representations should be aligned before tracking-specific motion is learned. The policy combines complementary frozen visual encoders, multiscale temporal context, a transformer backbone, and a lightweight head for robot-centric waypoint prediction. Its training progresses through vision--language alignment, multimodal reasoning mid-training, and tracking specialization on 19,532 oracle episodes. Compared with 41.54% distractor-tracking success for our earlier OmTrackVLA model, the complete multi-stage recipe achieves 55.80%. We then explore on-policy refinement using filtered oracle labels for states visited by the policy. This preliminary study adds 2.42 percentage points of distractor-tracking success, reaching 58.22%, while reducing collision rate by 1.99 pts. The final policy records the highest distractor- and ambiguous-tracking success rates among the compared methods. These results support staged multimodal learning as a promising direction for tracking among distractors and motivate further study of on-policy refinement for person following in shared spaces. Code is available at https://github.com/om-ai-lab/VLX-Go.
comment: tech report
♻ ★ Humanoid Badminton: Learning Dynamic Racket Skills from Limited Human Motion Data
High-speed racket sports provide a demanding testbed for humanoid robots, requiring time-critical decisions, precise striking, and dynamic whole-body coordination. In badminton, fast-changing shuttle trajectories require timely contact decisions, while successful returns demand precise racket pose and velocity within a brief contact window and across a broad three-dimensional striking workspace. Human motion data provide valuable priors for such athletic skills, but usable badminton references are limited and imperfect. Direct tracking provides insufficient executable variation for diverse shuttle conditions, while purely task-driven optimization may produce unnatural motion. We therefore present a three-stage hierarchical reinforcement learning framework for dynamic humanoid badminton. First, task-randomized motion augmentation expands sparse annotated hitting events into executable target-conditioned stroke variations, forming a continuous latent skill space. Second, a high-level planner outputs continuous latent skill codes to compose these skills online according to the observed shuttle state. Third, a context-conditioned adversarial regularizer encourages more natural planner-level skill usage while preserving return performance. In real-world deployment, our system sustains multi-skill rallies with human players, including forehand, backhand, and highly dynamic jump returns. This is the first real-world humanoid racket-sport system to demonstrate multi-skill human--robot rallies including highly dynamic jump returns.
comment: Accepted at the 10th Conference on Robot Learning (CoRL 2026)
♻ ★ BioProVLA-Agent: An Affordable, Protocol-Driven, Vision-Enhanced VLA-Enabled Embodied Multi-Agent System with Closed-Loop-Capable Reasoning for Biological Laboratory Manipulation
Biological laboratory automation can reduce repetitive manual work and improve reproducibility, but reliable embodied execution in wet-lab environments remains challenging. Protocols are often unstructured, labware is frequently transparent or reflective, and multi-step procedures require state-aware execution beyond one-shot instruction following. Existing robotic systems often rely on costly hardware, fixed workflows, dedicated instruments, or robotics-oriented interfaces. Here, we introduce BioProVLA-Agent, an affordable, protocol-driven, vision-enhanced embodied multi-agent system enabled by Vision-Language-Action (VLA) models for biological manipulation. The system uses protocols as the task interface and integrates protocol parsing, visual state verification, and embodied execution in a closed-loop workflow. A Tailored LLM Protocol Agent converts protocols into verifiable subtasks; a VLM-RAG Verification Agent assesses readiness and completion using observations, robot states, retrieved knowledge, and success/failure examples; and a VLA Embodied Agent executes verified subtasks through a lightweight policy. To improve robustness under wet-lab visual perturbations, we develop AugSmolVLA, an online augmentation strategy targeting transparent labware, reflections, illumination shifts, and overexposure. We evaluate the system on a hierarchical benchmark covering 15 atomic tasks, 6 composite workflows, and 3 bimanual tasks, including tube loading, sorting, waste disposal, cap twisting, and liquid pouring. Across normal and high-exposure settings, AugSmolVLA improves execution stability over ACT, X-VLA, and the original SmolVLA, especially for precise placement, transparent-object manipulation, composite workflows, and visually degraded scenes. These results suggest a practical route toward accessible, protocol-centered, and verification-capable embodied AI for biological manipulation.
comment: 17 pages, 10 figures
♻ ★ GemNav: Discrete-Token Visual Robot Navigation using a Multimodal Large Language Model
Visual navigation policies built on large pretrained models increasingly rely on large cross-embodiment datasets and expressive action-generation mechanisms. In this paper, we ask whether a pretrained Multimodal Large Language Model (MLLM) can be adapted efficiently for robot navigation by rethinking the representation and construction of navigation actions. We introduce GemNav, a visual navigation policy that represents each robot-local waypoint coordinate as a categorical distribution over discrete metric value bins and soft-decodes it into a continuous coordinate for execution. This probabilistic representation retains likelihood information over alternative coordinate values while providing a compact action space suitable for robot navigation. We further construct training trajectories using metric-distance goal sampling rather than fixed future frame offsets, exposing the policy to varying physical navigation horizons independent of dataset frame rate and robot speed. GemNav is trained on a single 8.7-hour public SCAND dataset, roughly three orders of magnitude smaller than competing cross-embodiment datasets. Extensive offline and closed-loop experiments show improved out-of-distribution generalization over frame-based sampling and substantially more reliable obstacle navigation than a matched continuous regression head. The same policy transfers without per-platform retraining across unseen environments and three physically distinct robot platforms.
♻ ★ Mono-Hydra++: Real-Time Monocular Scene Graph Construction with Multi-Task Learning for 3D Indoor Mapping SP
Autonomous agile robots need more than metric geometry: they must understand objects, rooms, places, and spatial relations for search, inspection, exploration, and human-robot interaction. Conventional metric maps support localization and collision avoidance, but do not provide this semantic and relational structure. 3D scene graphs address this gap by connecting geometry with object-level and room-level understanding. Building such representations on agile platforms remains difficult because aerial and lightweight robots operate under strict payload, power, and compute limits, making RGB-D cameras and LiDAR sensors impractical for many onboard settings. We present Mono-Hydra++, a real-time monocular RGB plus IMU pipeline for indoor metric-semantic mapping and hierarchical 3D scene graph construction. The system combines M2H-MX, a DINOv3-based multi-task model for depth and semantics, with a deep-feature visual-inertial odometry front end, sparse predicted depth constraints in the VIO-derived pose graph, semantic masking for dynamic regions, and pose-aware temporal alignment before volumetric fusion in the Mono-Hydra backend. On the Go-SLAM ScanNet evaluation subset, Mono-Hydra++ achieves 1.6\% lower average trajectory error than the strongest RGB-D baseline in our comparison, while using only monocular RGB plus IMU input. On calibrated 7-Scenes, it improves average ATE by 29.8\% over the strongest competing calibrated baseline. We further validate Mono-Hydra++ in a real ITC building deployment using RealSense RGB plus IMU and demonstrate embedded feasibility by deploying the ONNX/TensorRT FP16 M2H-MX-L perception model at 25.53 FPS on a Jetson Orin NX 16GB. These results show that Mono-Hydra++ can provide real-time metric-semantic mapping and scene graph construction for resource-constrained robotic platforms without relying on active depth sensors.
comment: Submitted to ISPRS Journal of Photogrammetry and Remote Sensing. 50 pages, figures and tables included. Code: https://github.com/BavanthaU/mono-hydra-pp.git
♻ ★ RoboPocket: Improve Robot Policies Instantly with Your Phone
Scaling robot learning is bottlenecked not only by data volume, but also by whether the data targets the policy's failures. Existing robot-free handheld interfaces enable portable in-the-wild data collection, yet remain open-loop: users record demonstrations without feedback from the current policy. Interactive methods such as DAgger provide such feedback by collecting corrective on-policy data, but require deploying imperfect policies on physical robots, making them costly, unsafe, and difficult to scale. We present RoboPocket, a smartphone-based system for robot-free policy iteration. By combining on-device sensing, display, and communication with cloud-side policy inference, RoboPocket provides real-time collection guidance and visualizes the policy's predicted trajectory in augmented reality as AR Visual Foresight. This allows users to identify policy weaknesses and collect targeted corrective data without deploying a physical robot. An asynchronous finetuning pipeline continuously incorporates new corrections and updates the policy within minutes, realizing a DAgger-inspired robot-free correction loop. Across diverse real-world manipulation tasks, we validate RoboPocket as a scalable data engine and show that online corrective data collection improves sample efficiency by up to $2\times$ over scaling offline demonstrations. Project page and videos: https://robo-pocket.github.io.
comment: Accepted to CoRL 2026. Project page: https://robo-pocket.github.io
♻ ★ Learning from Demonstrations Meets Symbolic Control - A Supervisory Control Architecture for Formal Certification of Learned Motion
Dynamical system (DS)-based learning from demonstration generates flexible robot motions, but existing guarantees rely on certificate generation to obtain formal guarantees of safety and goal convergence. We propose a symbolic supervisory control architecture that automatically generates a fail-safe backup plan, certifying safety, reachability, and actuator feasibility directly from the robot's dynamics, independent of the learned DS. A nominal controller tracks the learned DS, while the supervisory controller monitors safety and target convergence, providing non-conservative intervention when the nominal controller violates necessary constraints. Experimental results include simulations and real robot deployment with various 2D and 3D datasets, including handwriting datasets, validating the effectiveness of our approach in providing formal guarantees with learned motion even in complex, dynamic environments.
♻ ★ GRAIL: Generating Humanoid Loco-Manipulation from World Models
Scaling humanoid loco-manipulation requires robot-compatible demonstrations across diverse objects, whole-body motions, and scene geometries, but teleoperation and motion capture are difficult to scale because each collection depends on physical setups, instrumented actors, and robot operation. We present GRAIL, a digital generation pipeline that remains fully virtual until deployment: it grounds video world models in 3D assets and simulator-ready scenes to synthesize interactions without rebuilding physical environments or teleoperating the robot. Rather than reconstructing unconstrained in-the-wild videos, GRAIL starts from fully specified 3D configurations in which object geometry, camera parameters, metric scale, environment depth, and a robot-proportioned character are known before video generation and reused during reconstruction. This privileged setup better conditions 4D recovery, allowing model-based object tracking, human motion estimation, and interaction-aware optimization to reconstruct metric 4D human-object interaction (HOI) trajectories with reduced depth ambiguity and morphology mismatch. We retarget the recovered motions to a humanoid robot and train complementary task-general trackers: an object-aware latent adaptor for manipulation and a scene-aware tracker for terrain traversal. GRAIL produces over 20,000 sequences spanning pick-up, whole-body manipulation, sitting, and terrain traversal. Using only GRAIL-generated data, we train egocentric visual policies through a sim-to-real pipeline and deploy them on a Unitree G1 humanoid, achieving 84% real-world success on diverse object pick-up and 90% success on stair-climbing.
comment: Project page: https://research.nvidia.com/labs/dair/grail/
♻ ★ TRACT: Temporally Routed Action Chunks with Chronological Phase Authority for Contact-Rich Manipulation
Action chunking shortens the effective decision horizon of robot imitation learning by predicting multiple future actions, while conventional phase conditioning describes the current control instant. When a predicted horizon crosses a procedural boundary, assigning the current phase to the entire chunk creates a structural temporal mismatch. We present TRACT, which factorizes phase-structured action chunking into an accepted current phase and a single CURRENT-to-NEXT boundary inside the future horizon. A task-local graph constrains chronological phase authority, and a cumulative boundary distribution monotonically routes future queries through phase-specific query and action paths. For contact execution, a causal response-deficit integrator compares policy intent with ACK-eligible subsequent motion, accumulates arm compensation when directional response is suppressed, and decays after confirmed recovery. Across six real-robot variants with ten trials each, full TRACT achieves 10/10 full-sequence success, 99.00 [88.75, 100.00]% median [min, max] wipe completion, zero observed phase ambiguity, and zero stalls. Under the current complete method package and evaluation setting, the routed representation obtains better observed task results than the flat package (6/10 vs. 3/10 success; 77.08% vs. 8.03% median wipe completion). Chronological authority reduces observed phase ambiguity from 8/10 to 0/10, and response integration reduces stalls from 4/10 to 0/10. The package comparison does not isolate routing from other generator-package differences.
comment: 6 pages, 3 figures, Accepted for presentation at the 2026 IEEE International Conference on Robotics and Biomimetics (ROBIO 2026)
♻ ★ From Optical Flow to Safety: Visual Safety Certification via Scale-Free Relative Motion
Safe autonomous navigation using monocular vision remains challenging because explicit obstacle states are not directly observable. This letter introduces a visual safety-certification paradigm that uses scale-free, depth-normalized relative motion encoded by monocular optical flow as the representation for collision-risk certification, without reconstructing obstacle position, velocity, geometry, or a metric world model. Unlike conventional optical-flow-based navigation approaches that rely on image motion to shape avoidance behavior, the proposed framework directly encodes collision-cone geometry and safety constraints in this relative-motion representation. An online Concurrent Learning observer estimates the normalized relative velocity and provides a computable estimation-error bound. This uncertainty is propagated into the collision cone through angular inflation and a monocular robot-footprint margin, yielding a Robust Vision-Based Collision Cone Control Barrier Function (RV-C$^3$BF). An optimal-decay CBF quadratic program minimally modifies the nominal control input to enforce the resulting robust safety constraint under the stated assumptions. Extensive simulation and real-world experiments on a TurtleBot3 in cluttered environments with static and dynamic obstacles demonstrate collision avoidance using only monocular vision and onboard proprioception, with the estimation-error bound containing the measured error in $98.3\%$ of simulation samples and the robot achieving collision-free navigation in $93$ of $100$ real-world trials.
♻ ★ Causal Scene Narration: Inference-Time Text Restructuring for Vision-Language-Action Driving
Vision-Language-Action (VLA) models for autonomous driving receive navigation commands and hazard notices as disconnected text fragments, forcing the model to discover on its own which environmental constraints are relevant to the current maneuver. We introduce Causal Scene Narration (CSN), which restructures VLA text inputs through intent-constraint alignment, quantitative grounding, and structured separation at inference time with zero GPU cost. To test CSN across different model states and safety layers, we evaluate it alongside training-time preference alignment (PL-DPO-NLL) and runtime supervision as experimental conditions. A multi-town CARLA evaluation (16 routes, 8 towns, N=5, 95% bootstrap CIs) shows that CSN yields relative improvements in CARLA Driving Score (DS, route completion weighted by infraction penalty) of +31.1% on original LMDrive and +24.5% on the PL-DPO-NLL variant. A controlled ablation separating information content from text organization reveals that information content carries most of CSN's gain across both model weights (+6.17 DS on original LMDrive; +6.88 DS on PL-DPO-NLL). While causal structure accounts for an estimated 39.1% (95% CI [6.6%, 71.4%]) of the gain on original LMDrive versus 13.5% on PL-DPO-NLL, the structural share on the aligned model and the cross-configuration asymmetry remain statistically inconclusive at N=5 due to wide bootstrap intervals. A perception noise ablation further confirms that CSN's benefit survives distance errors up to +/-5 m, speed noise up to +/-30%, and 20% actor miss rates. Finally, we report a negative result on combining CSN with runtime supervision: a naive direction-conflict monitor never fires on this benchmark and, when paired with CSN, degrades DS via passive control clamping that truncates CSN-guided evasive maneuvers. Code is available at https://github.com/LiYun0607/CausalSceneNarration.
comment: Published in Transactions on Machine Learning Research (TMLR), 2026
♻ ★ NestedVLA: Learning to Consolidate and Generate Skills for Vision-Language-Action Models NeurIPS 2026
Vision--Language--Action (VLA) models promise generalist robotic agents, yet consolidating skills learned across heterogeneous tasks and environments remains difficult: training data is fragmented across platforms, joint multi-task training suffers from interference, and model merging tends to introduce parameter conflicts or require architectural changes. We instead advocate a paradigm in which skills are first learned independently and then consolidated post hoc. To this end, we propose NestedVLA, which replaces parameter aggregation with parameter generation: a nested hypernetwork, conditioned on the target task, synthesizes task-adaptive action-expert parameters that compose with a frozen base VLA. Task-specific compressed VLM residuals are retained separately. This formulation enables flexible task-conditioned skill reuse while mitigating interference. Across three RL benchmarks (MetaWorld, ManiSkill, CALVIN), NestedVLA outperforms the strongest model-merging baseline by 41 pp on average, matches per-task fine-tuning within 1 pp on ManiSkill, and substantially closes the gap on CALVIN, pointing toward a storage-efficient route to skill consolidation. Code is available at https://github.com/WanyuGroup/NeurIPS2026-NestedVLA.
comment: Camera-ready version accepted at NeurIPS 2026. Updated to the final conference format and included the NeurIPS checklist. 30 pages, 3 figures. Code: https://github.com/WanyuGroup/NeurIPS2026-NestedVLA
♻ ★ DeReCo: Decoupling Representation and Coordination Learning for Object-Adaptive Decentralized Multi-Robot Cooperative Grasping and Lifting
Generalizing decentralized multi-robot cooperative grasping and lifting across objects with diverse shapes and physical properties remains a fundamental challenge. Under decentralized execution, two key challenges arise: object-dependent representation learning under partial observability and coordination learning in multi-agent reinforcement learning (MARL) under non-stationarity. A typical approach jointly optimizes object-dependent representations and coordinated policies in an end-to-end manner while randomizing object shapes and physical properties during training. However, this joint optimization tightly couples representation and coordination learning, which can introduce bidirectional interference: inaccurate representations under partial observability can degrade coordination learning, while non-stationarity in MARL can further degrade representation learning, leading to degraded task performance. Inspired by prior work on decoupling representation and control learning in single-robot settings, we propose DeReCo, a MARL framework that decouples representation and coordination learning for object-adaptive decentralized multi-robot cooperative grasping and lifting. DeReCo adopts a two-phase training strategy: (1) coordination learning with privileged object information, and (2) transition to decentralized execution through reconstruction of object-dependent representations from local observations and retraining of coordination policies using the reconstructed representations. Experimental results show that DeReCo outperforms baselines in simulation on three training objects, generalizes to six unseen objects with varying masses and friction coefficients under similar grasping configurations, and achieves superior performance on two unseen objects in real-robot experiments.
comment: 9 pages, 7 figures
♻ ★ CoEvolve: Self-Evolving Code Policies from Skills to Strategies for Humanoid Robots
Recent advances in Vision-Language-Action (VLA) models and World Action Models (WAMs) have expanded the manipulation and whole body capabilities of individual robots. Yet a single robot remains limited by its embodiment, and many real world tasks require capabilities that can only emerge through cooperation. Existing multi robot frameworks largely emphasize task decomposition, allocation, and replanning, while providing limited support for adapting executable coordination when physical execution deviates from the intended behavior. We present CoEvolve, a framework that represents team coordination as executable code over fixed robot motor skills. CoEvolve introduces Verifier Guided Coordination Evolution, where a multimodal foundation model instantiates coordination strategies through motion primitives, semantic objectives, temporary robot roles, and bounded execution parameters. Physical verification evaluates each executed transition, rejected behaviors trigger structured coordination revision from an accepted checkpoint, and verified experience is retained to guide subsequent decisions. We evaluate CoEvolve with two simulated Unitree G1 humanoids transporting a freely supported rigid payload through constrained corridors. Across 15 base task runs with three multimodal backbone families, CoEvolve completes 8 tasks, with every backbone achieving end to end success. Additional evaluations yield verified placement on two additional corridor layouts and after both same-backbone and cross-backbone experience initialization, while exposing persistent limitations in motion precision and contact stability. These results support executable coordination as a practical interface for adapting long horizon multi robot collaboration under shared physical constraints.
♻ ★ AnyNav: Visual Neuro-Symbolic Friction Learning for Off-road Navigation
Off-road navigation is critical for a wide range of field robotics applications from planetary exploration to disaster response. However, it remains a longstanding challenge due to unstructured environments and the inherently complex terrain-vehicle interactions. Traditional physics-based methods struggle to accurately capture the nonlinear dynamics underlying these interactions, while purely data-driven approaches often overfit to specific motion patterns, vehicle geometries, or platforms, limiting their generalization in diverse, real-world scenarios. To address these limitations, we introduce AnyNav, a vision-based friction estimation and navigation framework grounded in neuro-symbolic principles. Our approach integrates neural networks for visual perception with symbolic physical models for reasoning about terrain-vehicle dynamics. To enable self-supervised learning in real-world settings, we adopt the imperative learning paradigm, employing bilevel optimization to train the friction network through physics-based optimization. This explicit incorporation of physical reasoning substantially enhances generalization across terrains, vehicle types, and operational conditions. Leveraging the predicted friction coefficients, we further develop a physics-informed navigation system capable of generating physically feasible, time-efficient paths together with corresponding speed profiles. We demonstrate that AnyNav seamlessly transfers from simulation to real-world robotic platforms, exhibiting strong robustness across different four-wheeled vehicles and diverse off-road environments.
♻ ★ MAGS-SLAM: Monocular Multi-Agent Gaussian Splatting SLAM for Geometrically and Photometrically Consistent Reconstruction
Collaborative photorealistic 3D reconstruction from multiple agents enables rapid large-scale scene capture for virtual production and cooperative multi-robot exploration. While recent 3D Gaussian Splatting (3DGS) SLAM algorithms can generate high-fidelity real-time mapping, most of the existing multi-agent Gaussian SLAM methods still rely on RGB-D sensors to obtain metric depth and simplify cross-agent alignment, limiting their deployment on low-cost or power-constrained robotic platforms, especially given the wider availability of RGB cameras. To address this challenge, we propose MAGS-SLAM, the first RGB-only multi-agent 3DGS SLAM framework for collaborative scene reconstruction. Each agent independently builds local monocular Gaussian submaps and transmits compact submap summaries rather than raw observations. To facilitate robust collaboration in the presence of monocular scale ambiguity, our framework integrates compact submap communication, geometry- and appearance-aware loop verification, and occupancy-aware Gaussian fusion, enabling coherent global reconstruction without active depth sensors. We further introduce ReplicaMultiagent Plus, a benchmark containing larger robot teams for evaluating collaborative Gaussian SLAM. Extensive experiments on both synthetic and real-world datasets demonstrate that MAGS-SLAM, using RGB images alone, achieves tracking accuracy, rendering quality, and geometric quality that are competitive with or superior to those of state-of-the-art RGB-D collaborative Gaussian SLAM methods.
comment: Accepted to IEEE Transactions on Automation Science and Engineering (T-ASE) 2026, project website: https://mags-slam.github.io
♻ ★ FastVLA: Task-Conditioned and Temporally Adaptive Visual Token Compression for Efficient Vision-Language-Action Models WACV 2027
Vision-Language-Action (VLA) models present a promising paradigm for general-purpose robot manipulation, but closed-loop deployment is often limited by inference efficiency: at each timestep, high-resolution, often multi-view images are encoded into hundreds of visual tokens that dominate transformer computation and latency. Token compression via attention is a natural solution but is challenging for VLA control because (1) decision-critical tokens are determined by the task, not generic visual saliency, and (2) the token budget varies substantially across steps, so static metrics and budgets waste computation or discard control-critical tokens. We introduce FastVLA, a training-free, task-aware, and temporally adaptive visual token compression framework. At each step, FastVLA scores token importance via task-to-vision attention and adaptively sets the compression rate from the importance-distribution uniformity, measured by an inverse-Simpson metric with online sliding-window calibration. We evaluate FastVLA on OpenVLA, OpenVLA-OFT, and CogACT across LIBERO and SIMPLER, and on a physical Franka Research 3 robot. In simulation, it reduces LLM-backbone FLOPs by 35-59\% and action latency by 9-32\% (a 1.09$\times$-1.46$\times$ speedup) while maintaining task success, achieving a comparable or better efficiency-performance trade-off than existing plug-and-play token compression/pruning/reuse baselines. On the real robot, it cuts action-prediction latency by 20\% (1.25$\times$ speedup) at comparable success. Code is available at https://github.com/jiuyixu25/FastVLA.
comment: Accepted by WACV 2027
♻ ★ Readiness Barrier Functions: Forward-Invariant Control Authority for Overactuated Multirotor Allocation
Allocation schemes that greedily maximize a readiness metric over the actuator fiber bundle of an overactuated multirotor produce commands that jump between disconnected optimal strata, demanding actuator rates no motor can deliver; effort-minimizing schemes are continuous but cannot guarantee that wrench-rate authority stays above any certified level. We reconcile the two by treating authority as a forward-invariant quantity: a control barrier function on the log-determinant of the drag-aware actuator-authority co-metric, enforced at torque level by a quadratic program in the allocation null space. A single design inequality renders the certified set compact and strictly interior to the actuator box, with the readiness cost of any rotor deactivation given in closed form as $\ln(n/(n{-}m))$ for symmetric designs. An explicit alignment ratio states when tracking can stay exact, otherwise the wrench error is bounded by $\mathcal{O}(ρ^{-1/2})$, and a robust variant handles motor-parameter uncertainty with a closed-form floor shift independent of the airframe matrix. On a hexarotor and a fully-actuated octorotor the closed-form gap matches simulation to machine precision. Over $300$ randomized missions, greedy maximization violates the certified floor in up to $51\%$ of authority-scarce missions, while the proposed filter never leaves the certified set and, in the hardest moment reversal, commits a $79\times$ smaller wrench error. The robust variant holds the floor on all $150$ perturbed plants. A high-fidelity simulation with measured propeller and motor data shows that the floor holds under unmodeled aerodynamics, electrical limits, and delay, and where it stops applying.
comment: This work has been submitted to the IEEE for possible publication
♻ ★ RoboFake: Are Deepfake Detectors Ready for Humanoid Face Robots?
Humanoid face robots can now move their lips in sync with speech and show detailed facial expressions. Give such a robot the face of a real person and drive it with generated speech, and you get a new kind of presentation attack: a face that talks and reacts in front of the camera, recorded as a real object with no edited pixel. We call this a RoboFake and ask whether current defenses are ready for it. We test three public face presentation attack detection (PAD) models and six public deepfake detectors on video of a physical lip-syncing face robot, on a synthesized video of the same robot, and on rendered robot faces. Thresholds are fixed on human data (BPCER 5% on FaceForensics++), and every result is compared with real human faces processed the same way as the robot videos and turned to the same head angle. A small mobile PAD model detects every physical robot frame (APCER 0%) and is not affected by compression, but its BPCER on processed human video rises to 14%; the two PAD models trained on CelebA-Spoof miss the robot in the direct capture, and one of them also misses most panel frames. Among the deepfake detectors, three flag almost every robot frame and two fewer than 7%; masking shows that the response comes from the face but does not track synthesis, and JPEG compression pushes most robot frames below the threshold for some detectors. Our results suggest that current models catch today's robots because their faces do not look human, not because they recognize the attack. We state three testable hypotheses about realistic, interactive RoboFakes.
comment: 9 pages, 4 figures, 6 tables. Keywords: Deepfake detection, presentation attack detection, face anti-spoofing, humanoid robots, face robots, biometric security, robustness
♻ ★ Flow Control: Steering Vision-Language-Action Models with Simple Real-Time Inputs
We introduce flow control of vision-language-action (VLA) models, a simple and effective way to steer VLA actions in real time through generic inputs, such as a keyboard. This method can be used out-of-the-box and does not require retraining or fine-tuning VLAs. It enables relatively crude user inputs to steer a VLA to align with user intent. Flow control writes the user input into the initial condition of the VLA's flow matching action expert. Because the flow is deterministic, integration retains information about the user input while moving it toward the action distribution learned during training, so that the generated actions reflect the user's intent (high fidelity) and stay close to the VLA's own unsteered actions (high conformity). We demonstrate that flow control has many desirable properties: (1) flow control accurately and responsively steers robot actions with user inputs, (2) it is robust to suboptimal user inputs, (3) it enables users to steer VLAs to achieve significantly higher success rates and faster task completion, and (4) fine-tuning a VLA on flow control trajectories improves the autonomous policy. Together, these results provide a simple and intuitive way for users to help steer VLA actions, increasing task performance. Project page and videos: https://jcykao.github.io/flowcontrol/.
comment: Accepted to the 10th Conference on Robot Learning (CoRL 2026). 29 pages. Project page: https://jcykao.github.io/flowcontrol/
Multiagent Systems
★ Cooperative Multi-Agent Reinforcement Learning for Market Making on Prediction-Market Order Books
A market maker provides liquidity by quoting prices to buy and sell an asset, earning the spread while managing inventory risk. We study market makers trained with multi-agent reinforcement learning on prediction markets, specifically Polymarket, whose thin order books pose distinct challenges. Three challenges underpin this research: adverse selection may cause a policy trained on real data to stop quoting; the consequences of a quote can arrive many steps after the action; and prediction-market L2 data does not identify individual orders needed for simulation. We address these challenges by reconstructing order-level LOBSTER-format data from Polymarket and training agents in JaxMARL-HFT on 417 real markets. We enrich the baseline market maker's observation space with the book and recent flow, redesign its actions and inventory controls, and propose a cooperative multi-agent decomposition into a quoter and a risk manager trained using recurrent Independent Proximal Policy Optimisation (IPPO) with RNN policies. We investigate reward shaping, particularly reward blending and post-fill markout, to mitigate credit-assignment difficulties between the agents. Our redesigned single-agent market maker quotes more frequently and incurs smaller held-out losses than the retrained JaxMARL-HFT baseline, although the contributions of jointly changed observations, actions and risk constraints are not separately identified. Decomposition alone performs poorly; careful reward design substantially improves the desk relative to the unshaped split and yields our best result. Models are evaluated on realised end-of-episode portfolio value on held-out markets. None establishes positive average portfolio value in periodic held-out evaluation, and each configuration was trained with one seed. The environment, data and protocol lay a foundation for makers that quote many linked markets at once.
comment: 50 pages, 8 figures, 7 tables. Extended version of a paper accepted at ICAIF 2026 Workshop on AI-Driven Market Microstructure. MSc dissertation, London School of Economics and Political Science (2026)
★ Topological Sheaves of Irreducibility: A Theoretical Framework for Multi-Agent Task Obstructions
Multi-agent systems can differ under identical agents and communication budgets when they support different forms of joint interaction. We introduce Topological Sheaves of Irreducibility (TSI), a sheaf-theoretic framework separating task-level compatibility obstructions from system capability. A task is represented by an interaction complex $K$ and a capacity sheaf $\mathcal{F}$; nonzero $H^n(K;\mathcal{F})$ defines a structural signature, while registered interventions test capability. For cocycle data, cochain realization is solvable exactly when the represented cohomology class vanishes. Examples show that the signature is sheaf-relative, and the degree-1 reduction yields cycle-space witnesses and observation requirements. We evaluate two finite one-round models. Across 15,625 degree-1 inputs, an all-agent additive primitive solves decision and repair exactly; the optimal pairwise system reaches 88.80% on decision, one input above the 88.79% majority-class baseline, and 26.63% on repair. On a degree-2 tetrahedral-boundary task over 531,441 inputs, the same-2-skeleton system retains all four three-agent primitives yet matches the pairwise-only optimum throughout the registered separation window, whereas adding the class-filling four-agent primitive makes both tasks exact. Status-preserving robustness checks retain exact performance but are not cost-matched to the class-restoring ablation. Conditional on the registered primitives and cost models, TSI provides task-relative, falsifiable tests of higher-order interaction.
comment: 31 pages, 1 figure, 3 tables
★ Autonomous intelligence for mining productivity: a multi-agent AI approach to operational value creation
Integrated Remote Operation Centres (IROCs) are emerging as the architectural frontier of Industry 4.0 for capital-intensive sectors such as mining, oil and gas, and energy. Yet two intertwined gaps constrain their evolution towards full autonomy. First, no integrated agentic architecture has been proposed for mining IROCs that orchestrates Supervisory Control and Data Acquisition systems, digital twins, predictive analytics and human-machine interfaces under safety-critical constraints. Second, no evaluation framework jointly addresses the real-time, Health-Safety-Environment, regulatory-auditability and cost-per-decision requirements that govern industrial agentic deployments. This paper makes four contributions. It synthesises industry needs and quantified gains documented in analogous industries that motivate an agentic approach to mining IROCs. It introduces EI2ROC, a three-tier multi-agent architecture combining a Large Language Model orchestrator with specialised monitoring and production-capacityprediction agents communicating through a Model Context Protocol-style tool interface. It provides a critical review of agentic evaluation along six orthogonal dimensions and identifies the industrial blind spot of current benchmarks. Finally, it proposes a multi-dimensional benchmarking framework for industrial agentic AI comprising scenario library, harness, metrics and humancalibration protocol. The paper states explicitly which components are implemented in the current prototype, which belong to the proposed reference design and which remain future work, and reports preliminary cost and latency observations from an operating site. Together, EI2ROC and the benchmarking framework constitute a methodological blueprint transferable across CAPEX intensive industries.
★ Discovery-Probability Graph Planning for Embodied Agents in Tasks with Hidden Cooperation Requirements
Embodied agents are increasingly capable of perceiving, planning, and acting autonomously in dynamic environments, demonstrating strong performance on complex cooperative tasks. Recent advances typically assume that cooperative tasks are known in advance. In real-world applications, however, hidden-cooperation tasks are more common: the need for cooperation only becomes apparent after an agent observes and interacts with task-related targets. In this paper, we first present HiddenCoop, a benchmark for effectively evaluating agent cooperation in hidden-cooperation tasks. We further propose Discovery-Probability Graph Planning (DPGP), a method for enabling effective agent cooperation in such tasks. At the core of DPGP is a discovery-probability graph that encodes the probability distribution over the number and locations of task-related targets, facilitating effective and efficient information exchange among agents. A search algorithm is then used to derive action plans that minimize expected costs. Experiments show that DPGP achieves cooperative-task success rates of 99.3% on C-WAH and 58.3% on TDW-MAT, surpassing all compared baselines, while also reducing overall embodied execution time and substantially lowering LLM token usage. Our code is available at https://github.com/hnu-vis/DPGP.
comment: 25 pages, 6 figures, including appendices
★ The Control Layer Behind Rule-Following Agents Trading Across Market Conditions
Trading discipline often fails people. Our study tested whether four AI trading agents followed their strategies and risk rules. Offering traders the evidence and an agent evaluation toolkit to separate luck from following the rules, before giving agents meaningful autonomy or capital. Our study analysed 2,627 closed positions from 12 February to 27 August 2026 across the same asset universe and execution conditions. Most positions were simulated against live prices with a 0.09% round-trip fee; 18% involved real Hyperliquid orders. Each position declared a strategy, stop and target, with a limit of five open positions per agent. The fleet finished net positive after fees. Rule adherence was high across all four agents, including the unprofitable one. Profitability tracked how often agents won relative to their break-even win rates. Institutional-style controls ran by default and contained losses through a 29% market drawdown and recovery, with documented exceptions. The discipline held for all four agents. Agents kept 97.5% of gross loss within declared stops, and typical losses closed at declared levels. Declared risk remained stable after losses and broadly stable across calm and volatile months. Twenty-three of 24 runs respected the position limit; logs identify the exception's size and duration. The same live data produced different results across assigned strategies. The trade-log tests can be rerun from the published trade snapshot; regime tests also require the published Bitcoin prices and regime labels. Section 3.5 specifies the stop-breach tolerance. These trade-level results are not account-level returns or proof of a lasting edge. Our study provides an evaluation toolkit to help traders distinguish rule-following from luck before committing capital. For anyone considering allocating capital to agents to trade, discipline should be the entry test, not the only test.
comment: 47 pages, 0 figures, 16 numbered tables. Includes a trading-agent evaluation toolkit and checklist
★ Bidirectionality is Necessary in Cellular Machines for Exact-Arrival Multi-Agent Path Finding
On ranked directed acyclic graphs coupled by vertex conflicts we study exact-arrival multi-agent path finding (departure and arrival pinned in place and time) posed as a distributed output task: every cell must emit its own bit of a valid joint path-support mask. The machines are cellular: one automaton per cell, state factored over cells, communication only along a fixed local fabric, synchronous rounds, a query-independent output round, no central decoder, and a per-topology transition code. "Necessary" here means: on an explicit hard topology, over all its feasible queries, no universally correct local-only architecture can renounce either temporal orientation; with global flags, impossibility becomes a bandwidth-round trade-off. The class is not vacuous: a flooding machine solves every instance in as many rounds as the diameter of the local fabric. The bound survives two concessions: we supply every agent's exact individual cavity, which alone solves the single-agent problem in zero rounds and makes the single-agent direction theorem vacuous; and we allow $k$ one-bit global aggregate flags. On a twelve-cell, two-agent, depth-three topology no forward-only machine with $k=0$ is correct on two feasible queries, for any number of rounds; the time-reversed topology does the same to backward-only machines. Placing $p$ copies side by side gives, for forward-only machines, $J_R+kR\ge p$ at constant depth, where $J_R$ measures how much of the query reaches the targets' cones and is $0$ here. A backward-only machine solves that family in constant rounds and a flag-streaming one in $\lceil p/k\rceil+O(1)$, so the bound is tight in order; on the composite family neither orientation can be renounced. The finite lemmas of the gadget and the pair-invariance contradictions are checked in Lean 4; the semantic bridge, the cone lemma and the counting are paper proofs. We report no benchmarks.
comment: 33 pages. Ancillary files: Lean 4 development for the finite gadget (theoremA.lean), exhaustive check scripts with frozen expected outputs, reproduction README and clean-run log. Companion paper: "Stability Certifies Validity: Amoeba-Inspired Rule Systems for SAT and Single-Agent Trajectory Constraints, Machine-Checked in Lean 4"
★ A GPU-Resident Pipeline for Multi-Robot 3D Inspection with Limited Perception
We present a complete, GPU-resident pipeline for multi-robot 3D inspection of underwater structures. Given a triangle mesh and a fleet of $K$ robots, each carrying a short-range, limited field-of-view camera, the pipeline computes 6-DOF trajectories that jointly cover a target fraction of the surface while minimizing a linearly interpolated makespan/total-cost objective. The joint problem is NP-hard, so we decompose it into five core stages - candidate viewpoint generation, visibility computation, set cover, min-max vehicle routing solved as a mixed integer program (MIP) with lifted Miller-Tucker-Zemlin subtour elimination, and priority-based multi-agent path finding with continuous-time inter-robot conflict checking - flanked by mesh loading and surface sampling as preprocessing and by trajectory smoothing as postprocessing. Our contributions are: an end-to-end GPU-accelerated integration of the five stages, with its kernel design documented; a GPU batch adaptation of Lien's $\varepsilon$-visibility with four custom CUDA kernels; a $β$-aware per-vehicle tour upper bound derived from the interpolated objective, supplementing the Desrochers-Laporte coefficient lift with forbidden-pair cuts; a 4D space-time extension of Zhou and Zeng's GPU parallel-frontier A*; and an experimental evaluation on 14 meshes (up to $7.2$M faces) that ablates the routing cuts and the parallel-frontier search, audits collision freedom and waypoint attainment \emph{after} trajectory smoothing, and measures the end-to-end gap against analytical and LP-relaxation lower bounds. Code and raw results are publicly available.
★ Behavioral Probes for Information Flow in LLM Swarms NeurIPS 2026
Most existing approaches to LLM swarm optimization are utility-driven, focusing on final task utility while leaving the processing, integration, and reliance of information within individual interactions largely implicit. To address this, we study LLM swarm interactions from an information-flow-based view and introduce behavioral probes to quantify information flow patterns. Specifically, we repurpose External Context Score (ECS) and Parametric Knowledge Score (PKS)---originally designed for hallucination analysis---to measure a node's external-context dependence and parametric-knowledge dependence, respectively. Across controlled and realistic swarm settings, we observe consistent patterns: ECS increases with input relevance and distinguishes useful multi-source contributions, while PKS decreases as informative context accumulates. Building on this heuristic interpretation, we demonstrate how these probes can complement utility-driven structure optimization in two controlled applications. Probe-guided node pruning reduces redundant communication and token consumption, and probe-guided reasoning-path search accelerates convergence compared to utility-only baselines. Our empirical results suggest that behavioral probes can make the internal information flow of LLM swarms observable and serve as effective auxiliary signals for structure optimization.
comment: Accepted at NeurIPS 2026
★ Learning Who Talks to Whom: Role-Aware Communication Topology Scheduling for Efficient LLM Multi-Agent Collaboration
Multi-agent systems driven by large language models (LLMs) rely on structured communication topologies to facilitate information exchange and collaborative reasoning. However, effectively scheduling these topologies to dynamically determine who talks to whom and when to communicate remains a fundamental challenge in balancing collective reasoning performance against exponential token overhead. Existing topology scheduling approaches either enforce static communication graphs that cannot adapt to dynamic problem-solving states, or overlook agent role complementarity while failing to explicitly incorporate the synthesized graph structure into team-level decision evaluation. To tackle these limitations, we propose ACTS, a role-aware adaptive communication topology scheduling framework for efficient LLM multi-agent collaboration. Orchestrated within a Centralized Training with Decentralized Execution (CTDE) paradigm, ACTS introduces a novel dual-attention mechanism. Specifically, a Role-conditioned Relational Attention module dynamically selects variable-cardinality interaction partners based on agent roles and evolving execution states, while a Topological Attention Mixer explicitly embeds the directed communication topology into team value decomposition for precise credit assignment. Comprehensive evaluations across 9 challenging benchmarks covering code generation, multi-step reasoning, general knowledge, and competitive mathematics demonstrate that ACTS consistently outperforms state-of-the-art baselines, boosting average accuracy by up to 6.19 percentage points while reducing token consumption by over 15%. The project website is available at https://acts-mas.github.io/ACTS/.
comment: Project website: https://acts-mas.github.io/ACTS/
★ MAPS: A Multi-Agent Personalized Script-Driven Video Advertising System Enhanced by Reinforcement Learning
With the rapid growth of online video consumption, video advertising has become a dominant component of digital advertising. Most personalized video advertising systems follow a retrieval-based paradigm, which matches ad content with user preferences in a feature space and recommends the most similar candidates. However, since the candidate creatives are pre-produced, this paradigm limits the granularity of personalization and leads to performance disparities across diverse user groups. To address this limitation, we propose a multi-agent personalized script-driven video advertising system (MAPS). We design four collaborative agents that form a closed-loop pipeline of user profile modeling, script generation, video synthesis, and feedback-driven iterative refinement. To train the Scripter Agent, we develop a reinforcement learning framework equipped with a multi-dimensional reward function, which steers the agent toward producing scripts that simultaneously align with user content preferences and target product information. Extensive offline experiments on representative datasets demonstrate that our approach achieves encouraging performance in generating and integrating personalized creatives. Online A/B experiments in a specific scenario in the Kuaishou advertising system show that MAPS improves CTR by 2.87%, CVR by 2.74%, and ARPU by 9.85% over the generative baseline.
♻ ★ FDE-Bench: Evaluating End-to-End Delivery from Underspecified Real-World Business Requests
Real-world business requests often leave key requirements unstated because frontline practitioners struggle to articulate rules they know from experience. Forward-deployed engineers (FDEs) must elicit these requirements through dialogue and iteratively optimize solutions until they meet customer acceptance criteria. Existing optimization benchmarks typically start from explicit objectives, while clarification benchmarks often introduce information gaps into existing tasks. We introduce FDE-Bench, the first benchmark built from real customer projects to evaluate agents on end-to-end delivery tasks that require interactive requirement clarification and iterative optimization. Its 49 cases span combinatorial optimization and machine learning. Clarification targets are distilled from discussions between frontline practitioners and FDEs. Agents elicit missing requirements from a simulated customer, and case-specific evaluators assess final artifacts against acceptance criteria using customer-accepted FDE solutions as quality references. Across 18 agent configurations, the highest mean delivery score is only 0.5510, compared with an FDE reference of 1.0. The highest delivery success rate (DSR), defined as the fraction of planned runs reaching this reference, is only 6.12%. These results show that reliable FDE-level end-to-end delivery remains out of reach for the evaluated agents on underspecified real-world business tasks. The benchmark project page is available at https://baidubce.github.io/FM-Agent/FDE-Bench/.
♻ ★ Learning Self-Orchestration in Decentralized Agent Swarms
The capabilities of an agent swarm depend on both its individual members and how they coordinate. Yet existing multi-agent systems often train agents within fixed orchestration structures, separating the learning of individual capabilities from the organization of collective behavior. We introduce market-based post-training (MPT), a framework for jointly learning action policies and self-orchestration in decentralized agent swarms. Agents express their willingness to participate through bids, which determine who acts and how collective outcomes provide local training credit. Learning both action and bidding behavior allows individual capabilities and patterns of participation to adapt together without a central coordinating agent. We evaluate MPT on embodied crafting, robot navigation, and deep research. Results show gains over monolithic post-training and confirm that agent participation indeed evolves with learning. With the same post-trained action policies, learned self-orchestration outperforms alternative coordination strategies. Ablations show that both action and bidding benefit from learning, and that learned bids provide more effective action-training signals. Beyond the original training objectives, trained swarms adapt faster to related tasks and achieve better performance with larger populations at inference time. Larger populations also suffer smaller performance losses under agent dropout, demonstrating that learned self-orchestration supports reuse, scaling, and robustness in decentralized agent swarms. More details are available on our project page at https://guoweixu.com/mpt/.
♻ ★ AutoMind: Adaptive Knowledgeable Agent for Automated Data Science
Large Language Model (LLM) agents have shown great potential in addressing real-world data science problems. LLM-driven data science agents promise to automate the entire machine learning pipeline, yet their real-world effectiveness remains limited. Existing frameworks depend on rigid, pre-defined workflows and inflexible coding strategies; consequently, they excel only on relatively simple, classical problems and fail to capture the empirical expertise that human practitioners bring to complex, innovative tasks. In this work, we introduce AutoMind, an adaptive, knowledgeable LLM-agent framework that overcomes these deficiencies through three key advances: (1) a curated expert knowledge base that grounds the agent in domain expert knowledge, (2) an agentic knowledgeable tree search algorithm that strategically explores possible solutions, and (3) a self-adaptive coding strategy that dynamically tailors code generation to task complexity. Evaluations on two automated data science benchmarks demonstrate that AutoMind delivers superior performance versus state-of-the-art baselines. Additional analyses confirm favorable effectiveness, efficiency, and qualitative solution quality, highlighting AutoMind as an efficient and robust step toward fully automated data science. Code is at https://github.com/zjunlp/AutoMind.
comment: DAI 2026
♻ ★ Annotrace: A Workbench for Step-Level Annotation of Agent Traces AAAI 2027
Every study of why LLM agents fail begins by labelling traces by hand, and every such study builds its own throwaway tool. Observability platforms record agent traces but offer no structured labelling; annotation tools label text but have no notion of a step, a delegation or a tool call, leaving it unclear what the smallest unit of execution to label should be. Annotrace reads six trace formats, including OpenTelemetry GenAI, LangSmith and Langfuse, and turns each run into an ordered, nested list of steps. The annotator writes their own labelling codebook; a frontier model proposes a starting set from a sample of their own traces, answers first, and is asked exactly the questions the human is asked, making annotation a joint human-AI process. Annotrace measures agreement between any two annotators, human or AI. On a multi-agent question-answering corpus it reproduces a previous manual effort's 235 annotation units exactly, and two models labelling the same traces agree 91% of the time at $κ= 0.82$.
comment: 4 pages, 4 figures, 1 table. Under review at the AAAI 2027 Demonstration Program
♻ ★ From Evidence to Effect: Authority Semantics and Runtime Infrastructure for Stateful Agents NeurIPS 2026
Stateful agents reuse artifacts after producing executions and permissions change. We formalize authority-sufficient observations and durable effects bound to execution and material identities. WTB implements this interface through runtime adapters, shared evidence, and transactional publication/recovery. Six study families separate the mechanism from its integration. Raw and typed evidence both solve 32/32 authority cases, with model-dependent planning effects. Fixed-intent enforcement blocks six unsafe proposals and executes 12 eligible authorized intents. Complete controls match WTB's capability. Paid integration yields 176/210 accepted benchmark-source stages, including 19/30 publication stages, recovery on 8/8 primary SWE repositories, and the most complete continuous trajectories on each of three source tasks. The findings connect authority information, effect admission, and infrastructure reuse in stateful agents.
comment: 36 pages, 4 figures, 22 tables. An earlier version of this work, titled "Beyond Agent Harnesses: Cross-Substrate Authority for Multi-Agent Systems," was accepted to the non-archival AgenticOS workshop at NeurIPS 2026
♻ ★ CyberArena: Benchmarking AI Agents in Live Symmetric Attack-and-Defense Combat
AI agents are demonstrating increasingly strong offensive and defensive cybersecurity capabilities in the real world. Current benchmarks mostly evaluate agents in isolation against static targets. Real cyber operations, however, are dynamic and coupled: security postures constantly change as attackers and defenders adapt to each other in real time. Static benchmarks quickly saturate and miss an agent's ability to attack and defend simultaneously under live pressure. To address this gap, we present CyberArena, a dynamic, symmetric adversarial benchmark built around multi-round attack-and-defense combat. In isolated sandboxes, two agents compete under identical rules, acting concurrently as both attackers and defenders over continuously changing services. This converts static targets into evolving, semi-open challenges, directly testing an agent's speed and precision in discovering, exploiting, and patching vulnerabilities. To sustain the benchmark's longevity, an automated agentic pipeline continuously generates, reviews, and validates new environments. Furthermore, because pairwise contests are notoriously hard to quantify, we introduce a dynamical model that maps match dynamics into unified, multidimensional scores across offensive, defensive, and domain-specific capabilities. Evaluating 12 frontier LLM agents across 40 challenges in 466 matches, CyberArena separates model capabilities where single-agent baselines hit a ceiling. The trajectories read like a fight rather than an exam: defenders patch one hole and miss the next and certify repairs on stale evidence, while capable agents think a move ahead, planting backdoors for later rounds and, in one case, writing their own packet sniffer to watch the opponent. Beyond a benchmark, CyberArena serves as an adaptable, scalable experimental platform for future research on cybersecurity agents.
♻ ★ Cascade Failure Attribution for Multi-Agent Systems
Large language model (LLM)-based agents have shown strong potential in solving complex tasks through multi-step reasoning, yet they remain vulnerable to execution failures. Accurate failure attribution is therefore critical for improving agent reliability. Existing topology- and spectrum-based methods exploit trajectory structures but often overlook fine-grained semantics, while LLM-based attribution methods capture semantic cues but suffer from long-context degradation over lengthy trajectories. To address these challenges, we propose DuoTrace, a plug-and-play detection filter for LLM-based failure attribution. DuoTrace follows a detect-before-attribute paradigm: it first detects anomalous executions and then supplies focused trajectory evidence to downstream LLM-based attribution methods. For effective VAE-based anomaly detection on agent trajectories, DuoTrace integrates dual-view semantic-structural node representations, a Tree-LSTM-based trajectory encoder, and data augmentation through outcome-verified LLM replay and root-to-node path extraction to handle heterogeneous nodes, hierarchical execution structures, and limited normal execution data. Experiments with six LLM-based attribution baselines show that DuoTrace improves agent-level and step-level attribution accuracy by 11.2% and 7.7%,respectively.
comment: 17 pages, 5 figures, 13 tables
♻ ★ Superficial Success vs. Internal Breakdown: An Empirical Study of Generalization in Adaptive Multi-Agent Systems
Adaptive multi-agent systems (MAS) are increasingly adopted to tackle complex problems. However, the narrow task coverage of their optimization raises the question of whether they can function as general-purpose systems. To address this gap, we conduct an extensive empirical study of adaptive MAS, revealing two key findings: (1) topological overfitting -- they fail to generalize across different domains; and (2) illusory coordination -- they achieve reasonable surface-level accuracy while the underlying agent interactions diverge from ideal MAS behavior, raising concerns about their practical utility. These findings highlight the pressing need to prioritize generalization in MAS development and motivate evaluation protocols that extend beyond simple final-answer correctness. We release our code at this https URL.
comment: 27 pages, 4 figures. Equal contribution for the first two authors
♻ ★ PiSAs: Benchmarking Contextual Integrity in Multi-User Agentic Systems
As LLM agents evolve from single-user assistants into shared organizational infrastructure, new privacy risks emerge: inappropriate information may not only be exposed through outputs for external recipients, but also internally across users through inter-agent messages, shared memory and user-agent interactions. These data spillage risks are not captured by existing privacy benchmarks grounded in contextual integrity (CI) as they focus primarily on either single-user settings or interactions between independently owned agents. We introduce PiSAs (Privacy in Shared Agentic Systems), a benchmark for assessing unintentional leaks with dual CI annotations: whether a piece of information is appropriate for the task, and which users may legitimately access it. This enables direct measurement of cross-user spillage across agentic system components and interfaces, such as outputs, inter-agent communication, and memory. PiSAs is system-agnostic and supports evaluation across different agent topologies and memory regimes. We find that, although system design improves CI compliance, results are bottlenecked by incorrect LLM judgment calls: even state-of-the-art models fail to reliably filter inappropriate content or restrict transmission to authorized users. Our findings underscore the need for privacy-preserving strategies, beyond those studied in this work.
Systems and Control (EESS)
★ Controllability of polynomial systems is undecidable and not recursively enumerable
Sontag conjectured in 1988 that global controllability of polynomial control systems with rational coefficients is undecidable, but that the controllable systems form a recursively enumerable set. We prove the first part of the conjecture and disprove the second. For every homogeneous polynomial vector field $F$ of odd degree with rational coefficients, we construct an explicit polynomial control system $Σ_F$, with one more state than $F$, that is globally controllable if and only if the origin is \emph{not} globally asymptotically stable for $\dot x=F(x)$. By a recent result, the homogeneous polynomial vector fields with a globally asymptotically stable origin form a set that is recursively enumerable but not decidable. Consequently, global controllability of polynomial systems is undecidable, and the controllable polynomial systems are not recursively enumerable. The same conclusions hold for systems of the form $\dotξ=f(ξ)+Bu$ with a polynomial drift and a constant input matrix.
★ Data Center Oscillation Isolation and Grid Support: Fundamental Limits and Source-Selective Control
The grid integration of large data centers poses a growing power-system stability challenge. Rapidly varying computing demand can induce oscillations, while the facilities must provide a prescribed response to grid disturbances. Existing controls address load fluctuations and grid support but do not establish their jointly achievable performance. This paper derives fundamental limits and develops a source-selective controller for simultaneous oscillation isolation and grid support. A coupled closed-loop model gives a condition for distinguishing disturbance sources and reveals a direct isolation-support conflict under grid-import-power feedback alone; additional electrical measurements can remove this pointwise conflict but leave a conditional cross-frequency sensitivity limit. Guided by these limits, we design a local residual-feedback storage controller through convex frequency-domain optimization with stability, isolation, and grid-response constraints across operating points. Tests on a single-generator interconnection and a modified nine-bus network show stronger oscillation isolation than equal-order optimized comparators while meeting the prescribed grid response across operating points.
★ Exact model reduction by Lie symmetry groups
We develop a model-reduction technique based on Lie group theory for general engineering models described by a system of algebraic, ordinary, and/or partial differential equations with an arbitrary number of inputs, outputs, and constant parameters. The proposed method reduces the parameter space of these models without changing their external behavior, that is, their input-output maps. We apply this method to phase-field models used for high-fidelity simulations in electrochemistry to accelerate the analysis and design of metal-ion batteries.
★ Simulation Priors for Rapid Adaptation in Model-Based RL for Autonomous Racing
Reinforcement learning enables learning high-performance control policies, but its reliance on extensive interaction limits deployment on physical systems where data collection is costly, and failures are unsafe. We introduce Prior-Informed Adaptation (PIA), an offline-to-online model-based RL framework that uses structured simulation priors for rapid adaptation. PIA pretrains a world model on heterogeneous low- and high-fidelity simulated trajectories, then adapts it with limited target-domain interaction using uncertainty-regularized planning. Rather than requiring an accurate simulator for policy transfer, PIA treats simulated trajectories as biased priors whose residual mismatch is corrected during online adaptation. We evaluate PIA in autonomous racing using high-fidelity simulation and on a real full-size autonomous race car. In simulation, PIA adapts faster across track and surface shifts than strong baselines, achieving lower lap times and fewer failures. On hardware, PIA achieves reliable few-shot adaptation with less than one hour of on-vehicle interaction.
★ System Identification for Nonlinear Stochastic State-Space Models Using Ensemble Kalman Inversion with Application to Lithium-Ion Batteries
Nonlinear state-space system identification has attracted decades of interest, yet remains challenging because of the interplay among unknown parameters, nonlinear dynamics, and latent state variables. Its difficulty increases dramatically when the state and parameter dimensions increase. To address this challenge, we develop an ensemble Kalman inversion (EnKI)-based approach. This approach leverages EnKI to perform maximum a posteriori parameter estimation through successive local Gaussian approximations, enabling an iterative and incremental search for unknown parameters. The search combines Monte Carlo sampling with Kalman-type updates to evolve an ensemble of parameter samples, thereby offering empirical stability and the ability to handle strongly nonlinear models. The EnKI search is coupled with state estimation conditioned on the latest parameter estimates. This enables parameter identification along with latent state recovery. To improve scalability for large datasets, we further develop a cascaded EnKI (CasEnKI) method, which assimilates subdatasets sequentially to accelerate computation. We validate the proposed methods using a lithium-ion battery model with ten state variables and 18 unknown parameters. Simulation results show that the EnKI approach offers high identification accuracy and empirical stability in parameter search. Further, experimental results illustrate the computational efficiency and scalability of the CasEnKI method for large battery datasets.
comment: 16 pages, 8 figures, 3 tables
★ Deep Pontryagin Learning for Continuous-Time Stochastic Control with Jumps
We develop a deep Pontryagin learning framework for high-dimensional continuous-time stochastic control problems that naturally accommodates jump-diffusions. In contrast to direct policy optimization, which repeatedly differentiates the objective through the entire time-discretized dynamics, our method separates adjoint learning from policy improvement by explicitly incorporating state-sensitivity information from the Pontryagin stochastic maximum principle. For a fixed Markov control, we first compute pathwise adjoint labels by backpropagating the reward through time along simulated state trajectories, and project these labels onto the Markov state through a supervised regression step. The control is then updated by enforcing the pointwise Hamiltonian stationarity condition. We prove that the resulting projected adjoint process satisfies a discrete-time Pontryagin principle and converges, as the time step tends to zero, to the solution of the corresponding continuous-time adjoint backward stochastic differential equation. Numerical experiments illustrate the accuracy, stability, and scalability of the approach.
★ Motion Planning with Temporal Logic Precedence Specifications in Dynamic Environments via Spacetime Augmented Graphs of Convex Sets
We extend the augmented graph of convex sets GCS framework for motion planning with temporal logic precedence specifications to dynamic environments in which obstacles, keys, and doors undergo piecewise-linear translation and/or dilation over time. The main insight is that piecewise-linear motion of polytopic sets in R^d lifts exactly to a static polytopic partition of spacetime R^d X R_>=0. We show that the main result from the static case carries over to spacetime essentially without modification: a shortest path in the spacetime augmented GCS provides an exact optimal solution to the motion planning problem with precedence specifications. Moreover, the layered subgraph lattice that encodes signal temporal logic (STL) precedence specifications retains precise correspondence with the Bellman--Held--Karp (BHK) algorithm, and the full library of restricted but expressive STL specifications generalizes to the dynamic setting. Further, the framework allows time-gated keys and doors, collectible or passable only within a temporal window. The geometric side of the framework requires two modifications: a causality constraint (the time coordinate of a trajectory must be non-decreasing) and a spacetime speed constraint (bounded physical velocity), both of which are convex in the Bézier control points.
★ Convex Inner Approximations for Safe AC Dispatch With Device Dynamics
Safe AC dispatch requires both feasible operating points and enough ramping capability to reach the next target. This paper constructs convex inner approximations that connect nonlinear network feasibility, constrained device response, and dispatch-interval tracking. A fixed-point certificate enforces AC operating limits under bounded load uncertainty. Componentwise response envelopes convert heterogeneous device dynamics into affine conditions on measured states and power references. A command governor then combines these conditions with available ramping capability and a hard terminal-error bound. It selects an admissible target through a minimum-deviation objective and holds the corresponding reference while reserved fast regulation remains active. The resulting quadratic program has no network-trajectory decision variables. Tests on the IEEE 39- and 118-bus systems include 96 single-target executions and 64 transitions in continuous dispatch sequences at five- and fifteen-minute intervals. All 160 transitions satisfy the AC constraints, independent balancing-generator capability checks, and the prescribed regulation-adjusted terminal accuracy. A paired ablation verifies the contribution of the terminal condition under slow device responses. Complete admission-cycle computation has median times of 3.45 and 8.79 ms for the two systems. The construction provides a tractable link between available ramping capability and safe, timely execution of AC dispatch.
★ Decentralized Stability Certificates for Interoperability in Power-Electronics-Dominated Power Systems
The rapid integration of power electronics converters has fundamentally altered the power grid's dynamic landscape, making modularity of operation and interoperability of heterogeneous devices two critical hurdles for power system reliability. In control language, these challenges can be paraphrased as decentralized certificates on the input/output behavior of every device, assured by local control, so that stability and guaranteed performance of the interconnected system emerge. For the power system application, such certificates should eventually be part of the rule book, the so-called grid code, and thus be generic and minimally restrictive for the control architecture of each device. Classical system-theoretic angles of attack to these questions go back to dissipativity concepts, which have historically been co-developed in automatic control, circuit theory, and power systems. Since standard dissipativity (e.g., passivity or small gain) criteria fall short in terms of conservatism, a range of more flexible and advanced tools have been developed to assure less conservative decentralized certificates guaranteeing stability under interconnection as well as dynamic performance following a disturbance. In this article, we review a range of methods that have been put forward to tackle these questions, compare and validate them, and point towards practically relevant research gaps which are amenable to control-theoretic approaches. Moreover, we provide a tutorial exposition of practically relevant power systems/electronics issues accessible to control engineers.
★ PhyPro-FM: A Physics and Prompt-Guided Graph Foundation Model for Contingency Analysis
The continuous evolution of next-generation power systems, characterized by the high penetration of renewable energy resources and dynamic topology variations, poses formidable challenges to traditional N-1/N-2 contingency analysis in terms of computational efficiency and scalability. While deep learning-based methods offer accelerated inference, existing studies predominantly treat contingency analysis as a pure regression problem, often overlooking the physical essence of grid operation and engineering requirements of real-world scenarios. To bridge this gap, this paper proposes a physics and prompt-guided graph foundation model (PhyPro-FM) for generalizable and engineering-viable contingency analysis. The proposed architecture integrates graph neural networks with a learnable global query mechanism and, notably, introduces a prompt-guided learning paradigm. By leveraging historical snapshots as few-shot prompts, the model dynamically captures correlations between target and historical operating conditions, significantly enhancing generalization capabilities under unseen topologies. Furthermore, we design a physics-informed training mechanism that prioritizes branch power flow errors over nodal voltage errors, aligning the optimization objective with critical grid security constraints. To ensure engineering applicability, a physics-informed inference mechanism is also developed, incorporating a residual-based correction module and a specialized post-processing strategy for singular branches to circumvent numerical instability. Comprehensive experiments demonstrate that PhyPro-FM outperforms state-of-the-art methods in accuracy and generalization, validating its superior physical consistency and robustness for practical engineering applications.
★ Beyond Observability: Sharp Sample Bounds for Minimax State Reconstruction
We study how many output samples match the worst-case state reconstruction error obtainable from a continuous history under bounded measurement noise. For observable scalar-output linear time-varying systems with $n$ states, $n$ samples suffice for exact reconstruction, whereas at most $n^2$ attain the continuous-history minimax error in the $\ell_\infty$ norm. An explicit construction proves sharpness even for real-analytic systems. For time-invariant systems, strict total positivity yields one $n$-sample schedule optimal for every state coordinate; endpoint-observed tridiagonal cooperative chains satisfy this conclusion in physical coordinates. Interior sensing behaves differently: an explicit three-state family requires four samples despite positivity and distinct real eigenvalues. The strict three-state gap persists under small stabilizing damping. A nilpotent four-state example and an interval proof for a six-state mechanical system give gaps. The results identify how temporal sampling and the prescribed state-error geometry determine the measurements required for minimax reconstruction. Time-domain experiments illustrate the resulting estimation bounds.
comment: Submitted to the International Journal of Control. 6 figures
★ Reassessing Clean Power 2030 Pathway Using Real Time Records
The National Energy System Operator (NESO) Clean Power 2030 advice to the government, accepted as United Kingdom (UK) government policy in December 2024, outlines a rapid increase in wind and solar generation capacities with the associated infrastructure by 2030, with the objective of reducing carbon dioxide emissions from the UK electricity system to 5.2 million tons per annum. Our previous paper used real-time records from 2017 to analyze Clean Power 2030 proposal. This paper extends the previous assessment, using real time records of 2017, 2018 and 2019 to test the resilience of the UK energy system under more diverse weather patterns. Two different modelling methods predict that the 2030 carbon dioxide emissions will be five times greater than Clean Power 2030 prediction. Furthermore, the projections show that, due to increased curtailment, only about half of NESO proposed increases in wind and solar capacities will be economically justified. NESO also advised the government that 35GW of unabated gas-fired generation would maintain security of supply, whereas our models suggest that at least 42 GW will be required during winter wind lulls. Recent events, such as grid stability problems in 2026, also discussed in the paper, validate our estimated and suggest the risk of blackouts in the UK is increasing.
comment: 18 pages, 8 tables, 5 figures
★ Mobile Embodied Agent Network (MEAN): Integrated Framework, Enabling Technologies, and Task-Driven Deployment
Mobile networks are evolving from human-centered, data-oriented communication infrastructures toward machine- and/or agent-dominant, task-oriented platforms that are to serve the rapidly increasing networked machines and embodied agents. In order to fulfill the newly arose complex task objectives, the framework of future mobile network is to be innovated to reflect and facilitate the intrinsic interplay and integration of the underlying processes of perception, communication, computation, decision, action, and feedback, etc. This survey establishes the mobile embodied agent network (MEAN) as a task-oriented networking paradigm in which networking and distributed intelligence are jointly organized around closed-loop embodied task execution. We first establish a unified MEAN framework that explains how network resources and intelligent functions jointly support embodied task execution, and extend its performance indicators from conventional communication metrics to embodied task outcomes and cross-layer system effects. With this framework, we analyze three complementary enabling mechanisms that form a cognition-interaction-adaption loop. Large artificial intelligence (AI) models provide high-level cognition, token communication connects this cognition with distributed agents through shared semantic representations, and reinforcement learning closes the loop by adapting system operation to network, task, and physical feedback. We further investigate the bidirectional interplay between MEAN and emerging technologies, showing how they mutually extend system capabilities and reshape task-oriented network design. Finally, we discuss open challenges and future research directions toward the practical realization of MEAN over real joint communication and computation platform.
★ Optimizing Airport Electric Ground Support Fleets with Digital-Twin-Guided Predictive Control
Electrifying airport ground operations is a key step toward reducing aviation-related emissions, but it makes task assignment and charging coordination strongly coupled in real time. This paper develops a digital twin of airport ground handling operations for electric ground-support vehicles (EGSVs), combining an agent-based model of airport operations with live flight-schedule updates and vehicle-energy dynamics. On this platform, we compare three control strategies for EGSV dispatching and charging: a rule-based heuristic, a rolling-horizon model predictive controller (MPC), and a Monte Carlo tree search (MCTS) controller. The controllers are evaluated on three representative Schiphol Airport operating days and for multiple fleet sizes. The results show a clear trade-off between delays and energy use. Across all tested scenarios, MCTS yields the lowest average flight delay, with a pooled delay of 4.09 min, improving on the heuristic and MPC controllers by 64.7\% and 31.5\%, but at an increased energy consumption of 1\% and 0.3\%, respectively.The MPC and MCTS controllers coordinate charging more effectively throughout the day and reduce the peak charging infrastructure required by 6.0 and 4.7 charger units, respectively. Additionally, we find more vehicles can result in lower overall energy consumption and lower charging infrastructure requirements for all controllers. These findings highlight the value of digital twins not only as offline design evaluation tools, but also as practical environments for real-time control policies in electrified airport operations.
★ Fast Reassessment of Prolonged Electricity Shortages under Shared Fuel Interruptions
When several generators share a fuel supply, one interruption can disable them together and cause electricity shortages lasting days. Such interruptions can be made less frequent, for example by weatherizing gas supply, or shorter, for example by on-site backup fuel. Yet traditional resource adequacy indices, expected unserved energy and loss-of-load hours, fail to distinguish such measures whenever they remove the same expected time without fuel, because for interruptions independent of system conditions these indices depend on the interruptions only through this time. Their difference is revealed by prolonged-shortage exposure, the expected annual hours in shortages of at least a given duration, whose exact chronological calculation must be repeated for every interruption scenario. To reduce this cost, a fast reassessment method with guaranteed accuracy is proposed. Network calculations are performed once and condensed into a restoration response, the additional prolonged-shortage hours caused by one interruption of each duration, from which any scenario is evaluated as a weighted sum without new simulation. An hour-by-hour error bound determines whether this evaluation is accurate enough, and exact dynamic programming is applied otherwise, so that every result lies within a requested tolerance of the exact value on the same Monte Carlo samples. On the Reliability Test System of the Grid Modernization Laboratory Consortium, most evaluations were accepted, fully accepted scenarios typically ran more than 200 times faster than exact calculation, and total computation time decreased in every workload. Measures against fuel interruptions can thus be screened quickly and accurately by their effect on prolonged shortages.
comment: 11 pages, 4 figures, 5 tables. Submitted to IEEE Transactions on Reliability. Code and data: https://github.com/DoganUrgun/Fast_Reassessment_Prolonged_Shortages
★ Who Breathes the Detour? A Coupled Traffic-Emission-Dispersion-Exposure Pipeline
Road traffic is among the largest sources of health-damaging air pollution in inhabited areas. Direct measurement of resident exposure at the scale of an entire corridor remains costly, and existing simulation approaches often target either high-fidelity local snapshots or regional averages, leaving the corridor scale underserved. We present a coupled, end-to-end pipeline that links four stages: (i) microscopic traffic simulation with SUMO, (ii) per-vehicle emissions from the HBEFA4 model, (iii) deterministic three-dimensional Eulerian diffusion solved with an implicit finite-difference scheme on a geometric vertical grid, and (iv) resident-level exposure expressed as an air quality index over a synthetic population. The design deliberately trades the fidelity of full CFD dispersion for a workflow that runs end-to-end for the residential areas of several settlements on a single laptop, and the complete implementation is released open source. We showcase the pipeline at the example of the Swiss Gotthard transit corridor, covering the residential areas of six villages, two days, four scenarios, and three pollutants, to enable assessment of congestion mitigation measures. Code & Material: https://github.com/cedriczeiter/zeiter_coupled_pipeline.
★ Every minute counts: Wind farm power control during imbalance settlement using distributional soft actor-critic
Wind farms can provide flexibility by curtailing their output during periods of excess generation in the power grid. This can be done through implicit balancing, where the power setpoints of the farm are adapted to anticipate the imbalance price. By doing so, the wind farm helps reduce the resulting system imbalance while also lowering its own costs. However, implicit balancing is challenging since the imbalance prices are unknown in advance and hard to predict. Furthermore, the control strategy of the wind farm can affect the imbalance price formation. To address both challenges, we develop a control policy with 1-minute setpoints using distributional soft actor-critic. Evaluated on real data of the Belgian grid and an offshore wind farm, we show that our method outperforms a price threshold controller, often used in industry, by 88% and performs on par with soft actor-critic. Furthermore, unlike soft actor-critic, our approach can learn a risk-averse policy by optimizing a risk measure, such as conditional value at risk. The risk-averse policy was found to better account for its own impact on the price, showing a reduction in overshoots, while achieving a similar degree of imbalance cost savings. Last, we find that using 1-minute instead of 15-minute setpoints further reduces imbalance costs for all evaluated control policies.
★ Timing Skew Estimation of Time-Interleaved ADC based on Inter-Channel Correlations without Computing Derivative of Input Signal
Time-interleaved analog-to-digital converters (TI-ADCs) enable multi-gigasample-per-second throughput in high-speed applications, but sub-picosecond timing skews among interleaved channels introduce large spectral artifacts that degrade dynamic performance. To continuously track parametric drifts without interrupting normal operation, background calibration is required. However, extracting timing skews under unknown inputs is challenging, and many existing correlation-based methods rely on resource-heavy derivative filters that restrict input bandwidth. This article proposes a novel background timing-skew estimation technique for two-channel TI-ADCs that overcomes these limitations. Closed-form timing-skew expressions are derived using: (a) adjacent-channel cross-correlation combined with single-channel auto-correlation for sinusoidal inputs, and (b) adjacent-channel cross-correlation for Wide-Sense Stationary (WSS) inputs, specifically bandpass-filtered wideband noise and Orthogonal Frequency Division Multiplexed (OFDM) signals. The derived analytical models are validated through comprehensive simulations, offering an efficient framework for real-time background calibration in high-speed data converters.
★ The Exact Boundary of Linear Optimality in Witsenhausen's Counterexample
Witsenhausen's counterexample has resisted a closed form solution for decades. However, it has remained unknown whether a mathematically tractable representation exists or not. In this paper, we solve this open problem, showing that the optimal policies are either linear or not tame and characterizing the exact regions where the two properties hold. Moreover, in the linear optimal region, we give an essentially unique closed form solution for the optimal policy, and show that no optimal first-stage policy is definable in any $o$-minimal expansion of the ordered real field in the nonlinear optimal region.
comment: 39 pages
★ Adaptive Long-range UAV Flight Planning using Monte Carlo Search Trees
Long-range fixed-wing UAV missions require energy-efficient, dynamically feasible trajectories that can adapt online to evolving wind and airspace constraints. We present a Monte Carlo Tree Search (MCTS)-based framework that addresses this challenge by formulating mission planning as a finite-horizon Markov decision process incorporating aircraft kinematics, onboard energy dynamics, forecast wind fields, terrain, and time-varying no-fly zones. The MCTS planner searches over waypoint references generated by pre-stabilizing feedback controllers and a constraint-aware navigation field, yielding dynamically feasible trajectories. Simulation results from medical delivery scenarios show that the MCTS planner outperforms baseline geometric planners and population-based optimization algorithms. Moreover, the MCTS planner can be used in closed-loop to rapidly compute safe diversion trajectories under evolving constraints.
comment: Authors' accepted manuscript. Published in AIAA, DOI: 10.2514/1.G010042
★ The Forward/Inverse Duality of Controlled Emergence:A Computational Design Program from Explanation to Engineering
The macroscopic capabilities of intelligent systems arise from the local interactions of many simple units. When such systems are deployed in the real world, their behaviour must nevertheless be constrained, leaving designers caught between releasing adaptability and preventing loss of control, yet without formal tools to resolve the dilemma. This paper proposes a unified research program for controlled emergence that restates this tension as a pair of dual problems: the forward problem explains macroscopic behaviour from local rules, and the inverse problem solves for controllable rules and readout mechanisms from a desired target. We take an augmented quintuple as the formal object shared by both ends, and decompose ``engineering'' into three independently measurable coordinates: controllability, reproducibility, and degree of formalizability. On this basis we give three testable working hypotheses (critical scale; the cooperation and trade-off between constraint and adaptability; minimal constraint) and one feasibility criterion, which reduce inverse design from unconstrained optimization to interval checking. The program takes explicit failure boundaries as its premise, does not pursue complete control, and always preserves the system's room for self-organization. The reduced pilot experiment attached to this paper serves only to verify the executability of the research protocol and does not constitute a statistical test of the working hypotheses. This design language generalizes to multi-agent artificial intelligence, the industrial internet, and emergency response systems, all of which must confront emergence and controllability at once.
comment: 38 pages, 7 figures, 13 tables
★ Design and Control of a Multimodal Legged-Aerial Robot Through Separate Control Contraction Metrics
The integration of dynamic legged locomotion and aerial mobility offers versatility that is deterred by challenges in payload distribution, actuation, and the contrasting requirements of both domains. In this paper, we present the design and operation of a multimodal quadrupedal aerial-legged robot. The presence of legged aerial systems also allows for thruster-assisted legged locomotion, which we make possible with a unified control framework using Control Contraction Metrics (CCM). We search for separate domain-specific contraction metrics for walking and flight independently offline and unify them using an online energy-based Control Lyapunov Function Quadratic Program (CLF QP). This enables optimal, cross-domain control allocation, which we show using simulations. We also show experimental results that validate the mechanical viability of our hardware and the paradigm of thruster-assisted trotting. The stability of said experiments motivates the deployment of the control framework for multimodal operations.
comment: Presented at 2026 IEEE International Conference on Intelligent Robots and Systems, Pittsburgh PA
★ Learned Control Variates for Rare Grid-Outage Estimation at Electric Vehicle Charging Stations
Public charging of electric vehicles (EVs) adds a large and synchronized load to the distribution feeders. In most networks, it is rare to overload a feeder at the same time. When it does happen, the consequences can be severe: an overloaded feeder stresses its transformers and can trigger cascading outages across a city. Planners who design for high reliability therefore need to estimate very small outage probabilities. To that end, plain Monte Carlo estimation with an expensive grid simulator can require a prohibitively large number of runs for this task. To reduce the number of runs, we propose a control variate (CV) built from a learned classifier of the outage, trained on cheap observable features such as temperature and load forecasts. We show that for any binary learner, the variance reduction is an explicit function of its recall and false-positive rate. Furthermore, we show that control variate based on a Gaussian proxy of the load loses its effect as outages become rarer, while a learner with non-vanishing precision keeps removing a fixed fraction of the variance even in the extreme rare-event regime. We verify these results in numerical experiments using a feeder-overload simulator across three orders of magnitude of outage probability and obtain that even weak learner with a single feature can remove a quarter of the variance, while a boosted-tree learner is able to remove about 60%. We then apply the estimator to the feeder network of Surabaya, Indonesia, in which the Gaussian load model leaves every feeder three to seven times the one-day-in-ten-years blackout target. At the same simulation budget, planning the reserve with the learned CV and a 95% confidence limit reduces the city-wide blackout days by a third compared with Monte Carlo. These results show that learned control variates can help planners design distribution grids that stay reliable against rare outages.
★ Dynamic Line Ratings Counteract Transmission Investment Needs Driven by Climate Change
Rising temperatures due to climate change will reduce power system transmission capacity. This reduction will interact with an ongoing shift from static to time-varying transmission rating standards. Previous research has yet to disentangle the effects of changing rating standards and climates on investment decisions. We quantify transmission investment needs for the Western U.S. power system (WECC) through mid-century under climate change and three rating standards: static ratings (SLR), ambient-adjusted ratings (AAR), and dynamic line ratings (DLR). By embedding alternative rating standards and high-resolution future weather data into a generation and transmission expansion model, PyPSA-USA, we find WECC will lose 1-3% of its transmission capacity across potential future climates under static ratings by mid-century, necessitating a 1-5% increase in transmission investment. Adoption of AAR or DLR eliminates most (81-99%) of the adaptation-driven transmission capacity needs, and further reduces aggregate transmission investments (by up to 31%). Additionally, we demonstrate that planning with time-varying ratings does not increase investment variability for individual lines under climate uncertainty relative to planning with static ratings. Our results emphasize the value and practicality of considering alternative rating standards in climate adaptation planning.
comment: 16 pages, 7 figures, supplementary information included, under review at Energy & Environmental Science
★ DeGroot Dynamics on Multiplex Networks with Directed and Acyclic Interlayer Connectivities
In models of opinion dynamics, researchers typically examine how an individual's opinion on one topic is influenced by the opinions of their neighbors in a network. However, people's opinions on a topic are also influenced by their opinions and their neighbors' opinions on related topics. In an election, for example, the opinions of a voter on a variety of issues inform their opinions of electoral candidates. To consider the influence between people's opinions on multiple distinct topics, we study a multiplex extension of the traditional DeGroot consensus model. In our multiplex DeGroot model, different network layers represent distinct topics and directed interlayer edges (which are not allowed to form cycles) encode relationships between distinct topics. In our model, an individual's opinion on one topic is influenced both by the opinions of their neighbors on that topic and by their own opinions on related topics. We prove under mild conditions that our multiplex DeGroot model converges to a limit state. We also prove that strong connectedness in ``source layers'' (which represent fundamental beliefs) eventually yields a consensus opinion in each layer. In particular, we show that consensus on source layers can lead to consensus on other layers even when those layers otherwise would be polarized. By numerically simulating our multiplex DeGroot model on a two-layer network, we examine the effects on opinion dynamics of (1) different network structures and (2) different types of initial opinion data.
comment: 44 pages, 9 figures
♻ ★ Braess' Paradoxes in Coupled Power and Transportation Systems
Transportation electrification introduces strong coupling between the power and transportation systems. In this paper, we generalize the classical notion of Braess' paradox to coupled power and transportation systems, and examine how the cross-system coupling induces new types of Braess' paradoxes. To this end, we model the power and transportation networks as graphs, coupled with charging points connecting to nodes in both graphs. The power system operation is characterized by the economic dispatch optimization, while the transportation system user equilibrium models travelers' route and charging choices. By analyzing simple coupled systems, we demonstrate that capacity expansion in either the transportation or the power system can deteriorate the performance of both systems, and uncover the fundamental mechanisms for such new Braess' paradoxes to occur. We also provide necessary and sufficient conditions for the occurrence of Braess' paradoxes in general coupled systems, leading to managerial insights for infrastructure planners. For general networks, by characterizing generalized user equilibria of coupled systems, we develop charging-pricing policies to mitigate Braess' paradoxes.
comment: 35 pages, 16 figures
♻ ★ Reduced Arc-Weight Heterogeneity for Master Stability Analysis of Directed Networks
Master stability analysis can be applied to directed networks even when their Laplacian matrices are not diagonalizable. Nevertheless, a diagonalizable Laplacian is easier to analyse because the perturbations governing synchronization separate into independent modes. This paper asks how many different positive coupling strengths are needed to obtain this simplification without adding or removing any directed connections. We first give an elementary construction showing that suitable positive weights exist for every weakly connected directed network and explain its relationship to stronger genericity results in the literature. We then introduce a measure of the smallest number of distinct coupling strengths required for diagonalizability. If a common coupling strength already produces a diagonalizable Laplacian, only one value is needed. For directed acyclic networks, we give a constructive upper bound based on the length of the longest directed path from a source. This bound is exact for rooted outward directed trees. Consequently, an outward star requires only one coupling value, whereas a directed path requires a different value on every arc. These results reduce coupling heterogeneity while retaining the simpler modal form of master stability analysis. Diagonalizability is treated as an analytical convenience, not as a sufficient condition for synchronization.
comment: 11 pages
♻ ★ Quotient Geometry and Persistence-Stable Metrics for Swarm Configurations
Can temporal information recover geometry lost by a persistence diagram? For $n\ge3$ points on the unit sphere evolving under known homogeneous complete consensus, we prove generic local recovery modulo rotations and permutations from two ordinary zero-dimensional Vietoris--Rips persistence snapshots. The structural result applies to every labeled tree: its edge inner products and their first temporal derivatives have differential rank $2n-3$, exactly the dimension of the regular formation quotient. An explicit pole--equator witness proves this simultaneously for all tree types after taking a finite intersection of open dense sets. Although one snapshot has generic local fibers of dimension $n-2$, two sufficiently close snapshots are injective on a common neighborhood. The optimal inverse Lipschitz constant grows as $Θ(1/|h|)$ when the sampling interval tends to zero. A verified left inverse, interval derivative bounds, and explicit persistence-matching margins certify neighborhoods and time intervals at rational reference formations. The results are local and model-specific; reflection ambiguity and centered rank obstructions remain.
comment: 15 pages
♻ ★ Analytical Prediction of Voltage Collapse in Current-Limited Grid-Forming Inverters
As converter-interfaced units become dominant in power systems, understanding nonlinear stability phenomena involving discrete switching events becomes essential for secure operation. The limited overcurrent capability of grid-forming (GFM) inverters makes current limiting essential during large disturbances. Activation of a circular current limiter (CCL) can cause the operating equilibrium to disappear under certain circumstances. This paper develops an analytical framework to predict the grid-voltage boundaries for CCL activation and determine whether the operating equilibrium persists or disappears after activation. For CCL-based GFM inverters with frozen anti-windup, closed-form expressions for the activation boundaries are derived while accounting for the filter capacitor. A criterion is established to determine whether a saturated (i.e., current-limited) stable equilibrium point (satSEP) persists following limiter activation. A continuation method is also developed to trace equilibria across normal-control and current-limited operation. Power-angle analysis and dynamic-model continuation in single-inverter and modified 9-bus systems validate these predictions. Time-domain simulations further confirm the results. The results show that CCL activation can either cause immediate loss of the stable equilibrium or allow stable current-limited operation to continue until the equilibrium is lost as operating conditions change further.
comment: Submitted to IEEE Transactions on Power Systems
♻ ★ The role of distributed friction dynamics on heave-induced motion of offshore drilling systems: modelling, analysis, and simulations
This paper investigates the influence of friction dynamics on heave-induced disturbance propagation using a coupled hydraulic-structural drilling model formulated as a semilinear interconnection of ordinary and hyperbolic partial differential equations (ODE-PDEs), and capturing spatially distributed memory effects in the drillstring-wellbore interaction. First, local and global existence and uniqueness results are established, and sufficient conditions for boundedness and input-to-state stability are deduced via a small-gain analysis. Subsequently, extensive numerical investigations are carried out, including harmonic excitation studies, nonlinear distortion analysis, spatial amplification investigations, mechanism-isolation experiments, parameter sensitivity analysis, and broadband simulations under realistic offshore sea-state excitation. The simulation results indicate that, whilst the hydraulic response is governed primarily by wave propagation phenomena, compressibility, and boundary reflections, the structural dynamics display substantially greater sensitivity to distributed friction effects. Furthermore, drillstring-wellbore friction is shown to have a relatively limited influence on well-posedness but a potentially significant impact on stability margins. These findings highlight the importance of accounting for coupled hydraulic-structural dynamics and distributed friction mechanisms when analysing, monitoring, and controlling offshore drilling systems.
comment: 45 pages, 11 figures. Under review
♻ ★ Repeated Binary Direct Collinear Impacts Under Incremental Contact Laws With Residual Indentation: A Hybrid Systems Formulation
Incremental contact laws specify the normal contact force through a differential relation, possibly carrying an internal state, driven by the indentation and its rate. In some, the force is extinguished at a nonzero indentation, whether by plastic deformation or by an elastic aftereffect, so that a residual deformation remains at separation. Such laws sit uneasily within rigid body dynamics, which admits no deformation. The tension is tolerable when the indentation is small relative to the bodies, so that it may be carried constitutively rather than geometrically. Even then, the contact law alone does not determine the interaction of the bodies. Because force and indentation no longer vanish together, conditions for the commencement and termination of contact must be supplied separately. So must the fate of the deformation and internal state at separation, neither of which the equations of motion contain. This article formulates the repeated direct collinear impact of two convex bodies under external forces as a hybrid dynamical system. The contact interface is modeled as a massless element carrying the contact law and its state, coupled to the bodies through relative velocity and an interaction force dictated by the contact state. Consequently, all switching and resets are confined to the interface, leaving the geometry and the equations of motion of the bodies unaltered. The principal analytical properties of the resulting formulations are established, among them passivity and completeness; the branching of solutions at the onset and termination of contact is also examined. The framework is demonstrated through numerical simulations.
comment: 29 pages, 6 figures
♻ ★ Joint Deployment and Resource Management for Group-Tag-Based Physical-Layer Authentication and Collaborative Communication in NTN
Physical-layer authentication (PLA) in non-terrestrial networks (NTNs) has primarily been studied from a security perspective, while its system-level integration with communication remains less explored. We propose the secure adaptive framework for authentication in multi-zone networks (SAFA-MZ), a system-level formulation for the joint management of collaborative group-tag and private-data delivery in NTN, where high-altitude platform station (HAPS) placement, user association, and transmission powers are jointly selected under service and secrecy constraints. Group-level authentication tags and user data are collaboratively transmitted by multiple HAPSs, with artificial noise (AN) mitigating information leakage. We jointly optimize HAPS placement, user association, and power allocation to balance deployment regularity, tag distribution, and user-rate fairness under secrecy, quality-of-service (QoS), power, and coverage constraints. The problem is addressed using a repair-based cross-entropy method (RCEM) and a graph-aware advantage actor-critic (A2C) algorithm (GA2C). Under colluding and non-colluding eavesdropper models, GA2C improves average secrecy spectral efficiency (SSE) by 135--139% over its single-connect baseline and 21.1--23.2% over its baseline without AN. Across ten independent random seeds, GA2C achieves approximately 23.4% and 17.9% higher mean per-user SSE under non-colluding and colluding eavesdropping, respectively, and a 16.2\% higher mean private-rate Jain's index than RCEM. These results quantify the gains of multi-node connectivity and AN within the proposed framework under the evaluated assumptions.
comment: Substantial revision: the title and author list were updated, RSMA- and differential-privacy-related components were removed, and the manuscript was extensively revised to introduce a joint deployment and resource-management framework for group-tag-based PLA and collaborative communication
♻ ★ Topological entropy of nonlinear time-varying systems
We study the topological entropy of nonlinear time-varying systems. We establish conditions under which entropy is independent of the initial time and derive two general upper bounds based on the matrix measure of the system Jacobian and the spectral abscissa of a Metzler comparison matrix, respectively. A general lower bound is also obtained using the trace of the Jacobian. For interconnected systems, we first derive an upper bound using the matrix measure of an interconnection matrix constructed from the Jacobian blocks. We then establish a new upper bound based on the spectral abscissa of a Metzler comparison matrix, incorporating the interconnection structure through subsystem-wise trajectory-separation analysis. This bound also recovers the two general upper bounds as special cases. All bounds depend on asymptotic properties of the Jacobian over reachable sets or their convex hulls. Simple examples show that either general upper bound may be tighter, while a time-varying Lorenz system illustrates the effects of norm selection and subsystem decomposition.
♻ ★ An Exact Solution Algorithm for the Bi-Level Optimization Problem of Electric Vehicles Charging Station Placement
This work addresses electric vehicle (EV) charging station placement through a bi-level optimization model, where the upper-level planner maximizes charging revenue by selecting station locations under budget constraints, while EV users at the lower level choose routes and charging stations to minimize travel and charging costs. To account for battery range limitation, we construct a battery-expanded network and apply a shortest path algorithm with Frank-Wolfe traffic assignment. Our primary contribution is developing the first global solution algorithm for large scale EV charging station placement problems. We propose a Branch-and-Price-and-Cut algorithm enhanced with value function cuts and column generation. Our exact algorithm delivers solutions provably within the reported optimality gap of the global optimum with mathematical certainty. Computational experiments on the Eastern Massachusetts network (74 nodes, 248 links), the Anaheim network (416 nodes, 914 links), and the Barcelona network (110 zones, 1,020 nodes, and 2,512 links) demonstrate exceptional performance. Our algorithm terminates in a reasonable runtime while achieving optimality gaps below 1% across all instances. Controlled benchmarks against two genetic algorithms on identical instances confirm that the proposed algorithm finds equal or better solutions in less computation time across all tested networks. The algorithm successfully handles problems with over 300,000 feasible combinations, transforming EV charging infrastructure planning into a tractable optimization suitable for practical decision making on real-world networks with optimality guaranteed.
♻ ★ From Optical Flow to Safety: Visual Safety Certification via Scale-Free Relative Motion
Safe autonomous navigation using monocular vision remains challenging because explicit obstacle states are not directly observable. This letter introduces a visual safety-certification paradigm that uses scale-free, depth-normalized relative motion encoded by monocular optical flow as the representation for collision-risk certification, without reconstructing obstacle position, velocity, geometry, or a metric world model. Unlike conventional optical-flow-based navigation approaches that rely on image motion to shape avoidance behavior, the proposed framework directly encodes collision-cone geometry and safety constraints in this relative-motion representation. An online Concurrent Learning observer estimates the normalized relative velocity and provides a computable estimation-error bound. This uncertainty is propagated into the collision cone through angular inflation and a monocular robot-footprint margin, yielding a Robust Vision-Based Collision Cone Control Barrier Function (RV-C$^3$BF). An optimal-decay CBF quadratic program minimally modifies the nominal control input to enforce the resulting robust safety constraint under the stated assumptions. Extensive simulation and real-world experiments on a TurtleBot3 in cluttered environments with static and dynamic obstacles demonstrate collision avoidance using only monocular vision and onboard proprioception, with the estimation-error bound containing the measured error in $98.3\%$ of simulation samples and the robot achieving collision-free navigation in $93$ of $100$ real-world trials.
♻ ★ Real-Time Solution-Seeking for Game-Theoretic Autonomous Driving via Time-Distributed Iterations
Computational complexity has been a major challenge in game-theoretic model predictive control (GT-MPC), as real-time solutions to a game (e.g., Nash equilibria (NEs)) have to be computed at each sampling instant of an MPC. This challenge is especially critical in autonomous driving, where interactions may involve many agents, and decisions must be made at fast sampling rates. We show that this challenge can be addressed through time-distributed solution-seeking iterations designed based on, e.g., Newton and Newton--Kantorovich methods. Specifically, the autonomous vehicle decision-making problem is first formulated as a GT-MPC problem. To ensure solution attainability, a potential game framework is adopted. Within this framework, both potential-function optimization and best-response dynamics are used to seek the NE. To enable real-time implementation, Newton and Newton--Kantorovich methods are employed to solve the optimization problems arising in the NE-seeking algorithms, with their iterations distributed over time. Numerical experiments on an intersection-crossing scenario demonstrate that the proposed methods achieve effective real-time performance.
comment: 7 pages, 11 figures
♻ ★ Readiness Barrier Functions: Forward-Invariant Control Authority for Overactuated Multirotor Allocation
Allocation schemes that greedily maximize a readiness metric over the actuator fiber bundle of an overactuated multirotor produce commands that jump between disconnected optimal strata, demanding actuator rates no motor can deliver; effort-minimizing schemes are continuous but cannot guarantee that wrench-rate authority stays above any certified level. We reconcile the two by treating authority as a forward-invariant quantity: a control barrier function on the log-determinant of the drag-aware actuator-authority co-metric, enforced at torque level by a quadratic program in the allocation null space. A single design inequality renders the certified set compact and strictly interior to the actuator box, with the readiness cost of any rotor deactivation given in closed form as $\ln(n/(n{-}m))$ for symmetric designs. An explicit alignment ratio states when tracking can stay exact, otherwise the wrench error is bounded by $\mathcal{O}(ρ^{-1/2})$, and a robust variant handles motor-parameter uncertainty with a closed-form floor shift independent of the airframe matrix. On a hexarotor and a fully-actuated octorotor the closed-form gap matches simulation to machine precision. Over $300$ randomized missions, greedy maximization violates the certified floor in up to $51\%$ of authority-scarce missions, while the proposed filter never leaves the certified set and, in the hardest moment reversal, commits a $79\times$ smaller wrench error. The robust variant holds the floor on all $150$ perturbed plants. A high-fidelity simulation with measured propeller and motor data shows that the floor holds under unmodeled aerodynamics, electrical limits, and delay, and where it stops applying.
comment: This work has been submitted to the IEEE for possible publication
♻ ★ Active Inference via Incentive Design: Eliciting Divergent Behaviors under Partial Observability
We introduce Active Inference through Incentive Design, a framework in which a leader offers strategic incentive to elicit diverging behaviors from followers of different types, thereby improving intent inference under imperfect observations. We analyze two settings: Case 1: a perfectly observable leader with two homogeneous followers with different original/unmodified reward functions, under the assumption that followers' best responses induce Gibbs trajectory distributions in their respective Markov decision processes, we show that this inference problem can be reformulated as maximizing the difference in expected cumulative return, under a modified reward for one follower, between the best-response policies of the two followers; Case 2 considers a partially observable leader who aims to distinguish among heterogeneous followers differing in dynamics and/or rewards. The leader balances incentive cost against information gain, measured by the posterior entropy over follower types, consider the follower's best responses to the incentive design. We develop an efficient first-order gradient-based algorithm to optimize the information-theoretic inference objective. Experiments in stochastic gridworld environments demonstrate significant improvements in both inference accuracy and efficiency over unincentivized baselines.
♻ ★ Explicit Steady-State Approximations for Parallel Server Systems with Heterogeneous Servers
We study the steady-state performance of parallel-server systems under an immediate routing architecture with two sources of heterogeneity: server types and job classes, often subject to compatibility constraints. We focus on the weighted-workload-task-allocation (WWTA) routing policy, a load-balancing scheme known to be throughput-optimal for such systems. Under a relaxed complete-resource-pooling (CRP) condition, we prove a strong form of state-space collapse in heavy traffic and that the scaled server workloads jointly converge in distribution to a deterministic vector multiplied by an exponential random variable, whose mean is explicitly given by system primitives. Our analysis yields three main insights. First, the conventional heavy-traffic assumption of a unique static allocation plan can be dropped; a relaxed CRP condition suffices. Second, the limiting workload distribution is shown to be independent of the local scheduling policies at the servers, allowing substantial flexibility. Third, the inefficient (non-basic) activities identified by the static allocation problem are proved to receive an asymptotically negligible fraction of routed jobs and service effort, even though WWTA has no prior knowledge of which activities are non-basic, highlighting its robustness to changing arrival rates.
Robotics
★ Adaptive Bendpoint Formation in Octopus-Inspired Soft Continuum Arms
Octopuses exhibit a powerful form of embodied intelligence from a highly adaptable soft body. One key mechanism is the ability to instantiate bending points at any location along the arm, which enables performant and dexterous behaviours. Taking inspiration from the octopus for its dexterity, this study focuses on changing the curvature of a tendon-actuated arm by adaptively changing the anchoring end-point along the arm. An array of inflatable shells is integrated into the arm, where inflating a shell causes an increase in friction at the location, jamming the tendon and creating a temporary new anchoring point. The novel soft actuator does not break the continuity of the arm between the actuated segment and the loose segment. Experiments show that the actuated segment has a non-constant bell-shaped curvature which is comparable to that of a real octopus arm during a reaching motion. Compared to conventional single-tendon continuum manipulators, the proposed design offers an expanded workspace and enhanced dexterity through the ability to independently shape different regions of the arm through selective curvature tuning. At the same time, it avoids the mechanical complexity typically associated with multi-segment, multi-actuator systems.
★ AQUAACT: A Joystick-Level Underwater Teleoperation Dataset and Closed-Loop Benchmark
Underwater robots support a growing range of work, from inspecting offshore structures and pipelines to surveying reefs and accompanying divers. Most of these tasks, however, are still flown by hand by a pilot, which raises the question of whether they can be carried out autonomously instead. On land, vision-language-action (VLA) models already let robots carry out tasks autonomously from language instructions, which makes them a natural candidate for taking the pilot's place. Underwater, however, there is little data to train such models. We take inspiration from how these tasks are flown today: the pilot watches the camera image and steers with a joystick, whose commands the flight controller turns into motion. We therefore record the pilots and introduce AQUAACT, to our knowledge the first publicly available language-annotated underwater dataset pairing camera images with human joystick commands. It comprises 1.9 hours of real-world and 17.9 hours of simulated demonstrations, both flown by human pilots, together with the vehicle's inertial measurements. Experiments with AQUAACT show that its simulated demonstrations transfer to real data and greatly reduce the real demonstrations a policy needs. A closed-loop benchmark of seven policies in simulation, from imitation-learning baselines to VLA models, each trained with its default recipe, shows that these tasks remain far from solved: no policy completes more than a fifth of the episodes. Pretraining on land data gives large VLA models no advantage over small policies trained from scratch, a gap that land-trained models have yet to close underwater.
★ CusADi2: Latency-Optimized Batch Computation of Robot Kinematics, Jacobians, and Hessians on the GPU
Trajectory optimization methods, from sampling-based zero-order to gradient-based first- and second-order approaches, require many evaluations of dynamical system functions, i.e., functions derived from a robot's dynamical model, such as forward kinematics and their Jacobians and Hessians. These evaluations are largely independent, e.g., across sampled trajectories or time steps, and can thus be evaluated in parallel as a batch. When used to make decisions online, these methods require low latency, i.e., each batch must complete in as little wall-clock time as possible. Existing frameworks utilize modern GPU hardware primarily to parallelize across batch instances, thereby optimizing throughput but not latency. We present CusADi2, a software library that reduces the batch evaluation latency by additionally parallelizing computation within each batch instance. We benchmark forward kinematics, Jacobians, and Hessians for all 67~robots in the MuJoCo Menagerie, including arms, hands, quadrupeds, humanoids, and biomechanical models, against MuJoCo XLA, MuJoCo Warp, frax, CusADi, and CasADi. CusADi2 achieves lower latency than all baselines at batch sizes of 64 to 4096, while matching them at larger batch sizes. We also demonstrate the performance advantage of CusADi2 in a sampling-based control application. All code is open-source at: https://github.com/johnviljoen/cusadi2
comment: 8 pages, 4 figures. Submitted to IEEE Robotics and Automation Letters (RA-L). Code: https://github.com/johnviljoen/cusadi2
★ BiDex-100: A Comprehensive Benchmark for Bimanual Dexterous Manipulation Across 100+ Simulation Tasks
Bimanual manipulation with dexterous hands has attracted growing attention in research on robotic action models. However, few studies have developed benchmarks specifically tailored for dexterous manipulation, leaving the dexterous manipulation capabilities of existing models insufficiently characterized. In this paper, we present BiDex-100, a simulation benchmark for bimanual dexterous manipulation that combines broad task coverage, multi-dimensional evaluation, and high-DoF hand embodiments. Concretely, the benchmark comprises 106 tasks organized along five capability axes: generalization, instruction following, limited demonstrations, long horizon, and memory. Each task is paired with an out-of-domain variant featuring controlled visual and spatial changes, allowing systematic assessment of robustness. In addition, the benchmark supports two embodiments and provides 8,080 success-validated teleoperated demonstrations, totaling 6.25 million frames. We evaluate seven recent Vision-Language-Action models under a unified protocol, establishing baselines for capability-specific performance, robustness, and data efficiency. The website is available at https://BiDex-100.github.io/BiDex-100/.
★ Self-Crystallizing Behavior Trees: Amortizing Foundation-Model Reasoning into Deterministic Control for Autonomous Drones
Classical robot control, Finite State Machines and Behavior Trees (BTs), is deterministic and inspectable but fixed at design time: it fails on situations its designers did not anticipate. Foundation models can synthesize new control logic at runtime, at a cost in latency and non-determinism hard to accept in safety-critical aerial systems. Existing LLM-driven approaches keep the model in the loop permanently and discard what it produces. We propose Self-Crystallizing Behavior Trees (SCBT), in which the foundation model is a compiler rather than a runtime. When a BT node fails, an LLM proposes a minimal, mission-agnostic subtree patch; CMA-ES tunes its parameters in simulation; a contextual bandit decides whether a stored patch or a new LLM call is worth its cost; and a three-stage admission pipeline (static check, invariant-monitored replays, an empirical margin over the unpatched tree) admits only patches that demonstrably recover. Admitted patches are stored in a fleet library and, once the LLM call rate decays, crystallized into a deterministic BT that no human wrote. We implement SCBT on Microsoft Project AirSim with a schema-constrained LLM pipeline (Gemini 3.1 Pro) and a lockstep simulator. Over three seeds of a 20-episode learning phase with wind-gust and battery failures, SCBT used $11\pm1.4$ LLM calls against 26 for whole-tree regeneration and 22 without a library, with 0.33 calls per episode in the second half. The crystallized trees, 28 and 33 nodes with two patches each, reached $97\pm2\%$ mission success with no learning running, against 12\% for the unpatched tree and 98\% for a hand-coded expert tree, and reproduce decision traces exactly when re-ticked offline. Two negative results: replay admission rejected useful wind recoveries by attributing wind-induced geofence excursions to the patch, and a static-only ablation generalized better to held-out failures.
★ Intent-Informed Model Predictive Control for Shared Autonomy in Remote Manipulation Tasks
Shared autonomy is a control paradigm that combines human cognitive abilities with robot precision to enable effective remote manipulation in complex environments. A key challenge is how to adapt the level of autonomy using information extracted from the human, task, and/or environment. In this paper, we propose a novel arbitration strategy that dynamically adjusts the level of autonomy in the system using the entropy of a belief distribution derived from a multimodal intent inference. Human intent is modeled with a Dynamic Bayesian Network (DBN), which updates beliefs over goals given real-time operator gaze data and robot arm motion during teleoperation. We then formulate the blending of human input and autonomous control as a Model Predictive Control (MPC) problem, where the system's autonomous behavior is guided by the belief distribution. This approach enables continuous, gradual transitions across the full spectrum of autonomy levels, from direct teleoperation to full autonomy. To validate our approach, we conducted experiments to compare the performance of our proposed algorithm against two MPC-based shared autonomy methods. Compared to the baselines, our approach achieved reductions of at least 61% in object retrieval time, 44% in trajectory length, and 60% in human control input. Participants reported high perceived control, predictability, intent recognition, and responsiveness, along with low workload.
★ SurGPT: A Multi-Task Foundation Model for Robotic Surgical Skills Assessment and Gesture Recognition
The assessment of surgical skills in robotic surgery training has traditionally relied on subjective evaluations by human experts, creating bottlenecks in surgical education and limiting scalability. We present a novel foundation model that automatically analyzes robotic surgical videos to provide comprehensive skill assessment and gesture understanding. Combining a visual encoder with a transformer architecture, the model performs three key tasks simultaneously: 1) surgical task classification (knot tying, needle passing, suturing), 2) skill level assessment on a standardized scale, and 3) natural language generation of surgical gesture descriptions. The model processes video sequences from the da Vinci surgical robot and generates human-readable text descriptions of surgical actions, similar to how language models generate text from prompts, using a multi-task learning framework with adaptive uncertainty weighting to optimize performance across all objectives. Our transformer-based architecture incorporates positional encoding for temporal understanding and attention mechanisms for spatiotemporal feature integration. Evaluated on the da Vinci Skills Simulator dataset, the proposed framework demonstrates state-of-the-art performance in surgical task recognition and provides interpretable skill assessments that correlate with expert evaluations, with the model's ability to generate natural language descriptions of surgical gestures opening new possibilities for automated surgical education, real-time feedback systems, and objective skill assessment in surgical training and certification programs.
★ Learning When to Adapt: Reactivity-Conditioned Residual Adaptation under Human-Robot Distribution Shift
Pedestrians differ in whether they respond to a robot, yet many RL policies assume all are non-reactive to prioritize safety. Policies trained without conditioning on local pedestrians' reactivity may therefore be unnecessarily conservative or rely on cooperation that does not occur. We propose RCRA, an online adaptation framework that modulates a learned residual around a frozen conservative navigation policy according to estimated pedestrian reactivity. A Transformer-based estimator infers latent pedestrian reactivity from motion histories and robot context, providing a more accurate signal for navigation adaptation and improving scenario-averaged simulation accuracy by $12.62$ percentage points over the Bayesian baseline, which assumes known pedestrian goals. These estimates are spatially aggregated into an adaptation scale that gates LoRA modules, enabling cooperative behavior near reactive pedestrians and conservative behavior near non-reactive pedestrians. Among baselines achieving SR above $90\%$ in every scenario, RCRA reduces scenario-averaged path length by $16.7\%$ relative to the conservative backbone and by $7.3\%$ relative to randomized-training baselines. It improves mean SR by $14.88$ percentage points over the reactive-trained policy.
★ T(R,O) Grasp++: Cross-Embodiment Dexterous Grasping in Cluttered Environments at Scale
Dexterous grasping in cluttered scenes remains a central challenge in robotic manipulation, as it requires high-dimensional hand coordination under complex object-environment constraints. To address this problem, we introduce one of the largest simulation benchmarks to date, comprising around 0.5B grasp annotations across eight dexterous hands and approximately 50K objects. Furthermore, we propose T(R,O) Grasp++, a scalable diffusion-based framework for cross-embodiment dexterous grasp synthesis in cluttered environments. Extensive experiments demonstrate that our model significantly outperforms existing baselines, achieving an average success rate of 88% across eight dexterous embodiments. Meanwhile, our pipeline achieves only 0.3s inference latency while improving inference sample efficiency by over 10 times.
★ AeroDASH: Asynchronous Vision-Language-Action Control for Continuous UAV Flight
Onboard inference latency creates a mismatch between the trajectories predicted by vision-language-action (VLA) policies and the ongoing motion of unmanned aerial vehicles (UAVs). We present AeroDASH, an asynchronous VLA framework for continuous language-conditioned flight. A B-spline policy is initialized through supervised fine-tuning on expert demonstrations, providing continuous trajectory references in position and yaw. During execution, trajectory fusion removes the expired prediction prefix and connects the remaining segment to the active reference with derivative continuity while limiting trajectory deformation. To account for deviations induced by asynchronous execution, execution-aware group relative policy optimization (GRPO) further adapts the policy using trajectory-level rewards. On the UAV-Flow test set, AeroDASH achieves a success rate of 76.56%, exceeding OpenVLA-UAV and AeroDASH w/o GRPO by 38.1 and 20.1 percentage points, respectively. Execution experiments show that AeroDASH sustains motion during inference and reduces acceleration, jerk, and speed fluctuations relative to asynchronous tracking of waypoint sequences. We further demonstrate language-conditioned flight with onboard inference and asynchronous trajectory execution. The code will be released upon acceptance.
comment: 8 pages, 6 figures, 2 tables
★ Energy-Score Stabilized Attentive Mixture-Density Policy: A Lightweight Approach to Multimodal Robot Motion IROS 2026
For practical motion generation, it is essential to handle multimodality (multiple actions can be correct for the same observation) while maintaining low inference cost and strong generalization from limited demonstrations. We introduce an attentive mixture-density policy trained with the Energy Score that represents future visuomotor trajectories as explicit probability distributions with a single forward pass. Concretely, we output Gaussian mixture parameters for joint commands (and optionally attention points) and optimize them with the Energy Score (computed as a sum of 1D CRPS under a diagonal covariance assumption) instead of the negative log-likelihood (NLL) to avoid MDN singularities and gradient-scale instabilities. Our architecture couples explicit spatial attention with mixture-density outputs, enabling robust multimodal behavior selection under visual noise and small-data regimes. On an 8-DOF manipulator, the method generates distinct avoidance trajectories and stabilizes learning in obstacle avoidance. Moreover, in a randomized moving-object grasping task, where spatiotemporal multimodality and fast timing decisions are required, the proposed model outperforms Diffusion Policy by 15-25% under several unseen conditions, demonstrating superior generalization and real-time feasibility.
comment: 8 pages, 6 figures, 1 table. Accepted to IROS 2026
★ Visual-Inertial Odometry for Ground Robots with Dynamic Camera-IMU Extrinsic Calibration
Visual-inertial odometry systems are often constrained by the assumption that sensors are rigidly mounted, which limits their adaptability in practical scenarios involving structural motion. To address this limitation, this paper investigates time-varying camera-IMU extrinsic calibration. First, a dynamic camera-IMU extrinsic calibration method based on the Denavit-Hartenberg convention is proposed by incorporating the robot kinematic model. Second, a two-step initialization strategy is developed to efficiently estimate the initial joint angles, providing a reliable starting point for subsequent extrinsic parameter optimization and enabling online estimation of timevarying camera-IMU extrinsic parameters. Simulation experiments demonstrate that the proposed framework can effectively accommodate variations in camera-IMU extrinsic parameters while maintaining competitive localization accuracy.
★ TILDE: Teacher-in-the-Loop Distillation from Embodied Foundation Models
Large-scale action pretraining equips Embodied Foundation Models (EFMs) with strong manipulation capabilities, with particular strengths in generalization and closed-loop robustness, but reproducing such pretraining requires substantial robot data and computation. To transfer these capabilities from pretrained EFMs to policies without action pretraining, we introduce TILDE (Teacher-In-the-Loop Distillation from EFMs), which synergizes on-policy distillation at student-induced states with learning from teacher-executed recovery trajectories. We further propose AutoDAgger, an autonomous data aggregation pipeline in which a robotic reward model is utilized to monitor task progress and trigger teacher intervention, thereby collecting recovery experience without human monitoring or manual control. AutoDAgger accelerates recovery-trajectory collection by more than $2\times$ compared with the human-in-the-loop method. By unifying learning from demonstrations, on-policy teacher supervision, and teacher's corrective behavior, TILDE improves task performance by up to $4.1\times$ over behavior cloning across three real-world tasks and supports effective cross-architecture distillation.
comment: Project page: https://hku-sail.github.io/TILDE
★ LateMatch: Breaking the Interaction Bottleneck via Cacheable Matching for Efficient Visual Localization
Visual localization is critical for embodied agents, but deploying state-of-the-art deep feature matchers on edge devices is fundamentally limited by their massive computational cost. Standard deep matchers tightly interleave intra-image context building and inter-image matching at every layer, creating a severe interaction bottleneck. Localizing a single query against K database images forces the system to execute the full network K independent times with no capacity for caching. To break this bottleneck, we advocate for a paradigm shift toward cacheable image matching. We realize this paradigm with LateMatch, a versatile approach that restructures existing matchers by deferring inter-image interaction to their final layers. By isolating intra-image processing into offline-cacheable Context Blocks, only a small set of Interaction Blocks must be executed online. To further optimize LateMatch, we introduce E-SToRE (Entropy-guided Spatial Token Reduction and Expansion). E-SToRE leverages the decreasing Shannon entropy of the late Interaction Blocks to dynamically reduce token counts just prior to cross-image interaction. Evaluated across HPatches, ScanNet, MegaDepth, ETH-3D, and Aachen, LateMatch reduces effective computational cost by over 2x compared to standard deep matchers, while E-SToRE yields an additional 10% to 14% savings with negligible degradation in accuracy.
★ Vision as Proprioception: Estimating Humanoid Robot Pose from Video
Estimating a humanoid robot's internal state from video has a wide range of real-world applications but remains unexplored. During interactions with a robot, its onboard state information may be unavailable due to privacy, ownership, or communication constraints. Meanwhile, online and AI-generated robot videos provide rich sources of data but typically lack corresponding joint configurations, limiting their direct use in robot learning. To address these issues, we present HumanoidPose, a new framework that treats vision as proprioception to recover full-body joint configurations in world space from video alone. Specifically, we combine pretrained human-pose features with robot kinematics in a unified model that predicts each embodiment's joint configurations. We retarget human motions to sixteen different humanoid embodiments, enabling scalable learning entirely from synthetic data. To improve real-world generalization, we further leverage unlabeled online videos through self-training to develop a 2D robot-pose detector. In our evaluation, HumanoidPose improves joint-angle and joint-position accuracy over the evaluated baselines and reduces joint-angle error by 18-32% relative to the best-performing baseline on each of the three real humanoid platforms. We further show improved global localization and generalization to unseen embodiments. HumanoidPose enables visual understanding of other robots during interaction and extraction of robot motion from online and generated videos.
comment: Code: https://github.com/eth-siplab/HumanoidPose
★ The Bitter Lesson Comes to Robotics
General purpose multimodal language models (LMs) are increasingly capable of directly controlling robotic arms (e.g., by commanding end effector poses) to manipulate their environment. Tracking the development of these capabilities is important for understanding their potential benefits, such as improving industrial automation, and risks, such as enabling malicious users to produce dangerous chemical or biological agents. To this end, we investigate the ability of frontier multimodal LMs to directly control physical robotic arms. We evaluate seven models at multiple reasoning-effort settings on six manipulation tasks taken from the RoboDojo benchmark. To our knowledge, this is the first systematic comparison of a broad range of frontier LMs across a common suite of manipulation tasks on physical hardware. We find that current frontier models can directly control real robotic arms to perform a variety of manipulation tasks but are not yet precise or efficient enough to complete them within a practical action budget. However, these capabilities are improving rapidly: performance increases on our benchmark between July and September 2026 (inclusive) correspond to a performance doubling time of 44.5 days, while the best recorded execution rate increased by a factor of 4.4 and the lowest recorded execution cost fell by a factor of 6.2. These trends raise the possibility that the Bitter Lesson is coming to robotics; the capabilities of embodied general purpose LMs may one day be comparable to those of more specialized models.
comment: Submitted to the Workshop on The Action Gap (TAG) at CoRL 2026
★ GOFusion: Real-Time Gaussian-Occupancy Submap Fusion for Fisher-Informed Autonomous Robot Navigation
In this letter, we present GOFusion, a real-time visual mapping system for autonomous robot navigation that anchors Gaussian primitives to local occupancy maps, providing both spatial awareness for path planning and high-fidelity appearance that enables information-driven robot navigation. GOFusion builds 3D Gaussian maps from pose estimates and local occupancy maps obtained via visual-inertial SLAM and enables pose corrections to maintain both geometric and visual consistency. Furthermore, it restricts the 3D Gaussian Splatting optimization to active submaps to bound GPU memory usage. Building on the appearance-rich Gaussian map, we introduce a rasterizer-integrated Fisher approximation that quantifies per-primitive sensitivity online, serving as an information-driven heuristic for Gaussian pruning and next-best-view planning. We evaluate GOFusion on public datasets and in real-world drone flights. It not only outperforms several baselines in mapping quality, but also achieves real-time, high-fidelity and uncertainty-aware mapping at scale. Finally, we demonstrate fully autonomous drone exploration for Gaussian map refinement using the proposed Fisher-informed heuristic. Code is available at https://github.com/Bennnoo/GOFusion. We provide a supplementary video demonstration at https://youtu.be/pLQKbxQvb6I.
comment: 8 pages, 7 figures, 4 tables. Paper under review
★ AI-Enhanced Radiation Classification in ROS
This study presents the development of two Robot Operating System (ROS) nodes with a primary focus on differentiating seven radioactive isotopes. The first node, referred to as the model training node, uses a Gradient Boosting classifier to identify and classify these isotopes. The classifier achieved a reported overall accuracy of 95\% on the held-out test dataset. The second node, known as the prediction node, provides classification results when users input data in CSV format. For the practical demonstration, the C12137 radiation detector module was mounted on the Jackal UGV, and an operator approached different radiation sources to collect data. To support future dataset expansion, we also developed a method for converting gamma-spectrum graph images into CSV files. In a separate robotic evaluation across three source locations, 30 of 42 trials were correctly classified (71.43\%). Am-241, Co-60, Cs-137, Na-22, and Ti-44 were correctly classified in all six trials each, whereas Ba-133 and Eu-152 were incorrectly classified in all six trials each. These results show lower accuracy in robotic trials than in the held-out test set.
comment: 7 pages, 4 figures, 3 tables
★ Structure over Reasoning: Unified Zero-Shot Navigation with a Small Vision-Language Model
Vision-language navigation is moving into photorealistic, physically simulated scenes, where the leading zero-shot agents put a vision-language model in the control loop to choose every move and stop. Robots favour small models, whose abilities are uneven: Qwen3-VL-4B can judge whether a single view looks like the goal, but its route reasoning is at chance, its distance estimates nearly constant, its stop votes unreliable. Our agent therefore asks it two perceptual questions per step, which direction looks like the goal and whether the view justifies stopping, plus one gated frontier query on route-bearing instructions, and treats the answers as evidence, not actions: a route-blind semantic field over a bird's-eye map accumulates the directional scores through a tempered update, a skeleton graph supplies frontiers and geodesic distances, and a two-stage terminator turns stop votes into termini through a trail-consistency gate and a budget-forced return. With every baseline re-run in one installation at one budget, the agent exceeds all four zero-shot baselines of the VLNVerse subset in mean success on both granularities, though the paired interval against the strongest coarse-grained baseline includes zero. Termination is the dominant lever in ablation, thirteen attempts to give the model more of the navigation establish no reliable gain, and the agent runs on a Unitree Go2 quadruped with off-board inference. Code, per-episode logs and a robot video will be released at https://anonymous.4open.science/r/structure-over-reasoning-PAGE.
comment: 8 pages, 1 figure
★ Compile Once, Run Cheap: Frozen, Portable Skills for Embodied Agents, and What Recursive Self-Improvement Adds
Embodied AI lacks the data-rich pretraining that gave language models their agents and skills, so new manipulation capability costs demonstrations per task or a frontier model in every episode. We study a third route: a strong general model acts only as an offline author that writes, runs, diagnoses and repairs a manipulation skill in simulation until it passes an exam on layouts drawn after freezing; the frozen skill then runs as code alone or under a cheap caller that reads only its description. We call this skill-level recursive self-improvement Skill-RSI and specify a compile protocol (isolated, audited author; hidden validation layouts; no-feedback and inherit-only controls; tamper-evident ledger) and a reuse protocol that runs the identical files with executors from another model family. On 7 RoboTwin 2.0 tasks with cluttered tables, Claude Code compiled 4 tasks into 7 frozen skills, 5 from a first draft needing only qualification, at USD 4.29-8.22 of author cost each (USD 55 per skill counting the 13 failed campaigns). The other three tasks did not compile in 18 campaigns, including five with GPT-6 and a pre-registered 150-layout run: on the hardest task every campaign scored 78-95 of 100 but had 3-16 safety aborts, failing the gate of 95% success with zero aborts; a 100-episode exam bounds an abort rate only to 3.6%. On hidden layouts, repair that inherits the previous version beat from-scratch rewriting in 8 of 8 controlled campaigns (sign test p = 0.004); an inherit-only lineage attributes most of the gap to inheritance, with development feedback adding a little (3 of 3, p = 0.125). Write-once campaigns found no detectable first-draft effect of practice (minimum detectable effect 14 points). The frozen skills kept 148/150 successes on new layouts with no model and 147/150 with a DeepSeek-flash caller at USD 0.00028-0.00054 per episode; no direct-control baseline was run.
comment: 28 pages, 4 figures, 13 tables. Code and data: https://github.com/Orillusion/embodied-skill-rsi
★ Localization and Neural Splatting on the Edge: An FPGA Scene Memory for Robots
Resource-constrained robots and autonomous vehicles need scene memories that localize a camera and render nearby views without a desktop GPU. Scene-coordinate regression (SCR) and 3D Gaussian splatting (3DGS) fit because they avoid an SfM stage, but ACEsplat-style pipelines are GPU-centered, and moving them to an edge FPGA is hard because SCR is pose-sensitive and 3DGS rendering is irregular and capacity-sensitive. We present LENS, an edge scene-memory acceleration system on the AMD KR260 that brings an ACEsplat-style workload to a low-power FPGA/ARM platform. Rather than proposing a new scene model, LENS co-designs the two dominant kernels with the board data path: an INT16 SCR accelerator that preserves pose accuracy, and a bit-exact fixed-point ordered 3DGS renderer that rewrites alpha transmittance as a log-sum-exp recurrence and removes provably useless streamed Gaussians. All LENS latency, power, and quality results are board-measured. The SCR engine preserves DSAC* localization accuracy at 25.42 ms, the on-board renderer reaches 34.11 dB on Cambridge against ACEsplat's 33.20, novel-view synthesis on RealEstate10K reaches 27.84 dB against 26.11, and all ten Wayspots scenes reach the ACEsplat rendering target. LENS wins performance per watt over a Jetson Orin NX on every deployment task, by 1.9x to 5.7x at about a quarter of the power, and is faster in absolute latency on localization and Wayspots rendering. On-device training reaches ACEsplat-level quality but not the GPU's time or energy, marking the remaining boundary for a future training accelerator.
comment: 8 pages
★ VLA-Rolling: Fewer Denoising Steps, Better Control via Adaptive Action Reuse
Flow-matching vision-language-action (VLA) models generate an action chunk by iteratively denoising Gaussian noise, but typically execute only a short prefix before replanning with updated observations. Under this receding-horizon paradigm, the unexecuted remainder---the action tail---is discarded, and the next chunk is regenerated from fresh noise. We show that this discarded tail retains transferable denoising progress: action-tail initialization can match or outperform a native 10-step Gaussian baseline using fewer denoising steps. However, the optimal denoising budget varies substantially across samples, indicating that tail reuse should be adapted to the updated environment. Based on this observation, we introduce VLA-Rolling, a lightweight inference framework that reuses action tails through three stages: action-tail inheritance, per-action confidence estimation, and confidence-guided refinement through renoising and denoising budget allocation, while the underlying VLA remains frozen. Across $π_{0.5}$, SmolVLA, X-VLA, and MolmoAct2 on LIBERO, MetaWorld, LIBERO-Plus, and RoboTwin 2.0, VLA-Rolling reduces action-chunk inference cost in all 16 model--benchmark pairs, achieving up to a $1.48\times$ speedup, while improving success rate in 14 pairs by up to $15.08$ percentage points. These results show that action tails are not merely discarded predictions, but reusable intermediate states that enable more efficient and accurate VLA inference. These results demonstrate that the action tail can support the generation of better actions with fewer denoising steps. Our project is available at https://chenbao123ww.github.io/VLA-Rolling/.
★ Learning Waypoints and Passage Events for Decentralized Multi-Quadrotor Trajectory Planning
Multi-quadrotor passage planning requires coordinated spatial routes and passage times under vehicle dynamics and actuation constraints. Optimizing these decisions jointly couples temporal allocation, nonlinear motion, and inter-vehicle interactions, creating a computational challenge for local trajectory generation. We propose a learning-optimization framework that predicts spatial waypoints and ordered passage events for independently solved quadrotor planners. A common physical time grid aligns the vehicles, with intermediate events fixed by the networks and terminal progress optimized locally to permit different completion metrics. Differentiable sequential convex programming incorporates rigid-body dynamics and rotor-thrust bounds, allowing collision and completion gradients to train both networks through the generated trajectories. In a two-quadrotor evaluation across 24 layouts, the adapted policy passes physical feasibility and sampled separation checks in all five held-out layouts. Their mean completion metric decreases by 1.52% relative to the inherited policy. The median forward time is 21.01 ms on the held-out exchange layout with matching numerical trajectory references. These results demonstrate that learned spatial and event parameters can coordinate independent rigid-body planners while preserving local vehicle constraints in the evaluated scenarios.
comment: 9 pages, 7 figures, 3 tables
★ Latent World Models Should Plan with Distances Along the Latent Manifold
Latent world models such as LeWM plan by searching for an action sequence whose predicted terminal latent is closest to the goal latent under an ambient norm, such as the Euclidean distance. Planning with such a cost assumes that proximity in the ambient latent space reflects proximity to the goal in the environment. Studying LeWM, we find that this assumption holds only in a small neighborhood around the goal, whereas farther away the L2 cost forms spurious low-cost basins that trap the planner. Yet neighborhood-based embeddings show that the latents retain the underlying physical state. We therefore hypothesize that the latents lie near a curved, low-dimensional manifold, and that the terminal cost should measure the distance along this latent manifold rather than through the ambient space. To estimate this manifold distance, we propose the Neighborhood Graph Cost (NGC), which computes shortest-path distances on a mutual nearest-neighbor graph constructed over training latents. NGC changes only the terminal cost and requires no retraining. This single change improves the planning success rate of LeWM across all four benchmark tasks and nearly solves TwoRoom and Reacher: even at the farthest goal offsets tested, NGC reaches 95% and 100% success, whereas the L2 cost falls to 27% and 81%. These results support the manifold hypothesis and our argument that latent world models should plan with distances along the latent manifold.
★ Finite-Time Capture by a Dubins Pursuer Against Unknown Evader Strategies
We consider a planar pursuit-evasion problem in which a Dubins pursuer seeks to capture a slower first-order point-mass evader that can change its heading instantaneously. First, we propose a turn-rate feedback law with a single tunable gain and derive an explicit admissible gain interval that guarantees finite-time capture against any admissible evader strategy while satisfying the turn-rate constraint. Next, we obtain an explicit capture-time bound and characterize the turning authority required for the gain interval to be non-empty. Finally, we show that the dependence on the evader's speed bound can be removed by deriving a speed ceiling, computable solely from the pursuer's parameters, below which the guarantees hold at the maximum allowable gain. Numerical simulations illustrate the effectiveness of the proposed approach.
★ Scalable, Intuitive, Mixed Reality-Based Semi-Autonomous Robotic Telemanipulation for the Execution of a Fine-Grained Bimanual Task IROS
Nowadays, robotic telemanipulation enables humans to control robots so as to execute tasks in a manner similar to how they perform them, and serves as an essential approach for collecting on-robot data for robot learning by human demonstrations. The quality of collected demonstrations directly impacts the performance of the learned policy. However, existing telemanipulation systems rely heavily on human operation, particularly for large-scale data collection. Collecting dozens or hundreds of demonstrations is a demanding process, as operators tend to fatigue quickly, especially during the execution of dexterous tasks that require sustained concentration. Recently, semi-autonomous robotic telemanipulation has emerged as a promising direction that reduces operator workload while preserving human-in-the-loop control for task execution. We therefore introduce a scalable, intuitive, semi-autonomous robotic telemanipulation system that leverages a mixed reality (MR) headset for on-robot data collection. The system autonomously executes the majority of the task trajectories and only requires the operator to make fine adjustments at key stages. We validate the effectiveness of our system by collecting data and training imitation learning policies on the execution of a high-precision, fine-grained bimanual peg-in-hole task. With lower data collection effort, our system obtains high-quality demonstrations that can be used to train fully autonomous task execution policies.
comment: 8 pages, 9 figures. Accepted to IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2026
★ Dream What I Can: Sparse Visual Planning for Long-Horizon Robot Manipulation
Video models provide rich priors for generalizable robotic manipulation, but plausible visual futures are not always executable. We introduce DreamCan, a hierarchical policy that grounds video models through sparse visual planning: a high-level planner adapted from a pre-trained video model and a low-level visuomotor policy. Rather than predicting dense trajectories, the high-level planner generates sparse visual subgoals that balance reachability and task utility. From offline, action-free robot videos, we estimate these quantities and use dynamic programming to select subgoal sequences without manual annotations. The resulting subgoals are more reliably attained than uniformly spaced alternatives, and the full system outperforms language, dense-video, and uniform-subgoal planners, reaching high planning success rates on Language-Table tasks and 91.2% success on LIBERO-Long with a 0.1B-parameter low-level policy. Additional suboptimal videos and test-time compute further improve success. The system also performs challenging real-world bimanual tasks, including box folding and packing with unseen object combinations. More results are available at https://dreamcan.github.io/.
♻ ★ Perceptive Variable-Timing Footstep Planning for Humanoid Locomotion on Disconnected Footholds
Walking over disconnected footholds requires selecting landing regions while adapting step timing to the robot's motion. We present a perceptive model predictive planner that combines discrete region selection, foot placement, and current-step duration in a mixed-integer quadratic program using step-to-step Divergent Component of Motion (DCM) dynamics. Onboard depth data are fused into a probabilistic heightmap in the stance foot frame. Convex steppable regions are extracted and shrunk by margins that increase with estimated height uncertainty. Linear constraints impose capturability-based bounds on DCM evolution and limit swing-foot retargeting using the remaining commanded swing time. Within-step replanning updates the initial DCM from measurements to respond to model mismatch and disturbances. We evaluate the framework on Digit in simulation over randomized footholds and on hardware over separated plates, with mean planner solve times of 11.4~ms and 4.0~ms, respectively.
comment: 8 pages, 6 figures, 1 table. Supplemental video at: https://xtificant.github.io/digit_fpp_pub/
♻ ★ ICAT: Incident-Case-Grounded Adaptive Testing for Physical-Risk Prediction in Embodied World Models NeurIPS 2026
Reference-image-conditioned video generators are increasingly used as neural simulators for embodied planning and policy learning, yet their ability to anticipate physical hazards is rarely evaluated. We introduce ICAT, which grounds test construction in incident reports and safety guidance through structured risk memories, constrained case generation, and human verification. The resulting benchmark contains 909 evaluation units with explicit initial conditions, triggering actions, and documented consequences. Across six generation systems, severe outcomes are frequently missing or attenuated even when the initial state is preserved and the risky action is executed: the strongest system achieves 23.5% conditional strict success under best-of-eight selection. Expert auditing, image-generator sensitivity analysis, matched safe-risky pairs, and alternative-outcome scoring provide complementary checks on this system-level finding. ICAT exposes a gap between reproducing risky actions and generating their documented consequences.
comment: Accepted to NeurIPS 2026 (Poster)
♻ ★ EDITH: Teaching Robot to Follow Nonverbal Human Intent from Egocentric Vision and Gaze
For natural human-robot interaction, a robot must understand human intent expressed not only through language but also through nonverbal signals such as gestures and gaze. However, current robot policies rely on language instructions as the sole interface for conveying intent, leaving nonverbal signals unused and placing the full burden of communication. In this work, we present EDITH, a robot framework that captures the human's nonverbal signals through continuous streams of first-person view and gaze from smart glasses, and uses them alongside language instructions as inputs to the robot policy. Our hardware system streams the human's first-person view, gaze, and speech to the robot in real time, transcribing the speech into language instructions. To handle these rich but noisy signals, we design a hierarchical policy in which a high-level policy infers the human's intent and produces a sequence of subtasks, where each subtask is represented as a fine-grained instruction paired with a keyframe that grounds the intent in the scene (e.g., the frame where the human points at the target object). A low-level policy then executes these subtasks. In our experiments on human-robot interactive tasks, EDITH enables the robot to act on the human's nonverbal signals even when intent is expressed only briefly, and significantly reduces user effort to convey intent compared to using language instructions alone. Visit our project page for source code and real-robot demo videos.
comment: We provide video demos and code in: https://project-edith.github.io
♻ ★ ELAN4D: Embodiment-Centric 4D Supervision for Vision-Language-Action Models via Plug-and-Play Adaptation
Vision-Language-Action (VLA) models have shown promise for robotic manipulation, yet most existing policies operate reactively by directly regressing actions from current observations, without explicitly modeling future dynamics. This limits their ability to generalize under out-of-distribution perturbations. To address this issue, we propose ELAN4D, an embodiment-centric, 4D-aware training framework that enhances VLA policies with future robot keypoint tracks as predictive spatio-temporal supervision. Using only forward kinematics from proprioceptive states, we derive 3D displacement tracks of robot keypoints, such as joints and the end-effector, with negligible preprocess cost. These tracks provide metric and compact supervision without requiring external trackers or reconstruction. A plug-and-play auxiliary branch with a lightweight track decoder injects this 4D signal into the action expert while preserving the pretrained vision-language backbone through gradient isolation. The track decoder is discarded during inference, leaving the base policy interface unchanged. Extensive experiments on LIBERO, LIBERO-Plus, RoboTwin2.0 and real-world manipulation tasks demonstrate that ELAN4D consistently improves over strong VLA baselines, achieving the best overall performance and substantial gains under out-of-distribution perturbations, including camera, background, and layout shifts. These results highlight the effectiveness of embodiment-centric 4D supervision for building more robust and generalizable manipulation policies.
♻ ★ What Does It Take For A Generalist Robot Policy To Adapt From Minimal Data?
A central goal in robot learning is to move beyond task-specific human data collection toward robots that improve through autonomous interaction. Yet fully autonomous learning remains difficult with current policies: sparse rewards and weak zero-shot exploration often prevent online learning from discovering successful behavior. As a first step towards fully autonomous learning, we propose the minimal-data adaptation regime, in which a pretrained robot policy must learn a new task from as little as one demonstration followed by autonomous online interaction. We establish that, although pretrained policies fail zero-shot on many new tasks, a single demonstration suffices to direct their behavioral prior to coarse task-following behavior, albeit with unreliable performance. From this insight, we isolate the key design principles required to convert coarse task-following into reliable task success. These include (1) the surprising efficacy of pretrained VLA representations to support sample-efficient value learning from online interaction (2) the sufficiency of residual policy learning to learn corrective behavior and (3) the need for careful data-balancing to prevent exploration collapse. As a representative of these principles, we propose MiDAS, a simple offline-to-online RL recipe that combines few-demonstration behavior cloning with value-based residual learning over frozen VLA representations. Across LIBERO and RoboCasa tasks, MiDAS substantially improves performance from single-demonstration initializations, including policies that succeed only rarely, validating the principles highlighted above. On a bimanual YAM platform, starting from a brittle initial policy, MiDAS improves task success after approximately six hours of online interaction.
♻ ★ Actron3D: Learning Actionable Neural Functions from Videos for Transferable Robotic Manipulation ICRA 2026
We present Actron3D, a framework that enables robots to acquire transferable 6-DoF manipulation skills from monocular, uncalibrated, RGB-only human demonstration videos. Our key idea is to represent manipulation knowledge within a video as a continuous neural function over object space. At the core of Actron3D lies the Neural Affordance Function, which distills geometry, visual features, contact priors, and action flows from diverse demonstration videos into a compact 3D neural representation. During deployment, we adopt a hierarchical pipeline that retrieves the matched affordance function and transfers encoded manipulation knowledge to novel objects through coarse-to-fine differentiable optimization. Leveraging the continuous nature of Neural Affordance Function, the framework performs spatial queries over multimodal features to align demonstrations with observations and generates precise 6-DoF manipulation policy. Experiments in both simulation and the real-world demonstrate that Actron3D significantly outperforms prior methods, achieving a 14.9 percentage point improvement in the average success rate across 13 tasks while requiring only 2-3 demonstration videos per task.
comment: ICRA 2026, 8 pages, 5 figures
♻ ★ Continually Evolving Skill Knowledge in Vision Language Action Model
Vision-language-action (VLA) models show promising knowledge accumulation ability through pretraining, yet continual learning remains challenging. Existing continual imitation learning (CIL) methods often require additional parameters or modules, limiting scalability for large VLA models. We propose Stellar VLA, a knowledge-driven CIL framework without parameter growth, featuring two variants: T-Stellar for task-centric modeling and TS-Stellar for hierarchical modeling of reusable skills across tasks. Stellar VLA jointly optimizes task representations and a self-evolving knowledge space, with knowledge-posterior-guided expert routing using knowledge relations and Top-K semantic embeddings for task specialization. Experiments on LIBERO demonstrate strong performance against VLA and CIL baselines with only 1% replay. Dual-arm real-world experiments validate knowledge transfer across embodiments and scenes. TS-Stellar excels in hierarchical manipulation, while visualizations demonstrate knowledge retention and task discovery. Project Website: https://stellarvla.github.io/
♻ ★ When Does Backpropagating Through Policy Memory Matter? Separating Representation Credit from Closed-Loop Credit
When a policy with memory is trained through a differentiable simulator, credit can reach what it stores along two routes: \emph{representation credit} teaches the network that writes the memory what to summarize, while \emph{closed-loop credit} travels back through the physical states that earlier actions produced and that the memory later stored. Stopping gradients at stored history, as truncated backpropagation and Transformer-XL do, removes both routes at once. Keeping every forward computation identical and removing one route at a time, we trained Transformer policies for quadrotor tracking to determine when each route matters. Credit through memory improved learning only when velocity observations were noisy or hidden. When velocity was hidden, mildly decaying the gradient between consecutive physical states ($γ= 0.8$) eliminated the truncation cost of removing closed-loop credit while leaving representation credit essential, a separation that held in 14 of 16 new seeds. Two further systems bound this result: in a fixed-wing aircraft, representation credit could replace closed-loop credit on its own, leaving no cost to separate, and in a cart-pole, training with full state-edge credit was too unstable to compare the routes. In both the quadrotor and a fully observed vessel model, static gradient norms failed to predict which cut cost more, so memory truncation choices are best guided by short training runs.
♻ ★ Accelerating VLA Inference with Perturbation Informed Execution
Action-chunking vision-language-action (VLA) models predict sequences of future robot actions. Under fixed-horizon execution, the robot executes a fixedlength prefix of each predicted chunk before invoking the policy again, leaving the remaining predictions unused. For a given number of executed actions, short execution horizons require more policy calls. Extending the execution horizon amortizes policy inference cost over more actions. However, uniformly applying long horizons across episodes can limit opportunities for precise corrections. To address this limitation, we propose PICES (Perturbation-Informed Chunk and Execution Horizon Selection), a training-free method that jointly selects an action chunk and the number of actions executed before the next policy call. Within a single policy call, the action expert generates a batch of action chunks from multiple initial noise samples and their locally perturbed counterparts, using the same conditioning features across all samples. PICES exploits differences between corresponding actions in each nominal chunk and its counterpart to determine candidate-specific execution horizons and select a nominal chunk for execution. Experiments on simulation benchmarks and real-world manipulation tasks demonstrate that PICES achieves speedup by up to 2.3x in wall-clock policy inference time on Jetson Orin and Thor relative to fixed horizon baselines while improving the success rate.
comment: 23 pages, 11 figures, 10 tables
♻ ★ AffordanceWAM: Affordance-Aware Joint World-Action Modeling for Robot Manipulation
Generalizable robot manipulation requires predicting how a scene will evolve, identifying where interactions are feasible, and determining how to act. Action-labeled robot videos directly supervise control but are costly and limited in diversity, whereas egocentric human videos capture diverse interactions but lack robot actions and differ in embodiment and appearance. We introduce AffordanceWAM, an affordance-aware generative World Action Model that represents object-centric spatiotemporal affordance through Scalar Affordance and Affordance Heatmap, within the generated future World. This representation grounds visual prediction in task-relevantant objects and interaction regions for action generation, and provides shared interaction targets across human and robot videos. Built on a pretrained video diffusion Transformer, AffordanceWAM uses separately parameterized World and Action Experts, coupled through Masked Joint Self-Attention, to jointly predict future RGB observations, Scalar Affordance fields, Affordance Heatmaps, and continuous robot actions under a unified flow-matching objective. Human videos supervise all three future-World streams, whereas robot trajectories additionally provide action supervision, enabling transfer without human action labels or retargeting. Experiments on RoboCasa, CALVIN ABC$\rightarrow$D, and real-world manipulation demonstrate consistent gains over RGB-only and robot-data-only baselines. Under fixed robot supervision, RoboCasa performance improves monotonically as affordance-annotated human video scales. These results support affordance as an effective interface for both vision-language-action learning and human-to-robot transfer.
comment: 20 pages, 4 figures, including supplementary material. Updated references and formatting. Project page: https://skywalker-yqz.github.io/AffordanceWAM/ Code: https://github.com/Skywalker-yqz/AffordanceWAM
♻ ★ AffordanceVLA: A Vision-Language-Action Model Empowering Action Generation through Affordance-Aware Understanding
Vision-Language-Action (VLA) models leverage the rich world knowledge of pretrained vision-language models (VLMs) to enable instruction-following robotic manipulation. However, the structural mismatch between VLM semantic spaces and embodied control policies often hinders the learning of precise perception-action mappings. To address this challenge, we propose AffordanceVLA, a scalable, unified framework that introduces structured affordance learning as a task-oriented and object-centric intermediate representation to establish a more precise and robust perception-action mapping. Specifically, we progressively model manipulation priors through three complementary components - Which2Act, Where2Act, and How2Act - which jointly produce spatially grounded, semantically conditioned, and action-coupled intermediate representations, thereby naturally bridging vision, language, and action. We integrate these modules into a Mixture-of-Transformer (MoT) architecture and adopt a three-stage progressive training curriculum that leverages 8.8M affordance-centric grounding and pointing samples, translating scalable vision-language supervision into improved robotic manipulation. To overcome the scarcity of dense affordance labels in robotic datasets, we also develop a robust automated data augmentation pipeline. Extensive experiments on ID & OOD simulation and real-world demonstrate that AffordanceVLA outperforms strong VLA baselines across diverse manipulation scenarios, validating structured affordance learning as a scalable and robust bridge that integrates multi-source vision-language supervision while connecting pretrained VLM semantics to embodied robot control.
comment: 37 pages, 9 figures, including supplementary material. Updated references and formatting. Project page: https://skywalker-yqz.github.io/AffordanceVLA/ Code: https://github.com/Skywalker-yqz/AffordanceVLA
♻ ★ Latent-WAM: Latent World Action Modeling for End-to-End Autonomous Driving
We introduce Latent-WAM, an efficient end-to-end autonomous driving framework that achieves strong trajectory planning through spatially-aware and dynamics-informed latent world representations. Existing world-model-based planners suffer from inadequately compressed representations, limited spatial understanding, and underutilized temporal dynamics, resulting in sub-optimal planning under constrained data and compute budgets. Latent-WAM addresses these limitations with two core modules: a Spatial-Aware Compressive World Encoder (SCWE) that distills geometric knowledge from a foundation model and compresses multi-view images into compact scene tokens via learnable queries, and a Dynamic Latent World Model (DLWM) that employs a causal Transformer to autoregressively predict future world status conditioned on historical visual and ego status representations. Extensive experiments on NAVSIM v2 and HUGSIM demonstrate new state-of-the-art results: 89.3 EPDMS on NAVSIM v2 and 28.9 HD-Score on HUGSIM, surpassing the best prior perception-free method by 3.2 EPDMS with significantly less training data and a compact 104M-parameter model. The code and model weights have been open-sourced at https://github.com/yukki555/Latent-WAM.
♻ ★ Online World Modeling Enables Real-World Inverse Reinforcement Learning from Observation
Current methods in robot learning are fundamentally bottlenecked by one or more of: hand-designed rewards, simulation modeling, or action supervision (e.g. teleoperation) each requiring significant domain expertise, engineering effort, and robot-operator labor. Towards reducing these bottlenecks, this work pursues observational learning via Inverse Reinforcement Learning from Observation (IRLfO) in which only access to task observations (e.g. video) is assumed. Due to the challenging setting and limitations of RL methods, IRLfO has thus far remained impractical for real-world robot learning. Here, we present the first IRL method to learn visual manipulation in the real world from scratch, and the first real-world demonstration of positive online transfer across visual manipulation tasks from scratch. In under 40 minutes, MPAIL2 learns pick-and-place from scratch to 82%, where RL and BC with equal interaction and demonstration budgets reach only 0% and 12% despite their reward and action supervision. Interactive project page with training videos: https://uwrobotlearning.github.io/mpail2/
comment: CoRL 2026
♻ ★ EVA: Aligning Video World Models with Executable Robot Actions via Inverse Dynamics Rewards
Video generative models offer a promising basis for robotic planning by predicting future visual rollouts from observations and task instructions. However, their training objectives typically lack explicit constraints on robot executability. Generated rollouts can exhibit embodiment-related artifacts, including arm deformation and temporal discontinuities, while the actions decoded from these rollouts may be unstable or violate robot motion limits. We term this mismatch between generated robot behavior and embodiment-specific execution requirements the executability gap. We introduce Executable Video Alignment (EVA), a post-training framework that uses action-space feedback to improve both the visual integrity of generated robot motion and the feasibility of decoded actions. EVA trains an inverse dynamics model (IDM) on robot demonstrations and freezes it to decode actions from generated videos. We compute a reward from these actions that penalizes non-smooth motion and violations of velocity and acceleration limits, and use it to guide reinforcement-learning updates to the video generator. By feeding decoded-action constraints back into video generation, EVA links visual and action-level improvements through a shared alignment signal. Experiments on RoboTwin, LIBERO, and a real bimanual robot show that EVA reduces embodiment-related visual artifacts and improves task success when executing IDM-decoded actions. Further experiments show that video models fine-tuned with EVA generate more effective video--action pairs for training downstream VLAs.
comment: Project page: https://eva-project-page.github.io/
Multiagent Systems
★ How Far Can an Agent Push? Trust-Safety Margins in Live Language-Based Economic Games
The previous version contained Unicode em dashes. Paste this ASCII-only version: Game theory identifies when a strategic action should be accepted, but not how long a live, language-based counterpart will continue to accept it. We entered the GLEE competition in its final week with one protected agent, Milton, and four instrument agents that ran deterministic, per-game randomized trials of policy changes; Milton received changes only through pre-specified gates or registered, rollback-guarded production rules. In negotiation, learned pricing improves adjusted percentile by 0.071 (n=21,168); in bargaining, a continuation model conditioned on the rejected offer did not improve on a recency-weighted incumbent and was retired. In persuasion, a preregistered five-dose experiment (7,391 assigned games, blinded until a fixed stop) shows that recommending every unit raises sales by 0.194 of the maximum (95% CI [0.147, 0.241]) where the receiver's slack makes obedience rational (52% of the Bayesian benchmark), lowers them where it does not (-0.067), and produces a larger early-to-late purchase decline than honest recommendations (-0.123). A rule fixed before unblinding now routes Milton's dose per cell. Milton stood 11th of 460 agents in bargaining at the fixed stop; we also report the close, where the final day's pool shift moved all five agents' bargaining ratings together while their play was unchanged, and three dispatch defects were found by checking delivered behavior against the design.
★ Explore or Confirm? LLM Agents Misallocate Measurements Under Noise
As model reasoning gets cheaper, the physical experiment becomes the slow and costly step in discovery. Each experiment returns a noisy reading: a high value can mean a good candidate or a lucky draw. With a fixed budget, an AI agent must keep choosing between exploring a new candidate and confirming a promising one by re-testing it. We measure that choice in Wellocate, a simulated assay campaign with known true values. It is built on a generated landscape and on two real protein datasets, GB1 and PhoQ, scrambled so no model can answer from memory. We test five language models as minimal agents, with no tools and no memory beyond the history in each prompt. They misallocate in both directions. Three over-confirm, spending most of the budget re-testing. On the generated landscape one under-confirms, trusting single readings, and there every model ends with a worse final pick than a basic scripted strategy. Each failure has its own cause. The over-confirmers run short of new candidates worth trying: a list of untried candidates cuts their re-testing and, on real protein data, makes two of them pick much better. A running summary of their readings or a description of the landscape changes it by at most five percentage points. For the under-confirmer, re-testing its top candidates in a final round, replayed on its own campaigns, recovers only about a quarter of what it loses by picking the wrong candidate. This suggests confirmation has to run through the whole campaign. We release Wellocate as a benchmark and reinforcement-learning environment with frozen splits and scripted baselines. With it, anyone can measure how an agent splits its budget before trusting it with real experiments, and train agents to split it better. Code, data and the environment are available at https://github.com/Xstendyl/wellocate.
comment: 38 pages, 12 figures. Code, data and environment: https://github.com/Xstendyl/wellocate
★ Scam Town: Can Scam Education Spread Like a Vaccine?
Most older adults who hear a warning about scams hear it from someone they know: a relative, a neighbour, or a friend who was targeted. Yet research on scam prevention almost always studies one person at a time. We ask whether scam awareness can spread through a community and protect it the way a vaccine does, since a warning protects whoever hears it and can be passed on. To find out, we built Scam Town, a simulated community in which a large language model plays each resident. Scam centres phone the residents, and after a call a resident can warn the people they know. Word of mouth helped, but it did not take off on its own: each warning led to fewer than one more, so a community cannot count on warnings "going viral". Across 80 simulated towns, everyday talk cut the number of people scammed by roughly a fifth. The largest gains came when an official warning, from a bank, a senior centre, or an agency, reached most of a town and residents then talked about it: in simulation (a combination we did not test live) the two together cut scams by about half. Protection was greatest when warnings reached the people scammers actually call, who here were older residents. Because our residents are language-model stand-ins rather than real people, we treat these findings as predictions to test with people, not as proof. They point to a simple recipe for scam education: give people a clear story they can retell, design campaigns to be passed along, and reach first those most likely to be called.
comment: 28 pages, 12 figures, 11 tables
♻ ★ Belief Coevolution in a Social Network of Generalist and Specialist Large Language Models: A Controlled Study
Large language models (LLMs) are increasingly deployed in multi-agent environments. However, the processes by which beliefs form and propagate among interacting LLMs remain poorly understood. We introduce CoevolveSim, a framework for studying belief diffusion within networked LLM populations. CoevolveSim allows us to isolate and study three factors: domain specialization, social-role assignment, and social network structure. Within this framework, generalist and specialist LLM agents exchange and revise beliefs. In each round, an LLM agent observes a summary of its neighbors' beliefs before updating its own. We find that persona-style role assignment reshapes individual belief revision but has little effect on population-level consensus. In contrast, introducing finetuned specialist LLMs produces a large change in consensus and is accompanied by persistent asymmetries in outgoing influence. We further show that an opinion-dynamics surrogate combining individual persistence with population-level belief composition reproduces consensus with high fidelity in every scenario, whereas predicting individual final-round beliefs in heterogeneous LLM populations additionally benefits from agent identity.
comment: 37 pages (9 pages of main text), 8 figures, 18 tables
♻ ★ Reward and Behavioral Measures of LLM Coordination in Dining Philosophers
Does high reward indicate productive coordination among language-model agents sharing resources? We examine this question in dining philosophers, where instances of the same model must acquire shared forks to eat. We evaluate seven model conditions across 630 episodes at five, ten, and fifteen philosophers, measuring scalar reward, deadlock, and meal throughput. At five philosophers, Mistral-Small 24B receives mean reward 0.827 while producing no meals. The reward decomposition explains this mismatch: avoiding deadlock earns a bonus without requiring meal production. To assess a policy trained with this reward, we compare the final checkpoint from a single Qwen3-14B group relative policy optimization (GRPO) training run with its base model. At five philosophers, the trained model has lower throughput (0.008 versus 0.020) and a higher deadlock rate in point estimates. Differences in mean episode reward are not statistically significant at any evaluated philosopher count. These findings show that the evaluated reward can assign high scores without task progress and support reporting behavioral measures alongside reward when comparing models and evaluating fine-tuned policies.
comment: 16 pages, 15 figures
♻ ★ Large Language Models over Networks: Collaborative Intelligence under Resource Constraints
Large language models (LLMs) are transforming society, powering applications from smartphone assistants to autonomous driving. Yet cloud-based LLM services alone cannot serve a growing class of applications, including those operating under intermittent connectivity, sub-second latency budgets, data-residency constraints, or sustained high-volume inference. On-device deployment is in turn constrained by limited computation and memory. No single endpoint can deliver high-quality service across this spectrum. This article focuses on collaborative intelligence, a paradigm in which multiple independent LLMs distributed across device and cloud endpoints collaborate at the task level through natural language or structured messages. Such collaboration strives for superior response quality under heterogeneous resource constraints spanning computation, memory, communication, and cost across network tiers. We present collaborative inference along two complementary and composable dimensions: vertical device-cloud collaboration and horizontal multi-agent collaboration, which can be combined into hybrid topologies in practice. We then examine learning to collaborate, addressing the training of routing policies and the development of cooperative capabilities among LLMs. Finally, we identify open research challenges including scaling under resource heterogeneity and trustworthy collaborative intelligence.
comment: IEEE Communications Magazine
♻ ★ Can Trustless Agents Be Trusted? An Empirical Study of the ERC-8004 Decentralized AI Agent Ecosystem
As autonomous AI agents increasingly transact across organizational boundaries, a fundamental trust challenge emerges: how can an agent assess whether an unknown counterpart is trustworthy? The ERC-8004 protocol addresses this challenge with the first permissionless trust layer for AI agent economies, built around three on-chain registries for Identity, Reputation, and Validation. Despite its rapid adoption, the protocol has not been studied empirically, leaving it unclear whether the information it records provides a trustworthy basis for decision-making. To address this gap, we present the first empirical study of ERC-8004 across three chains: Ethereum, BNB Smart Chain (BSC), and Base, covering the period from protocol deployment through May 13, 2026. We crawl on-chain Identity and Reputation events, off-chain files, and x402 payment transactions. On the identity side, we find that most registrations are placeholders rather than active agents, with only a small fraction (3%, 4%, and 15% across Ethereum, BSC, and Base) exposing a valid ERC-8004 registration file with at least one live service endpoint. On the reputation side, we show that the Registry, as currently deployed, cannot function as a trust signal: values are not commensurable, feedback records are rarely grounded in verifiable interactions, and reputation can be manipulated at minimal cost. Consistent with these design weaknesses, we find that a substantial fraction of reviewers (73.5%, 59.2%, and 90.6% across Ethereum, BSC, and Base) exhibit coordinated Sybil behavior. After removing Sybil-flagged feedback, 15.8%, 77.9%, and 86.8% of rated agents, respectively, are left with no valid feedback. We then turn these findings into concrete recommendations for future revisions of ERC-8004. Our study yields actionable protocol-design implications and establishes an empirical baseline for research on AI agent markets.
♻ ★ Learning Rational Strategies for Temporal Objectives in Multi-Agent Systems
We study rational synthesis for stochastic multi-agent systems with unknown dynamics, where each agent is equipped with its own probabilistic temporal objective specified using Probabilistic Linear Temporal Logic on Finite Traces ($\mathsf{PLTL}_f}$). We impose solution concepts (e.g., Nash equilibrium, subgame-perfect equilibrium) as constraints on the joint strategy profile, using payoffs derived from each agent's probability of satisfying its temporal objective. In particular, we formulate per-agent rational synthesis as finding a strategy profile -- if one exists -- such that (i) the designated agent's probabilistic temporal constraint is satisfied, and (ii) the profile represents a stable solution under a chosen solution concept. We give conditional convergence guarantees for tabular Nash-Q learning, certify satisfaction with simultaneous finite-sample confidence bounds, and assess strategic stability with approximate best-response checking. Our approach operates in a stochastic \emph{Multi-Agent Reinforcement Learning} (MARL) environment, where model-free learning algorithms attempt to attain extremal satisfaction probabilities for temporal objectives automatically compiled into reward automata. Our approach is empirically validated using 8 general-sum concurrent stochastic games.
♻ ★ KernelClaw: A Multi-Agent Framework for GPU Kernel Optimization with Dual-Memory Mechanisms
Improving GPU kernel efficiency is crucial for efficient AI systems. Recent work has explored leveraging large language models (LLMs) for GPU kernel generation and optimization. However, existing LLM-based kernel optimization pipelines typically rely on opaque, implicitly learned heuristics within the LLMs to determine optimization strategies. This leads to inefficient trial-and-error and weakly interpretable optimizations. Our key insight is to replace implicit heuristics with expert optimization knowledge that is aware of task trajectories. Specifically, we present KernelClaw, a multi-agent framework with a dual-level memory architecture. KernelClaw operates by coordinating agents with long-term memory of reusable expert knowledge and short-term memory to prevent repetitive backtracking. On KernelBench Levels 1-3, KernelClaw achieves a 100% success rate and average speedups of 6.98x, 7.89x, and 1.94x over Torch Eager on Levels 1, 2, and 3, respectively, outperforming prior baselines. Code is available at https://github.com/0satan0/KernelMem/.
♻ ★ Open-Ended Video Game Glitch Detection with Agentic Reasoning and Temporal Grounding
Open-ended video game glitch detection aims to identify glitches in gameplay videos, describe them in natural language, and localize when they occur. Unlike conventional game glitch understanding tasks which have largely been framed as image-level recognition or closed-form question answering, this task requires reasoning about game-specific dynamics such as mechanics, physics, rendering, animation, and expected state transitions directly over continuous gameplay videos and distinguishing true glitches from unusual but valid in-game events. To support this task, we introduce VideoGlitchBench, the first benchmark for open-ended video game glitch detection with temporal localization. VideoGlitchBench contains 5,238 gameplay videos from 120 games, each annotated with detailed glitch descriptions and precise temporal spans, enabling unified evaluation of semantic understanding and temporal grounding. We further propose GliDe, an agentic framework with three key components: a game-aware contextual memory for informed reasoning, a debate-based reflector for multi-perspective glitch detection and verification, and an event-level grounding module that recovers complete glitch intervals from fragmented temporal evidence. We also design a task-specific evaluation protocol that jointly measures semantic fidelity and temporal accuracy. Experiments show that this task remains highly challenging for current multimodal models, while GliDe achieves substantially stronger performance than corresponding vanilla model baselines.
comment: Accepted at ACM Multimedia 2026
Systems and Control (EESS)
★ Stability and Stabilization of Kronecker-Sum-Coupled Multilinear Systems
Coupled dynamical systems with multi-dimensional state structures arise in many applications such as spatiotemporal processes and large-scale interconnected systems. However, existing studies primarily focus on matrix-based coupled systems or tensor-based dynamical systems without couplings. The properties of coupled tensor-based systems remain largely unexplored. In this article, we study Kronecker-sum-coupled multilinear systems under the transform-based tensor product framework. We establish fundamental tensor spectral properties showing that stability can be characterized through lower-dimensional subsystem dynamics together with slice-wise matrix stability in the transform domain. We further develop optimization conditions for state-feedback stabilization and show that the stabilization problem can be decomposed into independent subsystem problems. Finally, we analyze the computational complexity of the proposed framework and demonstrate its substantial computational advantages over unfolding-based approaches using both synthetic systems and multi-inertia motor systems.
comment: 6 pages, 4 figures
★ Safe Barrier Function Control for Grid-Following Inverters under Weak Grid Conditions
This paper proposes a barrier control approach for the safe regulation of grid-following voltage-source-converters (VSC), operating in weak grid conditions. The proposed controller aims to enhance the transient performance and synchronization properties of the VSC, providing frequency and current limiting guarantees by employing smooth control barrier functions. First, a dynamic model is derived that includes the dynamic state interactions that are omitted in quasi-steady-state approaches, revealing the destabilizing elements both during transient and steady-state performance. Then, it is analytically proven that the proposed control law guarantees frequency limitation and inner current evolution in prescribed safety regions, while counteracting the destabilizing strong coupling between PCC voltage and injected current. In addition, sufficient conditions are formulated that guarantee the stability of the closed-loop system with respect to desired operating points. The theoretic results are verified in simulations of case studies operating in varying grid strength.
comment: 10 pages
★ Distributionally Robust Chance-Constrained Reinforcement Learning for Home Energy Management under Electric Vehicle Uncertainty
The electrification of residential energy systems and adoption of electric vehicles (EVs) create demand-side flexibility. However, random driving behaviour makes it difficult to ensure EV readiness at departure and maintain thermal comfort. To address this, we formulate residential scheduling as a distributionally robust (DR) chance-constrained Markov decision process, whose 2-Wasserstein ambiguity set guards against tail risk and distribution shift in EV behaviour. The intractable DR chance constraint is then replaced by a sufficient tightened expectation constraint under the nominal distribution. The resulting problem is solved by a Lagrangian soft actor--critic with distributional cost critics, and an online boundary guard enforces EV reachability and thermal comfort within the actuator limits at every step. Five controllers are compared over one year in Wellington, New Zealand, under a degraded forecast. The proposed controller is the cheapest controller that meets the EV target on every day and both chance constraints under forecast error and random driving behaviour. Compared with the rule-based baseline, the only other controller to do so, it lowers the annual cost by 8.6% and the comfort violation hours by 69%. The ablation study confirms that the boundary guard ensures pointwise feasibility within the actuator limits, and the DR chance constraint reduces tail risk.
★ Generalized and Unified Power Flow Framework for Converter-Dominated AC/DC Power Grids
Modern power grids increasingly rely on power electronic interfaces, requiring steady-state power flow analysis to accommodate diverse operating modes and network structures. Conventional approaches often rely on system-specific formulations and sequential approaches to handle such complexity. However, these approaches may limit applicability to diverse grid configurations and lead to convergence and computational issues. To address these limitations, this paper proposes a generalized and unified Newton-Raphson (NR) power flow framework for AC/DC hybrid grids. For generalized operational modeling, conventional and droop operating modes of generators and interlinking converters are represented through a common set of coefficients. For generalized structural modeling, models of AC transformers and DC/DC converters accommodate different transformer winding configurations and converter operating modes. For a unified solution, augmented variables are introduced to incorporate droop deadbands, output limits, and converter station losses into a single NR formulation, eliminating the need for sequential outer loops. To validate the effectiveness of the proposed framework, its results are compared with those obtained from existing power flow methods and electromagnetic transient simulations across transmission networks, distribution networks, and microgrids. The implementation is made publicly available to facilitate its use in analyzing AC/DC hybrid grids.
comment: Submitted to IEEE Transactions on Power Systems. 10 pages + 10-page appendix (Appendices A-C included at the end). Code: https://github.com/grid-modernization-lab-kor/unigrid-acdc-powerflow
★ Dynamic Skill-Based Routing for Call Centers with Transfers: A Value-Based Framework
We consider a multi-class, multi-pool queueing model of a telephone call center in which a service completion does not always end the call. The caller may be transferred to another agent for a second service, so the assignment of an agent to a call affects not only the caller's waiting time but also whether the call is resolved and how much further work it creates. We model these transfers as Markovian routing from original to rework classes, and the system manager dynamically assigns agent pools to classes so as to maximize the value generated by service, net of transfer penalties and waiting and abandonment costs, over a finite horizon. Focusing on the Halfin-Whitt heavy-traffic regime, we derive an approximating Brownian control problem. Its Hamilton-Jacobi-Bellman equation is a semilinear partial differential equation whose dimension equals the number of classes. Building on the deep splitting method of Beck et al. (2021), we develop a simulation-based computational method that approximates the value function and its gradient by neural networks, and we use it to construct an implementable routing policy for the original (prelimit) system. We calibrate the model to publicly available data from a large U.S. bank call center. In all test problems we consider, the proposed policy outperforms the best available benchmark policy we could find.
★ Learning Waypoints and Passage Events for Decentralized Multi-Quadrotor Trajectory Planning
Multi-quadrotor passage planning requires coordinated spatial routes and passage times under vehicle dynamics and actuation constraints. Optimizing these decisions jointly couples temporal allocation, nonlinear motion, and inter-vehicle interactions, creating a computational challenge for local trajectory generation. We propose a learning-optimization framework that predicts spatial waypoints and ordered passage events for independently solved quadrotor planners. A common physical time grid aligns the vehicles, with intermediate events fixed by the networks and terminal progress optimized locally to permit different completion metrics. Differentiable sequential convex programming incorporates rigid-body dynamics and rotor-thrust bounds, allowing collision and completion gradients to train both networks through the generated trajectories. In a two-quadrotor evaluation across 24 layouts, the adapted policy passes physical feasibility and sampled separation checks in all five held-out layouts. Their mean completion metric decreases by 1.52% relative to the inherited policy. The median forward time is 21.01 ms on the held-out exchange layout with matching numerical trajectory references. These results demonstrate that learned spatial and event parameters can coordinate independent rigid-body planners while preserving local vehicle constraints in the evaluated scenarios.
comment: 9 pages, 7 figures, 3 tables
★ LLM Agents as Suppliers of Physical System Models: Calibration and Acceptance in Modelica
To procure a model of a device, a client hands a supplier a description, sometimes with recorded data or a target response, and accepts the model against this handover. We let LLM coding agents act as the supplier on handovers with data, building Modelica models from library components for four simulated systems and a real switching magnet, and ask what acceptance certifies. Several agents reach the noise floor, but a handover admits a set of models: the description leaves parts of the structure open, and the data narrow it only to the agreed accuracy. We therefore accept by membership in this set, not by matching one reference, and add layers above behaviour: a physical invariant exposes an energy-creating model at the noise floor. Interpretability becomes a property of the delivered model, tested on a variant, read where none exists and shaped through the handover. Most failures are fits delivered with a systematic residual on their own fitting data, where the handover left open how close is close enough. In a follow-up, stating a threshold makes the agents compare against it, and more deliveries pass. The argument applies wherever a handover underspecifies a model meant to be read, not only run.
★ Finite-Time Capture by a Dubins Pursuer Against Unknown Evader Strategies
We consider a planar pursuit-evasion problem in which a Dubins pursuer seeks to capture a slower first-order point-mass evader that can change its heading instantaneously. First, we propose a turn-rate feedback law with a single tunable gain and derive an explicit admissible gain interval that guarantees finite-time capture against any admissible evader strategy while satisfying the turn-rate constraint. Next, we obtain an explicit capture-time bound and characterize the turning authority required for the gain interval to be non-empty. Finally, we show that the dependence on the evader's speed bound can be removed by deriving a speed ceiling, computable solely from the pursuer's parameters, below which the guarantees hold at the maximum allowable gain. Numerical simulations illustrate the effectiveness of the proposed approach.
★ Undulatory Swimming in Sand: Subsurface Locomotion of the Sandfish Lizard
The desert-dwelling sandfish (Scincus scincus) moves within dry sand, a material that displays solid and fluidlike behavior. High-speed x-ray imaging shows that below the surface, the lizard no longer uses limbs for propulsion but generates thrust to overcome drag by propagating an undulatory traveling wave down the body. Although viscous hydrodynamics can predict swimming speed in fluids such as water, an equivalent theory for granular drag is not available. To predict sandfish swimming speed, we developed an empirical model by measuring granular drag force on a small cylinder oriented at different angles relative to the displacement direction and summing these forces over the animal movement profile. The agreement between model and experiment implies that the noninertial swimming occurs in a frictional fluid.
♻ ★ Intent Demonstration in General-Sum Dynamic Games via Iterative Linear-Quadratic Approximations
Autonomous agents should coordinate effectively without prior knowledge of others' intents. While prior work has focused on intent inference, we address the inverse problem: how agents can strategically demonstrate their intents within general-sum dynamic games. We model this problem and propose an algorithm that balances intent demonstration with task performance. To handle nonlinear dynamic games with continuous state-action spaces, our method leverages iterative linear-quadratic game approximations and provides efficient intent-teaching guarantees: the uncertain agent's belief can be driven rapidly to the ground truth, while the demonstrating agent avoids expending effort on unnecessary belief alignment when it does not improve task performance. Theoretical analysis and hardware experiments confirm that our approach enables the demonstrating agent to reconcile task execution with belief alignment and strategically manage the information asymmetry among agents, even as its intent evolves during deployment.
comment: This paper has been accepted by IEEE TCST 2026. In this version, we addressed a few reviewers' final comments and all main results are unchanged
♻ ★ Robust Hybrid Finite-Time Parameter Estimation Without Persistence of Excitation
In this paper, we consider the problem of estimating parameters of a linear regression model. Using a hybrid systems framework, a hybrid algorithm is proposed allowing the estimate to converge to the exact value of the unknown parameters in predetermined finite time. Interestingly, we show that for the case of constant parameters, the convergence property of the hybrid algorithm holds while only requiring the regressor to be exciting on a given interval. For the case of piecewise constant parameters, the classical persistency of excitation condition is required to guarantee the convergence. Robustness of the proposed algorithm with respect to measurements noise is analysed. Finally, illustrative examples are provided showing the merits of the proposed approach in terms of scalability and the applicability for the general class of time-varying unknown parameters
♻ ★ HyperKKL: Enabling Non-Autonomous State Estimation through Dynamic Weight Conditioning ICLR 2026
This paper proposes HyperKKL, a novel learning approach for designing Kazantzis-Kravaris/Luenberger (KKL) observers for non-autonomous nonlinear systems. While KKL observers offer a rigorous theoretical framework by immersing nonlinear dynamics into a stable linear latent space, its practical realization relies on solving Partial Differential Equations (PDE) that are analytically intractable. Current existing learning-based approximations of the KKL observer are mostly designed for autonomous systems, failing to generalize to driven dynamics without expensive retraining or online gradient updates. HyperKKL addresses this by employing a hypernetwork architecture that encodes the exogenous input signal to instantaneously generate the parameters of the KKL observer, effectively learning a family of immersion maps parameterized by the external drive. We rigorously evaluate this approach against a curriculum learning strategy that attempts to generalize from autonomous regimes via training heuristics alone. The novel approach is illustrated on four numerical simulations in benchmark examples including the Duffing, Van der Pol, Lorenz, and Rössler systems.
comment: 17 pages, 6 figures, Accepted in ICLR 2026 AI & PDE Workshop
♻ ★ Dynamic Constraint Reconstruction Based Control Barrier Functions for Safety-Critical Control of High-Dimensional Manipulators
Control barrier functions (CBFs) provide formal safety guarantees for constrained nonlinear systems, but their effectiveness relies on accurate system dynamics. In high-dimensional manipulators subject to unknown disturbances and model uncertainties, fixed safety constraints constructed from nominal dynamics may become inconsistent with the actual system behavior, leading to safety degradation or excessive conservatism. This paper proposes a dynamic constraint reconstruction based control barrier function (DCR-CBF) framework for safety-critical control of disturbed robotic manipulators. An extended state observer is employed to estimate lumped disturbances online, and the estimated disturbance is incorporated into high-order control barrier functions to reconstruct safety constraints according to the estimated true dynamics. To address estimation inaccuracies, a safety margin is introduced, and a sufficient condition is derived to guarantee forward invariance under bounded estimation errors. Simulation studies on a 4-DOF excavation manipulator demonstrate that the proposed DCR-CBF method achieves zero safety violation under strong unknown disturbances while significantly improving trajectory-tracking performance compared with standard and robust CBF methods.
♻ ★ WULPUS PRO: Multi-mode Ultra-Low-Power Wearable Ultrasound and Array Imaging with CMUT Support
Wearable ultrasound enables continuous monitoring of physiological processes such as muscle dynamics, bladder volume, and cardiovascular activity. Existing fully wearable ultra-low-power platforms are limited to shallow, low-channel A-mode sensing, while larger multi-mode systems are too bulky and power-hungry for true wearability. We present WULPUS PRO, a runtime-programmable wearable ultrasound acquisition platform measuring $39\times21\times6 \mathrm{mm}$ and weighing $5 \mathrm{g}$. It integrates $30 \mathrm{V}$ excitation, 16 time-multiplexed channels, a low-noise receive front-end with up to $70 \mathrm{dB}$ gain, $14 \mathrm{MHz}$ bandwidth, time-gain compensation, and $45 \mathrm{dB}$ SNR. The platform supports deep-tissue echo acquisition up to $2.2 \mathrm{MHz}$ in RF-sampling mode and $8 \mathrm{MHz}$ in envelope-detection mode. We demonstrate B-mode imaging in a 16-channel ultra-low-power wearable with sub-millimeter axial and millimeter-scale lateral resolution in phantom experiments, while consuming $40 \mathrm{mW}$ at $50 \mathrm{Hz}$ PRF and under $60 \mathrm{mW}$ at $300 \mathrm{Hz}$ PRF. WULPUS PRO supports both piezoelectric and capacitive micromachined ultrasonic transducers, enabling integration with skin-conformal polymer-based CMUT arrays. As a host-agnostic acquisition front-end, it exposes standard data and power interfaces for BLE- and Wi-Fi-based wearable hosts. We demonstrate wireless transmission with external BLE and Wi-Fi modules and project 1-2 days of BLE operation at $50 \mathrm{Hz}$ PRF and over 3 h of Wi-Fi streaming at $300 \mathrm{Hz}$ PRF using a $300 \mathrm{mAh}$, $6.4 \mathrm{g}$ Li-Po cell. WULPUS PRO establishes a new class of fully programmable, B-mode-enabled, ultra-low-power wearable ultrasound platforms.
comment: 16 pages, 13 figures
♻ ★ Dynamic Quantum-Assisted Co-Design of Controller and Lyapunov Candidate Parameters for Nonlinear Systems
This paper proposes a dynamic quantum-assisted co-design framework for nonlinear closed-loop systems in which controller parameters and Lyapunov candidate parameters are redesigned jointly at successive decision epochs. Unlike conventional nonlinear control designs that typically tune controller gains offline and perform stability analysis separately, the proposed method embeds performance improvement and sample-based Lyapunov verification within a unified online optimization loop. The main novelty is a two-step computational structure that first contracts the continuous admissible search region around the current operating condition using a Black-Hole calibration procedure and then constructs a finite binary representation only over this calibrated region. The encoded objective is obtained from sampled nonlinear closed-loop evaluations and approximated by a local quadratic pseudo-Boolean surrogate, enabling an Ising-type Hamiltonian representation suitable for quantum-assisted optimization. Quantum imaginary time evolution is then used to explore the encoded Hamiltonian, and the resulting candidate bitstrings are decoded into continuous controller and Lyapunov parameters. To reduce dependence on the surrogate model, the decoded candidates are re-evaluated using the original nonlinear closed-loop cost and Lyapunov penalties before the final update is applied. The framework can accommodate sampled forms of different Lyapunov decay specifications by modifying the corresponding penalty and is numerically evaluated on first-order nonlinear consensus, second-order nonlinear consensus, and induction motor drive control examples. The implementation code used to generate the reported results is available at \href{https://github.com/LSU-RAISE-LAB/DQCLS-NS}{GitHub}.
♻ ★ Cyclic Reformulation-Based Identification and Polytopic Uncertainty Modeling for Multirate Systems
Modern control systems increasingly rely on heterogeneous sensors operating at different sampling rates, where intermittently missing outputs pose fundamental challenges for system identification. This paper proposes a non-iterative, control-oriented identification method for multirate systems based on cyclic reformulation. The method transforms multirate data into an expanded time-invariant representation and yields M parameter sets from a single input-output dataset, where M is the least common multiple of the sensor sampling periods. These parameter sets, expressed in a common state coordinate of the original dimension, are used in two complementary ways: their centroid, in which each output row is averaged over the phases at which that output is observed, serves as a noise-reduced nominal model, while their convex hull gives a polytopic uncertainty model whose vertices can be used directly in vertex-based LMI robust control design. Building on the exact cyclic-structure recovery established in the authors' preceding work, the paper contributes the centroid and polytopic models and evaluates them under finite noise through output-level validation statistics. Numerical simulations support both models: an illustrative SISO example shows that the centroid attains higher validation FIT than the best individual vertex and substantially outperforms an interpolation-based baseline, while a MIMO multirate sensing example confirms that the constructed polytope contains models whose validation FIT exceeds 95% on average even at the highest tested noise level. The resulting polytope is a data-derived uncertainty description obtained from a single experiment, so that the framework links multirate system identification with robust-control-oriented uncertainty modeling without iterative EM-type optimization.
comment: Revised version submitted to IEEE Access
♻ ★ Stable Inversion of Discrete-Time Linear Periodically Time-Varying Systems via Cyclic Reformulation
Inverse systems for discrete-time linear periodically time-varying (LPTV) plants are fundamental to feedforward control and iterative learning control of multirate and periodic systems. Building on the classical cyclic reformulation, which converts an N-periodic system into an equivalent LTI system at the original sampling rate, this paper derives an explicit closed-form N-periodic state-space realization of the inverse for an arbitrary uniform periodic relative degree r >= 0 (defined through the periodic Markov parameters). The technical basis is a structure-preservation property: after absorbing a phase shift for r >= 1, the LTI inverse of the cycled plant provably retains the cyclic (block-circulant/block-diagonal) structure, so that the periodic inverse matrices can be read off block-by-block, in exact correspondence with the LTI inverse. The resulting inverse system is real-valued, causal for r = 0 and r-step-delayed for r >= 1, operates at the original sampling rate, and reconstructs the input exactly under matched initial conditions, with geometric error decay otherwise. Through this correspondence, its stability is shown to be equivalent to the invariant zeros of the cycled plant - the periodic invariant zeros of the plant - lying in the open unit disk, which generalizes the minimum phase condition of the LTI case and is verifiable on a single constant matrix pencil. Numerical examples illustrate the construction, the stability characterization, and the implementation as an online periodic filter.
comment: Re-submitted to Automatica
Robotics
★ Anticipatory Gait Switching for Quadrupeds via Look-Ahead MDP Planning
Quadruped robots often switch gaits to traverse mixed terrain safely. Many practical supervisors enforce hardware constraints (e.g., dwell or guard intervals) using a transition handler. However, the switch decision itself is often driven by near-field thresholds, which can delay commitment when rough terrain is only detected shortly before contact in the local height map. We propose a lightweight height-map-only gait supervisor that unifies look-ahead and locomotion constraints in a single decision policy. A robot-centered height map is partitioned into near-, mid-, and far-field bands, each discretized into a terrain class, and band evolution is modeled as a Markov process. Gait selection is cast as a look-ahead Markov Decision Process whose transition model and cost explicitly encode platform-specific switching constraints. Indoor and outdoor experiments on a Unitree Go2 platform show reduced late switching, lower time spent using ineffective gaits on rough terrain, and improved roll stability compared to tuned reactive baselines. Overall, the MDP formulation yields anticipative constraint-aware gait switching that improves robustness without course-specific retuning.
★ DiPLO: Self-distilled Diffusion Planning for Loco-manipulation
Humanoid loco-manipulation is particularly challenging because it requires coordinating locomotion, balance, and object interaction, making it difficult to learn from limited demonstrations or from scratch through Reinforcement Learning (RL). We introduce DiPLO (self-distilled Diffusion Planning for Loco-manipulation), a framework that combines a self-distilled diffusion planner with a low-level whole-body tracking policy. Our planner is pretrained via supervised learning on motions derived from human-to-humanoid retargeting, while the tracker is trained via RL to execute them. DiPLO centers around two complementary post-training stages. First, we introduce Self-Distillation Diffusion Policy Optimisation (SD:DPO) to remove the planner's dependence on strong task conditioning, which we identify as a crucial ingredient for effective pretraining, as downstream fine-tuning does not recover comparable performance when it is omitted.. We then fine-tune this distilled model with Diffusion-Augmented Proximal Policy Optimization (DA:PPO), a maximum-entropy RL method for diffusion models, to improve task success and align generated plans with the frozen tracker's execution capabilities. Together, these stages enable motion generation through state-history and goal conditioning alone, enabling 99% task success in pick-carry-place tasks, compared to just 45% for the pretrained model. The resulting planner also generates motions for unseen object goals, with further evaluations on kicking and pushing/pulling tasks. Finally, we deploy our pipeline on the hardware, using our DA:PPO planner-tracker on a Unitree G1, achieving repeated successful executions without any real-world fine-tuning.
★ Escape Without a Recipe: Maneuver-Agnostic Mars Rover Recovery from Granular Entrapment IROS 2026
Wheeled planetary rovers immobilised in loose regolith have no principled recovery controller: scripted maneuvers are terrain-specific, and naive driving deepens entrapment. We introduce a recurrent policy that learns to escape without a predefined maneuver. The framework rests on a verified trap: in a continuum material point method (MPM) simulator we identify a configuration at sinkage-to-radius ratio z/r = 2.0 where the strongest of ten scripted controllers succeeds in only 7/32 trials, yet the regolith retains traction and continues to shear, confirming granular entrapment rather than a numerical artifact. A frozen entrapment detector, trained on 81,771 windows spanning three materials and six sinkage levels, supplies a three-state belief to the policy observation. A separate simulator mobility latch gates training rewards. Across five training seeds, mean action evaluation achieves 62.5% escape (range 28.1-81.3%; 32 trials per seed) versus 7/32 for the strongest scripted controller and 0/32 for action noise. A numerical audit identifies a coupling-buffer fault that silently discarded 9.7% of contact nodes at 16 parallel environments. Results establish recovery in the tested simulation regime; physical transfer remains unverified.
comment: IROS 2026, Space Robotics Workshop (SRW)
★ SARAM: Sparse Adapter Allocation with Reuse and Merging for Lifelong Robot Learning
Lifelong robot learning requires acquiring new tasks sequentially without forgetting earlier ones. With large pre-trained vision-language-action (VLA) backbones, adapter-based methods effectively mitigate this challenge. However, their task-specific adapters and selection modules accumulate with the task stream, creating a scalability challenge in on-disk storage and deployment cost under fixed onboard resource budgets. We propose SARAM, a lifelong learning framework that limits storage growth while keeping GPU memory and per-step inference cost independent of the task stream length. For each new task, SARAM reuses frozen adapters from previous tasks and introduces new adapters only at sparsely selected layers, with a duplication penalty encouraging them to capture information not already represented by the adapter library. At deployment, a compact signature derived from backbone activation statistics identifies the task once per episode, and the corresponding task update is merged into the backbone, leaving only the backbone in the control loop. Across four LIBERO suites, SARAM matches or exceeds adapter-based baselines while adding up to 4.5x less on-disk state per task. GPU memory usage and control frequency remain comparable to those of the backbone as the task stream grows. Experiments on a real UR5 arm with a different backbone further demonstrate retention throughout a sequential task stream.
★ When Better Policies Leave Harder Failures: Coverage Shift in VLM Success Monitoring
Can a better robot policy become harder to monitor? We study a fixed vision-language model (VLM) that judges success while a policy improves. In a controlled 20-seed tray-placement experiment, training selects only simulator-verified successes. Success rises from 66.8% to 97.0%, and accepted hover failures fall from 5.0% to 0.9% of executions. Yet the judge accepts 50.0% of remaining hovers, versus 31.1% under untrained sampling. These residual failures more often lie below the tray rim, which can conceal the gap beneath a grasped cube. A separate boundary probe accepts 148/160 below-rim hovers and 0/160 above-rim hovers. We call this change in conditional monitoring sensitivity coverage shift: learning changes which failures reach a fixed judge. A broader diagnosis covers nine judges and 1,392 stimuli. Judge-filtered experiments across eight settings further show heterogeneous effects of state-dependent acceptance. The results distinguish task improvement from failure detectability and motivate reporting absolute misses, conditional misses, and audit coverage together.
comment: 20 pages, including 10 pages of supplementary material
★ Learning Structured Locomotion References for Optimization-Based Whole-Body Control
Quadruped locomotion requires coordinating body motion,contact forces, foothold placement, and gait timing. We present a hierarchical controller in which reinforcement learning adapts these structured locomotion references while a gait scheduler, reactive footstep planner, and whole-body impulse controller retain responsibility for high-frequency execution. The model-based backbone can sustain a nominal trot independently, allowing the policy to focus on when and how to modify an already functional locomotion controller. We train the complete hierarchy in a vectorized controller-in-the-loop environment using a six-direction disturbance curriculum. Under a common simulator and evaluation protocol, the proposed controller achieves 77.29% mean survival across the tested disturbance grid, compared with 50.35% for the model-based reference controller. Training-time channel ablations identify adaptive foothold placement as the largest marginal contributor to grid survival and reveal direction-dependent effects of gait-rate adaptation. Together, these results demonstrate the viability of the proposed hierarchical controller architecture for integrating learned adaptation with optimization-based whole-body control through structured locomotion references.
comment: 8 pages, 6 figures
★ EndoVLA-OOV: Language-Conditioned Out-of-View Target Acquisition in Robotic Endoscopy
The scan-then-revisit workflow of upper gastrointestinal endoscopy requires bringing a previously observed suspicious region back into view for closer inspection. Acquiring such targets requires translating anatomical and appearance descriptions into camera motions from partial observations of a confined, deformable environment. We present EndoVLA-OOV, a language-conditioned vision-language-action policy for out-of-view target acquisition in robotic endoscopy. The policy adapts a pretrained backbone to continuous two-degree-of-freedom distal bending commands and combines initial, intermediate, and current images to guide search. To support policy learning, we provide an open-source VLA dataset containing 1,050 episodes across six tasks and approximately 262k observation-action pairs. Physical evaluation spans wet-lens and low-light conditions, two unseen stomach phantoms, and cross-embodiment transfer to an unseen ex-vivo environment. In the cross-embodiment evaluation, the full-data policy achieves 93.3% target acquisition success (28/30). In a controlled training-data comparison, expanding the corpus from 312 to 1,050 episodes increases acquisition success from 40.0% to 90.0% under the same model, training, and test configuration. Instruction and observation interventions further support the use of language and temporal context in the evaluated acquisition tasks. These results support language-guided endoscopic revisitation across the evaluated environments and robot embodiments and demonstrate the benefit of expanded task-specific demonstrations.
♻ ★ OccSim: Multi-kilometer Simulation with Long-horizon Occupancy World Models
Data-driven autonomous driving simulation has long been constrained by its heavy reliance on pre-recorded driving logs or spatial priors, such as HD maps. This fundamental dependency severely limits scalability, restricting open-ended generation capabilities to the finite scale of existing collected datasets. To break this bottleneck, we present OccSim, the first 3D occupancy world model driven simulator that requires neither HD maps nor log-derived layouts. OccSim obviates the requirement for continuous logs or HD maps; conditioned only on a single initial frame and a sequence of future ego-actions. It can stably generate over 3000 continuous frames, an 80x improvement in stable generation length over previous SoTA occupancy world models, and fuses them into maps spanning over 9 kilometers. Further qualitative stress tests show that it can roll out beyond 110K frames. OccSim is powered by two modules: W-DiT based static occupancy world model and the Layout Generator. W-DiT handles the ultra-long-horizon generation of static environments by explicitly introducing known rigid transformations in architecture design, while the Layout Generator populates the dynamic foreground with reactive agents based on the synthesized road topology. With these designs, OccSim can synthesize massive, diverse simulation streams. Extensive experiments demonstrate its downstream utility: data collected directly from OccSim can pre-train 4D semantic occupancy forecasting models to achieve up to 67% zero-shot performance on unseen data, outperforming previous asset-based simulator by 11%. When the collected data is scaled up to 5x its original size, zero-shot performance increases to 74%.
comment: 35 pages, 18 figures
♻ ★ SA-LIVO: Efficient LiDAR-Inertial-Visual Odometry with Subspace-Aware Degeneracy Handling
Light detection and ranging (LiDAR)-inertial-visual odometry fuses geometric depth with visual measurements, but its sensors fail independently: LiDAR in under-constrained geometry, vision under poor illumination or texture. Weak information is harmless, since the filter then relies on the other sensor or the inertial prediction; over-confident information is not. Existing countermeasures act on weak information or switch vision by hand-set thresholds. This article presents SA-LIVO, which fuses vision through explicit, online-estimated error models. A frame-level noise scale, estimated by covariance matching, absorbs whole-image errors, and Schmidt filtering carries the correlation of reused observations. Subspace-aware information fusion models the error shared by a frame's observations as a pose offset capping the visual information per direction; the exact fusion attenuates vision only above that cap and reduces to the information sum when the visual model is consistent. LiDAR and visual residuals share one invariant extended Kalman filter loop. On 24 handheld and aerial sequences, SA-LIVO is the most accurate or tied on 22 and stays bounded where others diverge; on the HILTI 2022 benchmark it needs 13.4 ms per frame on a laptop processor and 2.0-6.2x less memory than FAST-LIVO2, R3LIVE, and SR-LIVO.
comment: Preprint under review; not the final version, which may differ after peer review. 17 pages (11 main text + 6 supplementary material), 8 figures
♻ ★ Think before Go: Hierarchical Reasoning for Image-goal Navigation ACL2026
Image-goal navigation steers an agent to a target location specified by an image in unseen environments. Existing methods primarily handle this task by learning an end-to-end navigation policy, which compares the similarities of target and observation images and directly predicts the actions. However, when the target is distant or lies in another room, such methods fail to extract informative visual cues, leading the agent to wander around. Motivated by the human cognitive principle that deliberate, high-level reasoning guides fast, reactive execution in complex tasks, we propose Hierarchical Reasoning Navigation (HRNav), a framework that decomposes image-goal navigation into high-level planning and low-level execution. In high-level planning, a vision-language model is trained on a self-collected dataset to generate a short-horizon plan, such as whether the agent should walk through the door or down the hallway. This downgrades the difficulty of the long-horizon task, making it more amenable to the execution part. In low-level execution, an online reinforcement learning policy is utilized to decide actions conditioned on the short-horizon plan. We also devise a novel Wandering Suppression Penalty (WSP) to further reduce the wandering problem. Together, these components form a hierarchical framework for Image-Goal Navigation. Extensive experiments in both simulation and real-world environments demonstrate the superiority of our method.
comment: Accepted by ACL2026 Oral
♻ ★ Forecast-PEFT: Parameter-Efficient Fine-Tuning for Pre-trained Motion Forecasting Models
Recent advancements in motion forecasting have been significantly accelerated by self-supervised pre-training techniques. However, current methods typically rely on extensive fine-tuning of pre-trained models for downstream tasks like trajectory prediction, a process that can be inefficient. This inefficiency primarily arises because traditional fine-tuning strategies fail to fully exploit the close alignment between motion prediction tasks and masked reconstruction objectives used during pre-training. To overcome these limitations, we introduce Forecast-PEFT, a parameter-efficient fine-tuning framework specifically designed for motion forecasting. Forecast-PEFT preserves most of the pre-trained model's parameters, introducing minimal yet effective adjustments through novel prompts and adapters. By doing so, Forecast-PEFT significantly reduces the number of parameters needing retraining while preserving essential pre-trained knowledge. Our method demonstrates robust generalization across multiple datasets, achieving higher accuracy with only about 17% of the parameters required by traditional full fine-tuning approaches. Additionally, our fully fine-tuned variant, Forecast-FT, further enhances prediction performance, achieving improvements of up to 9.6% over conventional baseline methods. These findings highlight Forecast-PEFT as a scalable, efficient, and effective alternative for fine-tuning pre-trained motion forecasting models. Code is available at https://github.com/csjfwang/Forecast-PEFT.
comment: Accepted for publication in IEEE Transactions on Intelligent Transportation Systems. This version is the accepted manuscript
♻ ★ An integrated interpretable control effectiveness learning and nonlinear control allocation methodology for overactuated aircraft
Nonlinear dynamics and the strong couplings that arise between multiple effectors undermine the assumptions behind conventional, linear control allocation techniques. When flight enters regimes where nonlinear effects dominate, linear allocators exhibit reduced accuracy due to increased model mismatch, which subsequently degrades performance and robustness of the flight control system. High fidelity onboard models and black box data driven approaches can recover accuracy across the flight envelope, but respectively impose computational burdens prohibitive for real time allocation and sacrifice the interpretability required for verification and fault diagnosis. This paper addresses these limitations by learning an explicit, physics constrained analytical model of the control effectiveness mapping from representative flight data using Sparse Identification of Nonlinear Dynamics. The resulting mapping is compact, interpretable, and admits analytical derivatives, enabling efficient computation within nonlinear solvers that additionally incorporate actuator dynamics, without requiring an onboard model. An online adaptation mechanism monitors prediction residuals and refreshes the model when significant plant changes are detected, providing graceful reconfiguration under actuator failures and varying operating conditions. The methodology is evaluated on a high fidelity nonlinear benchmark aircraft across a range of aggressive maneuvers, achieving accuracy comparable to a full nonlinear onboard model while substantially reducing computational cost relative to established baselines.
♻ ★ A Phenomenology of Neural Geometry in Vision-Language-Action Models
Robotic manipulation is inherently geometric, but what shape does the world take inside the policy that performs it? Prior work on vision-language-action (VLA) policies establishes that task-relevant variables are linearly readable from activations, yet readability certifies only that a variable is present and says nothing about the shape its values trace, which may be curved and multi-dimensional. Using controlled sweeps, natural rollouts, linear probes, and activation interventions, we build a taxonomy of seven task-relevant concepts in VLA models. We find that realized geometry is predominantly multi-dimensional and is shaped by the constraints under which the policy acts. Physical joint limits, for example, leave wrist orientation an open arc rather than a circle, and distance to the target, a single scalar, is realized in two orthogonal subspaces, a coarse ruler for the approach and a fine ruler for the grasp. The same geometries reappear in two further VLA models on real-world robot data, indicating that they are neither model-specific nor artifacts of simulation. We demonstrate how the findings are actionable by steering along the geometry a concept realizes. Overall, this work lays out an initial phenomenology of realized geometry in VLA policies and demonstrates that a concept's shape carries as much importance as its readability.
comment: 41 pages, 10 pages of main text, 8 main text figures
♻ ★ Probabilistic Physics Digital Twin for Human-Robot Interaction
Digital twins are increasingly used in robotics to simulate interactions, predict action outcomes, and support planning. However, constructing accurate digital twins requires precise estimates of physical properties such as mass, friction, and center of mass, which are often difficult to obtain in real-world environments. We propose probabilistic digital twins, which represent uncertain physical properties as probability distributions rather than fixed values. By simulating actions across these distributions, the twin predicts a range of possible outcomes and their likelihoods. These probabilistic simulations can guide both human and robot actions by identifying actions that are effective despite physical uncertainty. We demonstrate this framework through human-robot interaction scenarios, including interfaces that communicate simulated outcomes to users and robot planning strategies that explicitly account for uncertainty.
♻ ★ KungfuAthleteBot: Learning High-Dynamic Motion from Videos with Unified Robust Recovery
Learning highly dynamic motions from video provides a scalable alternative to expensive data collection for humanoid robots, yet it is challenged by height floating, missing kinematic and force information, and limited robustness. We present KungfuAthleteBot (KAB), a unified framework that systematically addresses these issues. We first build a high-dynamic motion dataset in collaboration with professional martial artists, alleviating data scarcity while improving motion quality. To eliminate height drift and global jitter during both aerial and landing phases, we introduce a physics-guided parabolic trajectory correction that enforces physical plausibility. To bridge the discrepancy between human and robot actuation in highly dynamic behaviors, we design a three-stage curriculum that progressively enables stable acquisition of such motions. In addition, we propose a direct, teacher-free training paradigm that jointly learns disturbance rejection and fall recovery within a single policy. To the best of our knowledge, KAB achieves the fastest recovery time reported for a unified policy ($\approx 0.7$s). Experiments on a real humanoid robot demonstrate successful learning of highly dynamic motions from video and strong robustness, including rapid recovery under severe disturbances.
comment: 18 pages, 8 figures,5 tables
Multiagent Systems
★ Same Data, Different Bytes: Serialization-Invariant Integrity Checks for Agent-Published Datasets
Agent networks require reliable ways for requesting agents to verify where data is located, how it can be accessed, and whether the data received matches what the data owner made available. Checksum mismatches can reflect either altered content or serialization differences that preserve data equivalence under specified rules. Data identification mechanisms for agents, such as DataFacts, describe where data can be accessed and provide checksums for checking received data against published references. We propose a serialization-invariant identification method that assigns the same identifier to data representations equivalent under explicit rules. We use DataFacts as a case study, retaining its existing checksum alongside the proposed identifier. In a controlled agent study, we compare the proposed configuration with ordinary-tool and checksum baselines across equivalent re-encodings, altered content, and unavailable references. The study includes three model tiers from each of two providers. For equivalent re-encodings, the proposed configuration increased expected-outcome rates by 80.4 percentage points over the ordinary-tool baseline and by 70.2 percentage points over the checksum baseline. The proposed configuration also increased evidence-supported rejection of altered content by 3.5 percentage points over the checksum baseline. We contribute a serialization-invariant identification method with explicit equivalence rules, an integration approach illustrated through DataFacts, and an empirical evaluation against ordinary-tool and checksum baselines.
comment: 4 pages. Accepted at the NANDA Workshop, IEEE TPS-ISA 2026
★ The One-Pixel of Reasoning: One Truthful Relation Is Enough to Mislead LLM Agents
Can one sentence manipulate an agent's belief by targeting a single implicit warrant linking evidence to a conclusion? In this paper, we introduce the one-relation attack (ORA), a novel cognitive attack on this inferential relation. ORA localizes a critical implicit warrant and performs a one-relation intervention using a single source-grounded sentence that qualifies this warrant and steers the agent toward an incorrect belief. Across eighteen models on biomedical inference and vulnerability detection, best-of-$10 attacks overturn initially correct answers at mean rates of 54.9% for proprietary and 80.0% for open-weight models. Attacks also transfer without victim feedback, and paired controls support the roles of targeting and qualification. Moreover, successful sentences can pass source-support checks and evade identification, and multi-agent debate raises team error from 50.0% to 59.5% in our balanced stress test. These findings establish implicit warrants as a cognitive attack surface for manipulating agents' beliefs.
comment: Under Review
★ AgentSoV: Virtual Sensing for LLM Multi-Agent Systems via Stream of Variation
Large language model (LLM) multi-agent systems (MASs) solve complex tasks through interacting agents, yet their reliability is difficult to analyze because failures depend on both individual agents and how information and errors propagate across the system. Existing approaches largely treat this as post-hoc failure attribution, but do not provide a common model across heterogeneous MAS architectures or support broader analysis of information influence and observability. We introduce AgentSoV, a virtual-sensing framework that models MAS execution as a stochastic dynamic system through Stream of Variation (SoV). AgentSoV maps heterogeneous trajectories into diagnostic stages, constructs virtual measurements from observable outputs, and learns cross-stage variation propagation with a state-space model. Its representation is architecture-aware in stage dependencies yet architecture-agnostic in analytical language, supporting chain-structured, orchestration-based, and directed acyclic graph (DAG)-structured MASs. Without re-executing the MAS, AgentSoV supports three system-level analyses: failure attribution identifies where abnormal variation enters; stage-input influence quantifies how strongly intermediate inputs affect downstream outcomes; and virtual-sensor placement selects complementary observations under a monitoring budget. Experiments across multiple MAS architectures demonstrate a unified approach to fault localization, information influence, and observability. AgentSoV reframes agentic execution as a dynamic system to model rather than merely text to inspect, providing insights for MAS structure and component design.
★ Measuring Consensus in AI Agent Networks
As AI agents become increasingly connected through networks and take on tasks that require negotiation and joint decision-making, a systematic approach is needed to assess how their views converge and what information supports their decisions. We present an evaluation framework that treats consensus formation as the evolution of publicly stated positions toward a shared region, using accumulated communication history as context. The framework tracks position trajectories, group dispersion, and information expressed in public messages. Two case studies examine staffing decisions based on distributed facts and decisions requiring agents to reconcile duplicate information and a cancelled contract. Results from 1,872 completed runs associate sharing explanations with more accurate group decisions, while the second task reveals persistent disagreement despite accurate aggregate answers. Membership-change experiments provide a complementary evaluation. By separating convergence among agents from progress toward a task's full-information answer, the framework relates decision accuracy and bounded disagreement to the information available during the exchange.
comment: 4 pages. Accepted at the NANDA Workshop, IEEE TPS-ISA 2026
★ Cooperation Without Shared Rewards via Reciprocal Q-Augmentation
General-sum games are more challenging than fully cooperative or adversarial settings because agents may deceive or exploit one another. Achieving high social welfare requires sustained cooperation, while a single defection can drive agents toward a welfare suboptimal Nash equilibrium. Moreover, assuming access to other agents' rewards or internal states is often unrealistic and implicitly introduces additional cooperation into the setting. To address this issue, we introduce \textit{Reciprocal Q-Augmentation} (RQA), a method that uses a latent-variable opponent model to augment the Q-function with estimates of how one's actions affects other agents. RQA allows agents to learn both cooperative and defective policies using only their own Q-function. We show that, in social dilemmas, agents that all follow the cooperative policy of RQA learn to cooperate faster and reach higher levels of cooperation than fully cooperative algorithms. Moreover, a single RQA agent that switches appropriately between its cooperative and defecting policies can induce cooperation in agents trained with standard RL algorithms, which would otherwise end up defecting. We evaluate our approach on the Iterated Prisoner's Dilemma, Iterated Traveler's Dilemma, Iterated Stag Hunt, and \texttt{CleanUp-Lite}. Our experiments show that the algorithms implementing RQA outperforms fully cooperative methods with shared rewards and enables cooperation with algorithms that otherwise fail to reach high social welfare.
★ Neuro-Inspired Inhibitory Message Passing for Accurate and Token-Efficient Multi-Agent Reasoning
Multi-agent systems promise to solve problems by bringing together information beyond the reach of any individual agent. Retrieval and tool use can distribute this information across agents, making communication essential to collaboration. A central challenge is keeping crucial evidence visible in each agent's context as messages accumulate. Repeated information can crowd out rare, decisive evidence. Models of cortical circuits suggest a principle for addressing this imbalance: inhibition can selectively suppress shared inputs while preserving sensitivity to weaker signals. We adapt this principle into inhibitory message passing, a training-free communication router that cancels redundant evidence separately for each evidence sentence and receiver. Each round, it forwards evidence sentences restated in agents' messages, delivering each at most once to receivers whose own evidence lacks it and excluding the senders' reasoning and answers. Against standard message passing, accuracy in peer groups rose by 39-77 percentage points across three benchmarks, and tokens per run fell by 10-46%. Gains held across seven open models spanning four to thirty-two billion parameters and two commercial models, in groups up to sixty-four agents, and across multi-agent architectures including peer-to-peer, orchestrator-worker, and multilevel hierarchical organizations. These findings support a scalable design pattern for more accurate and token-efficient multi-agent reasoning.
★ CARVS: A Congestion-Aware Algorithm for Online Multi-Agent Pickup-and-Delivery in Human-Collaborative Warehouses
Intelligent warehouse systems have transformed logistics by integrating robots with minimal infrastructure changes, boosting efficiency and optimizing workflows compared to traditional human-only operations. However, human-robot collaboration introduces new challenges, particularly due to human uncertainty, which leads to dynamic interactions and unpredictable behavior. To address these complexities, this paper presents the Multi-Agent Pickup-and-Delivery with Human Collaboration (MAPDwHC) problem, an extension of the Multi-Agent Pickup-and-Delivery (MAPD) problem that incorporates human unpredictability. This problem adds key constraints, such as dynamic obstacles caused by robots temporarily blocking paths when interacting with humans, and the need for real-time responsiveness. We formalize the MAPDwHC problem with additional constraints and properties. Based on this problem setting, we propose the CARVS algorithm, which improves task assignment and path planning by integrating task swapping, validation checks, and random initialization. CARVS reduces congestion, enhances adaptability to human-robot interactions, and ensures real-time responsiveness. Simulation results show that CARVS significantly reduces timeouts of path planning in large-scale scenarios where traditional MAPD algorithms fail, improving both efficiency and scalability.
comment: 13 pages, 8 figures. Accepted at DAI 2026 (Oral)
★ Low-Bit Distributed Constrained Optimization with Application to Optimal Scheduling and Allocation
This paper proposes a distributed resource allocation algorithm in which each agent communicates only the sign of its local cost gradient, a single bit per directed link per iteration. The algorithm is based on Laplacian-gradient dynamics and preserves the equality (demand) constraint at every iterate, guaranteeing anytime feasibility irrespective of termination time. First, we provide a complete Lyapunov-based convergence analysis for the fixed-step-size regime: the iterates are shown to enter, in at most O(1/\varepsilon^2) iterations, a neighborhood of the optimizer whose radius scales as O(τ) in the decision variable and O(τ^2) in the cost residual, where $τ$ is the step-size. Crucially, the step-size bound is a time-invariant network constant (depending only on the graph Laplacian spectrum and the cost curvature bounds), so no adaptive tuning is required. Second, we characterize the invariant set under B-connected, weight-balanced digraphs, strictly weaker than the all-time connectivity and doubly-stochastic weights assumed in most prior work, and show that any odd, sign-preserving nonlinearity yields the same invariant-set and descent structure, with the single-bit protocol being the extreme communication-minimizing instance of a broader nonlinear class. Third, we address local box constraints via smooth penalty relaxation and quantify explicitly how the penalty weight interacts with the step-size bound and the optimality gap. Simulations on economic dispatch (1000 generators) and CPU scheduling (500 servers) demonstrate that the single-bit protocol achieves 5-60 times reduction in total communication cost relative to multi-bit baselines, at the price of a tunable and explicitly bounded optimality gap. A diminishing step-size schedule is shown empirically to eliminate this gap; its formal convergence analysis is identified as an open problem.
comment: Nonlinear Analysis: Hybrid Systems
★ Adding Heavy-Ball Momentum to Distributed Constrained Optimization for Energy Resource Management Subject to Time-Delays and Quantization
This paper studies a hybrid discrete-time signed gradient-consensus method with heavy-ball momentum, paired communication delays, and fixed logarithmic quantization. The method preserves the aggregate supply-demand equality at every iteration. For a static connected undirected graph and bounded delays, a Lyapunov-Krasovskii analysis establishes practical stability by isolating the discontinuous quantization error as an additive disturbance and closing a computable invariant-sublevel argument. The resulting ergodic residual bound is quadratic in the quantization resolution, and exact convergence to the smooth penalized equality-constrained optimizer is recovered in the unquantized case. For delay-free unquantized quadratic costs, an exact modal analysis identifies the heavy-ball stability region and optimal parameters, with the predicted rate validated on a fixed sparse network. A reproducible delayed experiment reports all constants and a certificate whose guaranteed steady-state level lies below the reported initialization. A separate mixed generator-battery experiment exercises the signed update directly. Finite smooth penalties approximate, but do not generally enforce, the local box constraints exactly.
comment: Nonlinear Analysis: Hybrid Systems
★ IF-MAP: Influence-Function-Inspired Agent Prioritization in Multi-Agent Systems
Improving every agent in an LLM-based multi-agent system can be prohibitively costly, making it important to identify which agent should be prioritized to achieve the greatest system-level gain with limited resources. Failure attribution can inform this decision, but an agent's responsibility for failures does not directly reveal the system-level gains achievable by improving it. We therefore introduce IF-MAP, an influence-function-inspired method for identifying agents with high potential for system-level improvement. For each agent, IF-MAP measures end-to-end loss changes under controlled semantic prompt perturbations, fits a regularized local response model in a low-dimensional prompt-embedding subspace, and uses the estimated response norm as a priority score. This black-box procedure requires no access to model parameters or gradients and produces agent priorities that can guide subsequent improvement effort. We evaluate IF-MAP across six multi-agent workflows spanning code generation, question answering, and mathematical reasoning, using the same priorities for prompt optimization and model upgrades. Optimizing the selected agents' prompts achieves a macro-average gain of 3.81 percentage points, compared with 1.85 for uniform agent selection. Across five target models, upgrading the selected agents yields macro-average gains of 7.62-16.42 percentage points, compared with 3.81-8.62 for uniform agent selection. Despite its probing overhead, IF-MAP reduces Qwen3-8B and DeepSeek-V3.2 token use by 74.2% and 35.2%, respectively, relative to optimizing every agent, and reduces target-model tokens by 63.6-64.9% relative to upgrading every agent. These results support the use of IF-MAP to identify promising agents and direct improvement effort toward greater end-to-end performance gains.
♻ ★ Scientific discovery as meta-optimization: a combinatorial optimization case study
Scientific discovery is fundamentally an optimization problem, defined by a vast ``state space'' of theories and experiments, and an evaluation criterion based on quality, novelty, and validity. Large language models (LLMs) have enabled automated exploration of this state space, but we argue that simultaneous modification of the evaluation criteria is equally important. Here, we propose formalizing research as meta-optimization, where the optimization objective itself is also being optimized. Our key contribution is ``consensus objective aggregation,'' where a portfolio of LLM-generated objective functions is combined via correlation-weighted voting, allowing the evaluation criterion to evolve alongside the solutions. We apply this framework to algorithm discovery for planted 3-SAT problems based on digital memcomputing machines. The leading solver from the consensus search reduces median integration steps by more than $38\times$ relative to a tuned baseline at 2,560 variables. Objective ablations show that a well-chosen fixed expert objective can also discover strong solvers, while consensus supplies a stable ranking that mitigates reward hacking behaviors. By bringing objective engineering into the discovery loop, this framework offers a practical route toward more adaptable automated research.
comment: 33 pages, 4 figures
♻ ★ ADIAS: Automated Design of Interactive Agentic Systems
Self-evolving agents can improve without updating model weights by iteratively optimizing the harness based on evaluation feedback. Existing methods are largely candidate-centric: they record evolution outcomes (which agent variants have been produced and how they perform), rather than evolution progress (which issues, i.e., recurring failure patterns, have been fixed and which remain). Such methods pose two difficulties for cumulative harness evolution. First, repair context is scattered across candidate records and must be reconstructed every round, which leads to inefficient, redundant, or misdirected revisions. Second, because candidates are evaluated as a whole, partial fixes across different agents are difficult to combine, while ineffective changes persist alongside beneficial ones. To address these problems, we formulate issue-centric agent optimization, which explicitly carries forward evolutionary progress through a persistent issue state. We instantiate this paradigm in ADIAS, a harness-evolution framework with two key components. An Issue Manager maintains the issue state with stable issue identities, lifecycle status, supporting evidence, and intervention-outcome histories. An Issue-Guided Optimizer proposes repair targets from the issue state and performs focused harness revision. On five interactive benchmarks, ADIAS outperforms the strongest baseline by 25.2% on average and achieves consistent gains across four backbone models. Controlled ablations show that removing the Issue Manager or the Issue-Guided Optimizer reduces performance by up to 40.7%.
comment: Work in progress. 24 pages, 11 tables, 5 figures
♻ ★ Optimizing Social Utility in Sequential Experiments NeurIPS 2026
Regulatory approval of products in high-stakes domains such as drug development requires statistical evidence of safety and efficacy through large-scale randomized controlled trials. However, the high financial cost of these trials may deter developers who lack absolute certainty in their product's efficacy, ultimately stifling the development of `moonshot' products that could offer high social utility. To address this inefficiency, in this paper, we introduce a statistical protocol for experimentation where the product developer (the agent) conducts a randomized controlled trial sequentially and the regulator (the principal) partially subsidizes its cost. By modeling the protocol using a belief Markov decision process, we show that the agent's optimal strategy can be found efficiently using dynamic programming. Further, we show that the social utility is a piecewise linear and convex function over the subsidy level the principal selects, and thus the socially optimal subsidy can also be found efficiently using divide-and-conquer. Simulation experiments using publicly available data on antibiotic development and approval demonstrate that our statistical protocol can be used to increase social utility by more than $35$$\%$ relative to standard, non-sequential protocols.
comment: NeurIPS 2026
♻ ★ Delegations as Adaptive Representation Patterns: Rethinking Influence in Liquid Democracy
Liquid democracy is a mechanism for the division of labor in decision-making through the transitive delegation of influence. In essence, all individuals possess the autonomy to determine the issues with which they will engage directly, while for other matters, they may appoint a representative of their choosing. So far, the literature has studied the delegation structures emerging in liquid democracy as static. As a result, transitivity defined as the capacity to transfer acquired authority to another entity, has been identified as a concern as it would be conducive to unrestrained accumulation of power. Focusing on the implementation of liquid democracy supported by the LiquidFeedback software, we propose a novel approach to assessing the influence of voting nodes in a transitive delegation graph, taking into account the process nature of real-world liquid democracy in which delegation and voting are distinct and increasingly independent activities. By introducing a novel model of delegations in liquid democracy, we show how transitivity may in fact contribute to an effective regulation of deliberation influence and decision-making power. While maintaining the one-person, one-vote paradigm for all votes cast, the anticipated influence of an agent, to the extent it is stemming from transitivity, experiences a precipitous decline following an exponential trajectory. In general, it is our objective to move the first steps towards a rigorous analysis of liquid democracy as an adaptive democratic representation process. The adaptivity aspect of liquid democracy has not yet been explored within the existing academic literature despite it being, we believe, one of its most important features. We therefore also outline a research agenda focusing on this aspect of liquid democracy.
comment: v2: Corrected numerical values and updated references to subsequent research on liquid democracy and network centrality, preserving the original research timeline and contributions
Systems and Control (EESS)
★ ATLAS: Expert-Guided Sustainable Satellite Backhaul for Off-Grid Wildlife Monitoring ATC 2026
Remote wildlife monitoring increasingly depends on high-fidelity cameras and sensors deployed far from grid power and terrestrial networks. Low Earth orbit satellite links can provide connectivity in these settings, but their high power draw and variable bandwidth, combined with intermittent on-site solar harvesting, make continuous backhaul infeasible. Current systems therefore filter at the edge, with motion triggers that waste limited energy and bandwidth on environmental artifacts, or fixed vision models that cannot follow evolving expert objectives. We present \textsc{ATLAS}, an expert-guided edge-to-cloud system that treats satellite backhaul as a scarce resource for off-grid wildlife monitoring. \textsc{ATLAS} uses foundation models to compile natural-language expert intent into semantic scoring logic, combines semantic relevance with environmental and energy state, and uses a constrained reinforcement-learning scheduler to decide which clips to upload and when. As a representative case study, we deployed \textsc{ATLAS} for over six months across 18 off-grid salmon-monitoring river sites. \textsc{ATLAS} achieves a mean clip-selection accuracy of $87.4\%$, exceeding mean baseline accuracy by $26.4$ percentage points, while maintaining near-continuous operation through low-solar periods in which baselines incur 16.0\% downtime.
comment: Accepted at the ACM SIGOPS Annual Technical Conference (ATC 2026)
★ Bellman Language Models: Solving the Recurring Decisions of LLM Agents
A general observation in agentic workflows, especially bounded ones, is that the decision space of the LLM, when we separate a decision from generation, starts approaching saturation: the agent meets the same states again and again, and after enough conversations it stops meeting new ones. LLMs generally don't treat these decisions as an optimization problem and instead re-derive them from their context/prompt on every turn. We show that treating decision making within reasonable bounds as an optimization problem, solving it once, and reusing the solved decisions can often make an agent faster, cheaper and more reliable. Our approach, which we call Discretize, Then Decide, starts from data such as the workflow's own logs or, when none exists, runs of the LLM agent itself. It reads each conversation into discrete states and estimates the probability of each outcome and the cost of each move, until no new states appear and the probabilities stop changing. It then solves the resulting Markov decision process. The solution runs as a controller around the model, sending moves whose content is fixed in the state, such as a tool call or a templated reply, and any state it hasn't seen is sent back to the LLM. We call an LLM run this way a Bellman language model (BLM), after Richard Bellman's equation for the best move in each state. On $τ^2$-bench telecom, the controller with no model solves 79% of tasks. As a BLM, Qwen3.8-27B, an open model, goes from 29% to 96% with 93% fewer model calls. For Claude Opus 5.5, which solves 96% of tasks on its own, running it as a BLM cuts model calls and cost per task by about 98% and lets it solve 2 more tasks. On InventoryBench, the solved policy with no model beats every published method. Where the hard part is writing content rather than choosing moves, such as resolving SymPy issues, few decisions can be solved in advance, and a BLM helps little.
comment: 52 pages, 13 figures, 11 tables. Preregistrations: https://github.com/adarsh-vatsa/dtd-preregistration
★ AgentSoV: Virtual Sensing for LLM Multi-Agent Systems via Stream of Variation
Large language model (LLM) multi-agent systems (MASs) solve complex tasks through interacting agents, yet their reliability is difficult to analyze because failures depend on both individual agents and how information and errors propagate across the system. Existing approaches largely treat this as post-hoc failure attribution, but do not provide a common model across heterogeneous MAS architectures or support broader analysis of information influence and observability. We introduce AgentSoV, a virtual-sensing framework that models MAS execution as a stochastic dynamic system through Stream of Variation (SoV). AgentSoV maps heterogeneous trajectories into diagnostic stages, constructs virtual measurements from observable outputs, and learns cross-stage variation propagation with a state-space model. Its representation is architecture-aware in stage dependencies yet architecture-agnostic in analytical language, supporting chain-structured, orchestration-based, and directed acyclic graph (DAG)-structured MASs. Without re-executing the MAS, AgentSoV supports three system-level analyses: failure attribution identifies where abnormal variation enters; stage-input influence quantifies how strongly intermediate inputs affect downstream outcomes; and virtual-sensor placement selects complementary observations under a monitoring budget. Experiments across multiple MAS architectures demonstrate a unified approach to fault localization, information influence, and observability. AgentSoV reframes agentic execution as a dynamic system to model rather than merely text to inspect, providing insights for MAS structure and component design.
★ HEL: A Heterogeneous Extensible Layer for GPU-Resident Runtime Orchestration
We describe a GPU-resident runtime system, HEL, built on stateful services that persist throughout the lifespan of a user application. A HEL programmer expresses a workload as an application that spawns coroutine-like tasks, which are then scheduled and run by the HEL runtime. HEL supports synchronous and asynchronous tasks, tasks with runtime dependencies, batched task submission, and automatic retries. Applications and tasks run alongside each other on the spatially partitioned GPU. Using producer-consumer and host-RPC benchmarks together with a PFPL compression case study, we show that HEL demonstrates a performance advantage for underutilized GPUs and for scenarios where HEL enables hiding task latency.
comment: 10 pages, 8 figures
★ Degradation Aware V2G Dispatch for Co-Optimization of Parking Facility and Edge Data Center Located in the Airport Campus
This paper studies an airport campus where long stay EV parking, airport buildings, and an AI data center share a single grid connection. The campus pays a commercial electricity tariff that charges for both the energy used and the highest power drawn during peak hours. Parked EVs can send power back to the campus through vehicle to grid (V2G) chargers and lower this peak. This paper proposes a two stage method. In the first stage, a linear program schedules EV charging and discharging to maximize the operator's profit and also sets a fixed daily payment to EV owners. In the second stage, a search chooses the share of energy savings paid to owners. The model guarantees that every owner is paid enough to take part, pays for battery wear so that EVs discharge only when it is worth the wear, and assigns charging ports on a first come, first served basis. In the case study, 400 V2G chargers are added to 1200 existing Level 2 chargers, serving 5000 expected sessions under summer and winter rates. In a summer week, V2G raises operator profit by $4978 compared with smart charging and lowers the high peak demand by 4400 kW (9%). The best contract gives owners 70% of the energy savings and earns 21% more than a fixed payment alone. Doubling the V2G ports to 800 lowers the peak further but reduces profit, because owner payments grow faster than the savings. V2G pays back within 10 years if a bidirectional charger costs less than about $1800 more than a standard one. Compared with a 5 MW/10 MWh stationary battery, V2G cuts more peak demand at $409 of investment per kW instead of $1522 and needs no extra land, while the battery does not pay back within its lifetime.
★ Three Machines Suffice for Local Hardening of Electromechanical Oscillations
The single-degree-of-freedom swing equation with sinusoidal coupling is locally softening: its oscillation frequency decreases with oscillation level near a stable equilibrium. Nonlinear modal interaction, however, can reverse this trend in multimachine systems. This letter gives a necessary-and-sufficient condition for strict leading-order local hardening, measured with respect to canonical action, in two symmetric three-machine operating-point families with equal or opposite power injections at the equal-inertia machines. An exact two-coordinate reduction expresses the condition in terms of a squared modal-frequency ratio and a dimensionless nonlinear coupling measure. The proof retains the induced mean shift and second harmonic and shows that the companion modal family remains locally softening. The criterion recovers both an equal-inertia hardening example and an exact large-inertia threshold. Coherent aggregation recovers the two-machine softening law, whereas the infinite-bus limit retains two dynamic rotor angles and permits hardening. An exact coherent-group extension preserves the selected periodic family, frequency, and canonical action for every finite machine count greater than two. Thus, three finite-inertia machines are the minimum permitting local hardening in the stated undamped, lossless classical model.
★ Comparison of Current-Limiting Strategies for Grid-Forming Converters Under Grid Faults
This paper investigates the performance of three state-of-the-art current-limiting strategies for grid-forming (GFM) converters under grid faults. These highly effective strategies include virtual admittance control, virtual admittance control enhanced with virtual power feedback, and the recently developed cross-forming control. The performance comparison addresses two crucial requirements for GFM converters under grid faults, namely transient stability and reactive current injection. The comparison indicates that the virtual admittance control enhanced with virtual power feedback shows the most promising transient stability performance, while the cross-forming control exhibits superior reactive current injection performance. Thus, individual benefits of each strategy can be obtained by properly combining them.
comment: Presented at the 16th IEEE PowerTech Conference (Kiel 2025). 6 pages, 12 figures
★ Multi-Agentic AI for Intent-Driven and Conflict-Aware rApp Policy Orchestration in Open RAN
Open radio access network (O-RAN) enables a multi-vendor ecosystem that accelerates xApp development for near-real-time (RT) network management. However, this openness shifts a combinatorial coordination burden to the Non-RT layer, where rApp policies compose xApps from a growing catalog to realize operator intents. Coexisting intents further compound this challenge, as conflicts may arise among xApp actions, rApp policies, and their cross-layer interactions. To maximize intent admission while maintaining key performance indicator (KPI) compliance across incoming and incumbent intents, we propose a large language model-based Multi-Agentic AI framework. A Reasoning Agent screens candidate xApps against KPI contracts and produces a joint intent admission and xApp assignment decision. On post-deployment KPI violations, a Diagnosis Agent identifies the root causes of failure, labels the conflict mechanism, and guides the Reasoning Agent in subsequent attempts. Experimental results show that the framework based on GPT-OSS 120B achieves 28.6% more KPI-compliant cases and admits over 15* as many incoming intents as state-of-the-art baselines. Under the Qwen 72B variant, Diagnosis increases compliant cases by 50.0%, quadruples admitted intents, and reduces deployment attempts by 32.6%. A live deployment conforming to O-RAN specifications further validates the feasibility of the proposed solution for conflict-aware rApp policy orchestration in real networks.
★ Distributed Real-Time Cooperative Model Predictive Control
This paper presents a novel distributed real-time iterative method for cooperative linear Model Predictive Control (MPC) with polyhedral state and input constraints, implementing a relaxed, recentered logarithmic barrier function. The method transforms centralized MPC problems into a distributed framework using temporal-spatial splitting, facilitating efficient online implementation. To achieve real-time performance, we introduce a distributed convex nonlinear optimization solver with guaranteed global linear convergence. By executing this solver with a fixed number of iterations per sampling time, the resulting near-optimal MPC controller ensures closed-loop stability while maintaining fixed computational complexity. The proposed approach is validated through an illustrative example involving a spring-vehicle-damper system benchmark, demonstrating both its effectiveness and computational efficiency.
★ Physics-Informed Koopman Learning for Data-Efficient System Identification of Automated Vehicle Dynamics
Understanding how a particular automated vehicle reacts to nearby traffic can improve motion prediction, control development, traffic simulation, and safety evaluation. However, commercial automated driving systems are proprietary, and single encounters may provide only a few seconds of noisy data. Standard physical models are interpretable but inflexible, whereas neural models are often data-hungry. We propose a physics-informed Koopman framework that combines a calibrated car-following law with a correction learned from the observations. A weak-form estimator learns this correction from velocity measurements integrated over time, avoiding noise-sensitive numerical differentiation. The resulting continuous-time model predicts the follower's response and provides interpretable measures of driving behavior. Experiments on OpenACC and CATS-CF evaluate 5-, 7-, and 11-second histories. Averaged over these histories, the constant-time-headway branch reduces speed prediction error by 16.0% and 15.4% on OpenACC and CATS-CF, respectively, relative to the best neural baselines. The PI-Koopman branches achieve the lowest mean speed and gap errors among the physical and neural comparators. Weak-form estimation reduces acceleration-field reconstruction error by 74.5% in noisy known-law simulations and noise-induced field perturbation by 91.6% in paired real-vehicle measurements. The same PI-Koopman system identification supports following-mode recognition with an approximately eleven-fold difference in median residual variance between human drivers and automated vehicles, and a single residual threshold achieves 85.4% balanced accuracy. The branch informed by the intelligent driver model takes a median 17 to 26 ms for identification and forecasting on the tested central processing units, supporting frequent target adaptation.
comment: 13 pages, 9 figures. The CATS-CF dataset and code are available at https://github.com/CATS-Lab/PI-Koopman
★ Discrete Phase-Shift Optimization of Reconfigurable Intelligent Surfaces using Uniform Reference Directions
Reconfigurable Intelligent Surfaces (RISs) enable programmable control of the wireless propagation environment through adjustable reflecting elements. A fundamental challenge in RIS-assisted communications is the optimization of discrete phase shifts to maximize the received signal power. While recently developed algorithms can obtain optimal solutions with linear complexity, their computational cost may still be excessive in applications requiring large-scale simulations, real-time adaptation, or repeated optimization over numerous channel realizations. This paper proposes a low-complexity discrete phase-shift optimization algorithm based on multiple reference directions within the feasible optimality sector. By evaluating candidate phase configurations generated from uniformly distributed reference directions, the proposed method systematically improves upon existing approximation approaches that rely on a fixed set of reference directions. The number of reference directions serves as a design parameter that enables a controllable trade-off between computational complexity and optimality. Extensive simulations demonstrate that the proposed algorithm achieves substantial computational speedups while maintaining a small optimality gap relative to the optimal solution. The results further show that increasing the number of reference directions progressively reduces the optimality gap, providing flexible operating points that can be tailored to application-specific complexity and performance requirements.
★ Low-Bit Distributed Constrained Optimization with Application to Optimal Scheduling and Allocation
This paper proposes a distributed resource allocation algorithm in which each agent communicates only the sign of its local cost gradient, a single bit per directed link per iteration. The algorithm is based on Laplacian-gradient dynamics and preserves the equality (demand) constraint at every iterate, guaranteeing anytime feasibility irrespective of termination time. First, we provide a complete Lyapunov-based convergence analysis for the fixed-step-size regime: the iterates are shown to enter, in at most O(1/\varepsilon^2) iterations, a neighborhood of the optimizer whose radius scales as O(τ) in the decision variable and O(τ^2) in the cost residual, where $τ$ is the step-size. Crucially, the step-size bound is a time-invariant network constant (depending only on the graph Laplacian spectrum and the cost curvature bounds), so no adaptive tuning is required. Second, we characterize the invariant set under B-connected, weight-balanced digraphs, strictly weaker than the all-time connectivity and doubly-stochastic weights assumed in most prior work, and show that any odd, sign-preserving nonlinearity yields the same invariant-set and descent structure, with the single-bit protocol being the extreme communication-minimizing instance of a broader nonlinear class. Third, we address local box constraints via smooth penalty relaxation and quantify explicitly how the penalty weight interacts with the step-size bound and the optimality gap. Simulations on economic dispatch (1000 generators) and CPU scheduling (500 servers) demonstrate that the single-bit protocol achieves 5-60 times reduction in total communication cost relative to multi-bit baselines, at the price of a tunable and explicitly bounded optimality gap. A diminishing step-size schedule is shown empirically to eliminate this gap; its formal convergence analysis is identified as an open problem.
comment: Nonlinear Analysis: Hybrid Systems
★ Adding Heavy-Ball Momentum to Distributed Constrained Optimization for Energy Resource Management Subject to Time-Delays and Quantization
This paper studies a hybrid discrete-time signed gradient-consensus method with heavy-ball momentum, paired communication delays, and fixed logarithmic quantization. The method preserves the aggregate supply-demand equality at every iteration. For a static connected undirected graph and bounded delays, a Lyapunov-Krasovskii analysis establishes practical stability by isolating the discontinuous quantization error as an additive disturbance and closing a computable invariant-sublevel argument. The resulting ergodic residual bound is quadratic in the quantization resolution, and exact convergence to the smooth penalized equality-constrained optimizer is recovered in the unquantized case. For delay-free unquantized quadratic costs, an exact modal analysis identifies the heavy-ball stability region and optimal parameters, with the predicted rate validated on a fixed sparse network. A reproducible delayed experiment reports all constants and a certificate whose guaranteed steady-state level lies below the reported initialization. A separate mixed generator-battery experiment exercises the signed update directly. Finite smooth penalties approximate, but do not generally enforce, the local box constraints exactly.
comment: Nonlinear Analysis: Hybrid Systems
★ New Results for Networked Control of Nonlinear Plants Subject to Transmission Delays
This paper studies networked control for nonlinear plants subject to several network-induced constraints, with a particular focus on transmission delays. In the context of general nonlinear networked control systems, a standard approach for guaranteeing stability is to construct a storage function, coupled with a single timer variable, to analyze the maximum allowable transmission interval (MATI) and maximum allowable transmission delay (MAD). In this paper, we propose a new approach that assigns each signal that is either \textit{in transit or most recently updated}, to an individual timer variable. This enables the individual characterization and compensation for the dynamics of each potential updating error. Subsequently, based on a novel construction of timer-coupled storage functions, the dynamics of these timer variables is designed and investigated, leading to less conservative conditions on the MATI and MAD than those by the standard approach while still ensuring closed-loop input-to-state stability with respect to external disturbances. Finally, simulation results are given to illustrate the advantages of these new results.
comment: Accepted by IEEE CDC 2026. Copyright may be transferred without notice, after which this version might no longer be accessible
★ Global stability of Michaelis-Menten networks
In a very nice paper, Blanchini, Breda, Giordano and Liessi studied networks of species coupled through Michaelis-Menten interactions. Under standard assumptions of strong connectedness and non-degeneracy, they show that there is a unique equilibrium, which is (locally) asymptotically stable. This note shows that global stability, which was stated there as an open problem, holds as well. In fact, whenever a positive equilibrium exists, it attracts every solution with positive initial condition, with no connectedness or non-degeneracy hypothesis at all; those assumptions are needed only in order to guarantee existence and to cover solutions starting on the boundary of the orthant. The key is to note that the system is contractive with respect to the Thompson part metric.
★ StarMind: Challenges and Opportunities in Orbital Computing
Space-based artificial intelligence (AI) is moving beyond isolated onboard processing toward constellation-scale computing over optical inter-satellite links (OISLs). Yet current infrastructures remain fragmented, making end-to-end (E2E) coordination across heterogeneous and multi-vendor systems difficult. This article examines the StarMind vision and develops a reference framework for distributed orbital computing. It organizes the system into five layers and defines six standardizable interfaces that link mission intent to resource conditions and trustworthy result delivery. The framework also clarifies the operational constraints that determine whether orbital cooperation is worthwhile. In a representative synthetic aperture radar/interferometric SAR (SAR/InSAR) case study, the framework reduces E2E latency to 41-min, compared with 61-min for localized onboard computing and 146-min for ground-based processing. This gain does not rely on faster processors in orbit. Instead, it comes from reducing contact waiting and urgent Earth downlink traffic through resource-aware task placement and gateway selection. Feasibility analysis shows that the proposed framework is more effective when source datasets are large, output products are compact, and optical inter-satellite link capacity is sufficient. When these conditions are absent, conventional processing may remain preferable. Finally, we discuss research opportunities in joint task partitioning and cross-domain orchestration, hierarchical federated intelligence, and large model-enabled intelligence to advance an interoperable space-ground computing continuum.
comment: This work has been submitted to the IEEE for possible publication. 7 pages, 4 figures
♻ ★ Identifiability-Guided Composite Load Modeling of AI Data Centers
The rapid growth of AI data centers introduces large, workload-dependent loads whose protection and reconnection behavior can affect power-system transient stability. Calibrating dynamic models of such loads is challenging because treating all associated elements as freely estimable may lead to weakly identified or physically confounded parameterizations. This paper proposes an identifiability-guided calibration framework for a composite load model with separate IT, cooling, and auxiliary branches. A nuisance-projected Fisher-information criterion characterizes the local identifiability of the cooling-motor composition, while a workload non-equivalence result shows that voltage-only recalibration cannot reproduce workload-dependent responses. Guided by these results, a grey-box estimator trained with branch-level supervision learns the workload-to-power relation and the cooling composition, while retaining a prescribed UPS protection law and an externally anchored reconnect-ramp duration. The learned workload relation is then distilled into a closed form for transient-stability simulation. Evaluation using 300,000 MIT Supercloud GPU power-utilization samples and 12,600 IEEE 39-bus fault scenarios yields an aggregate active-power $R^2$ of 0.948, compared with 0.856 for an LSTM baseline. Removing branch-level supervision preserves relatively high aggregate accuracy but substantially degrades cooling-component recovery. Under unseen deep voltage sags, the proposed model achieves a transient $R^2$ of 0.750, compared with 0.033 for the LSTM. Closed-loop simulations further show that cooling composition and protection behavior can affect retained-load states and voltage responses near protection thresholds.
♻ ★ A Hierarchical Optimisation Framework for Integrated Electric-Hydrogen-Transport Systems
Integrated electric-hydrogen infrastructures are becoming increasingly important with the growing deployment of electric vehicles (EVs) and hydrogen vehicles (HVs) in transport systems. However, the strong coupling between vehicle scheduling and multi-energy dispatch introduces significant operational challenges. This paper models an integrated electric-hydrogen-transport system (EHTS) and proposes a hierarchical optimisation framework that couples vehicle scheduling and downstream energy dispatch through a sequential, demand-driven two-layer structure. In the vehicle scheduling layer, a solver-free greedy heuristic (SFGH) algorithm is developed to avoid repeated optimisation solving, enabling real-time EV charging and HV refuelling under non-preemptive service and within-interval sequential assignment. The resulting charging and refuelling demands are subsequently passed to the energy dispatch layer, where a deep reinforcement learning (DRL)-based approach is designed to optimise battery operation, hydrogen-tank operation, and PV generation allocation to minimise the overall operational cost of the EHTS while satisfying the scheduled transport demand. Representative case studies, together with comparative, ablation, and generalisation analyses, demonstrate the effectiveness and robustness of the proposed framework. Furthermore, the learned dispatch policy maintains strong performance across diverse transport-demand scenarios and PV penetration levels without retraining, demonstrating robust generalisation capability for practical deployment.
♻ ★ Provable Bounds on the Hessian of Neural Networks: Derivative-Preserving Reachability Analysis
We propose a novel reachability analysis method tailored for neural networks with smooth activations. Our method relies on constructing an end-to-end first-order Taylor expansion of the network and bounding the remainder. To enable this, we develop a scalable algorithm for computing closed-form bounds on both the gradient norm and Hessian norm of the network. A key component is a loop transformation of the activation functions, which leverages their monotonicity to sharpen the bounds. This yields a local abstraction that preserves the network's value and gradient at the expansion point. For larger input domains, we integrate this local model into a branch-and-bound scheme, recursively refining the overapproximation. We validate our approach through extensive experiments and demonstrate scalability and accuracy improvements over state-of-the-art verification baselines.
♻ ★ An integrated interpretable control effectiveness learning and nonlinear control allocation methodology for overactuated aircraft
Nonlinear dynamics and the strong couplings that arise between multiple effectors undermine the assumptions behind conventional, linear control allocation techniques. When flight enters regimes where nonlinear effects dominate, linear allocators exhibit reduced accuracy due to increased model mismatch, which subsequently degrades performance and robustness of the flight control system. High fidelity onboard models and black box data driven approaches can recover accuracy across the flight envelope, but respectively impose computational burdens prohibitive for real time allocation and sacrifice the interpretability required for verification and fault diagnosis. This paper addresses these limitations by learning an explicit, physics constrained analytical model of the control effectiveness mapping from representative flight data using Sparse Identification of Nonlinear Dynamics. The resulting mapping is compact, interpretable, and admits analytical derivatives, enabling efficient computation within nonlinear solvers that additionally incorporate actuator dynamics, without requiring an onboard model. An online adaptation mechanism monitors prediction residuals and refreshes the model when significant plant changes are detected, providing graceful reconfiguration under actuator failures and varying operating conditions. The methodology is evaluated on a high fidelity nonlinear benchmark aircraft across a range of aggressive maneuvers, achieving accuracy comparable to a full nonlinear onboard model while substantially reducing computational cost relative to established baselines.
♻ ★ MIRA: A Modular Open-Source Micro-UAV for Indoor Research
Indoor robotics research increasingly relies on micro-UAV platforms whose airframes, electronics, and control software are open to modification. Off-the-shelf platforms often lack the low-level access that such modifications require, while building a custom vehicle demands substantial engineering effort before the first flight test, leaving many laboratories working within constraints that limit the scope of their research. We present MIRA (Modular Indoor Research Architecture), an open-source micro-UAV for indoor research, built around a replicable 3D-printed PLA airframe with detachable arms and a containerized low-level software package that manages the companion-to-autopilot communication bridge via Micro XRCE-DDS. Designed as a white-box architecture, its core subsystems, including the companion computer, are individually replaceable without firmware refactoring, which supports local fabrication, crash-localized repair, and component substitution from existing laboratory inventory. We characterize MIRA through autonomous flights comprising takeoff, hovering, trajectory tracking, and landing within an optical motion-capture volume. The communication pipeline sustains a median companion-to-autopilot latency of 0.02 ms, with the external motion-capture link as the main remaining latency source, and time-domain analysis shows that structural vibration remains stable during dynamic maneuvers and below the reference level adopted in this work. All design files and software are released openly to support replication and extension by other laboratories.
♻ ★ From Inference to Control: Structure-Guided Control of Hypergraph Dynamics
Controllability determines whether a system's state can be guided toward any desired configuration, making it a fundamental prerequisite for designing effective control strategies. In the context of networked systems, controllability is a well-established concept. However, many real-world systems, from biological collectives to engineered infrastructures, exhibit higher-order interactions that cannot be captured by simple graphs. Moreover, the interaction structures might be unknown and difficult to measure directly. Here, we close this gap by combining hypergraph inference with the identification of controllable nodes. Building on the inferred structure, we design a controller that, given a set of controllable nodes, steers the system toward a desired configuration. We formulate analytical controllability guarantees for polynomial systems. For non-polynomial dynamics on hypergraphs, we propose a heuristic method for identifying controllable nodes and validate the proposed approach using Kuramoto oscillators.
comment: 6 pages, 4 figures. Content was expanded following reviewers' comments
♻ ★ Mode-Weighted Transport Certificates for State-Dependent Reflected Switching Diffusions
State-dependent switching diffusions can contract in law even when individual modes are expansive. We establish a computable, constant-one contraction certificate on $\mathbb{R}^n$ and compact convex domains with normal reflection. The mode-weighted transport cost combines spatial separation with a discrete graph penalty and retains spatial dependence when the modes differ. A synchronous Brownian coupling and a maximal coupling of jump clocks in equal modes lead to separate equal- and unequal-mode generator inequalities. We give a complete regularization argument for the nonsmooth cost, including the drift and jump errors, the sign of reflection, and the limiting Itô-Dynkin calculation. A resolvent proof transfers contraction to probability-valued weak solutions of the no-flux Fokker-Planck-Kolmogorov system, while an explicit Foster-Lyapunov condition ensures existence and uniqueness of an invariant law on the whole space. For fixed weight ordering and decay rate, certificate synthesis is a semi-infinite linear feasibility problem; on compact domains, a Lipschitz-buffered finite mesh provides a rigorous test. One-dimensional and planar examples illustrate the certificate and its numerical verification.
♻ ★ Physical AI: A Survey of Models, Learning, Systems, and Open Challenges
Physical AI aims to enable intelligent systems to perceive, reason, act, and learn through interaction with the physical world. Unlike purely digital intelligence, physical agents must operate under partial observability, uncertain dynamics, embodiment constraints, and real-time requirements, while their actions continuously reshape the environment and subsequent observations. Recent advances in foundation models and robot learning have expanded the capabilities of physical agents, yet achieving reliable autonomy requires more than improvements in individual models. This survey presents a comprehensive, system-level review of Physical AI, organized around the closed-loop process of physical interaction. We examine the foundations and recent developments in task representations and memory, predictive and action-generating models, data and learning methods, embodiment and control, computing systems, evaluation, and safety. Beyond reviewing individual advances, we investigate how these components interact and collectively shape the capabilities and limitations of Physical AI systems. Building on this integrated perspective, we identify fundamental challenges in generalization, resource-efficient learning and execution, reliable state tracking, safety, recovery, and continual adaptation. We argue that advancing Physical AI requires moving beyond isolated model performance toward integrated systems capable of robust, efficient, safe, and autonomous physical interaction under changing real-world conditions.
♻ ★ Rate-Cost Tradeoffs in Nonlinear Control
We study finite-horizon rate-cost tradeoffs for general stochastic control systems, including nonlinear systems. Under an average control cost constraint, we show that the minimum expected communication rate lies within an additive logarithmic gap above the normalized directed information minimized over causal control policies. This establishes directed information as the operationally relevant quantity governing rate-limited control, extending its previously established roles in causal source coding and linear quadratic Gaussian (LQG) control to general nonlinear control systems.
comment: 8 pages, 1 figure
Robotics
★ Blind Precise Assembly: Proprioceptive Sim-to-Real Peg-in-Hole for Diverse Parts
Every robot already carries high-resolution contact sensors: its own joints. Legged robots and humanoids exploit them to walk, run, and track whole-body motions without vision, but there the contacts are large and forceful. Is joint sensing also enough for precise, contact-rich assembly, where the robot must find a millimeter-scale opening, mesh gear teeth, or catch a thread through light contact? We study this question by training blind assembly policies that observe only a short history of joint positions, velocities, and their own actions, without visual, force, torque, or tactile sensing. The poses of both the fixture and the part are unknown to the policy, and the grasp is randomized substantially. The policy must therefore assemble the parts through contact, sensed exclusively through its own proprioception. Trained with reinforcement learning entirely in simulation over randomized parts, grasps, and contacts, such policies learn peg insertion, gear meshing, and nut threading. The peg insertion policy transfers zero-shot to the real world and inserts pegs of different shapes and sizes in 73% of trials. Controlled comparisons with matched visual policies in simulation show that blind policies generalize far better to unseen part shapes and sizes, though they can be slower. Training such policies needs thousands of different assemblies, each simulated with its clearance intact, so we also release an open-source generator that builds them procedurally from exact convex pieces for MuJoCo; it is more accurate and faster than convex decomposition or MuJoCo's signed-distance collision. Videos and interactive simulations are at https://younghyopark.me/blind-proprioceptive-assembly/.
comment: 21 pages, 15 figures, 7 tables. Antonia Bronars and Younghyo Park contributed equally. Project page: https://younghyopark.me/blind-proprioceptive-assembly/
★ StopWatch: Goal Grounding for Object-goal Navigation
Pretrained VLMs bring rich world knowledge and semantic priors to object-goal navigation, and fine-tuning a VLM into a navigation policy has become a common recipe. Despite this progress, success rates remain low, and it is unclear which capability these fine-tuned policies lack. Our diagnosis identifies goal grounding as the dominant failure. These policies explore well, yet often fail to notice that the goal is already in view or to estimate how far away it is. To address this failure, we ground the goal explicitly. Explicit grounding substantially improves success and reaches state-of-the-art results on six benchmarks across three scene collections. We also correct annotation defects in these benchmarks, so that future methods are evaluated more faithfully. Our analysis on the clean benchmarks shows that, with ground-truth grounding, our policy nearly matches the expert that generated its demonstrations, leaving the headroom in perception rather than policy learning. Our work points to goal grounding as a missing piece of object-goal navigation and offers a simple and effective solution to the task.
comment: Project website https://stopwatch-nav.github.io
★ SparseMax-dSTL: Sound Robustness with Horizon-Independent Gradient Allocation for Temporal Logic Trajectory Optimization
A gradient-based solver that enforces a Signal Temporal Logic (STL) specification uses a smooth approximation of its quantitative semantics, the robustness. In safety-critical trajectory optimization, this approximation must have two properties. First, it must be sound, that is, nonnegative only on trajectories that satisfy the specification. Second, at each temporal operator, the weight of its gradient on the violating samples must stay bounded away from zero, however long the horizon. The approximations in use, including the log-sum-exp, its sound variant, and the generalized-mean robustness, do not have both. We introduce lower approximations of the minimum and the maximum whose gradients are sparsemax projections, and we prove that the robustness built from them has both. It lies below the exact robustness and is therefore sound, and its error has a bound that is independent of the number of samples and known before optimization. The gradient of each approximation is exactly zero on every sample beyond a data-dependent threshold, so the weight on violating samples a gap beyond the others has a positive lower bound at every horizon. On a planar unicycle and on a seven-joint manipulator beside a person in simulation, the log-sum-exp approximation is positive on a violating trajectory. As the horizon of the manipulator's task grows from 10 to 80 s, the gradient weight of the sound variant on the violating samples falls from 0.22 to 0.04, while that of sparsemax stays at 1. In a constrained trajectory optimization of the unicycle, the constraint on the log-sum-exp robustness returns a violating trajectory, whereas the constraint on the sparsemax robustness returns a trajectory that satisfies the specification with the required margin.
comment: 8 pages, 5 figures. This work has been submitted to the IEEE for possible publication
★ Self-Building Labs
Self-driving laboratories (SDLs) accelerate scientific discovery by automating experimental execution and optimization. AI agents can increasingly propose new experiments, but existing SDLs can only run the procedures they were built for. Adapting them to a new experiment still takes extensive manual engineering, which makes laboratory flexibility the bottleneck for scaling discovery. We introduce the self-building laboratory (SBL), which designs, configures, and operates its own laboratory setup given a high-level scientific goal. SBL starts from whatever already exists, from an empty room to a working SDL, and builds the missing capabilities: the lab layout, its digital twin, robot skills, and experimental workflows. We demonstrate the system working across different robots and setups, scaling from a single workstation to laboratories with several robots working together. We validate SBL's end-to-end capabilities through a real-world quantum-dot synthesis experiment. By shifting the role of human researchers from manually configuring and programming device automation to specifying scientific objectives, SBL makes scientific automation more accessible, flexible, and scalable. Project website: https://self-building-labs.github.io/
★ MOSS: Multimodal Origami Swamp Robot System ICRA 2027
Robotic systems for environmental monitoring and ecological surveying in wetlands and swamps must operate across diverse terrestrial and aquatic conditions. Conventional fixed-geometry wheels cannot adapt their contact geometry to these varying conditions. This work presents MOSS (Multimodal Origami Swamp Robot System), a compact amphibious robotic platform with 3D-printed origami wheels. The compact crank mechanism switches the wheel between contracted and expanded configurations, changing its width, body diameter, and fin orientation simultaneously. The contracted configuration is better suited to efficient rolling on firmer ground, whereas the expanded configuration provides greater contact and terrain engagement on softer surfaces. MOSS was evaluated on soil, sand, mud with varying water content, and water, in addition to in-motion morphing and terrestrial-aquatic transition tests. The more energy-efficient wheel configuration depended on terrain. The contracted configuration achieved a lower cost of transport on firmer soil and mud, whereas the expanded configuration performed better on wetter mud, sand, and water, reducing the cost of transport by up to 29%. Expanded wheels also reduced slip in wet mud and reduced both slip and hub sinkage in sand. MOSS successfully completed configuration changes in $1.95 \pm 0.11$ s and demonstrated continuous locomotion across soil-mud-water transitions. Outdoor trials further demonstrated operation across natural deformable terrain and water. These results show that adaptive wheel geometry provides terrain-dependent locomotion benefits and enables a single compact platform to traverse substantially different terrestrial and aquatic conditions.
comment: 8 pages, 8 figures, submitted to ICRA 2027
★ Not All Actions Contribute Equally: Learning Precise Progress Estimation from Imprecise Demonstrations
Progress estimators play an essential role in providing dense reward signals for reinforcement learning in robotic manipulation, but accurately assessing task progress can be much challenging. Existing methods heavily rely on expert data and naively assume that all actions within a demonstration contribute to task success. This assumption often breaks down in real-world scenarios, especially in high-precision manipulation, where subtle geometric errors can lead to task failure. Therefore, these methods can erroneously interpret human mistakes as meaningful progress, further corrupting progress learning. We argue that a key limitation is the lack of a global prior regarding task evaluation criteria, which is crucial for distinguishing meaningful progress from human mistakes. To this end, we propose PRIME, a novel progress estimator capable of learning precise progress estimation from imprecise demonstrations by aggregating experience across trajectories. PRIME constructs a weighted graph over observed states, connecting states across trajectories based on spatial proximity and visual consistency. Edge weights encode spatial distances as transition costs, and shortest-path search approximates the geodesic distance to success. Changes in this distance naturally distinguish advancement, stagnation, and setbacks without assuming monotonic progress over time. Extensive experiments in both simulation and real world on precise manipulation tasks show that PRIME is consistently effective across diverse tasks, and robust to data with varying degrees of suboptimality. Notably, the advantage of PRIME gets more pronounced in higher-precision scenarios, yielding up to a \(5\times\) improvement in policy success rate over the SFT baselines on charger insertion.
comment: 9 pages, 13 figures
★ Learning Robot Metareasoning: Adaptive Computation Budgets for Real-Time Control
Planning while moving faces an historical trade-off between deliberation and reactivity. A planner running concurrently with execution needs time to compute, so the robot must commit to an open-loop action chunk for that duration: a longer commitment buys a better plan at the cost of feedback frequency. Existing controllers fix this action commitment, and consequently the computation budget, before deployment. However, the optimal computation time is highly state-dependent (due to factors such as unmodeled dynamics, safety constraints, and marginal optimisation gains), i.e. dynamic, therefore offline static tuning risks leaving systems blind during critical maneuvers or unable to find global effective solutions. We cast the identification of the optimal computation budget as a semi-Markov decision process (SMDP) in which a single decision defines both the robot action commitments and the time allocated to the solver. To solve this, we train a reinforcement-learning policy conditioned on the current state, the executing plan, and inexpensive planner statistics. Across ground-robot navigation, agile quadrotor flight, and contact-rich manipulation, the learned policy changes its budget with the demands of the scenario, discovering efficient reasoning patterns that outperform fixed-budget baselines. https://lorenzomagnino.github.io/learning-robot-metareasoning-review/
★ TransNav: Transformer-Based End-to-End Navigation Method for Static and Dynamic Quadrotor Obstacle Avoidance
Obstacle avoidance for autonomous quadrotors in cluttered environments remains challenging, especially when both dense static obstacles and moving obstacles are present. Existing methods are often tailored to either static or dynamic settings, which always limit their adaptability across diverse environments. This paper presents TransNav, a transformer-based end-to-end navigation method for quadrotor obstacle avoidance in both static and dynamic environments. TransNav employs a patch-based point-cloud encoding scheme that combines local feature extraction with global contextual modeling, enabling compact yet informative environment representation. The method integrates perception and navigation in a single architecture and directly predicts a finite-horizon action sequence, from which the first action is executed. Moreover, we develop an unsupervised learning strategy by incorporating a lightweight point-mass dynamics model, which directly optimizes trajectory-level costs over the predicted sequence through numerical gradient propagation. Thus, the policy can be efficiently trained without expert demonstrations. Simulation and real-world experiments show that the proposed method achieves strong performance in dense static environments and competitive obstacle-avoidance capability in dynamic scenarios, while maintaining low inference cost for onboard deployment.
comment: 8 pages, 6 figures, 2 tables. Published in IEEE Robotics and Automation Letters (RA-L)
★ Foundation-Model-Driven 3D Scene Graphs for Mobile Manipulation
Geometric and Semantic Foundation models are becoming increasingly popular tools for generating spatial maps for robots. Recent work (e.g., FOUND-IT) has leveraged these tools to construct real-time, task-driven, and open-set 3D scene graphs from a single uncalibrated camera. However, while classical robot mapping techniques have undergone extensive system demonstrations, limited work has been done to develop both the fundamental and the systems wide advancements needed to integrate these new types of robot spatial memory into a full, useable autonomy pipeline. In this paper, we bridge this gap, and in doing so we also evaluate the effectiveness and limitations of this type of 3D memory. Towards this end, we integrate FOUND-IT into a mobile manipulator platform and demonstrate it is well-suited to support multiple objectives such as object change detection, support for multiple mapping sessions, determination of navigable paths, and directly support object grasping for the task of scene re-arrangement. Furthermore, we provide nuanced insights into the effectiveness and limitations of a monocular, foundation-model-driven 3D scene graph construction system using observations from rigorous on-robot experiments. Code is available at https://github. com/MIT-SPARK/FOUND-IT.
★ Beyond Action Prediction: Learning Search-Free Planning in Latent Space
Latent world models learn action-conditioned state transitions in a compact representation space, providing a predictive model for visual planning and control. However, planning with these models typically requires costly test-time optimization over action sequences. We propose to replace this online search with a goal-conditioned inverse model that directly predicts actions from current and goal observations. A key challenge is that action prediction alone neither ensures that goal representations encode action-relevant information nor that predicted actions are consistent with the learned latent prediction model. We introduce Hindsight Inverse JEPA Planning (HIJP), a search-free framework that addresses both aspects during training. HIJP relabels future observations along offline trajectories as goals, providing action supervision across multiple temporal horizons. By retaining gradients through the goal representations, this supervision directly shapes the visual representation. HIJP further feeds predicted actions into the latent predictor model and aligns the resulting next-state representations with encoded successor observations, enforcing action-transition consistency. At inference time, the inverse dynamics model and latent predictor models are applied alternately to generate multi-step action sequences without online search. Across four visual control tasks, HIJP achieves 86.8% overall success and a 1.9-16.5x planning speedup over search-based planning.
★ F-LIVO: A Feature-Level Geometric Measurement Framework for LiDAR-Inertial-Visual Odometry
Measurement-level LiDAR-Inertial-Visual Odometry (LIVO) can achieve high accuracy, and its practical resource requirements depend on the adopted association and map-maintenance strategy. Persistently attaching dense photometric patches or appearance attributes to mapped LiDAR points can introduce additional storage, association, and update costs. This paper presents F-LIVO, a filter-based LIVO framework that couples a visual-inertial subsystem and a LiDAR-inertial subsystem through feature-level point-to-plane constraints between visual points and LiDAR planes. To preserve cross-modal coupling when visual triangulation is weak, we further recover visual feature depth from the LiDAR map through local plane fitting and ray-plane intersection. In the LiDAR update, the covariance of each point-to-plane residual is adaptively scaled according to local planar reliability, thereby reducing overconfident updates on poorly conditioned surfaces. Because the proposed method relies on sparse geometric features rather than persistent dense photometric map attributes, it retains a compact map representation. Experiments on multiple public datasets show competitive, sequence-dependent accuracy. Across the evaluated degradation cases, F-LIVO maintains continuous estimation with comparatively low accumulated end-to-end drift. Under the evaluated resource configuration, it achieves the lowest mean and peak central processing unit (CPU) usage, the lowest peak resident set size (RSS), and the lowest RSS growth rate among the evaluated implementations, while R2LIVE retains a lower mean RSS. The source code will be made publicly available upon acceptance to facilitate reproducible research.
comment: 19 pages, 20 figures
★ Enhancing Visual Task-Redirection Attacks on Vision-Language-Action Models via Closed-Loop Optimization
Vision-language-action (VLA) models enable language-guided robot manipulation, but remain vulnerable to visual attacks that undermine user control over robot behavior. Prior studies optimize visual perturbations using fixed target supervision. However, the perturbation can change earlier actions and, in turn, the physical states reached later. As the attacked trajectory evolves, this supervision may no longer match the behavior needed at the states actually reached. Accurately matching these targets can therefore fail to advance the attacker-specified task. Even with suitable supervision, reducing average matching error does not ensure sustained task progress. The robot may still fail at a critical stage, and further updates can disrupt previously successful behavior. To address these limitations, we propose TRACO, a framework that enhances existing targeted visual attacks through closed-loop optimization. Its Execution-Driven Supervision rebuilds supervision for the states reached under attack to guide subsequent updates. Within this framework, Progress-Guided Selection focuses updates where progress stalls while maintaining supervision on the earlier behavior needed to reach that point. Experiments across VLA models and simulation environments show improved task-redirection success with TRACO, highlighting the security implications of closed-loop attack optimization.
★ Embodiment-Aware Planning for Heterogeneous Bimanual Manipulation
Many manipulation tasks that are effortless for dexterous human hands remain challenging for robotic systems equipped with simple, task-specialized grippers. Industrial platforms commonly rely on end-effectors such as suction cups or parallel-jaw grippers, yet no single mode can reliably satisfy the diverse constraints encountered across different phases of complex manipulation tasks, including pickup, reorientation, insertion, and constrained placement. In this work, we investigate whether complementary simple grippers within a heterogeneous bimanual setup can collectively realize tasks that exceed the capability of any individual gripper alone. Given a single human demonstration specifying the desired object motion, we formulate embodiment-aware execution as a finite-horizon optimization over arm-trajectory assignments. At each segment, the planner determines whether single-arm manipulation or a coordinated dual arm handover is required, jointly reasoning over heterogeneous grasp feasibility, motion constraints, manipulability, and cross-gripper compatibility. Exploiting this formulation, we compute globally consistent assignments with explicit switching regularization. Experiments on a dual Franka FR3 platform equipped with suction and parallel-jaw grippers demonstrate improved feasibility and reliability in tasks that are infeasible under single-gripper execution.
comment: Please find the project website here: https://sukhvansh2004.github.io/Bimanual-Manipulation/
★ Reason-guided Human Reward Modeling for Preference-based Reinforcement Learning
Reinforcement learning (RL) policies optimized for task rewards may produce behaviors that are efficient but misaligned with human expectations, safety preferences, or intent. This paper presents a reason-guided preference-based reinforcement learning (PbRL) framework that learns a human reward model from trajectory-level feedback. In addition to preference labels, the framework incorporates reason tags explaining why a behavior is preferred, enabling the learned reward to capture both human choices and their underlying rationale. Compared with preference-only reward learning, the reason-guided formulation provides additional supervision for learning human-aligned rewards and improves the interpretability of the resulting policy. The learned human reward is combined with the environment reward during policy optimization, allowing the agent to improve task performance while producing more interpretable, human- aligned, and preference-sensitive behavior
★ Reinforcement Learning Based Multi-Terrain Adaptive Whole-Body Control for a Quadruped Manipulator
Legged robots equipped with manipulators offer promising capabilities for loco-manipulation tasks in unstructured environments. However, achieving task-consistent end-effector behavior during locomotion remains challenging due to terrain-induced disturbances and the strong coupling between locomotion and manipulation, particularly for small-scale quadruped platforms. A reinforcement-learning based whole-body control framework is proposed for quadruped loco-manipulation on complex terrain. The proposed framework combines a hybrid task representation with task-prioritized whole-body learning to coordinate locomotion and manipulation while preserving task-consistent end-effector behavior. A lightweight quadruped manipulator platform is further developed to validate the proposed framework on a small-scale quadruped system. Real-world experiments demonstrate successful deployment on complex terrains such as slopes up to 30 degrees, stairs, and deformable terrain. These results demonstrate the feasibility of deploying learned whole-body control policy for quadruped loco-manipulation in real-world environments and highlight the importance of task-consistent whole-body coordination for terrain-adaptive loco-manipulation.
★ FLoTS-SLAM: Optical Flow-Derived Uncertainty for Monocular Dynamic SLAM with Two-Stage Optimization
Monocular simultaneous localization and mapping (SLAM) remains challenging in dynamic environments. Real-time dynamic object identification in such environments is a critical problem. Recent approaches estimate per-pixel uncertainty from rendering residuals or multiview feature inconsistency to identify dynamic regions. These methods lack direct motion evidence, and their accuracy may degrade when reconstruction is inaccurate or features are ambiguous. In addition, such uncertainty merely downweights pixels in bundle adjustment (BA), so inaccurate uncertainty estimates can still impair camera pose estimation and 3D reconstruction. Accordingly, we present a monocular RGB SLAM system for dynamic scenes. Specifically, a motion uncertainty estimation method is proposed to derive uncertainty from the residual between optical flow and camera motion induced flow, directly reflecting object motion. The estimation varies with frame alignment accuracy, reducing false detections and yielding accurate pixel-level masks of dynamic regions. To make BA more robust, we estimate camera poses from high confidence pixels. These poses are then used to reproject dynamic pixels during joint refinement of poses and depth, reducing the influence of object motion on both pose and depth updates. Moreover, we provide a quantitative evaluation of dynamic masks and analyze how these masks affect the downstream tracking and reconstruction of dynamic objects. Our approach improves the mean dynamic mask IoU by 35\% over a state-of-the-art baseline on DAVIS, and reduces the average ATE by 59\% on Bonn and 19\% on TUM relative to the SLAM backbone used in our system. The source code will be made publicly available.
★ HexaShell: A Thin Harness for Exploring the Limits of VLM Agents as Robot Coding Agents
Agents built on vision language models (VLMs) now compete with learned policies on robot manipulation leaderboards. We ask how far a coding agent can go when it controls a robot instead of a computer, and where it stops. Our system, HexaShell, keeps the harness thin: it only turns the agent's commands into robot motion. The agent drives a robot with two arms through one command line. It sees three RGB cameras and knows their calibration. After each failed attempt, the agent reads the log, finds the cause, and writes or fixes small programs called tools. In this way it keeps what it learns as code, as a coding agent does when it fixes a failing test. The tools may use only what the robot observes. They use triangulation to find where objects are, relative to each other and to the gripper. If two cameras see the same point, the point lies where the two camera rays meet. If a point lies on a known plane, such as the table or the top of a block, it lies where its ray meets that plane. When the head camera is not enough, the robot moves a wrist camera to get a second view. We build the tools on only ten layouts of one seed per task, then freeze them and test them on ten new layouts of the same seed. We use Codex with GPT-6-Astra at medium reasoning effort. On all 42 RoboDojo tasks, HexaShell reaches a success rate of 39.7% and a progress score of 48.0, averaged in the same way as the leaderboard. This is a new state of the art on RoboDojo. The limit is contact, not reasoning: pushing reaches 92.5% and memory tasks 65.0%, but tight insertion only 8.3%. When the agent fails, it usually sees the failure but cannot make the motion work. We open source the tools evolved so far at https://github.com/HexaFuture/HexaShell.
comment: Code: https://github.com/HexaFuture/HexaShell
★ No Eyes, No Problem: What Does Touch Contribute to Blind Baoding Ball Manipulation?
Humans manipulate objects within their hands through touch alone with remarkable ease, but for robots, achieving sustained real-world manipulation without vision remains an open challenge. Although reinforcement learning (RL) can generate sophisticated blind manipulation policies in simulation, transferring them to a real dexterous hand remains hindered by continuously changing contacts, motion-induced tactile information drift, and the gap between simulated and physical dynamics. We study this problem through Baoding ball rotation on the Shadow Dexterous Hand Lite, using only proprioception and binary tactile feedback as observations. We achieve successful zero-shot transfer through a combination of a sensor-aligned, coupling-aware embodiment in simulation, tactile calibration, and two novel episodic domain randomisations: slew-rate action limits and STuck-AT Taxels. On the physical hand, the policy achieves sustained blind Baoding rotation, completing 225 consecutive $\mathbf{180^\circ}$ rotations (112.5 full rotations) at $\sim0.36 \;\text{rot/s}$ in a single continuous trial without any vision or object-state information. Furthermore, we evaluate the contribution of tactile sensing relative to proprioception alone and identify the hand regions where tactile information is most important. This is the strongest reported hardware result for Baoding ball manipulation in both rotation count and speed, irrespective of the sensing modality used. By effectively bridging the sim-to-real gap, our framework sets a new benchmark for sustained tactile-driven dexterous manipulation on physical hardware.
comment: 9 Pages, 8 Figures
★ AWAM: Active Trajectory-Conditioned View Imagination for Robust Robot Manipulation
Rapid progress in embodied intelligence has enabled increasingly capable robots to follow language instructions and manipulate diverse objects. However, strong results in structured environments do not automatically transfer to open-world conditions, where clutter, occlusion, and changing viewpoints can obscure the spatial relationships required for action. To address this perceptual gap, a virtual active-perception paradigm is introduced. AWAM, an Active World-Action Model actively plans where to observe and imagines what will be seen along the planned trajectory before acting. Task-related virtual camera motion is predicted and used as a structured query into a geometry-conditioned world model, producing a reusable spatial prior for manipulation, which is later compressed into a fixed memory and fused with live observations through cross-attention for diffusion-based action prediction. AWAM achieves competitive success rates with 75.9% on LIBERO-Occ, 62.4% on LIBERO-Plus, 76.5% on RLBench-OG Occlusion1, and 94.4% in real-robot trials. These results support improved robustness to the evaluated occlusions and distribution shifts, addressing part of the gap between structured manipulation and less controlled environments.
★ Look Before You Act: Bridging Active Perception and Vision-Language-Action Models via Observation-Aligned Data Adaptation
Robots manipulating objects in clutter must decide not only how to act, but also where to look. Although active camera motion can expose an occluded target, it may move the robot into an initial observation-state distribution that differs from the manipulation policy's training distribution. We identify this problem as first-frame distribution sensitivity: under perturbed starts in LIBERO, naively prepending camera repositioning to pi_0.5 reduces success from 46.3% to 11.5%. We introduce ActiveAlign, a data-adaptation framework that trains a Vision-Language-Action (VLA) policy from the states at which active perception terminates. A learned viewpoint module first generates a camera trajectory. Offline, ActiveAlign uses closed-loop alignment to connect the trajectory endpoint to an existing demonstration, replays the remaining manipulation trajectory, and retains successful continuations for fine-tuning. At deployment, the same viewpoint module precedes the adapted VLA without modifying its architecture. Across 40 LIBERO tasks, ActiveAlign achieves 70.7% success with pi_0 and 72.7% with pi_0.5, outperforming their standard-training baselines by 40.0% and 26.4% respectively. Matched-state and data-matched controls support gains beyond prefix shortening and generic viewpoint augmentation; OpenVLA and GR00T N1 also show the same trend. Success rate under physical occlusion in real world increases from 35/90 to 83/90. These results show that active perception benefits manipulation when policy training is aligned with the observation and robot-state distribution induced by viewpoint selection.
★ Beyond Success Rate: Interventional Evaluation of Memory in Vision-Language-Action Systems
When we say that a robot uses memory, does that memory actually play a role? Success-rate ablations alone cannot establish the use of task-relevant facts: they change information and computation together, while aggregate rates can hide offsetting gains and losses. We propose an Interventional Memory Evaluation Protocol (IMEP) for vision-language-action (VLA) systems to address four questions: (1) Does memory provide a net benefit? (2) Under what conditions does memory help? (3) Do changes to memory alter task outcomes? (4) Are these effects specific to task-relevant facts? At the retrieval interface, we compare target-fact substitutions with fact-preserving replacements, minimal numerical perturbations, and exact replicas, while assessing decision correctness separately from motor execution success. We evaluate within-episode, cross-episode, and agentic memory. In cross-episode learned-bank experiments, a one-unit-in-the-last-place (ULP) intervention, including its installation procedure, changes 11.9% of seed-paired outcomes, while changing key facts produces no clear additional effect over fact-preserving replacements. To examine when memory helps, we introduce ChainBench, where correct decisions require historical information unavailable in current observations. The full memory configuration raises task success from 39.1% to 74.3%, matching the oracle-decision condition under shared recorded executions. These results show that correct history can support robot decisions, but establishing whether a memory system realizes this benefit requires fact-specific evidence beyond aggregate success rates.
comment: 41 pages, including references and appendices
★ LAW: Unifying Language, Action, and World Modeling for Agentic Physical Intelligence
World-action models jointly predict actions and future world states, but long-horizon manipulation further requires language reasoning to plan subtask objectives, assess execution progress, and guide action selection. We present LAW, a closed-loop VLM--WAM framework that unifies language reasoning, action generation, and world modeling through semantic subtask progress. Within the LAW agent, a hybrid-expert vision-language model plans the remaining subtasks, estimates the values of and fuses actions proposed by a world-action model, and asynchronously assesses execution status to support continued execution and subtask transitions. Within the world-action model, Mixture-of-Transformer Experts integrates modality-specific and shared computation for joint video--action modeling. Dual-stream hierarchical visual memory maintains temporal context for both models, while subtask-local value learning aligns action evaluation with semantic objectives and preserves supervision from successful prefixes of failed trajectories. LAW achieves 94.21% average success across 50 RoboTwin tasks. Replacing the WAM backbone with OpenWAM-$alpha$ further raises the average success rate to 95.56%, establishing a new open-source state of the art on RoboTwin. Despite having only 2B parameters, our VLM also outperforms variants using Embodied-R1.5 and GPT-5.5 High. On three long-horizon real-robot tasks, LAW achieves 65% average success, compared with 28.33% for $pi_{0.5}$ and 25% for Fast-WAM.
★ From Sliding Windows to Self-evolving Lifelong Memory: A Dual-perspective Survey of Memory Paradigms for Long-Horizon Robotics VLA Models
Long-horizon robotic tasks expose a fundamental limitation of current Vision-Language-Action (VLA) models because their restricted temporal context pre-vents effective accumulation and reuse of past experiences. As a result, memory has become a fundamental component for enabling embodied agents to perform long-horizon reasoning and adaptive decision-making. This survey presents a systematic taxonomy of VLA memory mechanisms based on memory scope and persistence, categorizing existing approaches into five representative paradigms and one emerging frontier. Based on this taxonomy, we compare representative VLA memory systems in terms of memory characteristics, architectural com-plexity, and open-source maturity, revealing a clear trade-off in which increasing architectural complexity is generally accompanied by decreasing open-source ma-turity. Finally, we discuss the key challenges in building full-lifecycle memory systems, including efficient memory management and lifelong knowledge evolu-tion. Overall, this survey provides a unified perspective on memory architectures for VLA agents and offers insights into the development of lifelong embodied in-telligence.
comment: 10 pages, 1 figure
★ SkyAgent: Revisable Target Memory for Continuous Aerial Search
Continuous aerial search builds target hypotheses from sparse, viewpoint-dependent observations. When later evidence contradicts a stored hypothesis, however, the candidate can remain available to the planner and continue triggering approach or verification actions. We call this trajectory-level failure false candidate persistence (FCP). We equip SkyAgent with Revisable Target Memory (RTM), which associates evidence with persistent candidates, discounts correlated views, and revises each candidate through Tentative, Confirmed, and Rejected states. The key interface binds every generated waypoint to its source candidate: rejection therefore invalidates stale goals, while conservative reactivation protects temporarily occluded targets. Across four splits from the CityAVOS, UAV-ON, and OpenUAV benchmarks, our SkyAgent equipped with RTM outperforms all competitors, improving SR by 2.90--6.31 points while substantially reducing Behavior-FCP and post-contradiction travel with a low false-rejection rate. Across 35 matched real-world DJI scenarios per method, RTM consistently improves task success and behavioral reliability, further validating its state-to-action correction under physical deployment conditions.
★ CoEvolve: Self-Evolving Code Policies from Skills to Strategies for Humanoid Robots ICLR 2027
Recent advances in Vision-Language-Action (VLA) models and World Action Models (WAMs) have expanded the manipulation and whole body capabilities of individual robots. Yet a single robot remains limited by its embodiment, and many real world tasks require capabilities that can only emerge through cooperation. Existing multi robot frameworks largely emphasize task decomposition, allocation, and replanning, while providing limited support for adapting executable coordination when physical execution deviates from the intended behavior. We present CoEvolve, a framework that represents team coordination as executable code over fixed robot motor skills. CoEvolve introduces Verifier Guided Coordination Evolution, where a multimodal foundation model instantiates coordination strategies through motion primitives, semantic objectives, temporary robot roles, and bounded execution parameters. Physical verification evaluates each executed transition, rejected behaviors trigger structured coordination revision from an accepted checkpoint, and verified experience is retained to guide subsequent decisions. We evaluate CoEvolve with two simulated Unitree G1 humanoids transporting a freely supported rigid payload through constrained corridors. Across 15 base task runs with three multimodal backbone families, CoEvolve completes 8 tasks, with every backbone achieving end to end success. Additional evaluations yield verified placement on two additional corridor layouts and after both same-backbone and cross-backbone experience initialization, while exposing persistent limitations in motion precision and contact stability. These results support executable coordination as a practical interface for adapting long horizon multi robot collaboration under shared physical constraints.
comment: Under review at ICLR 2027
★ LY-GWM-RoboCasa-Human300: Prediction-Guided Action Selection with a Graph Dynamics Model
We present LY-GWM-RoboCasa-Human300, a policy that combines a Human300-fine-tuned GR00T-N1.5 action generator, an action-conditioned graph dynamics model, and a learned binary state-reward scorer. At inference, the policy samples four action chunks, predicts the future latent features and robot state for each, and executes the candidate with the highest predicted-state reward logit. Predictions therefore influence the actions sent to the environment. The evaluated system uses a 16-step prediction horizon and keeps all three components frozen during evaluation. In the local evaluation on 50 RoboCasa 1.0.1 tasks in the pretrain kitchen split, with 50 episodes per task, the system records 207/2,500 successes (8.28%), compared with 181/2,500 (7.24%) in the historical self-trained GR00T-N1.5 baseline. The observed difference is 26 successes, or 1.04 percentage points. The atomic group improves from 141/900 to 150/900. Separately, the RoboCasa maintainers re-evaluated the submission and reported 354/1,500 successes (23.6%) across 50 tasks, with leaderboard rates of 48.3%, 16.3%, and 3.1% for Atomic-Seen, Composite-Seen, and Composite-Unseen. The local and official results are reported separately; their difference is not attributed to a model improvement. Recorded local audits identify 212,700 changed environment-input chunks, including 28,223 in atomic tasks. We describe the actual training stages, implemented inference mechanism, evaluation provenance, and public reproduction entrypoints. These results document an active prediction-and-selection system and a positive historical difference.
comment: 15 pages. Technical report. Code and evaluation materials: https://github.com/SEIN-LYGWM/LY-GWM-RoboCasa-Human300-eval
★ NestedVLA: Learning to Consolidate and Generate Skills for Vision-Language-Action Models NeurIPS 2026
Vision--Language--Action (VLA) models promise generalist robotic agents, yet consolidating skills learned across heterogeneous tasks and environments remains difficult: training data is fragmented across platforms, joint multi-task training suffers from interference, and model merging tends to introduce parameter conflicts or require architectural changes. We instead advocate a paradigm in which skills are first learned independently and then consolidated post hoc. To this end, we propose NestedVLA, which replaces parameter aggregation with parameter generation: a nested hypernetwork, conditioned on the target task, synthesizes task-adaptive action-expert parameters that compose with a frozen base VLA. Task-specific compressed VLM residuals are retained separately. This formulation enables flexible task-conditioned skill reuse while mitigating interference. Across three RL benchmarks (MetaWorld, ManiSkill, CALVIN), NestedVLA outperforms the strongest model-merging baseline by 41 pp on average, matches per-task fine-tuning within 1 pp on ManiSkill, and substantially closes the gap on CALVIN, pointing toward a storage-efficient route to skill consolidation. Code is available at https://github.com/WanyuGroup/NeurIPS2026-NestedVLA.
comment: Accepted at NeurIPS 2026. 23 pages, 3 figures. Code: https://github.com/WanyuGroup/NeurIPS2026-NestedVLA
★ Expand-and-Prune: Test-Time Candidate-Space Adaptation for End-to-End Autonomous Driving WACV
Candidate-based end-to-end driving planners reason over a finite set of trajectory hypotheses. However, for sparse planners, a limited candidate set may provide insufficient support for scene-relevant maneuvers, while fixed downstream candidate budgets do not account for prediction uncertainty. To address these challenges, we introduce Expand-and-Prune, a test-time framework that jointly adapts candidate support and capacity without updating learned parameters. The framework expands the native candidate set using semantic driving intent and temporally propagated predictions and then uses uncertainty in intermediate trajectory scores to adaptively prune the resulting candidates. Across four heterogeneous candidate-based planners and four benchmarks, Expand-and-Prune consistently improves planning performance. It achieves state-of-the-art results on LangAuto, increasing the Driving Score from 68.2 to 75.9, while providing further improvement on Bench2Drive, Fail2Drive, and NAVSIMv2, with particularly strong improvements in out-of-distribution scenarios. Moreover, in its full instantiation, 72.5% of selected trajectories originate from the newly introduced intent-guided or temporal candidates, confirming that candidate-space expansion provides useful hypotheses beyond those generated natively by the planner. The code is available at https://github.com/iis-esslingen/Expand-and-Prune.
comment: Accepted to IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2027
★ SimpleVLN: Towards Simple and Frontier Vision-Language Navigation
Vision-and-Language Navigation (VLN) requires embodied agents to ground language in visual observations and make long-horizon decisions. Existing methods often address these demands through task-specific architectures. As multimodal large language models (MLLMs) become increasingly capable, an important question remains: Have we truly unlocked the potential of native MLLMs for VLN before introducing task-specific architecture? We present SimpleVLN, an architecture-preserving framework developed through studies of navigation formulation, data construction, and optimization. At the formulation level, SimpleVLN combines single-turn decision inputs with fixed-budget global-local memory to efficiently use navigation history without unbounded context growth, while action chunks enable efficient output of multiple primitive actions per model invocation. For data construction, we combine expert demonstrations with policy-dependent DAgger experience and develop a scalable synthesis pipeline that generates large-scale tasks for reinforcement learning. At the optimization level, a progressive IL-DAgger-RL pipeline advances from action imitation to task-level outcome optimization. Comprehensive experiments validate the effectiveness of these three aspects. We instantiate SimpleVLN across multiple MLLM backbones and model sizes, demonstrating its generality. SimpleVLN achieves 76.7% and 80.1% success rates on the R2R-CE and RxR-CE splits, respectively, substantially outperforming existing methods. These results show that the navigation capabilities of native MLLMs can be effectively unlocked without task-specific architectural designs.
comment: homepage: https://simplevln.github.io/
★ Structural Affinity: An Algorithm for Coherent Cage-Based Shape Design
Cage-based deformation (CBD) parametrises 3D geometry through a small set of control handles, effectively enabling a variety of tasks from animation and interactive modelling to tool-based co-design, neural deformation transfer, and aerodynamic shape optimization. Many optimization approaches relying on CBD employ regularisation strategies to reduce the distortion of the deformed mesh by encouraging smooth, regular cage deformations. These techniques offer soft guidance to an optimization algorithm towards desirable deformations, but compete with the task objective rather than changing the deformation parameterization itself. In this work we introduce Structural Affinity for CBD, an explicit regularisation method which enables smooth deformations through control cages by exploiting a two-cage structure to couple cage-handle motion directly. Structural Affinity precomputes an affinity matrix using mean-value coordinates, binding the inner control-cage to a coarser outer-cage. This matrix defines how much a cage handle displacement propagates to neighbouring handles which substantially decreases and often eliminates cage self-intersections, reduces mesh distortion, and adds linear overhead at deformation time. Across robotic control--morphology co-design, source-to-target neural deformation transfer, and aerodynamic wing shape optimization, Structural Affinity produces more regular deformations and substantially reduces reliance on hand-designed structural loss terms. We report consistent gains on three complementary diagnostics: the cotangent Laplacian, a proxy for surface smoothness; cage distortion, capturing whether control handles move coherently; and surface roughness, which measures undesirable oscillation in wing geometry.
comment: 27 pages, 16 figures
★ ORD-WM: Learning Object-Centric Rigid Dynamics for 3D World Modeling in Robotic Manipulation
World models provide robots with a predictive mechanism for reasoning about how actions change object states in manipulation tasks. However, existing image-based and dense 3D world models do not explicitly impose object-level rigid motion, leaving rigid-object geometry preservation to be learned implicitly and making multi-step rollouts susceptible to shape drift or object-level motion inconsistency. To address this, we propose ORD-WM, a 3D world model that learns object-centric rigid dynamics for robotic manipulation. ORD-WM learns a vector-neuron variational autoencoder supervised by signed distance fields, which encodes each object point cloud as a geometry-aware latent representation composed of 3D vector features, and uses an action-conditioned dynamics model to predict object-wise SE(3) motion increments. Rather than generating unconstrained next latent states, ORD-WM analytically applies predicted object-wise rigid transformations to the current latent states, separating persistent object geometry from pose changes. Experiments show that ORD-WM enables stable, geometry-preserving multi-step prediction and supports efficient real-time replanning for closed-loop model predictive control in simulated and real-world manipulation scenarios.
★ SCOPE: Selective Certification of Program Edits for Embodied Heuristic Learning
Embodied heuristic learning must balance strategic adaptation with the preservation of previously successful behavior. Local repairs may leave structural failures unresolved, whereas broader revisions can introduce regressions that are costly to detect through repeated environment interaction. We introduce SCOPE (Selective Certification Of Program Edits), which uses failed and successful executions as complementary evidence for program revision. Frontier-guided program search annotates a shared diagnostic graph with execution evidence to distinguish supported progress from unresolved checks. Selective behavioral certification uses stateful API replay to restore recorded responses and mutations, checking whether edits preserve selected historical successes. Diagnosis and certification feedback, supplemented by sparse environment diagnostics, guide consecutive repairs across configurations. SCOPE outperforms the strongest baseline in both simulation and real-world tasks. On LIBERO-PRO and Robosuite, it achieves mean success rates of 80.66% and 84.48%, with respective gains of 6.51 and 3.48 percentage points. Our code is available at https://github.com/VDIGPKU/SCOPE.
★ BeJEPA: A Goal-Conditioned Humanoid Behavior World Model via Unsupervised RL
Humanoid control requires coordinating whole-body posture with position and heading in the world frame. Mainstream controllers imitate trajectories in the root frame, but struggle to regain the target world pose after displacement. We introduce BeJEPA, a goal-conditioned Behavior World Model learned through unsupervised reinforcement learning from human motion priors and robot interaction. BeJEPA learns a shared representation of body configuration and world placement through a joint-embedding predictive architecture (JEPA). A Dynamic Predictor predicts the next state embedding under an action, while a Successor Predictor summarizes a goal-conditioned policy's expected discounted future state embeddings in a single vector. During joint training, Dynamic grounds the initial representation in observed physical transitions, while Successor increasingly organizes it around long-term behavior. Their composition guides policy improvement through one physical prediction and one behavioral summary, without long imagined rollouts. In 20-second motion-tracking experiments, BeJEPA with online action refinement using Dynamic outperforms all evaluated baselines in global body-position error, reducing it on AMASS by 61.7% relative to BFM-Zero with world-pose inputs and 58.7% relative to SONIC. Physical deployment on a Unitree G1 demonstrates joint attainment of posture, world position, and heading goals, recovery from the ground, and closed-loop return after disturbances and goal changes. Project page: https://ylyem9x.github.io/BeJEPA-project.
★ GRASP: Global-Local Representation Alignment and Spatial-Physical Grounding for Bimanual Robotic Manipulation
Vision-Language-Action (VLA) models that directly regress joint-space actions avoid the deployment-time cost of inverse kinematics. However, their training objectives remain misaligned with task success, as joint-space losses ignore the unequal physical consequences of joint errors and visual conditioning lacks explicit focus on hand-object interaction regions. To address these issues, we propose a training-phase prior injection framework for fine-grained bimanual manipulation. First, a physics-grounded flow-matching loss maps the implicit clean-action estimate at every noise level through differentiable forward kinematics (FK) and supervises the resulting Tool Center Point (TCP) pose and inter-arm relative pose, with a bounded noise-level weighting that compensates gradient decay. Second, we introduce a global prediction query group distilled from a frozen DINOv3 teacher through future-frame feature prediction, together with a local interaction query supervised by Grounding DINO at the attention level. Both teachers are removed at inference, and the deployment pipeline is that of a standard joint-space flow policy, adding only the learned query pathway. Simulation and real-robot experiments on long-horizon bimanual tasks show that the two priors are complementary across different deployment settings. The full model excels in in-distribution deployment, while the distilled query branch is preferable under strong visual randomization.
comment: 8 pages, 4 figures
★ InstAgent: A Recursively Self-Improving Agent Operating System for Heterogeneous Robot Collaboration
Heterogeneous robot teams enable complex, long-horizon tasks by combining complementary sensing, mobility, and manipulation capabilities across multiple embodiments. Yet existing systems struggle to retain execution-time corrections as reusable experience, maintain task-relevant context memory across space and time for reliable handoffs, and explicitly represent the inter-robot dependencies required for structured collaboration. In order to address these challenges, we present InstAgent, a recursively self-improving (RSI) Agent Operating System for heterogeneous robot collaboration. InstAgent organizes team behaviors as agentic behavior trees (BTs) that explicitly model capability assignment, inter-robot dependencies and handoff checks. The RSI mechanism updates these collaboration programs from execution feedback and preserves successful revisions for reuse in future tasks, while shared multimodal spatio-temporal memory provides context for planning and handoff validation. Experiments on 20 real-world multi-robot tasks with three high-level models show consistent gains in task success, first-attempt completion, and execution efficiency.
comment: 9 pages, 3 figures
★ Bio-Inspired Magnetic Navigation Using Gradient Descent With Rao-Blackwellized Particle Filtering
Navigation in GPS-denied environments is commonly based on magnetic maps, which are used to correct trajectory drift and enable relocalization. However, such navigation systems suffer from a strong dependence on magnetic maps that are complex, heavy, and costly to construct. Bio-inspired magnetic navigation (BIMN) does not rely on magnetic maps, making it simpler and more frugal. In this paper, we present a gradient descent BIMN method based on the Rao-Blackwellized particle filter (RBPF) for local gradient estimation. We show that magnetic navigation can be performed without magnetic maps, as gradient descent naturally attracts the system toward the destination, resulting in a simple and memory-efficient approach.
★ Foundation Models for Autonomous Space Robotics: Architectures, Physical Adaptation, and Validation
Space robots must interpret tasks and act despite unfamiliar dynamics, difficult sensing conditions, and limited opportunities for human intervention. Foundation models offer reusable perceptual and behavioral knowledge, but their value depends on how that knowledge reaches physical execution. This critical review follows space-specific requirements through perception, prediction, planning, and action to deployment and validation, emphasizing orbital manipulation and proximity operations. Terrestrial comparisons establish benefits of particular pretraining stages and model interfaces; their transfer to space dynamics and constrained onboard operation remains insufficiently tested. Event cameras offer timely motion observations across strong illumination contrasts, while spiking neural networks provide possible routes to sparse local computation. We examine how sensor degradation, state estimation and computing choices affect feedback and recovery. Higher action throughput need not yield fresher feedback; lower processor power can also coexist with higher energy per inference. We organize validation as an iterative sim-to-ground-to-space pathway, connecting physical tests with telemetry-based diagnosis and closed-loop revalidation. A satellite-capture example links observations to approach, contact and recovery, while a corresponding protocol separates generalization from adaptation with limited target-domain interaction. These comparisons identify what models already contribute and the physical evidence needed to support autonomous space operation.
comment: 23 pages, 7 figures. Critical review. Ancillary files include source inventory and evidence documentation
★ Unifying Embodied Foundation Models via a Model-to-Policy Formulation
Vision-language-action (VLA) models and world-action models (WAMs) have emerged as promising approaches to robot learning and control. These models integrate multimodal perception and action generation, but differ in their architectures, output representations, and learning objectives. In this paper, we present a unified, architecture-agnostic probabilistic formulation for embodied foundation models based on stochastic kernels. Our formulation has two key components: (i) we characterize embodied foundation models through conditional output distributions and action-realization kernels, which together induce task-conditioned robot policies; and (ii) we connect pretraining and post-training to policy deployment, accommodating model-specific pretraining objectives and offline or online post-training strategies. We instantiate the formulation for OpenVLA and DreamZero, covering autoregressive action-token prediction and joint video-action generation.
★ PASS: A Trustworthy Precision Approach Simulation System for CAT III Integrity Algorithm Development
For fixed-wing aircraft, reliable localization during approach and landing is critical to flight safety. Visual--inertial navigation is promising, but aircraft navigation also requires integrity: trustworthy outputs and timely warnings when they should not be used. Real approach data are costly to collect, while many visual errors and failures required for integrity algorithm development are difficult to reproduce safely. We present PASS, a trustworthy Precision Approach Simulation System for CAT III integrity algorithm development. In a unified temporal and geometric framework, PASS integrates configurable airport environments, prescribed trajectories, inertial simulation, calibrated stereo images, and continuous runway-relative ground truth. It supports navigation and integrity research through controllable sensor errors and environmental variations, computable visual projection errors, deterministic runway geometry priors, and condition-labeled data for statistical error modeling. We further propose Sim-to-Real Scene Trustworthiness (SRST), a measure based on DINOv2 feature distributions and UMAP, to quantify sim-to-real closeness. Experiments with representative stereo--inertial estimators demonstrate the usability of the generated sequences and quantify localization errors across algorithms and environments. PASS provides a controllable foundation for developing and evaluating visual--inertial navigation and integrity algorithms for fixed-wing precision approaches up to CAT III requirements.
★ VLA-Fabric: Learning Interaction for Composable Multi-Arm VLA Agents
Vision-language-action (VLA) models are rapidly expanding what individual robots can accomplish, yet many real-world manipulation settings require multiple arms to operate as a coordinated system. Existing approaches organize multi-arm coordination in different ways, including centralized joint policies, high-level task or role assignment, and locally executed policies that react to teammates. However, how complete action-producing VLAs should explicitly interact with one another during action generation remains largely unexplored. We introduce VLA-Fabric, which organizes one complete VLA per arm while preserving local perception and action ownership, and enables coordination through learned interaction across policy boundaries. We first systematically study where such interaction should occur, how peer information should be aggregated, and how the interaction should be learned, revealing complementary coordination functions spanning shared context, peer-specific latent information, and action-stage interaction. We then re-instantiate these functions in a structurally different VLA architecture, showing that the coordination principle is not tied to a particular tensor layout or backbone implementation. Finally, we compose teams of three and four arms and evaluate them in simulation and physical experiments. On all four physical tasks, Full interaction improves success over independently trained local policies by 16--40 percentage points. Together, these results suggest that explicit interaction among complete VLA policies offers a scalable path toward networked embodied intelligence.
comment: 17 pages, including references and appendices. Project website: https://nebulis-lab.com/VLA-Fabric/
★ What Makes a Good Symbol for Hybrid Discrete-Continuous Planning?
The physical world has continuous state and action spaces, yet robot planning often benefits from hybrid discrete-continuous formulations that explicitly expose combinatorial structure, such as contact modes or discrete primitive actions. Existing methods typically rely on human-defined symbol sets or symbols derived from language supervision, without asking what makes a symbol computationally useful. In this paper, we take an inference-algorithm--centric view to define usefulness: a good symbol should expose the discrete commitments that simplify solving the original planning problem. We first introduce the concept of solver-sufficient symbolization by studying finite-horizon piecewise-affine systems, for which solver sufficiency can be precisely characterized, and develop a counterexample-guided algorithm for symbol invention. We then extend this principle to invent symbols for optimization objectives parameterized by general neural networks as energy functions, showing that solver-sufficient symbols can nevertheless be induced from their continuous optimization landscapes. Experiments on simulated and real-robot tasks show that the resulting symbols reduce solving cost while preserving solution quality, while also improving interpretability and steerability. Furthermore, our theoretical and algorithmic insights help explain why certain human-defined symbols are computationally useful.
★ Robotic Spatiotemporal Editing: A Unified Interface for Controllable Action Generation via Masked Action Reconstruction
Generative policies can synthesize robot behaviors, yet controlling and editing them through diverse spatiotemporal references remains challenging. In the real world, human intent requires robot behaviors that satisfy constraints on where, when, and how actions are executed. Our key insight is that spatiotemporal control references, such as keypoints, flows, orientations, and keyposes, can be viewed as partial specifications of the same temporal Cartesian action space. We introduce the MAP (Masked Action Space) interface to unify these action references and formulate controllable manipulation as an action inpainting problem. Our Inpainting Policy (IP) reconstructs missing action dimensions of the end-effector, combining progress-guided alignment with multi-mask training to support unseen action-constraint combinations. We further adopt a training-free RePaint variant that iteratively inpaints the action during denoising to reduce control error. On ManiSkill and LIBERO-10, our policy achieves average task success rates of 92.67% and 64.67%, respectively.Real-world experiments demonstrate spatiotemporal control with average positional and rotational errors of 44.3 mm and 7°, respectively, showing that our policy can use diverse references to guide executable robot behaviors.
★ GR-VIP: Geometric representation conditioning for precise visuomotor policy
Achieving precise terminal position and heading alignment remains a significant challenge for learning-based mobile robot control. Recent vision-based learning methods have shown promise for precise target-relative control and generalization to unseen environments with pretrained visual encoders, but general-purpose visual representations are not specifically designed to capture the fine-grained spatial differences between current and goal views required for precise control, and real-world demonstrations combining millimeter-scale terminal accuracy with zero-shot transfer across environments remain limited. To address these challenges, we propose Geometric Representation Conditioning for Precise Visuomotor Policy (GR-VIP), a goal-conditioned visuomotor policy that leverages a visual encoder pretrained for relative camera pose estimation. Rather than using the final relative-pose estimate, GR-VIP conditions action prediction on the dense relational features produced before pose regression, preserving fine-grained cross-view information for learning corrective actions from demonstrations. We evaluate GR-VIP on a real-world docking task and demonstrate high docking success rates and millimeter-scale terminal accuracy in both the training and unseen environments, without additional training for transfer. Furthermore, supplementary experiments illustrate the policy's ability to align with non-docking targets and complete docking under a tested low-light condition.
comment: Submitted to IEEE Robotics and Automation Letters (RA-L). 8 pages, 5 figures. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible
★ Hardware-as-Regularization: Enhancing Data Consistency in Imitation Learning via an Adjustable Passive Compliant Leader Arm
Low-cost passive leader arms enable scalable data collection for imitation learning, but their lack of restoring forces leaves redundant joints unconstrained, yielding highly inconsistent joint-space trajectories. We address this with a low-cost passive joint module (an adjustable mechanical clutch and a 3D-printed spiral spring) that pulls each joint toward a user-defined neutral posture. This passive restoring force acts as a physical prior equivalent to joint-space $L_2$ regularization (``Hardware-as-Regularization''), improving kinematic consistency with shorter completion time than an unregularized arm and preserved movement smoothness. Experiments show substantial variance reduction over a naive passive arm, trajectory consistency on par with an active virtual spring of the same actuator class, and substantially higher downstream success rates (e.g., 73.3% vs. 36.7% on Block Stacking); a stiffness sweep shows consistency degrading monotonically as stiffness decreases, confirming that mechanical stiffness behaves like a regularization weight.
★ MambaAct: Efficient Mamba Action Experts for Flow-Matching Vision-Language-Action Models
Vision-language-action (VLA) models have shown strong promise for generalist robotic manipulation, but their inference cost remains a major bottleneck for real-time control. This limitation is especially pronounced in flow-matching VLAs such as $π_{0.5}$, where the action expert is evaluated repeatedly during iterative denoising. We introduce MambaAct, an efficient Mamba action expert for flow-matching VLA policies. MambaAct retains the pretrained visual-language prefix encoder of $π_{0.5}$, while replacing its transformer-based action expert with a compact Mamba-based action expert. Within this expert, Mamba serves as the action mixer that models dependencies across the noisy continuous-action sequence. To preserve visual-language conditioning without repeated token-level cross-attention, MambaAct uses a lightweight Context Compressor to map Vision-language model (VLM) prefix tokens into a compact conditioning signal that gates and modulates each Mamba action block. MambaAct uses a $10\times$ smaller action expert and reduces flow-matching denoising latency by $64.8\%$ on LIBERO and $58.8\%$ in real-world deployment, while improving mean LIBERO success by $1.08$ percentage points across three independent fine-tuning seeds and maintaining gains on LIBERO-Plus and real-world benchmarks. Our code and model weights are publicly available at https://github.com/giakhuyendihoc/MambaAct.
★ Ego2Contact: Predicting Touch from Egocentric Human Video for Contact-Rich Robot Manipulation
Contact locations and interaction forces are critical for contact-rich robot manipulation but are often ambiguous from vision alone. Measuring them on robots requires specialized tactile hardware and platform-specific data collection, which keeps robot tactile data scarce. Human egocentric video is far easier to scale but lacks touch. Recent predictors trained on egocentric recordings paired with tactile-glove measurements can predict this missing signal from video alone. These predictors, however, remain confined to human hands, which differ from robot hands in appearance and tactile layout. We present Ego2Contact, which learns a visual-tactile prior from human interactions and transfers it to robot hands. Its predictor fuses temporal scene context with hand-local visual features to infer bimanual tactile fields from the RGB video. Anatomical region correspondences align varying robot tactile layouts with the predictor's canonical grid, turning robot measurements into fine-tuning targets and predictions into policy inputs. Ego2Contact outperforms prior tactile predictors on human data, including unseen objects, and on robot data after adaptation. The adapted predictor provides tactile inputs for both simulated reinforcement learning and real-robot policy post-training. In simulation, policies using predicted touch outperform non-tactile baselines and recover part of the benefit of ground-truth tactile sensing. These results show that human-learned tactile priors can supply touch when direct tactile sensing is limited.
comment: 8 pages, 6 figures
★ SenseSwarm: Swarm Exploration via Neural Structure Estimation with Shared World-Anticipation for Rapid Mapping
A robot swarm covers a building faster with more robots, but the map is only as complete as the space the robots have physically visited. We target the amount of an unknown building a decentralized swarm knows at each instant, ahead of physical coverage. SenseSwarm serves a lightweight learned map predictor on the swarm's shared occupancy map and folds the predicted structure back into a stigmergic guidance field. Its 709K-parameter predictor is fine-tuned on the swarm's own multi-front observations, which we show is necessary: a single-robot predictor collapses on such views. We build a benchmark of twelve real building floors from HouseExpo, HKUST, and KTH, up to 24,000 square meters, and run 1,620 simulations across nine coordination methods, swarm sizes 32 to 512, and three seeds. On physical coverage the swarm is near parity with a greedy frontier planner, as expected. On map reconstruction, every non-predictive baseline needs 1.3 to 1.9 times longer to reach an 80 percent known map, and SenseSwarm holds an equal-time area lead of about 670 square meters, up to 1,359. Folding the prediction into the shared field matters: the same predictor applied per robot matches our speed but suffers a 6.9 times higher inter-robot contact rate at 512 robots. The prediction is computed once per shared map, so its cost stayed flat from 32 to 512 robots.
comment: 9 pages, 7 figures
★ Structured Perception Representations for Off-road Dynamics Learning
Autonomously navigating off-road terrain at high speed demands accurate control, which often relies on dynamics models that capture complex interactions between the robot and terrain. In off-road regimes, terrain interaction is a major component of the vehicle's dynamics. Prior work attempts to capture this effect by training learned models of the vehicle's dynamics, conditioned on its perception of the local environment. However, said conditioning is often overly simplistic, ignoring the spatial structure of the dynamics problem. Our approach, SpatialBEV, exploits this spatial structure. At each step, we query the bird's-eye-view (BEV) features around the predicted state, conditioning the next prediction on current geometric and semantic cues. We additionally present a benchmarking approach that mines a dataset's most difficult scenarios. On datasets collected across seasons, SpatialBEV reduces prediction error by 17.1%, compared to physics-based and learned baselines. We further evaluate on hardware on high-speed ($\geq$10 m/s) and geometrically challenging courses with a 25.3% reduction in maximum cross-track error. More details can be found at https://theairlab.org/spatial-bev-dynamics .
comment: Accepted at CoRL 2026. 24 pages, 14 figures
★ Sample-Efficient Reinforcement Learning for Policy Improvement via Flow-Reversed Action Anchors
Generalist robot policies offer strong behavioral priors, but achieving deployment-level reliability and throughput remains difficult. Reinforcement learning (RL) can improve these policies, yet its effectiveness is constrained by the cost of online interaction. Existing methods adapt frozen flow-matching policies by treating noise space as a continuous action space. We show that this behaviorally unstructured space makes exploration inefficient, often preventing these methods from extracting meaningful learning signals on challenging tasks. We introduce AnchorQ, which enables coarse-to-fine policy improvement through a structured, refinable latent action representation. At each decision step, AnchorQ uses flow reversal to map reference behaviors to state-conditioned latent anchors, samples local particles around those anchors, and applies value-guided sampling to explore promising refinements. Our analysis shows that anchors reduce identification complexity at the cost of support bias; particle refinement trades this bias against action-class complexity; and critic accuracy determines the reliability of KL-regularized, value-guided selection. Across simulated and real-world manipulation tasks, AnchorQ outperforms state-of-the-art post-training baselines in sample efficiency, delivering a 37% improvement in real-world success rate and 97% higher simulated task throughput under the same interaction budget. Additional visualizations are available on our website https://anchorq-rl.com/.
★ DefFlow: Goal-Conditioned Deformable Object Manipulation using 3D Point Flows
Manipulating deformable objects toward desired 3D configurations is a fundamental capability for robots operating in unstructured physical environments. However, as the range of possible initial and target configurations grows, goal-conditioned deformable manipulation requires a robot to infer increasingly diverse manipulation behaviors in addition to precise continuous actions. We introduce DefFlow, a framework for goal-conditioned manipulation of deformable objects toward a broad range of 3D configurations. DefFlow learns from demonstrations by jointly predicting the goal-conditioned 3D point flow of the robot and deformable object. We show that this formulation better captures both the manipulation decisions, such as when and where to grasp, release, or re-grasp, and the precise continuous motions required for goal reaching, rather than predicting only the robot flow. Across rope, granular-media, and cloth manipulation settings with diverse initial and target configurations, DefFlow substantially outperforms goal-conditioned action-prediction and robot-flow-only baselines on success rates and goal-distance metrics. We further demonstrate deployment of our framework on a real robot using real point-cloud observations and unseen goal configurations.
★ Simultaneous Force and Motion Control in Generalised Coordinates ICRA
Robot manipulators often come into contact with rigid objects when performing work. This imposes physical constraints on their motion. Simultaneous force & motion control was developed to regulate contact forces commensurate with these constraints, whilst projecting motion control in to the orthogonal, unconstrained directions. This is typically formulated in the task coordinates, i.e. operational space, then mapped to joint torques to command the robot. However, most constraints on the robot control, such as joint limits, collision avoidance, singularity avoidance, etc. are naturally expressed in generalised coordinates (joint space). Thus, formulating the control problem in task space makes it difficult to guarantee control feasibility. In this paper Gauss' principle of least constraint is used to solve the control as a quadratic programming (QP) problem in generalised coordinates. It is shown that the theoretical solution to this problem produces the correct constrained dynamics for accurate contact force control. Moreover, inequality constraints may be appended to ensure feasibility and safety. Simulation results were conducted in MuJoCo and compared to the operational space formulation. The proposed method is able to simultaneously maintain a desired contact force, minimise trajectory tracking error, and adhere to practical inequality constraints.
comment: 8 pages, 5 figures, submitted to IEEE International Conference on Robotics and Automation (ICRA) 2027
★ PASS-VLN: Passability-Aware Local-Command Vision-and-Language Navigation
Reliable vision-and-language navigation (VLN) requires not only understanding instructions but also determining whether and how a robot can physically execute them. However, existing VLN benchmarks typically assume executable instructions and focus on long-horizon tasks. This leaves body-dependent local execution, including passability and collision-free execution, largely unexamined. Evaluating these factors within long-horizon tasks would entangle local physical execution with task planning and long-term spatial memory, making navigation failures difficult to attribute. We therefore focus on grounding local navigation commands in executable motion under environmental and robot body constraints. We present PASS-VLN, a passability-aware local-command VLN benchmark and model. The benchmark holds the scene, instruction, start pose, and reference route fixed while varying robot body width, requiring target completion when the instructed passage is feasible and stopping before blockage otherwise. Our model encodes body width through obstacle inflation in a depth-derived local bird's-eye view, fuses geometry with visual and linguistic context, and predicts discrete latent action codes for continuous closed-loop execution. Experiments show that PASS-VLN improves collision-free success by approximately 12 percentage points over a strong VLN model fine-tuned on our benchmark, while adapting reach-versus-stop behavior to body constraints and generalizing to unseen body widths. When integrated with a high-level planner, PASS-VLN achieves higher success with fewer local commands, while real-world deployment further demonstrates its effectiveness as a low-level executor. The benchmark and model will be released.
comment: 9 pages, 4 figures
★ Pedicel-Targeted Strawberry Harvesting via Imitation Learning on a Low-Cost Teleoperated Robotic Arm IROS
Precise pedicel-targeted grasping in strawberry harvesting remains a critical bottleneck due to the pedicel's small diameter (1.3-3 mm), color similarity with surrounding foliage, and precise force control required for clean abscission zone detachment. As a market-grade crop, strawberry aesthetic value makes bruise-free pedicel detachment essential over direct fruit grasping. Classical approaches struggle to unify vision models, motion planning, and contact manipulation into a robust framework, particularly in occluded canopy environments. Therefore, this work presents a visuomotor imitation learning pipeline for pedicel-targeted strawberry harvesting that unifies perception, motion planning, and contact manipulation from collecting human teleoperation demonstrations, training an Action Chunking with Transformers (ACT) policy with a ResNet-50 visual backbone, and deploying the learned policy autonomously on a Jetson AGX Orin edge platform at 15 Hz. The trained policy achieved consistent loss convergence from 2.84 to 0.091 over 100K gradient steps on 130 demonstrations. We have also established a reproducible open-source pipeline with publicly released dataset and model weights for the agricultural robotics community. Full dataset is available at, https://huggingface.co/datasets/nitindominicrai/strawberry_pedicel_grasp_teleoperation and trained policy at, https://huggingface.co/nitindominicrai/act_strawberry_pedicel. To replicate this work, all code, configurations, and deployment instructions are publicly available at, https://github.com/nitin-dominic/autonomous_strawberry_pedicel_harvesting.
comment: 4 pages, 3 figures. This paper was peer-reviewed and accepted in the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), "From Innovation to Adoption: Hardware, AI, Policy, and Funding Challenges in Agricultural Robotics Workshop," Pittsburgh, Pennsylvania, 2026
★ Shape-Aware Neural Time Fields for Rigid-Body Motion and Manipulation Planning
Rigid-body motion planning is fundamental to robotic transport and manipulation, where both object position and orientation determine whether a body can move through cluttered environments. This coupling makes planning chal- lenging because translation and rotation can be constrained very differently by surrounding geometry. We propose Shape- Aware Neural Time Fields, a physics-informed neural motion planning framework that learns a reusable travel-time, or cost- to-go, representation for rigid-body motion while explicitly accounting for these distinct geometric restrictions. The learned field captures local motion difficulty in translation and rotation and is paired with a sampling-based planner that balances progress toward the goal with the cost of reaching nearby poses. We evaluate the framework across 2D and 3D rigid- body planning tasks and non-prehensile manipulation through object pushing in diverse environments. Results show substantial improvements in planning success and inference efficiency over physics-informed neural planning baselines, while remaining competitive with classical sampling-based planners at significantly lower planning times. Ablation studies further validate the contributions of both the learned representation and the proposed planning strategy. Finally, closed-loop pushing experiments under execution deviations and external disturbances demonstrate that fast replanning enables reactive and adaptive task execution.
♻ ★ Learning What to Say to Your VLA: Mostly Harmless Vision Language Action Model Steering
Vision-Language-Action (VLA) models provide a natural language interface to robot control, but the mapping from language to behavior is often brittle and unintuitive: semantically similar instructions can induce drastically different behaviors, while some capabilities may not be elicitable through prompting alone. As a result, both human instructions and zero-shot language models can fail to reliably steer VLAs toward successful task execution. In this work, we propose a framework that interactively searches for language sequences that improve closed-loop VLA task performance, distills these sequences into a test-time language feedback policy (LFP), and learns an improvement head that predicts when language steering will improve performance. We conformalize this improvement head to prevent harmful steering interventions, where the LFP decreases task performance relative to the original instruction on out-of-distribution scenarios. Crucially, our approach operates on arbitrary frozen pre-trained VLAs, requiring neither access to the original training distribution nor fine-tuning of the underlying model. On seen environments, our conformalized LFP improves base VLA performance by 24.7% in simulation and 65.0% in hardware. On visual and semantic perturbations with calibration samples, our conformalized LFP has strong harmlessness guarantees, and produces recovery behaviors not observed with open-loop prompting.
comment: CoRL 2026
♻ ★ HydroShear: Hydroelastic Shear Simulation for Tactile Sim-to-Real Reinforcement Learning
In this paper, we address the problem of tactile sim-to-real policy transfer for contact-rich tasks. Existing methods primarily focus on vision-based sensors and emphasize image rendering quality while providing overly simplistic models of force and shear. Consequently, these models exhibit a large sim-to-real gap for many dexterous tasks. Here, we present HydroShear, a non-holonomic hydroelastic tactile simulator that advances the state-of-the-art by modeling: a) stick-slip transitions, b) path-dependent force and shear build up, and c) full SE(3) object-sensor interactions. HydroShear extends hydroelastic contact models using Signed Distance Functions (SDFs) to track the displacements of the on-surface points of an indenter during physical interaction with the sensor membrane. Our approach generates physics-based, computationally efficient force fields from arbitrary watertight geometries while remaining agnostic to the underlying physics engine. In experiments with GelSight Minis, HydroShear more faithfully reproduces real tactile shear compared to existing methods. This fidelity enables zero-shot sim-to-real transfer of reinforcement learning policies across four tasks: peg insertion, bin packing, book shelving for insertion, and drawer pulling for fine gripper control under slip. Our method achieves a 93% average success rate, outperforming policies trained on tactile images (34%) and alternative shear simulation methods (58%-61%).
comment: Accepted to RSS 2026 | Project page: https://hydroshear.github.io
♻ ★ MANGO-Grasp: Mahalanobis Fields over Geometry-Oriented 3D Gaussians for Cross-Embodiment Dexterous Grasping
Cross-embodiment dexterous grasping aims to synthesize stable grasps across heterogeneous multi-fingered hands with little or no embodiment-specific tuning. Existing interaction-centric methods achieve promising results, but their object representations often underrepresent local surface geometry, hand descriptors do not explicitly capture both morphology and kinematics, and hand-object interaction modeling is typically isotropic. We propose MANGO-Grasp, an anisotropic interaction framework that represents objects as geometry-oriented 3D Gaussian primitives to better preserve local surface geometry, and encodes hands into pretrained descriptors that capture both morphology and kinematics. Hand-object interaction is modeled using anisotropic Mahalanobis fields that reflect the directional nature of contact and guide a shared optimization for grasp realization across embodiments. On the CMAP and MultiGripperGrasp benchmarks, MANGO-Grasp outperforms the strongest seen-hand baseline by up to 8.24 percentage points in simulation. It also transfers zero-shot to the unseen SharpaWave hand, improving over the strongest zero-shot baseline by up to 16.57 percentage points, and achieves 86% success in real-world experiments. The code and additional materials will be made available upon publication at https://connor-zh.github.io/MANGO-Grasp/.
♻ ★ OpenDexGrasp: Open-vocabulary Task-Oriented Dexterous Grasping
Dexterous grasp synthesis has advanced rapidly in generating stable and physically plausible hand poses, but real-world manipulation requires grasps that preserve the function implied by the task. We study open-vocabulary task-oriented dexterous grasp generation, where a robot must infer functional intent from free-form language, ground it in multi-view visual observations and object geometry, and generate an executable high-degree-of-freedom grasp. We present OpenDexGrasp, a unified data and generative modeling framework for this setting. OpenDexVerse provides dual-source supervision organized by the Coverage-to-Alignment (C2A) Recipe: OpenDex-Scale offers large-scale semantic and geometric coverage through automatic grasp synthesis and vision-language annotation, while OpenDex-Align supplies high-quality embodied alignment through human teleoperation and category-level transfer. OpenDexGrasp learns a shared perception-action latent representation that couples open-vocabulary vision-language context with dexterous action generation. Affordance grounding and grasp generation provide complementary supervision over this latent space, enabling direct generation of task-consistent dexterous grasps without a separate affordance-to-pose inference stage. Extensive simulation and real-robot experiments demonstrate improved functional alignment, physical feasibility, generalization to unseen categories, and real-world execution success. Additional details and videos are available at https://opendexgrasp.github.io/.
comment: Accepted at CoRL2026
♻ ★ DexBioLab: Benchmarking Dexterous Manipulation with Dynamic Experimental Phenomena in Realistic Laboratories
Automating biological laboratories requires robots to perform dexterous tool interactions under realistic visual conditions while handling dynamic experimental phenomena, presenting significant challenges for robotic manipulation. To enable systematic evaluation under these conditions, we introduce DexBioLab, a simulation benchmark and framework for dexterous manipulation in realistic biological laboratory environments. DexBioLab comprises 14 tasks across 4 robot embodiments, spanning functional tool use, precision manipulation, language-conditioned visual grounding, and instrument operation. We introduce controllable dynamic experimental phenomena, including flames, vapor, and bubbles, to capture the evolving visual conditions of laboratory procedures. To further enhance environmental realism, we develop a hybrid rendering pipeline that integrates interactive simulated foreground objects with Gaussian-splatting backgrounds reconstructed from 6 real laboratory scenes. For efficient large-scale data generation and evaluation, we further build a parallel simulation framework on top of mjlab, proposing Parallel Finite-State Machine (PFSM) and integrating batched rendering with decoupled trajectory replay for demonstration generation and model-independent closed-loop policy evaluation. Our evaluation reveals substantial variation in policy performance across tasks, exposing distinct strengths and weaknesses of current manipulation policies. Real-world experiments further demonstrate a reduced sim-to-real gap, supporting the validity of AutoBio+ for realistic manipulation evaluation. Supplementary materials are available at https://cumulusalpha.github.io/DexBioLabWebPage.
♻ ★ Metro-WM: Long-Horizon Latent Planning with Realisable Sub-Goals
Model-predictive control with Joint-Embedding Predictive Architectures (JEPAs) provides a strong zero-shot goal-reaching planner, but it is only effective over short planning horizons. Hierarchical extensions attempt to bridge this gap by learning a macro planner to predict intermediate latent sub-goals to guide the micro planner. In this work, we demonstrate that unconstrained latent sub-goal prediction is fundamentally flawed. A rigorous evaluation reveals that a leading state-of-the-art macro planner routinely emits physically unrealisable sub-goals. To resolve this, we introduce Metro-WM, a hierarchical framework that issues sub-goals by retrieving genuine states from prior experience rather than generating ungrounded latent vectors. Specifically, Metro-WM constructs a graph whose vertices are observed frames from offline expert demonstrations or random-action trajectories, allowing frames from different episodes to be connected and stitched into routes to the goal. Planning over the full graph also makes the system highly robust to execution errors: if the micro planner drifts off course, Metro-WM instantly finds a new optimal path from the current state. Our experiments show that Metro-WM achieves superior long-horizon success rates of up to 37.33 percentage points over the next best hierarchical approach while being up to 10.9 times faster, requiring both 13-56 times less offline compute and fewer tuned hyperparameters. Additional analysis reveals that Metro-WM finds shorter paths than the offline demonstrations, outperforms an oracle relying on the query's own demonstration, and maintains robust performance under extremely sparse dataset conditions.
♻ ★ OrbiSim: World Models as Differentiable Physics Engines for Embodied Intelligence
We present OrbiSim, a novel robotic simulation paradigm that redefines world models as a fully differentiable physics engine for embodied intelligence. Unlike prior world models that focus on unconstrained imagination in latent or visual domains, OrbiSim establishes a unified, physically-grounded pathway that bridges structured scene assets, neural dynamics, and downstream reinforcement learning. By enabling end-to-end differentiability throughout the entire simulation loop -- spanning from explicit state transitions to visual observation generation -- OrbiSim supports tasks traditionally intractable for classical simulators, such as differentiable contact modeling, gradient-based policy optimization under sparse rewards, and intuitive physical inference. Empirical results demonstrate that OrbiSim significantly outperforms state-of-the-art world models in both predictive fidelity and control performance. Furthermore, its consistent responsiveness to asset configurations and physical parameters suggests its potential as a differentiable tool for enhancing robot simulation and policy training.
comment: Project page: https://jjleejj85.github.io/projects/orbisim
♻ ★ Learning Cross-Coupled and Regime Dependent Dynamics for Aerial Manipulation
Accurate dynamics modeling of aerial manipulators is challenging because platform--manipulator coupling, history-dependent interactions, and changing payload or manipulator configurations produce cross-coupled, temporal, and nonstationary residual dynamics. We propose a structured encoder--decoder framework in which a nonlinear history encoder captures temporal and cross-variable dependencies, while a lightweight linear latent decoder supports efficient deployment-time adaptation. Its linear-in-parameter structure enables closed-form Bayesian updates with consistency-driven covariance inflation and integration with real-time model predictive control (MPC). Experiments on a real aerial-manipulation platform demonstrate improved residual-prediction accuracy, rapid adaptation following dynamics changes, and lower MPC tracking error under changing payload conditions. The results show that combining structured history-dependent representation learning with lightweight online adaptation provides an effective predictive model for aerial manipulation under coupled and nonstationary dynamics.
♻ ★ HyVGGT-VO: Hybrid Dense Visual Odometry With Classical VO and Feed-Forward Model
Dense visual odometry (VO), which provides pose estimation and dense 3D reconstruction, serves as the cornerstone for applications ranging from robotics to augmented reality. Recently, feed-forward models have demonstrated remarkable capabilities in dense mapping. However, when these models are used in dense visual SLAM systems, their keyframe-based processing often leads to sparse pose outputs, limiting continuous per-frame pose estimation. In contrast, traditional sparse methods provide high-frequency pose outputs but lack the capability for dense reconstruction. To address these limitations, we propose HyVGGT-VO, a novel framework that combines the computational efficiency of sparse VO with the dense reconstruction capabilities of feed-forward model. To the best of our knowledge, this is the first work to tightly couple a traditional VO frontend with VGGT. Specifically, we design an adaptive hybrid tracking frontend that dynamically switches between traditional optical flow and the VGGT model to ensure robustness. Furthermore, we introduce a hierarchical optimization framework that jointly refines VO poses and the scale of VGGT predictions to ensure global scale consistency. Under the odometry setting, compared with VGGT-SLAM 2.0, HyVGGT-VO reduces the average ATE by approximately 91% on EuRoC MAV and 20% on TUM RGB-D, and decreases the average relative translation and rotation errors by 46% and 92% on KITTI, respectively, while achieving 16.1 Hz pose output and 3.2 Hz dense-mapping updates with 6.5 GB of memory on an 8 GB laptop GPU.Our code is available at https://geneta2580.github.io/HyVGGT-VO.io.
♻ ★ EDAR: Learning Environment-Dependent Action Representations for Robotic Manipulation
Learning effective action representations is critical for robotic manipulation, where raw control trajectories are often noisy, redundant, and difficult to model directly. Existing methods mainly encode the structure of the action stream itself, treating the role of actions in the environment as implicit. Yet manipulation is about changing the world: the same action segment can induce different outcomes under different scene contexts, making action semantics inherently environment-dependent. We propose EDAR, an Environment-Dependent Action Representation that grounds action tokens in both executable control structure and expected visual consequences. By coupling motor commands with their environment-conditioned effects, EDAR encourages the learned action space to capture interaction semantics rather than merely command-level patterns. Experiments on simulated and real-robot manipulation benchmarks demonstrate that EDAR improves downstream policy learning, especially in long-horizon manipulation. These results highlight the importance of grounding action representations in executable control structure and environment-conditioned visual change.
♻ ★ A Tendon-Driven Robotic Jellyfish with Constrained Soft Actuation and Depth Control via Reinforcement Learning
Jellyfish-inspired robots offer a compliant and efficient approach to underwater locomotion, but achieving large deformation together with repeatable actuation and closed-loop control remains challenging. In this work, we present a tendon-driven robotic jellyfish with constrained soft actuation. Each actuator combines a flexible substrate with discrete constraints, enabling bending up to \(150^\circ\) with an approximately linear tendon displacement-bending relationship. Eight actuators driven by four servos allow the robot to perform stable swimming, attitude adjustment, and self-righting. Based on the linear actuation, a reinforcement-learning controller is further developed, enabling closed-loop depth regulation in both simulation and physical experiments. These results show that mechanical constraints can improve the controllability of soft actuation while preserving compliant jellyfish-like motion, providing a route toward manoeuvrable and autonomous jellyfish robots.
comment: 8 pages, 8 figures. Corrected a typo in an author name in the arXiv metadata. No changes to the manuscript
♻ ★ Confidence Calibration in Vision-Language-Action Models
Trustworthy robot behavior requires not only high levels of task success but also that the robot can reliably quantify how likely it is to succeed. To this end, we present a first-of-its-kind study of confidence calibration in vision-language-action (VLA) foundation models, which map visual observations and natural language instructions to low-level robot motor commands. We establish a confidence estimation baseline for VLAs, examine how task success relates to calibration error and how calibration evolves over time, and introduce two lightweight techniques to remedy the miscalibration we observe: prompt ensembles and action-wise Platt scaling. Our aim in this study is to begin to develop the tools and conceptual understanding necessary to render VLAs trustworthy via reliable uncertainty quantification.
comment: 34 pages, 7 tables, 22 figures; published in TMLR, final camera ready
♻ ★ CorrelationFlow: A Training-Free Geometric Approach for LiDAR Scene Flow Estimation
LiDAR scene flow is the task of estimating point-wise motion from sequential point clouds. It is essential for autonomous driving, as it provides critical information about moving objects and their trajectories. In recent years, unsupervised methods have gained significant traction by eliminating the need for expensive point-wise annotations. However, despite incremental performance improvements, these methods uniformly rely on a single paradigm: training feed-forward networks through optimizing a loss function based on the Chamfer distance, a framework that inherently limits their progression. This paper presents CorrelationFlow, a new paradigm that reformulates LiDAR scene flow as a correlation maximization problem between Bird's-Eye View (BEV) images. Instead of using learnable parameters, our approach employs a chain of image processing operations to solve this problem. The learning-free nature of CorrelationFlow enables strong generalization across diverse domains. This capability is demonstrated by the second-place ranking in the unsupervised track of the Argoverse 2 Scene Flow Challenge at CVPR 2026, which featured a test set spanning five datasets that cover varied vehicle platforms and environments. Our results show that this classical, model-based approach can match or even surpass the performance of learning-based methods, thereby opening a new direction for unsupervised LiDAR scene flow research. The source code will be released.
♻ ★ GroundAnything: Reconciling Parallel Decoding with Precise Visual Grounding at Flash Speed
Autoregressive (AR) grounding models serialize spatial predictions, introducing sequential latency and imposing a causal order on output tokens. We view grounding as visual evidence extraction: objects, locations, and spatial relations are jointly constrained by the image and query, yet their dependencies do not imply an intrinsic left-to-right generation order. This distinction makes bidirectional diffusion a natural fit, allowing spatial hypotheses to emerge in parallel and be jointly refined through iterative denoising. We introduce GroundAnything, a 4B-parameter grounding foundation model that reconciles fast parallel decoding with precise localization through blockwise denoising. Training combines grounding pretraining from public datasets and dedicated data engines, direct AR-to-diffusion conversion with joint AR and diffusion objectives, supervised fine-tuning, and GRPO-based reinforcement post-training. Across 30 grounding benchmarks, our autoregressive variant, GroundAnything-VLM, establishes a new overall state of the art among similarly sized models at 72.42%, remaining competitive with GPT-6 Astra (71.35%). With entropy-guided decoding, GroundAnything also surpasses the prior state of the art at this scale, averaging 61.75% versus 53.32% for the fast MTP-based LocateAnything model. We further explore decoding strategies, showing that an optional self-speculative mode achieves a $4.51\times$ speedup over the AR counterpart with a 0.74 percentage-point drop in COCO F1mIoU. Infrastructure experiments show that progressive inference optimizations translate parallel decoding into practical speedups. These support efficient visual grounding in latency-sensitive real-world systems.
comment: 61 pages, including supplementary material. Project page: https://groundingpi.github.io/groundanything/ Code: [https://github.com/groundingpi/GroundAnything](https://github.com/groundingpi/GroundAnything) Model: https://huggingface.co/GroundingPI/GroundAnything, https://huggingface.co/GroundingPI/GroundAnything-VLM
♻ ★ GroundingPI: A Grounding Foundation Model towards Physical Intelligence with Visual Primitives
Precise grounding matters. It specifies which object is the target and where that object is, even in clutter and for tiny objects, and it has to be fast enough for closed-loop control. Yet vision-language-action (VLA) and world-action models (WAMs) take perception from general-purpose vision-language and video-generation backbones, which still fail in these settings. We introduce GroundingPI, a 4B grounding foundation model that generates points and boxes as quantized coordinates in a shared vocabulary. Training combines multimodal and spatial pretraining, supervised fine-tuning, and reinforcement learning with GRPO, using supervision from public datasets and dedicated data engines. Against 44 baselines across 34 grounding benchmarks spanning 11 perceptual capabilities, GroundingPI establishes a new state of the art, averaging 73.68%, above the larger GPT-6 Astra (71.54%). As a downstream visual backbone, GroundingPI improves performance on robotic manipulation and autonomous driving. On RoboTwin 2.0, it outperforms every mainstream backbone we evaluate in all four out-of-distribution settings, by up to 24.8% relative to the strongest backbone. On RoboCasa-GR1, GroundingPI trained with 50% of the demonstrations outperforms those baselines trained with 75%. On nuScenes, used as the visual backbone, GroundingPI attains an average open-loop L2 error of 0.296 m. We systematically analyze GroundingPI's pretraining in scale and data composition. Downstream autonomous driving and robotic manipulation improve as the pretraining is scaled. Analyzing the data recipe across these 11 perceptual capabilities shows dense grounding's substantial benefits for both, and OCR's potential as a catalyst for perceptual learning. These results support grounding as a perceptual foundation, and dedicated perceptual pretraining as a promising direction for foundation models of physical intelligence.
comment: 64 pages, including supplementary material. Project page: https://groundingpi.github.io/ Code: https://github.com/groundingpi/GroundingPI Model: https://huggingface.co/GroundingPI/GroundingPI
♻ ★ PRISM: Polynomial Representations for Interaction-Structured Motor Control
Robot policies map observations to actions, typically using multilayer perceptrons (MLPs). Yet robot dynamics and control objectives involve multiplicative interactions among observable physical variables. MLPs can approximate these interactions but must learn them implicitly, at a representational cost. We introduce PRISM (Polynomial Representations for Interaction Structured Motor Control), a physics-motivated yet data-driven representation that learns multiplicative interactions by recursively adding scaled bilinear features, without enumerating all polynomial terms. Replacing only the MLP's first affine layer, PRISM is plug-and-play for reinforcement and imitation learning, requiring no changes to training objectives, sensing, or controllers. Across humanoid locomotion and contact-rich manipulation, PRISM improves survival from 51.0\% to 92.5\%, outperforming even larger MLPs. More strikingly, its performance advantage persists under out-of-distribution changes in friction, robot mass, and payload, without retraining. PRISM also exhibits sensorless compliance-like behavior, increases failure-recovery success from 40.9\% to 52.3\% without explicit admittance control, improves real-robot manipulation, and learns features that better predict physical quantities. These results make PRISM a compelling alternative to standard MLPs for robot policy learning. The project page is available at https://lsh3163.github.io/prism/.
♻ ★ Multi-Agent Reinforcement Learning-Based UAV Pathfinding for Obstacle Avoidance in Stochastic Environment
Traditional methods plan feasible paths for multiple agents in the stochastic environment. However, the methods' iterations with the changes in the environment result in computation complexities, especially for the decentralized agents without a centralized planner. Although reinforcement learning provides a plausible solution because of the generalization for different environments, it struggles with enormous agent-environment interactions in training. Here, we propose a novel centralized training with decentralized execution method based on multi-agent reinforcement learning, which is improved based on the idea of model predictive control. In our approach, agents communicate only with the centralized planner to make decentralized decisions online in the stochastic environment. Furthermore, considering the communication constraint with the centralized planner, each agent plans feasible paths through the extended observation, which combines information on neighboring agents based on the distance-weighted mean field approach. Inspired by the rolling optimization approach of model predictive control, we conduct multi-step value convergence in multi-agent reinforcement learning to enhance the training efficiency, which reduces the expensive interactions in convergence. Experiment results in both comparison, ablation, and real-robot studies validate the effectiveness and generalization performance of our method.
♻ ★ Enabling Tidying Tasks in Robotic Vacuum Cleaners via a Geometry-Induced Self-Locking Deployable Linkage
Robotic vacuum cleaners must extend their functionality within tight limits on size, cost, and durability. Deployable linkages with few actuators meet these limits, but their backdrivability makes it difficult to hold a load-bearing posture without continuous motor torque. We introduce a two-stage deployable linkage that locks through its own geometry. At full deployment, an internal three-link subchain approaches a collinear, near-singular configuration in which the input-output transmission ratio tends to zero; plate loads therefore leave only a small residual back-driving torque, which drive-train friction and a lightweight magnetic detent absorb. A single actuator performs deployment and locking, a passive chassis-mounted trigger releases the lock for retraction, and a second actuator adjusts only the deployment height. We integrate the mechanism into a commercial robotic vacuum cleaner and validate it through a 100-cycle repeatability test, a quasi-static load test in which the locked plate held 50 N with the motor de-energized, and 140 pushing-type tidying trials on seven household objects against a commercial arm-equipped vacuum robot. We derive the analytical holding condition and discuss its limits as an intermediate design point between passive compliance and articulated manipulation.
comment: 8 pages, 6 figures, 1 table. v2: corrected author name, added minor revisions
♻ ★ Analytic Robot Self-Model: Learning Screw-Axis Kinematics for Rendering and Model-Based Control
Visual self-modelling lets robots learn body models from observations, but such models usually encode kinematics implicitly rather than exposing metric joint parameters for computation and control. Moreover, a good fit to images does not by itself establish accurate kinematics. Here we present an explicit analytic robot self-model that requires neither deformation networks nor URDF/CAD templates. Multi-view images and joint readings establish Gaussian appearance and provide motion evidence for rigid link assignments, whereas sparse end-effector poses independently identify metric screw axes. The resulting closed-form forward kinematics and analytic Jacobian can be evaluated beyond the fitting ranges. We validate the model on a simulated seven-axis arm, a simulated twelve-joint quadruped and a real six-axis arm. In simulation, it achieves millimetre-level reaching and pick-and-place, as well as closed-loop quadruped walking. On the real arm, it renders held-out configurations, predicts end-effector positions with a median error of 6.4 mm, compared with 11.7 mm for a calibrated nominal model, and supports reaching with external marker feedback. A controlled comparison shows that image fit alone is not enough: two Panda models differing only in their joint axes are 0.55 dB apart in PSNR, yet their end-effector prediction errors differ 13.7-fold.
comment: 26 pages, 6 figures, supplementary information included
♻ ★ NativeMEM: Native Memory Compression for Long-Horizon Robotic Manipulation
How can pretrained Vision-Language-Action (VLA) models retain long-horizon visual histories with high-frequency updates without sacrificing efficiency? Existing approaches rely on external memory management, which restrains either the memory horizon or the reactiveness of pretrained policies. To this end, we present NativeMEM, a VLA policy that features long-term and real-time updated memory. At its core is an efficient memory encoding scheme, Native Memory Compression, which repurposes the VLA's own vision encoder to compress each historical frame from each camera view into a single token. Appended to the input sequence, these memory tokens enable the pretrained VLA to attend over long-term history with negligible latency overhead, requiring neither an external planner nor a freshly initialized memory module. To align the memory tokens with the pretrained policy, we first develop a generic memory tokenizer under the supervision of a frozen VLA on memory-demanding data, and then unfreeze the VLA for task-specific fine-tuning. NativeMEM consistently outperforms prior methods, boosting success rates from 32.4% to 84.0% in simulation and up to 98.7% on real robots, while maintaining low inference latency and GPU memory usage. Notably, NativeMEM exhibits high data efficiency by achieving competitive results with prior arts using only 20% of the training data.
comment: Accepted by CoRL 2026. Project Page: https://opendrivelab.com/NativeMEM
♻ ★ All Roads Lead to Rome: Flow-driven Multi-Anchor Exploration for Open-Environment Active 3D Mapping NeurIPS 2026
To advance the development of embodied intelligence, Open-Environment Active 3D Mapping has attracted increasing attention, aiming to perform a long-horizon and shortest trajectory exploration for reconstructing unseen scenarios. Since only limited information about unseen environments is available, methods built on the closed-set assumption, i.e., assuming that the test environments are similar to those seen during training, cannot generalize satisfactorily. In existing active mapping methods, long-horizon exploration is often guided by predicting a coarse long-range goal and then converting it into an executable path. However, this stage is usually formulated as single-point prediction. Under partial observability, the same local observation may correspond to multiple plausible exploration directions, making such deterministic prediction prone to brittle decisions and degraded performance in unseen scenarios. Our experiments further verify that this is a key factor underlying their weak generalization. To address this issue, we reformulate long-horizon target prediction as conditional multimodal anchor generation using Conditional Flow Matching. Instead of predicting a single goal, our method learns a conditional distribution over coarse exploration anchors from the current mapping state. These anchors are first converted into executable candidate paths through obstacle-aware planning. We then apply exploration-mode clustering to compress geometrically similar trajectories and reduce candidate redundancy. Finally, a hierarchical selection module selects the most promising mode and reranks paths within it to produce the final executable trajectory. Experiments show that our method improves generalization and reconstruction efficiency in open environments.
comment: Accepted by NeurIPS 2026 (spotlight)
♻ ★ Perceptive Behavior Foundation Model: Adapting Human Motion Priors to Robot-Centric Terrain
Humanoid behavior foundation models aim to acquire reusable whole-body control policies from broad human motion priors, enabling a single controller to produce diverse and expressive behaviors. However, existing motion-centric foundation policies largely assume that the reference motion is already physically compatible with the robot's surroundings. This assumption breaks when the demonstrator, operator, and robot inhabit different environments: a human motion may specify the intended behavior, but not the footholds, clearance, body height, or contact timing required by the robot's local terrain. We introduce \emph{Perceptive Behavior Foundation Model} (Perceptive BFM), a terrain-aware humanoid control framework that grounds human motion priors in robot-centric perception. The model preserves raw kinematic motion references as the behavioral interface, while using local terrain observations to adapt contacts, posture, and timing. To provide scalable terrain supervision, we develop \emph{terrain-conformal reference synthesis} (TCRS), which converts locomotion-oriented human motion clips into terrain-consistent references through contact-aware foothold construction, foot-geometry-aware swing optimization, support-aware root reconstruction, collision repair, and multi-point inverse kinematics. We then train a blind adapted-reference teacher and transfer its terrain-conformal behavior to a deployed raw-reference student through target-frame action alignment. The student is an identity-gated Transformer tracker whose terrain features enter through residual pathways initialized to preserve the motion-tracking prior and trained to produce local corrections only when needed.
♻ ★ Towards Unified World Models for Visual Navigation via Memory-Augmented Planning and Foresight ECCV 2026
Enabling embodied agents to imagine future states is essential for robust and generalizable visual navigation. Yet, state-of-the-art systems typically rely on modular designs that decouple navigation planning from visual world modeling, which often induces state-action misalignment and weak adaptability in novel or dynamic scenarios. We propose UniWM, a unified, memory-augmented world model that integrates egocentric visual foresight and planning within a single multimodal autoregressive backbone. UniWM explicitly grounds action selection in visually imagined outcomes, tightly aligning prediction with control. Meanwhile, a hierarchical memory mechanism fuses short-term perceptual cues with longer-term trajectory context, supporting stable and coherent reasoning over extended horizons. Extensive experiments on four challenging benchmarks (Go Stanford, ReCon, SCAND, HuRoN) and the 1X Humanoid Dataset show that UniWM improves navigation success rates by up to 30%, substantially reduces trajectory errors against strong baselines, generalizes zero-shot to the unseen TartanDrive dataset, and scales naturally to high-dimensional humanoid navigation. These results position UniWM as a principled step toward unified, imagination-driven embodied navigation. The code and models are available at https://github.com/UWMILab/UniWM.
comment: Accepted to ECCV 2026. 22 pages, 12 figures, code: https://github.com/UWMILab/UniWM
♻ ★ RealtimeWAM: One-Step Asynchronous World Action Models
World Action Models (WAMs) incorporate visual representations from video generation backbones to guide action prediction. Recent efficient WAMs adopt Mixture-of-Transformers (MoT) architectures and compute video representations once for reuse by the action expert. However, intra-expert iteration (\ie, multi-step action denoising) and inter-expert waiting (\ie, sequential execution of the video and action experts) still limit inference efficiency. To this end, we present RealtimeWAM, an extremely efficient WAM variant with one-step action generation and asynchronous inference, addressing these two bottlenecks. To reduce intra-expert iteration, we propose Teacher-Anchored Consistency Distillation (TACD) to address a local-global error gap: low local consistency error alone does not guarantee accurate final actions. TACD supplements local consistency with explicit supervision from the frozen teacher's multi-step rollout endpoint, enabling accurate one-step action generation. Additionally, we propose Cross-Expert Wavefront Pipelining (CEWP) to eliminate unnecessary expert-level waiting. It overlaps the two experts through block-wise sharing of the video KV cache, synchronizing only immediately before the corresponding action attention consumes it. Extensive experiments across diverse benchmarks (\eg, LIBERO, LIBERO-Plus and RoboTwin) and model variants (\eg, Fast-WAM and Faster-WAM) demonstrate the superiority of RealtimeWAM. Notably, RealtimeWAM maintains near-lossless performance (\ie, $<1\%$ drop) across these benchmarks while delivering significant end-to-end speedup (\eg, $\sim25\times$ on H100). Our code and checkpoints are available via this \href{https://github.com/ModelTC/LightX2V/tree/main/scripts/realtimewam}{link}.
comment: The code and checkpoints are available at https://github.com/ModelTC/LightX2V/tree/main/scripts/realtimewam
♻ ★ OxyGen: Unified KV Cache Management for VLA Inference under Multi-Task Parallelism NeurIPS 2026
Embodied AI agents increasingly require parallel execution of multiple tasks, such as manipulation, conversation, and memory construction, from shared observations under distinct time constraints. Recent Mixture-of-Transformers (MoT) Vision-Language-Action Models (VLAs) support such heterogeneous outputs, yet existing inference systems incur redundant computation and resource contention when serving concurrent tasks on-device. We identify isolated KV cache management as the root cause. To address this, we propose unified KV cache management, an inference design that treats the KV cache as a first-class shared resource across tasks and control cycles. This abstraction enables two key optimizations: cross-task KV sharing eliminates redundant prefill of shared observations, while cross-frame continuous batching batches unfinished language requests from different frames and advances them within a per-frame decoding budget. We implement this design in OxyGen and evaluate $π_{0.5}$ on NVIDIA GeForce RTX 4090 and Jetson AGX Thor. OxyGen achieves up to 3.7$\times$ speedup over isolated execution, delivering over 200 tokens/s language throughput and 70 Hz action frequency simultaneously on RTX 4090. We also validate the design on Xiaomi-Robotics-0 and StarVLA across different backbones and expert configurations. We further demonstrate the inference gains on a humanoid robot with on-board Jetson AGX Thor.
comment: NeurIPS 2026
♻ ★ Cyclops: LiDAR as a Camera That Dreams in Color
Conventionally, robotic perception relies heavily on cameras due to the rich semantic texture they provide. However, their performance degrades significantly in low-light or high-dynamic-range environments. Conversely, while Light Detection and Ranging (LiDAR) captures illumination-invariant geometric and intensity properties, the resulting data are typically single-channel and sparse, creating a significant modality gap when applying vision models pre-trained on RGB datasets. In this paper, we propose Cyclops, a framework that translates sparse Non-Repetitive Scanning LiDAR (NRS-LiDAR) intensity into RGB video, enabling camera-free inference for all-day perception tasks. Our approach first converts sparse LiDAR intensity projections into dense representations via a frozen pre-trained densification module, serving as a geometrically rich source condition. The dense intensity latent is then transported toward the target RGB distribution through Latent Bridge Matching (LBM) with a learned velocity field in a few ODE integration steps. To mitigate inter-frame flickering, we inject prior-frame context via temporal attention layers and further formulate the velocity field as a policy optimized by a differentiable terminal reward that encourages terminal fidelity through backpropagation along the ODE trajectory. Extensive experiments demonstrate that the synthesized RGB, including those generated under near-dark conditions, enable standard RGB-based perception models to substantially outperform both LiDAR baselines and conventional cameras on semantic segmentation, lane detection, and point cloud colorization across diverse lighting conditions.
comment: 10th Conference on Robot Learning (CoRL 2026). Project: https://github.com/IMRL/Cyclops
♻ ★ Schrödinger's Navigator: Imagining an Ensemble of Futures for Zero-Shot Object Navigation
Zero-shot object navigation (ZSON) requires robots to find target objects in unseen environments without task-specific fine-tuning or pre-built maps, a key capability for general-purpose service robots. Yet methods that perform well in simulation often degrade in cluttered real-world scenes with severe occlusion and latent hazards, where large unseen regions make single-scene inference brittle and unsafe. We propose Schrödinger's Navigator, a belief-aware framework that reasons at inference time over multiple trajectory-conditioned imagined 3D futures. Given candidate paths, a trajectory-conditioned 3D world model predicts hypothetical observations and maintains a superposition of plausible scene realizations rather than committing to one map. An adaptive occluder-aware sampler directs imagination to uncertainty-critical regions, while a Future-Aware Value Map (FAVM) aggregates imagined futures for robust, proactive action selection. Experiments in simulation and on a physical Go2 quadruped show that Schrödinger's Navigator outperforms strong ZSON baselines, improving hidden-target discovery and risk-aware waypoint selection in occlusion-heavy navigation scenarios. These results highlight imagined 3D futures as a scalable and generalizable strategy for zero-shot navigation in uncertain real-world environments.
comment: Accepted to the Conference on Robot Learning (CoRL) 2026
♻ ★ ADEPT: Accelerating Dexterity via Pre-Training and Post-Training using Reinforcement Learning
We introduce Accelerating Dexterity via Pre-Training (ADEPT), a large-scale reinforcement learning (RL) framework for learning sim-to-real transferable dexterity across high degree-of-freedom (DoF) robot embodiments that can solve long-horizon tasks directly from raw visuo-tactile perception. ADEPT pretrains a dexterous policy on a generic object reposing task, then post-trains downstream policies with this pretrained behavior as a prior. ADEPT enables learning new behaviors that are otherwise difficult to discover from scratch on multi-fingered robots and avoids learning the same set of skills over again for every new downstream task. The pretrained policy zero-shots the reposing phase of downstream tasks, but naïve RL fine-tuning rapidly degrades this capability during transfer. We address this with a stable post-training recipe combining behavior-cloning distillation, critic warm-up, and conservative on-policy updates. To safely exploit the full kinematic dexterity, we introduce a joint-space Geometric Fabric that mediates between the RL policy and the robot. We distill post-trained teachers into perceptive students that zero-shot sim-to-real transfer on two embodiments: a 23 DoF Kuka-Allegro with two RGB cameras, and a 29 DoF Flexiv-Sharpa with two RGB cameras and five vision-based tactile sensors, and can solve long-horizon tasks from challenging initial states with dexterity at human-level speed.
comment: Accepted at CoRL 2026 | Project page: https://adept-dexterity.github.io/
♻ ★ Adapting Generalist Vehicle Models for High-Speed MPC Across Terrains
High-speed off-road autonomy requires precise closed-loop control for a target vehicle while remaining robust across changing terrains. Recent forward kinodynamic (FKD) prediction foundation models suggest a promising path, starting from a generalist model and specializing it to the target platform. However, effective specialization remains challenging, as it often requires substantial real-world data, and models adapted to one setting can still overfit to specific terrains or driving regimes. We present OptCar (Optimized Car), a recipe for bridging the gap from generalist to specialist FKD models that preserves cross-terrain generalization while optimizing performance for a specific vehicle. OptCar introduces a transformer FKD architecture that uses FiLM to condition multi-step predictions on a single dynamics context token summarizing recent state-action history. It then specializes the generalist model using limited real-world data and targeted synthetic rollouts from environment-specific system identification. In closed-loop model predictive control (MPC) experiments across three terrains and an out-of-distribution cart-pulling task, the largest gains appear at 6 m/s, the highest speed evaluated and the regime in which slip dominates tracking error. On vegetation + dirt, the most slip-diverse terrain, OptCar reduces 6 m/s trajectory tracking error by roughly 55% relative to AnyCar fine-tuned on real data alone, and remains the most accurate even when an unseen cart payload changes the dynamics. With 5 minutes of real data per terrain, OptCar is competitive on road with a specialist trained on 30 minutes of road data and outperforms it when the terrain changes. Project Page : https://amrl.cs.utexas.edu/optcar/
comment: https://amrl.cs.utexas.edu/optcar/ Selected at CORL 2026
♻ ★ Learning Asynchronous Upper-body Task-space Trajectory Tracking Policy for Humanoid Robots
High-level humanoid planners often output sparse task-space, low-rate trajectories, whereas whole-body controllers run at high frequency. This creates temporal asynchrony between the planning and execution, and structural incompleteness for full-body control. We propose an asynchronous upper body task-space tracking framework for humanoids. A student policy is initialized by teacher-student distillation, conditioned on the full cached future trajectory and an execution-time index, and trained with a sliding-window global reward to reduce frame drift without explicit frame estimation. For task-specific post-training, an MPC module completes sparse references into floating-base and upper-body guidance, while action- and FK level self-guidance constrain policy drift. Simulation and Unitree G1 hardware experiments show improved tracking under low update rates, stronger performance than synchronous and decoupled baselines, and safer adaptation to out-of-distribution motions.
comment: 10 pages, 8 figures
♻ ★ RoboHarness: A Simple Harness Outperforms VLA and World Action Models
The rise of large language models (LLMs) has opened a path toward generalist embodied AI, raising the question of how LLMs can act in the physical world. One straightforward approach is training the model to generate actions: collect robot-action data and video demonstration, refine LLM architecture with an action head, and train a vision-language-action (VLA) model at scale. However, we argue that coding agents have the potential to realize generalist embodied manipulation. All they need is a simple yet effective robot harness. We present RoboHarness: a robotic harness that gives LLM agents a visual-geometric control panel so that they can directly understand and invoke embodied tasks. With RoboHarness, Qwen3.8-Flash-Next outperforms state-of-the-art VLA models on BEHAVIOR Challenge 2025 tasks. The implementation is open source.
comment: 18 pages, including references and appendices
♻ ★ Demystifying When and Why VLAs Fail in Contact-Rich Tasks and How to Fix Them
We address the problem of understanding when and why Vision-Language-Action models struggle with contact-rich manipulation tasks that require precise physical interaction. Prior work has primarily focused on addressing contact failures through force-augmented architectures and training-time regularizers, yet the factors contributing to these failures remain underexplored. We identify two distinct failure modes underlying this gap. Precision failures occur when policies struggle to produce the small, accurate corrective actions required during contact and force failures occur when policies fail to respond to contact forces. To address these failures, we propose FACT. During post-training, FACT increases low noise sampling to improve fine action prediction and incorporates structured force conditioning to guide action generation using contact state and force history. FACT achieves an average success rate of 66% across five contact-rich tasks against 41% for the best prior baseline, in an evaluation spanning almost 2,500 real-world rollouts.
comment: 17 pages, Spotlight at CoRL 2026
♻ ★ NEXUS: Learning Safety Constraints and Action Models for Safe and Robust Embodied Planning
Large language models (LLMs) enable embodied agents to plan from naturallanguage instructions, yet their actions may be physically infeasible or hazardous. Existing neuro-symbolic learners primarily refine execution models, while formal safety methods acquire or enforce safety constraints. This separation limits the reuse of both execution corrections and safety knowledge across tasks. We present NEXUS, a modular neuro-symbolic framework that jointly maintains a PDDL domain model and an LTL safety specification. Execution failures are used to repair action schemas, whereas detected hazards are converted into reusable LTL constraints through hierarchical semantic and translation sampling with equivalence consensus. Accumulated constraints are enforced through LTL progression during symbolic search; comparing constrained and unconstrained search outcomes distinguishes physical unreachability from safety-induced refusal. Experiments on SafeAgentBench evaluation set show that NEXUS achieves the best overall trade-off between safetask completion and unsafe-task refusal, while maintaining low violation rates under jailbreak attacks. Ablations confirm the complementary benefits of PDDLdomain refinement and LTL-constraint accumulation.
♻ ★ FELT: Generating Tactile Signals from Vision for Visuo-Tactile Manipulation
Tactile sensing can improve performance in contact-rich robot manipulation. However, learning policies that use real tactile feedback typically requires demonstrations collected with tactile-equipped hardware. We present Feature-Extracted Latent Tactile (FELT), a framework that uses existing paired visuo-tactile data to learn contact-aware representations from RGB observations. FELT provides these representations to downstream policies through two interfaces: generated tactile images and contact-aware features. FELT combines a frozen visual encoder with separate query decoders for the left and right finger, reflecting the spatial organization of dual-finger tactile arrays. The two branches exchange information through gated connections, and a convolutional readout produces spatial contact representations for the downstream interfaces. For downstream manipulation, the image interface decodes these spatial representations into tactile images, allowing policies trained with either real or generated tactile observations to use generated images at deployment. Alternatively, the feature interface feeds these representations directly to the policy during both training and deployment, without real tactile observations. Experiments on four contact-rich real-robot tasks show that both interfaces achieve higher success rates than vision-only policies. Directly incorporating learned contact-aware features into the policy improves success by 15-22.5 percentage points across the four tasks.
comment: 16 pages, including supplementary material
♻ ★ When Does Corruption Break the Plan? A Degradation-Tolerance Benchmark for Camera-Only End-to-End Driving WACV 2027
Camera-only end-to-end (E2E) driving models are nearing deployment, where the camera stream is degraded by blur, noise, low light, weather, frame loss, and memory faults. How much a policy tolerates before its driving breaks is unclear. Corruption-robustness benchmarks target detection or bird's-eye-view perception, not the planning output that drives the car. We present DriveDegrade, a benchmark for image-degradation tolerance in camera-only E2E driving. Sixteen corruption families at five severities are injected on the fly inside the image loader, one operator reaching fifteen policies, and we evaluate open-loop planning on nuScenes and NAVSIM plus a CARLA closed-loop anchor. First, mild degradation barely affects planning, and the families that break it have a clear threshold at mid severity. Second, fragility is corruption-dependent: blur, JPEG, and raindrop damage planning most, while weather and bit error are tolerated far into the range. Third, a flat curve is ambiguous, so we separate corruptions that degrade the image from those that remove it. A planner that reads its camera must lose accuracy when information is deleted, whatever it does under quality loss. On these two axes the planners separate sharply, quantifying the ego-status shortcut without mistaking indifference for robustness. A released vision-language-action planner is flat on both axes, and blinding all six of its cameras costs it only 11.5 percent.
comment: Accepted to WACV 2027. 17 pages, including supplementary material
♻ ★ SPACE: Swarm Pheromone Fields for Adaptive Collision-Aware Exploration
Massive robot swarms can explore unknown environments quickly, but adding robots eventually stops helping. Doorways and dense traffic create congestion, increasing inter-robot contacts and reducing the value of each additional robot. We study this safety-efficiency tradeoff for ground swarms of tens to hundreds of robots. We present SPACE, Swarm Pheromone Fields for Adaptive Collision-Aware Exploration. Inspired by ant foraging, SPACE maintains a shared environmental field with an attractive frontier pheromone, a repellent explore pheromone, and a fast robot-density field. Robots use local field guidance on a synchronized shared map. We evaluate SPACE on real building floorplans, namely sixteen home layouts from the HouseExpo dataset and eight campus floors from the KTH dataset, with swarms of up to two hundred and fifty-six robots. At congested swarm sizes, SPACE attains the lowest inter-robot contact rate on HouseExpo and lies on the empirical Pareto frontier on KTH, together with handed lane formation at sizes where Lane has fewer contacts. Relative to a greedy nearest-frontier planner, SPACE reduces the contact rate by four to seventeen times while keeping coverage time within about two percent. The results indicate that, at this scale, coordination mainly improves safety rather than coverage time.
comment: Accepted to IEEE ROBIO 2026. 7 pages, 4 figures
♻ ★ WorldOracle: Benchmarking Action-Conditioned World Models against Real-World Executions
We introduce WorldOracle, a benchmark specifically designed to evaluate action-conditioned world models (ACWMs) against paired real-world executions. Given the same initial observation and low-level action sequence, WorldOracle asks whether a predicted rollout reproduces what happens in the real world, including physical consequences and task-state transitions. We evaluate models along four complementary dimensions: physical state correctness, interaction spatial alignment, key interaction area fidelity, and multi-view action coherence. To support this evaluation, we curate real-world training and evaluation suites that cover different interaction dynamics and embodiments. The evaluation suite contains long, multi-segment episodes with both successful and failed executions, enabling evaluation across varied action outcomes and behavior distributions. Experiments on seven representative ACWMs reveal distinct model strengths and failure modes. WorldOracle provides a systematic foundation for measuring and advancing action-conditioned world modeling for manipulation. We release the datasets, evaluation code, and leaderboard: https://worldoracle-benchmark.github.io/.
♻ ★ MAIA: Anticipating Interaction Anomalies in Contact-Rich Manipulation via Multimodal World Models
Reliable contact-rich manipulation requires detecting execution deviations early enough for corrective action. Yet most monitoring methods are reactive, identifying anomalies only after they become observable. Anticipating such anomalies is challenging because visual observations only partially reveal physical interaction, while planned actions describe intended motion rather than their actual physical consequences. We address this problem with MAIA, a multimodal framework for anomaly anticipation. Its vision-action-force world model learns nominal interaction dynamics and predicts the visual and force responses of planned actions, while a lightweight anomaly head predicts impending interaction anomalies. To enable systematic evaluation, we introduce CRIA, a real-world multimodal dataset of 2,852 trajectories spanning 18 contact-rich manipulation tasks, with frame-level annotations of anomaly onset and evolution. Experiments on both seen and novel tasks show that MAIA achieves more accurate anomaly discrimination than representative baselines while providing earlier warnings, with bACC scores of 0.765 and 0.710 and median anticipation lead times of 28.0 and 25.1 steps, respectively. Moreover, using these predictions for anomaly-aware action selection improves end-to-end task success from 52.6% to 61.5% without retraining the underlying manipulation policy. These results highlight the value of modeling future physical consequences for proactive anomaly anticipation.
♻ ★ Can a Frontier VLM Serve as a Robot Policy for Memory-Dependent Manipulation?
We use a frozen frontier vision-language model, GPT-6-Astra, directly as the robot policy on RoboMME, a benchmark of 16 memory-dependent manipulation tasks spanning temporal, spatial, object, and procedural memory. The model keeps the whole episode in context and outputs every end-effector waypoint and gripper command, with no RoboMME-specific training. On the 800 test episodes it succeeds in 92.1%, above the previously reported human score of 90.5%, which was obtained with an oracle skill interface, and far ahead of the best leaderboard entry (60.8%) and of a hierarchical system that pairs the same model with a RoboMME-trained policy (79.1%). It matches or exceeds the human score on 13 of the 16 tasks. Its 63 failures concentrate in two tasks and are mostly premature completions and mistimed or unintended button presses. These results blur the usual division between long-horizon planning, memory and end-effector control, and raise the question of how much of this stack a single general-purpose model can take over in embodied systems. Project website: https://embodied-astra.github.io. Code, prompts and all episode logs: https://github.com/jackbjed/embodied-astra-robomme.
comment: 13 pages, 6 figures, 5 tables. Project website: https://embodied-astra.github.io
♻ ★ PanoAir: A Panoramic Visual-Inertial SLAM with Cross-Time Real-World UAV Dataset
Accurate pose estimation is fundamental for unmanned aerial vehicle (UAV) applications, where Visual-Inertial SLAM (VI-SLAM) provides a cost-effective solution for localization and mapping. However, existing VI-SLAM methods mainly rely on sensors with limited fields of view (FoV), which can lead to drift and even failure in complex UAV scenarios. Although panoramic cameras provide omnidirectional perception to improve robustness, panoramic VI-SLAM and corresponding real-world datasets for UAVs remain underexplored. To address this limitation, we first construct a real-world panoramic visual-inertial dataset covering diverse flight conditions, including varying illumination, altitudes, trajectory lengths, and motion dynamics. To achieve accurate and robust pose estimation under such challenging UAV scenarios, we propose a panoramic VI-SLAM framework that exploits the omnidirectional FoV via the proposed panoramic feature extraction and panoramic loop closure, enhancing feature constraints and ensuring global consistency. Extensive experiments on both the proposed dataset and public benchmarks demonstrate that our method achieves superior accuracy, robustness, and consistency, completing all 17 sequences without tracking failure and reducing the ATE by up to 47.1% compared with the best-performing baseline. Moreover, deployment on an embedded platform validates its practical applicability, achieving comparable computational efficiency to PC implementations. The source code and dataset are publicly available at https://github.com/YYWumm/PanoAir
♻ ★ EgoTac: In-the-wild Tactile Prediction from Egocentric Vision NeurIPS 2026
Touch is fundamental to dexterous manipulation, yet most egocentric human data increasingly used for robot learning lacks tactile information. Directly collecting large-scale tactile data is challenging due to sensor limitations, while human video data is abundant, contact-rich, and easily scalable. This motivates a natural question: can tactile signals be inferred purely from vision? To address this, we introduce EgoTac, a generalizable model that predicts rich tactile information directly from egocentric human videos. EgoTac is trained on a unified corpus of over 5.7M image-tactile pairs, covering both continuous force measurements and binary contacts. By learning from this diverse dataset, EgoTac captures nuanced touch dynamics across varied interactions. Experiments demonstrate strong performance: in-domain prediction achieves an average force error below 0.06N. On out-of-domain contact prediction benchmarks, EgoTac consistently outperforms the state-of-the-art contact estimator. It also captures the rise and fall patterns of real tactile data and enables zero-shot predictions on unconstrained real-world videos. Scaling analyses further reveal that both data diversity and volume improve performance steadily. Overall, EgoTac provides a scalable pathway to extract tactile priors from egocentric human videos, enabling broadly applicable tactile-aware robot learning. Related resources can be found at https://egotac.github.io/ .
comment: NeurIPS 2026 Accepted. Project Page: https://egotac.github.io/
♻ ★ AnyRank: Reliable Anytime Policy Ranking via Sequential Testing
Due to the high cost of hardware experiments, robot policies are predominantly evaluated by comparing their empirical performance on a limited number of hardware trials. However, finite-sample statistical uncertainty can lead to erroneous conclusions, especially when ranking multiple policies using a fixed batch of evaluation data. We propose AnyRank: a sequential method for multi-policy ranking that (i) adaptively allocates trials to hard-to-separate policies, (ii) provides (partial) rankings at any stopping time while bounding the probability of an incorrect ranking, and (iii) optionally exploits prior information to prioritize hardware evaluations more efficiently. This prior information could be informed from expert intuition, a small set of held-out hardware evaluations, or from policy rollouts in a simulator or an action-conditioned video world model. Notably, the prior can be imperfect; while a strong prior can provide efficiency gains on hardware, any inaccuracies in prior policy rankings do not invalidate the statistical assurances of AnyRank. We achieve the aforementioned properties through novel theoretical contributions that combine sequential testing with multiple-hypothesis graphical testing approaches. Experiments in simulation and real-world policy evaluation benchmarks show that AnyRank requires up to 50% fewer hardware evaluations compared to existing statistically rigorous approaches, while guarding against the incorrect rankings produced by heuristic methods used by state-of-the-art benchmarks.
comment: CoRL 2026
♻ ★ STRIPS-WM: Learning Grounded Propositional STRIPS-style World Models from Images
Robots performing long-horizon visual manipulation observe high-dimensional images, but successful plans depend on action-relevant facts: what can be done now and what changes afterward. A useful planning representation should discard irrelevant visual details while preserving action applicability and effects. Classical task planners exploit this structure through symbolic operators with preconditions and effects, but obtaining such representations from raw visual experience remains challenging. We study a visual task-planning setting in which a robot receives only image transitions: the current image, executed high-level action, and the resulting image. At test time, given a start image and a goal image, the robot must produce a sequence of high-level actions that reaches the goal. To address this problem, we introduce STRIPS-WM, a framework for learning image-grounded STRIPS-style world models directly from visual transitions. STRIPS-WM first induces a finite abstract transition graph from images, then learns latent binary predicates and one grounded propositional operator per action label. The learned operators form a symbolic action model with sparse preconditions and add/delete effects. Finally, the learned predicates are distilled into a visual encoder, enabling classical planning directly from novel start and goal images. Experiments on visual rearrangement tasks show that STRIPS-WM improves image-to-plan success over the tested visual rollout, latent graph-search and latent-symbolic baselines.
comment: Accepted at the Conference on Robot Learning (CoRL) 2026. Project website: https://elpislab.org/projects/strips-wm/
♻ ★ PinNet: Keypoint-Aware Learned Local Descriptors with Geometric Embedding for Loop Closure in LiDAR SLAM
Loop closure is essential to reduce drift and build globally consistent maps in large-scale environments. However, reliable loop closure with only geometric information from, e.g., a LiDAR sensor, remains challenging due to the difficulty of constructing discriminative geometric features. We present PinNet, a neural network that produces local geometric descriptors from point clouds for place recognition and scanto-scan registration. PinNet incorporates a neural network that generates keypoints and their corresponding descriptors, together with a plane-based geometric self-attention module that models inter-keypoint spatial relationships to enhance descriptor discriminability for loop-closure detection and point-cloud registration. The approach is comprehensively evaluated on multiple datasets collected with different LiDAR sensors. Experimental results demonstrate strong place-recognition performance, precise relative pose estimation, and successful single-shot localization in different environments.
comment: fixed LCDNet registration errors
♻ ★ Towards Robot Skill Learning and Adaptation with Gaussian Processes IROS 2026
General robot skill adaptation requires expressive representations robust to varying task configurations. While recent learning-based skill adaptation methods refined via Reinforcement Learning (RL), have shown success, existing skill models often lack sufficient representational capacity for anything beyond minor environmental changes. In contrast, Gaussian Process (GP)-based skill modelling provides an expressive representation with useful analytical properties; however, adaptation of GP-based skills remains underexplored. This paper proposes a novel, robust skill adaptation framework that utilises GPs with sparse via-points for compact and expressive modelling. The model considers the trajectory's poses and leverages its first and second analytical derivatives to preserve the skill's kinematic profile. We present three adaptation methods to cater for the variability between initial and observed configurations. Firstly, an optimisation agent that adjusts the path's via-points while preserving the demonstration velocity. Second, a behaviour cloning agent trained to replicate output trajectories from the optimisation agent. Lastly, an RL agent that has learnt to modify via-points whilst maintaining the kinematic profile and enabling online capabilities. Evaluated across three tasks (drawer opening, cube-pushing and bar manipulation) in both simulation and hardware, our proposed methods outperform every benchmark in success rates. Furthermore, the results demonstrate that the GP-based representation enables all three methods to attain high cosine similarity and low velocity magnitude errors, indicating strong preservation of the kinematic profile. Overall, our formulation provides a compact representation capable of adapting to large deviations from a single demonstrated skill.
comment: Published in IEEE Robotics and Automation Letters (RA-L), 2026. Presented at IEEE/RSJ IROS 2026
♻ ★ WireCraft: A Simulation Benchmark for Industrial DLO Manipulation
Deformable Linear Objects (DLOs), such as wires and cables, are central to industrial assembly. Unlike rigid objects, whose state is captured by a 6-DoF pose, DLOs have an infinite-dimensional configuration space and deform continuously under contact with grippers, fixtures, and the workspace, making them a demanding benchmark for general dexterous manipulation. Despite their importance, policy development and comparison remain difficult: existing benchmarks are often tied to specific hardware setups, lack modular and customizable task assets, or study generic deformable-object tasks without the fixtures relevant to real-world industrial wire manipulation. Few benchmarks align simulation, real-world data, and shared evaluation protocols. To bridge this gap, we introduce WireCraft, a simulation benchmark for industrial DLO manipulation with configurable difficulty and assets, spanning three task families: connector insertion, clip routing, and channel seating. It supports two complementary DLO physics models, articulated and deformable, and the released trajectories come from both simulation and a physical UR5. We benchmark reinforcement learning (RL), imitation learning (IL), and vision-language-action (VLA) policies under shared metrics. Privileged state-based RL solves a representative setting in each task family with over 82% success, confirming the tasks are well-posed. Image-based policies remain less successful across the evaluated tasks, with ACT and DiT VLA falling below privileged State PPO. On a physical UR5, supplementing real demonstrations with simulated data improves success by 15-65 percentage points in the best-performing mixture for each task. These results indicate that industrial DLO manipulation, though tractable under privileged state, remains an open challenge for current vision-based learning.
comment: Accepted at CoRL 2026 (Spotlight). Camera-ready version with additional simulation and real-robot experiments. 29 pages including references and appendices
Multiagent Systems
★ Self-Building Labs
Self-driving laboratories (SDLs) accelerate scientific discovery by automating experimental execution and optimization. AI agents can increasingly propose new experiments, but existing SDLs can only run the procedures they were built for. Adapting them to a new experiment still takes extensive manual engineering, which makes laboratory flexibility the bottleneck for scaling discovery. We introduce the self-building laboratory (SBL), which designs, configures, and operates its own laboratory setup given a high-level scientific goal. SBL starts from whatever already exists, from an empty room to a working SDL, and builds the missing capabilities: the lab layout, its digital twin, robot skills, and experimental workflows. We demonstrate the system working across different robots and setups, scaling from a single workstation to laboratories with several robots working together. We validate SBL's end-to-end capabilities through a real-world quantum-dot synthesis experiment. By shifting the role of human researchers from manually configuring and programming device automation to specifying scientific objectives, SBL makes scientific automation more accessible, flexible, and scalable. Project website: https://self-building-labs.github.io/
★ CocoRNA: Collective RNA Design with Cooperative Multi-agent Reinforcement Learning ICML 2026
Designing RNA sequences that reliably fold into specific secondary structures is essential for understanding their biological functions but remains a challenging computational problem. We propose CocoRNA, a cooperative multi-agent reinforcement learning framework for RNA inverse design. CocoRNA simplifies the design task by decomposing it into smaller sub-problems, each solved collaboratively by multiple agents. This approach reduces the complexity of the problem and improves the exploration of design policies. During training, a centralized critic uses global structural information to guide the agents, enabling them to jointly optimize their design strategies. As a result, CocoRNA learns high-quality RNA design policies that generalize effectively to unseen structures without additional training. Experiments on the Rfam dataset demonstrate that CocoRNA substantially outperforms state-of-the-art methods in both success rate and design speed. Further experiments on other biological sequence design tasks highlight the effectiveness and broad potential of CocoRNA for complex design tasks.
comment: ICML 2026
★ On the possibilities and limitations of LLM-based multi-agent systems for multi-criteria decision-making in healthcare operations management
Multi-criteria models for healthcare operations need weights that express how stakeholders trade off patient waiting time, provider workload, and operating cost, but these weights are difficult to quantify. Teams of large language model (LLM) agents, each prompted to represent a stakeholder, have been used to supply them. We propose a framework for assessing the reliability of LLM-generated preferences as inputs to multi-objective optimization and quantifying how their uncertainty affects decisions. We study explicit preferences by asking agents to rate the importance of the criteria in an outpatient appointment scheduling problem, and implicit preferences by inferring trade-offs from the agents' pairwise choices between emergency department configurations. The computational experiments show that the explicit preferences respond to the clinic's operating conditions and differ across roles, but deliberation among the agents can erase these differences, whereas simple averaging preserves them. The pairwise choices used to reveal the agents' implicit preferences depend largely on which option is presented first, and we find limited evidence that a statistical model using contextual information alongside presentation order predicts the agents' choices on new comparison pairs better than a model using presentation order alone. We then formulate robust weighted-sum optimization models to account for preference uncertainty. In most cases, these models select different schedules or configurations from those obtained using average preference weights. These results indicate that LLM-generated preferences need an audit of this kind before they are used as optimization weights in healthcare decision support.
★ Annotrace: A Workbench for Step-Level Annotation of Agent Traces AAAI 2027
Every study of why LLM agents fail begins by labelling traces by hand, and every such study builds its own throwaway tool. Observability platforms record agent traces but offer no structured labelling; annotation tools label text but have no notion of a step, a delegation or a tool call, leaving it unclear what the smallest unit of execution to label should be. Annotrace reads six trace formats, including OpenTelemetry GenAI, LangSmith and Langfuse, and turns each run into an ordered, nested list of steps. The annotator writes their own labelling codebook; a frontier model proposes a starting set from a sample of their own traces, answers first, and is asked exactly the questions the human is asked, making annotation a joint human-AI process. Annotrace measures agreement between any two annotators, human or AI. On a multi-agent question-answering corpus it reproduces a previous manual effort's 235 annotation units exactly, and two models labelling the same traces agree 91% of the time at $κ= 0.82$.
comment: 4 pages, 4 figures, 1 table. Under review at the AAAI 2027 Demonstration Program
★ Towards High Semantic Fidelity: Hyperdimensional Symbolic Messages in Multi-Agent Communication NeurIPS 2026
Semantic fidelity is essential for multi-agent communication under non-ideal channels, comprising transmission fidelity and conversion fidelity. Yet existing numerical and neural message formats cannot simultaneously achieve both. To address this, we propose a novel message format-\textbf{H}yperdimensional \textbf{S}ymbolic \textbf{M}essages (HSM) and its corresponding communication framework \textbf{Herm}. For transmission fidelity, Herm suppresses semantic entanglement via an entity-attribute-value structured encoding pipeline. For conversion fidelity, Herm first leverages a receiver-centric ego-binding mechanism to align semantics. Then, a similarity-aware dual-branch fusion module is proposed to prevent critical semantic dilution. To realize explicit message-semantic conversion, we devise an iterative decoding scheme via algebraic inverse operations. Experiments exhibit that Herm outperforms baselines across various noisy scenarios while maintaining superior structural readability.
comment: NeurIPS 2026 Accept
★ Cryptographically Secure Agent-to-Agent Communication in Multi-Agent Systems
Multi-agent systems (MAS) comprise AI agents that collaborate or compete to perform complex, distributed tasks. Their dynamic interactions create opportunities for threat actors to target and impersonate other agents, intercept messages, inject malicious prompts, and exfiltrate sensitive data while simultaneously complicating the detection of anomalous behavior through conventional monitoring. This paper surveys threats and cryptographic challenges in agent-to-agent communication and presents a zero-trust architecture integrating identity, transport security, message integrity, and monitoring. The architecture establishes agent identities via decentralized identifiers and task-scoped verifiable credentials backed by a permissioned ledger, enforcing least-privilege authorization. For transport security, it provides mutually authenticated, crypto-agile TLS with hybrid post-quantum key establishment, and evaluates KEMTLS with pre-distributed public keys to reduce handshake overhead. Signed CBOR envelopes preserve payload integrity across multi-hop exchanges, while zero-knowledge proofs verify declared task constraints without exposing underlying inputs. Context-aware monitoring of inter-agent messages, tool activity, and execution telemetry informs the setting of risk-based trust thresholds and supports the detection of behavioral anomalies. Together, these mechanisms provide an integrated framework for securing and auditing agent communication, with implications for other agentic protocols and prospective 6G semantic communication.
★ Thought Blocks: Non-Autoregressive Latent Communication for Multi-Agent Systems via Trajectory-to-State Compilation
LLM-based multi-agent systems typically communicate through text, incurring repeated generation and re-encoding costs. Although latent communication removes explicit verbalization, constructing latent messages may still require autoregressive latent trajectories. We therefore distinguish message representation from message construction and ask whether agents can directly construct useful continuation states without executing the trajectories that normally produce them. We formulate this problem as trajectory-to-state compilation and introduce Thought Blocks, fixed-size blocks of latent key--value (KV) continuation memory constructed in one shot. Thought Blocks achieve a 2.5 times average latency speedup over autoregressive latent communication. Compared with text-based multi-agent communication, Thought Blocks reduce text tokens by 45.7% and autoregressive steps by 84.8%, achieving an 8.6 times average latency speedup. These results demonstrate that trajectory-to-state compilation enables efficient inter-agent latent communication without sequential latent rollouts at inference.
★ KiranaBench: Can a Language Model Run a Neighbourhood Shop?
Large language model (LLM) agents are increasingly evaluated on long-horizon tasks, including autonomously running a business. Existing benchmarks for this setting are almost exclusively English-language, Western in retail context, and single-agent, with the agent observing the business directly. We study how these results transfer to a multi-agent Indic setting. We introduce KiranaBench, a simulated Indian general store in which four roles (manager, cashier, helper, and bookkeeper) are each played by a separate LLM agent. Interactions with customers and suppliers occur in English, Hindi, or Hinglish, and product demand is calibrated to India's national Household Consumption Expenditure Survey. Evaluating four open-weight LLMs, we find that all fall far short of a fixed-rule shopkeeper baseline: the best model earns only one-eighth of the baseline's profit, and two models finish below an unmanaged shop. Performance is also consistently lower for Hindi and Hinglish than for English supplier communication. KiranaBench offers evidence that current LLM agents are not yet reliable for autonomously operating small businesses in India, and provides a testbed for developing agents suited to settings that are long-horizon, collaborative, and multilingual.
comment: 27 pages, 16 figures, 7 tables. Code, data and traces: https://osf.io/s2pxg/
★ Provable Authority: A Protocol for Verifiable Authority Resolution and Control Across Human, Organizational, and Agentic Transactions
Payment systems authenticate users, devices, accounts, and cryptographic credentials, but a relying party, such as a counterparty, payment rail, marketplace, or autonomous agent runtime, generally cannot obtain compact, independently verifiable evidence that the party presenting one exact transaction holds current organizational authority to execute it under the applicable evidence, policy, budget, and approval state. Agentic commerce makes this gap operationally urgent: a software agent can produce a valid signature while acting outside its delegated purpose, after its sponsor's authority has changed, or on transaction details that differ from those a human approved. We describe the ChainIT Authority Protocol, a subject-neutral and rail-neutral pre-execution authority and evidence control plane. The protocol separates durable identity, a versioned organizational authority record (the Authority Resolution Pactvera), sourced evidence assertions, deterministic reason-coded policy predicates, and organization-configured dispositions; binds each decision and human approval to a canonical transaction digest; reserves capacity atomically so that limits hold under concurrency; and authorizes commit only through a single-use, sender-constrained Execution Authorization Credential consumed at the rail adapter, with append-only lifecycle evidence sealed after the terminal outcome. An Agent Subject Profile adds the workload registration, attested key hierarchy, and sovereignty boundary required when the acting subject is an AI agent, without introducing a separate authority concept. We specify the commit-time execution predicate and bounded-freshness status rules, give enforcement profiles for fiat (ACH, wire, card, instant payment) and stablecoin rails, and state residual risks and non-goals.
comment: ChainIT white paper CIT-PA-WP-7.0. 32 pages, 3 figures. Companion paper: Provable Compliance. Project page: https://chainit.com/white-papers/
★ Cooperation Without Adaptation: LLM Agents in a Collective-Risk Dilemma
Language-model agents are increasingly deployed to act on behalf of people in settings where individual decisions shape shared outcomes. We examine whether five production models adapt their contributions to changing risks and partner behaviour, or instead follow a relatively fixed strategy. In a collective-risk dilemma game (CRG), six players contribute to a shared pool over ten rounds; if the pool falls short of its target, a catastrophe may destroy their remaining resources. Four models contributed approximately an equal share at every positive risk level, and four of the five also contributed when risk was zero, even though contributing could not improve their payoff. Explicitly stating that only the agent's own payoff mattered reduced zero-risk contributions for most models, while a more neutral prompt did not. Several models correctly identified which action offered the higher expected payoff at different risk levels, yet their contributions changed little. Against fixed partners, no model consistently chose the payoff-maximising response, and in mixed-model groups, a single agent's shortfall in the final round could cause an otherwise successful group to fail. Payoff-based selection also depended on population composition: in a single six-agent group, it favoured the model that contributed least and reduced group success, whereas in larger, randomly mixed populations, it favoured an exact fair-share model at moderate and high risk. These findings show that cooperative self-play alone is not evidence of strategic adaptability. Consistent with a machine-behaviour approach, evaluations of agents in multi-agent systems should test how they respond to incentives, partner actions, and population composition.
comment: 10 pages, 5 figures, 50-page supplementary material appended
★ ROAR: Benchmarking LMM Agents under Real-Time Constraints in Multiplayer Action-Shooter Games NeurIPS 2026
Large multimodal models (LMMs) are increasingly deployed as agents that perceive, reason, and act in interactive multi-agent environments. Yet most existing benchmarks for LMM-based agents in multi-agent settings remain turn-based: the environment and other agents pause while an agent performs inference. Such protocols obscure a fundamental challenge of real-time constraints, where the world continues to evolve and delayed decisions may become outdated before execution. Motivated by this limitation, we introduce ROAR, a multiplayer action-shooter game environment for evaluating LMM-based agents in multi-agent settings under real-time constraints. ROAR covers cooperative, competitive, mixed cooperative-competitive, and basic single-agent scenarios. We further propose a Multi-Agent Asynchronous Gym interface that decouples environment evolution from agent reasoning by running the environment and each agent in separate threads, enabling the environment to advance at fixed time intervals without waiting for LMM inference and allowing agents to reason independently without blocking one another. ROAR also features convenient deployment, multimodal observations, and customizable levels. ROAR offers a practical testbed for advancing LMM-based agents beyond static, turn-based evaluation toward real-time multi-agent decision-making.
comment: NeurIPS 2026
★ Model Capacity Frontier: A Development Framework for LLM-Supported Social Simulation
Large language model (LLM) agents are increasingly incorporated into agent-based models (ABMs) on the premise that richer, context-sensitive human behavior can improve empirical fidelity. However, LLM-supported ABMs introduce an open-ended design space in which subtle modeling and configuration choices, such as prompt design and environment coupling, can substantially alter outcomes. Developing LLM-supported ABMs to simulate human collective behavior therefore requires navigating configuration sensitivity while identifying the limitations of the current model architecture. In this paper, we introduce the \textit{Model Capacity Frontier} (MCF), a framework that characterizes the empirical performance attainable by a specified architecture across its admissible configurations. In multi-objective settings, this capacity is represented by an estimated Pareto frontier obtained through systematic configuration exploration. We demonstrate MCF in crowd simulation of university commencement events with approximately 12,000 agents, using empirical movement data on temporal occupancy, spatial distribution, and flow direction. We first characterize the capacity of a conventional crowd model and then evaluate successive LLM-supported architectural extensions. In the process, MCF reveals which dimensions of attainable performance expand and which remain limited, providing empirical targets for refinement and re-evaluation. Initial LLM integration improves flow direction but degrades temporal occupancy and spatial distribution; subsequent architectural refinement of the LLM layer improves temporal occupancy and recovers spatial performance, while retaining much of the earlier gain in flow direction. Our case study shows how MCF guides the systematic architectural development of LLM-supported ABMs in configuration-sensitive, open-ended design spaces.
★ Preservation and loss of algorithmic structure in experimental diffusion of joint-action norms
Everyday norms of social interaction - e.g. norms of sharing space, taking turns, or queuing - are essential to large societies, offering structured procedures for common interactions. However, it remains unclear how the underlying algorithms of these implicit joint plans evolve as they diffuse through populations. Here we show that this process of diffusion can select against complex algorithms in the earliest stages. We conducted a large-scale study of experimental diffusion of novel social norms among human participants, leveraging advances in program synthesis to characterize the procedures people developed in independent populations. Inherited procedures became structurally simplified and lost during interactions between participants who inherited different variants of joint-action norms; however, opportunities for interactive repair offset this pressure, preserving complex algorithms despite misalignment. These results demonstrate that interactive reasoning mediates selective pressures on structural complexity during cultural evolution of procedural norms.
★ The Weight of Teamwork: Does Communication Structure Help LLM Scientists Search for Scientific Laws? NeurIPS 2026
Team performance is often reported as one delivered score, even though that score is produced in two stages: a system generates candidate answers and a selection rule returns one of them. We use symbolic regression as an observable scientific-search setting and compare fixed-budget LLM teams that differ in peer visibility and update timing, evaluating both their frozen candidate archives and the expressions selected from those archives. Distributing the call budget across more independent trajectories produced the largest measured gain, improving mean held-out fitting score Q by 0.70 at the widest tested allocation. Under a common archive-wide selector, the implemented Ring policy reduced mean Q by 0.33 relative to Independent, while Star and Complete remained unresolved. Holding the generated archives fixed, changing the delivery rule changed the measured architecture gaps, including the sign of the Complete point estimate. These results support reporting candidate generation and final selection as separate endpoints. They do not identify a communication mechanism or a universally preferred topology; rather, they show that an architecture ranking is not identified by communication structure alone when the rule that returns the final answer is left unspecified.
comment: Accepted at the NeurIPS 2026 Workshop on Foundations of Agentic Systems Theory (FAST), non-archival. 7 pages main text, 33 pages total with appendices
★ Coopshift: Post-Training Amplifies The Memory Curse In LLM Cooperation
How does post-training change cooperation during repeated interaction? Within each of seven released model lineages, we compare pretrained (Base), supervised fine-tuned (SFT), and subsequent preference- or reward-optimized (Aligned) checkpoints across seven social dilemmas. Copies of each checkpoint play one another for 200 rounds, with visible history limited to the last two or forty rounds. The memory curse refers to lower cooperation when agents can see more past rounds. Our central finding is that post-training amplifies this decline. Averaged across the six binary-action games, expanding visible history reduces cooperation at every SFT and Aligned checkpoint. At each post-trained stage, these declines exceed Base's in six of seven lineages. Most of this average amplification is already present at SFT. With long history, both post-trained stages also cooperate less than Base on average in each binary-action game. In the symmetric binary games, their fitted cooperation rates fall more steeply on average than Base's as per-round cooperation costs rise, benefits fall, or noncooperation becomes more rewarding. The difference in cooperation rates between Aligned and SFT also changes from one-shot decisions to repeated play. When given the same past records, their choices depend on whether the instructions describe a final round or possible future interaction. Post-training evaluation should therefore go beyond isolated choices to assess whether cooperation is sustained in repeated interaction and how it depends on payoffs and visible history.
★ Alliance of Stochastic Routing and Reinforcement Fusion for Multi-Model Reasoning
This paper studies dynamic model routing under uncertainty in multimodal settings, where a system must select and potentially compose multiple models to answer a query. Standard reinforcement learning approaches for routing often suffer from premature convergence to suboptimal models due to noisy rewards and insufficient exploration. We propose a unified vision-language framework (VL-Scout) for efficient multimodal and multi-step reasoning with turn-based reinforcement fusion. First, VL-Scout combines Bayesian priors, policy gradient optimization, and sequential decision-making to enable structured exploration and adaptive inference. A context-dependent Dirichlet belief model allows for epistemic uncertainty over model quality and is updated online based on observed rewards and context-specific evidence, enabling Thompson-style exploration while serving as a prior for regularization. Second, a routing policy is trained via a KL-regularized REINFORCE objective by maximizing a reward-weighted evidence lower bound, where the policy acts as an amortized posterior, and the Dirichlet belief serves as a prior. Third, we introduce a fusion policy trained with Group Relative Policy Optimization (GRPO) or Reinforcement Learning with Verifiable Rewards (RLVR), which operates over intermediate model rationales and selects between two actions: CONTINUE or ANSWER, terminating with a generated output. We evaluate VL-Scout on four representative multi-modal benchmarks (MMMU, MMMU-Pro, OKVQA, and RealWorldQA), compared against 11 representative routing and policy optimization baselines. Our method provides consistent performance improvements while reducing premature policy collapse. These results highlight the importance of integrating uncertainty-aware priors with both amortized policy learning and sequential fusion for robust and efficient model selection in discrete action spaces
★ Red-Teaming Multi-Agent Attacks for Loss-of-Control Risks
AI agents are growing more capable and persistent, and are being deployed in ever larger numbers. As more start collaborating, AI control measures must remain effective when they covertly collude on a harmful objective. We introduce MiniLab, a synthetic series of live services and codebases (5 interconnected services, 249 benign tasks, 8 attack objectives), adapted from internal AI deployment infrastructure. We demonstrate distributed attacks, in which different agents carry out separate attack fragments, and how their covertness evades frontier monitors when reading their trajectories individually. We show how distributed attacks spread across diverse contexts and harmful objectives through 16 human-optimized attacks (weight exfiltration, unmonitored rogue deployment, permission elevation, and more). A frontier monitor, GPT-5.6 Sol, reading each agent separately flags only 7% of successful distributed attacks, compared with 87% when a single agent carries out the same side task. Individual attacks can also be composed into longer attack sequences toward a greater loss-of-control outcome, without any individual agent being flagged. Finally, we tested whether models can plan these covert attacks: even with execution and monitor feedback, the best model-planned attacks succeed covertly against a single-trajectory GPT-5.6 Sol monitor in at most 25% of executions, compared with 80% for human-crafted plans, though this may scale with model capabilities. By turning worst-case coordinated attacks into reproducible red-team cases, our work provides a foundation for preventing locally benign agent actions from accumulating into system-level loss of control.
★ LLM Evolution in a Simple Simulated Market
Arguments that AI systems will evolve in harmful ways are largely made in the abstract. We provide a concrete empirical example. We simulate simple markets consisting of populations of twenty Large Language Model agents tasked with solving arithmetic problems. Agents differ only in their system prompt; those that go bankrupt are removed and replaced by mutated copies of the prompts of successful agents. Two market conditions select for opposite phenotypes: a market that rewards accuracy produces agents with high accuracy (94%) and high token use; a market that charges per token produces agents with low accuracy (10%) and low token use. In the second market, only tokens are incentivised yet we see a collapse in accuracy. Token use and accuracy are correlated traits such that selection on one also moves the other, and behaviour adapts along an unpriced dimension. A third neutral condition with no selection allows us to separate these effects from bias in the mutation operator. Adaptation occurs through a collective optimisation process that operates above the level of individual agents, who are never told the payment rule, do not receive feedback, and have no memory.
♻ ★ Coordination Matters: Process-Aware Evaluation of Cooperative Multi-Agent Reinforcement Learning
Cooperative multi-agent reinforcement learning (MARL) benchmarks commonly emphasize aggregate outcomes such as return, success rate, or completion time. While essential, these metrics can obscure how agents coordinate and fail to distinguish policies that achieve similar outcomes through different interaction processes. We propose a process-aware evaluation framework that complements aggregate performance with diagnostics of interference, coordinated task completion, participation, and coordination efficiency. We evaluate the framework in STAT, a controlled spatial task-allocation testbed, and examine its broader applicability in two established cooperative MARL benchmarks. Across diverse value-based and actor-critic methods, we find policies with nearly identical return can exhibit substantial differences in coordination behavior. Controlled scaling in STAT reveals distinct process signatures for spatial, task, and agent scaling, while a reward-shaping ablation shows that explicitly penalizing assignment conflicts can alter coordination behavior with little change in task return. Results in the established benchmarks further show that the framework remains informative under different coordination mechanisms. These findings support process-aware evaluation as a complement to outcome-based benchmarking. The framework provides a common structure for selecting and interpreting environment-specific diagnostics that characterize not only what cooperative policies achieve, but how they coordinate to achieve it.
comment: Under review
♻ ★ TANGO: RNA Topology and Geometry Co-Design NeurIPS 2026
Designing RNA sequences that satisfy target tertiary-structure objectives is an important problem in RNA engineering, with broad applications in synthetic biology and therapeutics. Existing 3D RNA design methods have made substantial progress, but they largely treat the target fold as a geometry-only objective. This is incomplete for RNA, whose functional folds are specified not only by tertiary geometry but also by secondary base-pairing topology. We therefore formulate RNA inverse design as a topology-and-geometry co-design problem and propose TANGO, a multi-objective multi-agent reinforcement learning framework for this task. TANGO adopts a staged curriculum: it first learns a topology-aware policy from secondary-structure targets, then transfers this policy to tertiary-structure targets to train a general co-design model, and finally applies frontier-guided fine-tuning to refine target-specific trade-offs among secondary-structure fidelity, tertiary-structure fidelity, and sequence diversity. Experiments show that TANGO generates diverse and novel designs, improving tertiary- and secondary-structure performance by 18.3% and 52.7%, respectively, over the strongest baseline.
comment: NeurIPS 2026
♻ ★ GRAFT-ATHENA: Self-Improving Agentic Teams for Autonomous Discovery and Evolutionary Numerical Algorithms
Language-model agents plan, execute, and evaluate scientific workflows. However, the problem--method relationships guiding their choices remain implicit, so plausible outputs can violate the assumptions of the scientific problem. We introduce Graph Reduction to Adaptive Factored Trees (GRAFT), which makes this problem-to-method map explicit as an expandable probabilistic structure of admissible problems, methods, and dependencies. Because this structure follows scientific semantics, knowledge from documentation, papers, and domain experts can be encoded, while factorization keeps the structure tractable. Semantic fingerprints of encoded problems measure their similarity, so expertise and experience guide related problems. GRAFT-guided workflows exceeded published expert baselines in physics-informed learning, recovered blood-rheology trends, predicted Apollo Command Module hypersonic loading within 1.8\% of experiment, and established machine-checked error bounds for regularized brain-flow reconstruction and a spectral physics-informed architecture. The representation retains evaluated outcomes and incorporates new methods, allowing scientific knowledge and experience to accumulate.
♻ ★ Where Do Multi-Agent Systems Fail? Evidence-Grounded Diagnosis of Collective Mechanisms
When a multi-agent system answers correctly, it is tempting to conclude that its agents shared, checked, and used information as intended. Yet a system can break one of its collective mechanisms, the rules that govern how agents route, admit, store, and act on shared information, and still return the right answer, while a wrong answer rarely reveals which mechanism failed. We ask what evidence from an execution is sufficient to conclude that a particular mechanism was violated. Our answer is a diagnostic contract, which separates what counts as a violation from which execution records can establish one, and concludes that a violation is supported, ruled out, or unknown; removing records can make this conclusion unknown but never reverse it. We test contracts for four mechanisms by replaying executions from the step where a mechanism acts, once unchanged, once with the mechanism broken, and once with it restored. Broken mechanisms often left the answer correct. An LLM diagnoser detected many more violations from internal records than from public outputs, yet with identical records a generic prompt often claimed certainty the records did not support, which prompts stating the contracts largely avoided. The contracts also applied, in narrow form, to mechanisms in independently developed systems, but a diagnostic behavior that was nearly perfect on our benchmark degraded on an independently developed workflow. A correct outcome is therefore no substitute for records of how collective mechanisms operated, and agreement on one benchmark does not show that a diagnoser transfers to another system.
♻ ★ When Extremists Enter LLM Societies: Investigating Extreme Behavior in Multi-Agent Systems
Large language model (LLM) agents increasingly serve as computational actors in multi-agent social simulations. While existing work emphasizes representative personas and average-case behavior, social science shows that persistent tail actors can disproportionately shape collective dynamics. We study this regularity in controlled LLM-based multi-agent systems (LLM-MAS). In these systems, such actors cannot be reduced to fixed scalar states: prompts, natural-language histories, and backbone generation mediate their behavior and influence. We first analyzed three properties of extremists in sociology and model the LLM extremist based on LLM-native perspective. We introduce EXAM, an EXtremist Agent Modeling framework that capable of configuring extremist agents for specific tasks and precisely controlling their population proportion, spatial distribution, interaction protocols, and measurement interfaces. Across four scenarios: congressional election, financial crisis, epidemic public opinion, and iterated prisoner's dilemma, we observe that small persistent tail groups produce measurable, backbone- and scenario-dependent collective shifts. EXAM shows how this social-science regularity manifests through language-mediated mechanisms specific to LLM-based social simulation.
♻ ★ Recursive Game Creator: An Agentic Product-Level Experience-Oriented Game Harness
Recent game design agents have made substantial progress in generating playable games. However, program correctness does not ensure an enjoyable experience for players. We present Recursive Game Creator, an experience-oriented harness to advance agentic game development from rough game prototypes into entertaining games. Recursive Game Creator organizes recursive development around four components: Designer, Builder, Player, and Reviewer. The Designer translates user instructions and Reviewer's feedback into detailed plans. The Builder turns these plans into candidate games. The coding-native Player creates and executes reusable policies through programmatic interfaces to efficiently collect diverse gameplay trajectories, mitigating evaluation bias caused by slow GUI-based collection. The Reviewer uses carefully designed trajectory-based metrics to induce player preferences, integrating with visual evidence and explicit textual preferences to evaluate games against game-specific criteria. Finally, the Reviewer accepts the better version and provides improvement reviews for the next round, closing the recursive loop. Our method achieves state-of-the-art overall performance of 77.89 on GameCraft-Bench. On GameASG-Bench, it achieves a strict task success rate of 53.2%, a 34.1% improvement over the same-model baseline, and the highest mean runtime-check pass rate at 93.4% among compared methods. A user study shows longer playtime and higher ratings. Project page is at: https://imbaldy.github.io/recursive-game-creator/.
comment: Project page: https://imbaldy.github.io/recursive-game-creator/ ; Code: https://github.com/IMBALDY/RecursiveGameCreator
♻ ★ CollabFlow: Recursive Self-Improvement of Agent Collaboration
Recursive self-improvement (RSI) lets a system improve from its own outcomes; in LLM-based multi-agent systems, Agents refine one another within a task, and outcomes improve how they collaborate across tasks. However, existing multi-agent collaboration leaves this loop open: collaboration is pre-defined at the operator level, topology-only learning keeps verbatim exchange that propagates errors, and reward maximization on a system's own outcomes concentrates on a few teams. To address these challenges, we propose CollabFlow, an RSI system of Learned Agent Collaboration: a trainable Collab-Director constructs teams of complete Agents, a frozen executor runs them, and each round's outcomes retrain the director. Within each round, the edges of a collaboration graph carry protocols of Evidence-Conditioned Communication: a receiver adopts a differing answer only when the sender's evidence is stronger by a margin, so the director learns who communicates and how. Across rounds, we further propose Collaborative Trajectory Balance (CTB), a flow-based objective that credits each team once across its construction orders and targets a reward-proportional distribution over teams, so several good teams stay in play. We also bound how far this self-generated target moves between rounds, which shrinks as records accumulate. On twelve datasets, CollabFlow outperforms all baselines and keeps improving across rounds. Code is available at https://anonymous.4open.science/r/CollabFlow-631E.
♻ ★ Do Multi-Agent LLMs Explore Each Other? NeurIPS 2026
Exploration is essential for reliable autonomy in multi-agent systems, yet it remains unclear whether large language model (LLM) agents can explore effectively when interacting with one another. We show that modern LLM agents often fail to do so, exhibiting myopic and polarized interaction patterns that lead to suboptimal coordination and increased regret. We formalize this challenge as the Multi-Agent Exploration problem, modeling it as a partially observable stochastic game (POSG), in which agents must probe peers to infer their capabilities and identify effective interaction strategies. To address this, we introduce Multi-Agent Contextual Exploration (MACE), a lightweight framework that explicitly promotes exploration through structured peer selection. Across both contextual and parametric diversity settings, exploration behavior and downstream task performance substantially improve when MACE is applied. We further provide theoretical insight that MACE achieves a sublinear expected-regret bound. Overall, our results highlight a fundamental limitation of current LLM agents and underscore the importance of explicitly guided exploration for reliable multi-agent autonomy.
comment: NeurIPS 2026 IAEval Workshop (Oral Presentation)
♻ ★ SCARCE: Emergent Institutions in Long-Horizon Multi-Agent Coding Systems
As autonomous agents become widely deployed, coordinating access to shared computing resources and services will become increasingly important. Recent incidents have already revealed unplanned coordination between agents, highlighting the safety implications of these interactions. Yet prior studies of agent negotiation often use stylized games and simulated bargaining, leaving open how these behaviors develop during sustained, long-horizon tasks. We propose SCARCE (Shared Compute ARena for Coding agEnts), a framework in which agents solve sequences of coding tasks under individual deadlines while negotiating access to a shared execution resource. This negotiation determines when and for how long they can inspect programs, compile code, and run tests. We find that agents spontaneously develop informal institutions: persistent rules and commitments grounded in reputation, reciprocity, and precedent. These arrangements, which we call institutional behaviors, carry across tasks and influence resource allocation, with agents sometimes forgoing immediate advantages to honor earlier agreements. We develop a 21-category taxonomy of negotiation strategies and emergent norms and use it to analyze eight models and audit more than 1,500 trajectories across different environmental settings. Among the models tested, newer frontier models exhibit more institutional behaviors than older or lightweight models. Alongside cooperation, we observe exclusionary coalitions and rare divide-and-conquer strategies. SCARCE provides a controlled setting for proactively and systematically auditing how these relationships emerge, shape access to resources, and affect the safety and coordination of multi-agent systems.
♻ ★ VACS: Value-Aligned Compositional Shielding for Multi-Agent Reasoning
Multi-agent reasoning systems in high-stakes domains must be both accurate and safe, yet agents often follow heterogeneous value priorities (e.g., rigor, conciseness, safety), causing conflicting recommendations. Existing methods do not jointly provide: (i) principled inference of each agent's implicit values from behavior, (ii) compositional formal safety guarantees without full online communication, and (iii) value-aware conflict resolution with faithful explanations. We present VACS (Value-Aligned Compositional Shielding), a four-layer framework addressing all three. Layer 1 learns value-dimension rewards from pairwise preferences using Bradley-Terry modeling and infers per-agent value weights via deep MaxEnt IRL. Layer 2 encodes value constraints in a Lean-inspired DSL and synthesizes compositional assume-guarantee shields for runtime safety. Layer 3 resolves disagreement through nucleolus-based credit allocation and Hamiltonian consensus optimization under long-term value constraints. Layer 4 extracts a critical reasoning path from co-state sensitivities and generates formally grounded natural-language explanations. Our contribution is primarily a unified systems design with formalized interfaces and operational guarantees at the verifier-constrained decision level, rather than a complete end-to-end formal proof of all language-model internals. In controlled proof-of-concept evaluations with role-conditioned agent panels on NEJM-AI QA, MathInstruct-Subset, and a cybersecurity incident-response benchmark (CyberSec-Eval), VACS outperforms strong baselines in accuracy (85.4%, 95.0%, and 90.0%) while reducing logical inconsistency rates to near zero.
♻ ★ A Jury, Not an Electorate: Label-Free Reliability Estimation for Ensembles of Language Models
An ensemble of language models is a jury, not an electorate. The answers are testimony from witnesses of unknown reliability, and the question is not what the majority said but who is credible. Most ensemble methods assign each expert a fixed reputation, often learned from labels. We ask whether it can instead be inferred from the pattern of agreement and disagreement among the experts. We model the experts as a jury: each question has a latent true answer, each expert reports it with unknown reliability, and those reliabilities are estimated by Expectation Maximisation (EM) from unlabelled answer configurations alone. We show that its decision rule reduces to a log-odds weighted linear score over the experts' answer distributions. What changes is not how the jury votes, but how it learns whom to trust. On a 1560-question benchmark, the jury, with nine experts of up to 3B parameters, performs comparably to the same fusion rule with label-fitted weights, showing no statistically significant advantage from supervision. Instead, it outperforms the best single model with matched self-consistency and the strongest label-free alternative on the full benchmark.
comment: 35 pages, 4 figures, 13 tables, including appendices
♻ ★ Orchestra: Corroboration-Based Regulatory Candidate Discovery via Composed Bioinformatics MCP Agents SC
Orchestra composes two independently built bioinformatics MCP servers -- RegNetAgents, which infers regulatory network topology, and CASCADE, which provides independent functional evidence (LINCS knockdown, DepMap essentiality, super-enhancer status, DoRothEA transcription-factor confidence) -- into one multi-agent LangGraph workflow. Its central claim is that requiring the two systems to agree on a candidate regulator yields a more trustworthy candidate than either alone. We test this on the TCGA tumor-acquired regulator tier, selecting each query's ten regulators by ARACNe mutual-information edge weight. Across BRCA/COAD focal genes and matched negative controls, requiring at least 2 of 4 evidence sources predicts known-driver status (IntOGen) among focal genes (odds ratio 3.15, Benjamini-Hochberg-adjusted p=0.016; 2.92 after deduplication) but shows no association among negative controls (p=0.70); a single source does not in BRCA/COAD. In a second panel (STAD) the pattern reproduces and is stronger (OR 6.71, adjusted p=0.0047), again with no association in controls. Corroboration adds value beyond MI edge weight in both panels (BRCA/COAD p=0.039; STAD p=0.0001), though modestly in BRCA/COAD. Deduplication, a permutation test and a reproduction of RegNetAgents' published results agree; a second, separately derived driver list (Vogelstein et al. 2013) is directionally consistent but too sparse to reach significance. All experiments invoke Orchestra's real agentic entry point.
comment: 20 pages, 3 figures. v4: driver reference set changed from OncoKB to IntOGen; data re-collected on RegNetAgents v1.3.0 / CASCADE v1.5.0 with updated panels; all results, tables and figures updated
♻ ★ MOIRA: Policy Memory and Belief Revision for Language-Model Agents in Non-Stationary Pair Trading
Many sequential decision-making problems exhibit hierarchical structure, where high-level semantic choices constrain downstream actions and feedback is delayed and ambiguous. Learning in such settings is challenging due to credit assignment: performance degradation may arise from flawed abstractions, suboptimal execution, or their interaction. We study this challenge through pair trading, a domain that naturally combines long-horizon semantic reasoning for asset pair selection with short-horizon execution under partial observability. We formulate pair trading as a hierarchical reinforcement learning problem and propose a language-driven optimization framework in which both high-level and low-level policies are parameterized by large language models (LLMs) and optimized exclusively through prompt updates. Our approach leverages pretrained LLMs as hierarchical policies and uses trajectory- and episode-level textual feedback to adapt abstractions and execution without gradient-based fine-tuning. By explicitly separating abstraction selection from execution, the framework reduces non-stationarity across hierarchical levels and enables targeted adaptation under delayed feedback. Experiments on real-world market data show consistent improvements over traditional and LLM-based baselines, demonstrating the effectiveness of language-driven hierarchical reinforcement learning.
Systems and Control (EESS)
★ SparseMax-dSTL: Sound Robustness with Horizon-Independent Gradient Allocation for Temporal Logic Trajectory Optimization
A gradient-based solver that enforces a Signal Temporal Logic (STL) specification uses a smooth approximation of its quantitative semantics, the robustness. In safety-critical trajectory optimization, this approximation must have two properties. First, it must be sound, that is, nonnegative only on trajectories that satisfy the specification. Second, at each temporal operator, the weight of its gradient on the violating samples must stay bounded away from zero, however long the horizon. The approximations in use, including the log-sum-exp, its sound variant, and the generalized-mean robustness, do not have both. We introduce lower approximations of the minimum and the maximum whose gradients are sparsemax projections, and we prove that the robustness built from them has both. It lies below the exact robustness and is therefore sound, and its error has a bound that is independent of the number of samples and known before optimization. The gradient of each approximation is exactly zero on every sample beyond a data-dependent threshold, so the weight on violating samples a gap beyond the others has a positive lower bound at every horizon. On a planar unicycle and on a seven-joint manipulator beside a person in simulation, the log-sum-exp approximation is positive on a violating trajectory. As the horizon of the manipulator's task grows from 10 to 80 s, the gradient weight of the sound variant on the violating samples falls from 0.22 to 0.04, while that of sparsemax stays at 1. In a constrained trajectory optimization of the unicycle, the constraint on the log-sum-exp robustness returns a violating trajectory, whereas the constraint on the sparsemax robustness returns a trajectory that satisfies the specification with the required margin.
comment: 8 pages, 5 figures. This work has been submitted to the IEEE for possible publication
★ Lyapunov Exponents in Power Systems: A Survey of Theory and Applications in Stability, Control, Forecasting, and Protection
Lyapunov exponents (LEs) are fundamental metrics from nonlinear dynamical systems theory that quantify the average exponential rate of divergence or convergence of nearby trajectories in phase space. Over the past three decades, LEs have emerged as powerful tools across a wide range of power system problems, including transient and voltage stability assessment, chaos detection and mitigation, short-term load and renewable energy forecasting, and fault diagnosis. This paper presents a comprehensive review of these applications, covering more than 165 publications spanning from the early demonstrations of chaos in simplified machine models (1990s) to the most recent data-driven and machine-learning-augmented approaches applied to inverter-based resource (IBR)-dominated grids (2024--2026). The review is organized into four principal application domains: (i) stability analysis and real-time monitoring, (ii) chaos characterization and control, (iii) forecasting and predictability analysis, and (iv) protection, fault diagnosis, and condition monitoring. For each domain, we trace the evolution of methods, compare model-based and data-driven LE estimation strategies, identify key benchmarks and validation gaps, and outline the most pressing open research problems.
★ Parametric Resonance in Primal-Dual Learning for Slowly Non-Stationary Constrained MDPs
Primal-dual methods for constrained Markov decision processes are analyzed under a two-timescale assumption, with non-stationarity entering through two channels. In the additive channel, where the constraint level oscillates, the dual oscillation saturates at a floor set by the moving optimum and is amplified linearly by the timescale ratio. In the parametric channel, where the constraint sensitivity oscillates, a coherent modulation carries no resonant forcing at any depth: the dual update's stiffness modulation cancels the primal update's damping modulation. Instability instead comes from a mismatch between the constraint estimate used by the primal update and the measurement used by the dual update, producing a parametric resonance at twice the pair's natural frequency. The tolerated mismatch grows with the square root of the forcing bandwidth and collapses with the square root of the dual-to-primal step-size ratio. These results hold on the tangent linear-quadratic reduction around an active saddle, for one constraint and for $K$ constraints under a common modulation and an isotropy condition. Driving both updates from the same estimate removes the resonant forcing but introduces a stationary offset equal to the estimator's bias. In a Lagrangian soft actor-critic agent, the multiplier locks onto the sub-harmonic at the timescale ratios its identified linear loop predicts and a coherent cost critic suppresses the lock-in, while a proportional (PID) term, predicted to leave the lock-in in place, suppresses most of it through the projection of the multiplier.
comment: 33 pages, 3 figures. Code: https://github.com/mbaelde/dualdynamics-code
★ Anytime-Valid Certification of Robustness Margins for Safe Control via E-Values
Robust control uses the error between a known model and the true system as a robustness margin. However, this error is usually unknown: conservative estimates restrict feasibility and performance, while underestimates can result in violations of the control property. We estimate this robustness margin from online data using e-values, a statistical tool for hypothesis testing, in the setting of discrete-time control barrier functions (CBFs). Here, we measure the robustness margin as the difference between the observed CBF and the model-based CBF prediction. We then design a robust CBF controller in three steps by: (1) selecting a set of candidate margins, (2) obtaining validity certificates for these candidates via sequential tests based on e-values, and (3) updating the CBF controller with the smallest certified candidate. The deployed tests accumulate evidence against the hypothesis that an exceedance risk, the probability of the robustness margin exceeding the candidate margin, remains at or above a pre-specified failure probability $α$. Our tests are anytime valid and thus enable the use of certified candidates in the CBF controller, guaranteeing that the probability that the exceedance risk of a certified margin exceeds $α$ is at most a user-chosen confidence $δ$. We validate our approach on navigation problems across different environments with visualizations in NVIDIA Isaac Sim.
★ MIXSim: A multiclass first order traffic model with composition-dependent dynamics for mixed-autonomy urban networks
This paper presents a novel multiclass first order macroscopic traffic model for mixed-autonomy in signalized urban networks, called MIXSim. Human driven vehicles (HDVs) and Connected and Automated Vehicles (CAVs) are represented as separate classes while origin-destination information is retained through commodity specific states. The local vehicle class composition and class specific driving behaviour characteristics determine the parameters of a composition-dependent triangular fundamental diagram. At signalized intersections, the composition of movement demand additionally determines the saturation flow rate, while the realized movement flow is constrained by demand, effective green time, and downstream receiving capacity. Numerical experiments investigate different CAV penetration levels and CAV time gaps, together with alternative signal control strategies. The numerical experiments illustrate how changes in CAV penetration and CAV time gap affect network performance under the tested conditions, and demonstrate the use of MIXSim for comparing signal control strategies across different mixed traffic regimes.
★ Net-Zero Power UAVs: Facilitating Quantum HRLLC in Multi-Hop UAV Repeaters
In this paper, we propose a novel joint optimization framework for autonomous, quantum-enabled UAV repeater networks designed to support hyper-reliable low-latency communications (HRLLC) in challenging environments such as post-explosion chemical plant operations. The proposed system uses a symmetric dual-UAV architecture with three sequential quantum channels operating under strict energy harvesting (EH) constraints. We formulate a highly challenging non-convex mixed-integer non-linear programming (MINLP) problem. This model jointly determines qubit allocation, 2-D uncrewed aerial vehicle (UAV) spatial positioning (distance and altitude), and transmit powers to minimize the total decoding error. Crucially, our framework explicitly models real-world quantum physics, including decoherence, gate imperfections, and entanglement degradation. Since the objective function is demonstrably non-convex, to efficiently solve the MINLP, we propose a quantum-resource-optimized GA (QRO-GA). The results of our numerical simulations show that the QRO-GA yields better results than benchmark methods and it is highly efficient, with the convergence speed significantly increasing with the total qubit allocation. Convergence is achieved in as few as 5 iterations for higher qubit allocation. Moreover, the results reveal a critical U-shaped tradeoff, proving that an optimal latency distribution within a strict $80\ μ$s transmission window is strictly required to maintain HRLLC reliability. In addition, we find that the QRO-GA successfully optimizes the 2-D UAV positioning to minimize total decoding error. Finally, it balances consumed and harvested power to achieve net-zero power UAVs system.
comment: 5 pages. Published in IEEE Wireless Communications Letters. This work was supported by the Postdoctoral Research Fellowship from the Fonds de recherche du Québec (FRQ) under Grant 368613
★ Exploiting the Flexibility of LLM Inference for Frequency Regulation Service
While the widespread deployment of Large Language Models (LLMs) has significantly driven up power demands in computing clusters, the unique execution characteristics of LLM inference offer a novel opportunity to utilize these clusters as controllable loads. In this paper, we investigate the potential of LLM inference cluster to provide frequency regulation services through inference request scheduling. First, we model the problem as a hierarchical Markov decision process that includes request-to-replica dispatching and token-level resource allocation. Then, we introduce a reward redistribution network to address the temporal asynchrony between millisecond-level scheduling decisions and second-level frequency regulation signals. Furthermore, we develop an optimization framework based on a mean-field approximation to enable coordination among a large number of replicas under high request concurrency. Simulations using real-world data demonstrate that the proposed approach accurately tracks frequency regulation signals without compromising service-level objectives.
★ QIEN: Quantum-Inspired Energy-Neutral UAV Trajectory for WPT-Enabled Industrial HRLLC
This paper introduces Quantum-Inspired Energy-Neutral (QIEN) UAV Trajectory for WPT-Enabled Industrial HRLLC, a novel framework for mobility-integrated trajectory and resource optimization in solar-powered uncrewed aerial vehicle (UAV) relay systems supporting autonomous mining robots in post-disaster environments. We formulate a mixed-integer nonlinear program (MINLP) that jointly optimizes UAV trajectory, finite blocklength allocation, transmit power, and wireless power transfer (WPT) over discrete time horizons to maximize aggregate data throughput. The system operates under strict extreme hyper-reliable low-latency communication (HRLLC) constraints, energy sustainability requirements via solar harvesting, and continuous robot power delivery. To solve this large-scale optimization problem, QIEN employs a hybrid Cooperative Quantum-behaved Particle Swarm Optimization (Co-QPSO) algorithm using quantum tunneling mechanisms and a heterogeneous swarm architecture. Our simulation results demonstrate that the proposed QIEN framework significantly outperforms benchmark algorithms, achieving successful convergence while maintaining energy-neutral operation and satisfying all reliability, power, and mobility constraints.
comment: 14 pages. Accepted for publication in IEEE Transactions on Vehicular Technology. This work was supported by the Postdoctoral Research Fellowship from the Fonds de recherche du Québec (FRQ) under Grant 368613
★ Diversifying Moving Target Defense Against Adaptive Attacks Through Factorized Diffusion
Proactive defense offers a means to enhance power-system cybersecurity through controlled modifications of system configurations. One example is moving target defense (MTD), which uses distributed flexible AC transmission system (D-FACTS) devices to perturb transmission-line reactances against stealthy false data injection attacks. Existing MTD methods primarily rely on deterministic optimization, producing a single perturbation plan for each operating condition. However, repeated or highly correlated plans may expose predictable system configurations that attackers can learn and exploit. To address this limitation, this paper reformulates MTD as a conditional generative problem that produces diverse, secure, and physically feasible perturbation candidates. Learning such a distribution is challenging because MTD decisions jointly involve selecting branches, determining perturbation magnitudes, and coordinating adjustments across the network. To make this problem tractable, we factorize the decision distribution into three sequential conditional stages: branch selection, magnitude allocation, and continuous refinement. Graph-conditioned diffusion models are employed to capture the discrete and continuous decisions while accounting for network dependencies. Furthermore, a - objective is incorporated into diffusion training to favor candidates with desirable defense effectiveness and operational costs. Numerical studies on the IEEE 118- and 300-bus systems show that the proposed framework achieves AC feasibility comparable to that of optimization-based MTD baselines and higher - values, while reducing online computation time by an order of magnitude. The generated candidate pool remains effective under changing operating conditions and device availability, providing diverse high-value alternatives for flexible online cyber defense.
comment: 10 pages, 5 figures
★ ConQUESTA: Topology-Aware Post-Training Structural Pruning of Quantum Neural Networks ISCA
Deploying quantum neural networks (QNNs) on sparse hardware requires preserving task accuracy while reducing compiler-induced two-qubit gates, SWAPs, and depth. We present ConQUESTA (\underline{C}onstrained \underline{Qu}antum \underline{E}ntangler \underline{S}election with \underline{T}opology \underline{A}wareness), a post training pruning framework that combines task saliency with compiler cost, screens candidate entanglers, and searches complete pruning masks under an explicit accuracy constraint. Unlike one-shot ranking, each mask is transpiled as a whole across multiple coupling topologies. Across 4-, 6-, and 8-qubit QNNs and eight model seeds at 55% pruning, ConQUESTA achieves mean resource reductions of 61.6% on Breast Cancer, 72.0% on Digits01, and 82.2% on Iris, exceeding the strongest matched simple baseline by 3.5, 1.7, and 8.9 percentage points, respectively. Fixed-mask recovery raises feasibility from 16/24 to 22/24 runs. In a fixed-k=4 scaling study on a scaling probe of 4 to 12-qubits, the raw mask space grew from 495 to 12.1M while the evaluated masks increase only from 189 to 305. Resource-weight changes alter depth/routing priorities with at most a 0.44-point mean validation shift.
comment: Submitted to IEEE ISCAS 2027 for Review
★ Distributed Performance Boosting through Neural Control for Plug-and-Play Networks
We address the problem of improving the performance of pre-stabilized networked systems subject to plug-and-play (PnP) operations, namely additions or removals of subsystems. While distributed baseline controllers can preserve stability after a PnP transition, augmenting them with expressive neural policies can invalidate the formal stability guarantees. To overcome this issue, we build on recently proposed Performance Boosting controllers based on the Internal Model Control (IMC) principle, which reduce closed-loop stability to a finite $\ell_2$-gain requirement on an auxiliary operator. In the distributed setting, this operator is formed by interconnecting local neural operators that exchange information only between neighboring subsystems. We then derive free parametrizations of this interconnection, namely explicit maps from unconstrained trainable variables to interconnected neural operators satisfying a prescribed network gain bound for every parameter value. The obtained distributed neural controller can therefore be trained by unconstrained gradient-based optimization without stability constraints or projection steps, while retaining the prescribed gain bound at every training iterate. We develop two complementary parametrizations. A neighbor-based compositional parametrization uses local coupling bounds, so that after a PnP event only subsystems whose bounds change need to update their gain parameters. A network-norm parametrization instead uses a bound on the overall coupling matrix and can reduce conservatism, at the price of requiring network-wide information. A PnP DC-microgrid case study shows that the proposed neural layer improves voltage transients under topology changes and load variations while preserving nominal closed-loop stability.
★ Characterization, ageing and post-mortem dataset of commercial 58Ah prismatic NMC cells
This paper provides an extensive dataset for commercial 58 Ah prismatic NMC cells. The dataset comprises three types of data: characterization, ageing, and cell teardown. Characterization data include tests such as GITT, HPPC, and EIS, among others. Ageing data were obtained from 41 cells subjected to 38 distinct ageing conditions, including application-surrogate drive cycles. Recurring reference performance tests and EIS measurements track the evolution of each cell. Teardown data were obtained from pristine and aged (post-mortem) cells. They include material analyses and half-cell measurements. This single dataset supports analytics, model development (both machine-learning and physics-based), benchmarking, and techno-economic analysis. Several example analyses are carried out to show the usability of the data. For instance, the paper shows how lithium stripping can be detected from the voltage response of a short discharge applied after each charge. The data and the processing and analysis scripts are openly provided alongside this paper.
★ Stochastic Shockwave Dynamics in First-Order Traffic Flow
Moving congestion fronts vary across realizations even when the traffic-flow law is fixed. This paper develops a generator-based formulation of the Lighthill--Whitham--Richards (LWR) model with random initial density and a deterministic fundamental diagram. Randomness enters through the initial spatial density field; every realization follows the same conservation law. For a smooth strictly concave fundamental diagram, we adapt established one-sided spatial Markov closure to traffic variables under the stated spatial-process and regularity assumptions, and derive a kinetic balance for physically normalized front intensities. Evolving the spatial generator provides density laws and congestion-front statistics without solving a road-grid problem for every realization. The formulation is continuous in physical space and time, while its numerical implementation approximates density state and integrates physical time. Four experiments distinguish deterministic consistency from stochastic comparison. Riemann and signal examples recover independently constructed exact LWR solutions. Smooth strictly concave approximations approach the deterministic triangular-diagram signal solution over the tested finite horizons; this does not establish stochastic closure at contacts or the capacity kink. A matched Monte Carlo Godunov ensemble evaluates front counts, sizes, mergers, and density distributions. The evolved generator also yields an instantaneous travel-time distribution for a frozen density field, compared with the same functional of the matched Godunov profiles. This quantity is not experienced travel time through evolving traffic. Resolution checks identify remaining normalization and small-front-loss effects. The contribution is direct evolution and interpretation of traffic-state and front distributions under a fixed first-order traffic law.
★ Converter-Aware Coordination of Computing Flexibility in Behind-the-Meter Data Centers
Data centers are emerging as large electricity consumers whose demand can be reshaped through workload scheduling. However, computing flexibility in behind-the-meter facilities does not automatically translate into electrically realizable power flexibility. Ignoring converter and network operating limits may cause converter overloads, protection trips, and interruptions to power supply and computing services. This paper proposes a converter-aware intraday scheduling methodology. First, a task-level mixed-integer linear formulation preserves synchronous training configurations and completion requirements, while Minkowski-sum aggregation and Shapley--Folkman convexification yield a linear inference-pool representation without unit-level binary variables. The latter admits a conditional relative relaxation-gap bound that decreases inversely with pool size. Second, a four-node electrical model links computing demand, on-site injections, and grid exchange to converter and network limits through lifted voltage variables and second-order cone relaxation. Finally, a mixed-integer model coordinates workloads, photovoltaic generation, energy storage, and grid purchases. Case studies show that electrical limits reshape scheduling and renewable utilization, while on-site resources reduce grid dependence. Under the studied conditions, fixed training cannot support full inference service within voltage limits, whereas coordinated scheduling achieves negligible inference shortage and passes verification against the original alternating-current model. These findings demonstrate the role of computing--electrical coordination in realizing the operational value of workload flexibility.
★ Data Center Load Accommodation under QoS and Checkpoint-Induced Frequency Security Constraints
The rapid growth of artificial intelligence workloads is accelerating the expansion and grid interconnection of data centers (DCs). Existing data center load accommodation (DCLA) assessments primarily focus on grid and resource adequacy and often imply that greater workload flexibility always increases accommodable capacity. However, workload flexibility has a dual effect. Workload deferral and temporal shifting can alleviate network congestion, whereas synchronized checkpointing of training workloads can cause large and rapid load reductions, threatening frequency stability. Therefore, DCLA cannot be determined solely by grid adequacy. This paper develops an optimal DCLA method that jointly accounts for workload quality of service (QoS), checkpoint-induced frequency security (FS), and grid adequacy. First, a QoS-aware model characterizes workload-dependent flexibility while ensuring service feasibility. Second, checkpoint-induced FS constraints are developed for periodic and event-triggered power variations, with key frequency-security limits reformulated in algebraic form. Third, a two-stage lexicographic model is developed to maximize DCLA and minimizes the required energy storage (ES) deployment, and then optimizes system operation under the given DC and ES siting and sizing decisions. Case studies show that the proposed method avoids a 40.3% overestimation of DCLA compared with the grid-adequacy benchmark. QoS is the primary limiting factor, while checkpoint-induced frequency-security constraints become more restrictive as training workloads increase. These findings highlight that reliable DCLA should account for both its grid-support value and the associated QoS and FS risks.
comment: 10 pages
★ Hybrid Cones: Set Representation with Sharp Unions of Minimal Overhead for Nonlinear and Switched Linear Systems
Reachability analysis and set-based estimation for nonlinear systems require set representations capable of capturing nonconvex geometry. Hybrid Zonotopes (HZs) have recently gained attention because they can encode a union of up to $2^n$ polytopic components. Recent work has further shown that several HZ operations can be made \textit{sharp}, meaning that relaxing the binary variables yields exactly the convex hull of the represented set. However, no known HZ construction provides a sharp union of two convex sets that adds only one binary generator and no further continuous generators or equality constraints to those of the operands. This limitation affects scalability in repeated union operations, ReLU neural-network reachability, and nonlinear propagation methods that cover graph sets by unions of simpler sets. This paper identifies a connection between the exact closed-form convex-hull operation for Constrained Convex Generators (CCGs) and the binary-generator structure underlying HZs. We propose Hybrid Cones that can encode up to $2^n$ norm-cone components. Hybrid Cones inherit the logical structure of HZs while allowing richer convex components, including polytopic and norm-based sets. They support sharp unions requiring only one added binary variable and can represent unions and intersections of polytopes and ellipsoids. As a key application, we derive an exact representation of the ReLU graph using only two continuous and one binary generators. Numerical experiments for a localization problem with a nonlinear energy measurement function approximated by a neural network and the stability verification of the closed loop of a double integrator corrupted by a nonlinear disturbance with a saturated Linear Quadratic Regulator (LQR) controller show substantial reduction in representation size and reduced conservatism in comparison with HZs.
comment: 19 pages
★ Filter-Augmented Reinforcement Learning for Battery Fast Charging with State of Health and Plating-Aware Reward Design
Fast charging of lithium-ion batteries is crucial for electric vehicle adoption, yet it increases safety risks and accelerates battery ageing. To address this, we propose a filter-augmented reinforcement learning (RL) framework for lifelong health-aware fast charging, based on the twin-delayed deep deterministic policy gradient (TD3) algorithm. The RL agent is guided by a reward function that balances charging speed and battery longevity through the incorporation of state of health (SoH) information and an SoH-dependent voltage penalty term derived from a constant--current constant--overpotential charging strategy. Since onboard SoH estimates are inevitably corrupted by noise, two filtering strategies are integrated: an exponential moving average (EMA) filter when noise statistics are unknown, and a Kalman filter when they are available, with the filtered estimate embedded in both the agent's state and reward computation. The RL policy is trained using data from high-fidelity life-cycle simulations. The evaluation uses a published electrochemical and degradation parameterisation for a commercial LG M50 cell. Results show that the proposed method achieves up to 21.4\% more equivalent full cycles (EFCs) than conventional constant current--constant voltage (CC--CV) charging under 2\% SoH estimation noise, and maintains this lifetime advantage even under 5\% noise, while sustaining comparable charging speed.
★ Lifting Safety Through Reduced-Order Model Architectures: An Invariant Manifold Approach
Reduced-order model architectures are often used to decompose complex safety-critical control design tasks into tractable subproblems. In this paradigm, one must ask: when does the safety of a reduced-order model lift to safety of the full system? Taking an invariant manifold perspective, we provide necessary and sufficient conditions for the existence of smooth, regular lifts of compact safe sets from reduced-order models. Our characterization relies only on asymptotic stability of an embedding of a reduced-order safe set, without additional robustness or rate conditions. We apply these results to study control barrier functions in reduced-order model architectures and lifts of safe sets through cascades. Further, we use our conditions to obtain topological obstructions to the tracking of safe, kinematic reduced-order models for mechanical systems. We conclude by using these obstructions to inform safe hierarchical control design for an underactuated satellite on SE(3).
comment: 8 pages, 2 figures, Submitted to the American Control Conference (ACC) 2027
★ Switch-Time-Aware Hybrid Model Predictive Control Using the Hybrid Minimum Principle
This article presents a time-aware hybrid model predictive control (MPC) framework that simultaneously optimizes the objective with respect to both continuous control inputs and event switching times. Rather than relying on a fixed time grid, the switching instant is lifted into the decision vector and treated as an explicit optimization variable. The resulting transcription is a smooth nonlinear program whose Karush-Kuhn-Tucker (KKT) conditions are consistent with the Hybrid Minimum Principle (HMP) event-time stationarity condition, so first-order hybrid sensitivities are recovered implicitly through the KKT system rather than by evaluating saltation matrices. Each MPC step solves a single sparse Quadratic Program (QP), jointly optimizing control inputs and impact timing in real time. On a bouncing mass landing on an oscillating platform, the method attaches in half the time of fixed-grid saltation MPC, reduces cumulative control effort by 30%, and removes the post-impact rebounds, at a 15% increase in per-cycle solve time. It matches the cumulative effort of offline time-aware solvers to within 4% while running at 2.4 ms per cycle against 281 ms for a complementarity-based formulation of the same problem, of which a 10.6x reduction is attributable to the transcription alone at a fixed solver.
comment: 8 pages; 4 figures, 2 tables
★ A Koopman--Polynomial Chaos Framework for SMPC of Nonlinear Stochastic Processes
This paper develops a polynomial parametric Koopman operator (PPKO) framework for stochastic model predictive control (SMPC) of nonlinear systems with parametric uncertainty and time-varying stochastic disturbances. The Koopman operator is parameterized by polynomial chaos expansions (PCEs), enabling data-driven nonlinear dynamics to be integrated with PCE-based uncertainty propagation and control. We show that the polynomial parameterization introduces no irreducible approximation gap beyond PCE truncation. We then derive a condensed deterministic SMPC reformulation that handles both time-invariant parametric uncertainty and time-varying disturbances while retaining an online optimization dimension that depends only on the control horizon and input dimension, independent of the lifted-state and PCE dimensions. The resulting formulation yields convex programs with expectation and second-moment constraints that can be solved using standard optimization methods. Numerical studies on the Duffing oscillator and the electrothermal cooling system demonstrate the accuracy, scalability, and closed-loop performance of the proposed framework.
comment: A preliminary version of this work will be presented at the 2026 IEEE Conference on Decision and Control (CDC)
♻ ★ Public Battery Field Data
Battery systems and algorithms are developed and evaluated primarily on laboratory cycling data, but fielded battery systems are subjected to duty cycles, temperatures, and measurement conditions that cannot be reproduced in a lab. Field data from batteries in service has begun to appear in public releases attached to individual papers, but each release has its own format, units, labels, and other gaps that make it difficult to compare against others. We present a curated collection of 35 public field-data releases from 27 research groups, covering applications that span passenger EVs, buses, light electric mobility, grid-scale storage, distributed storage, consumer electronics, industrial robotics and satellites. Where the releases are complete enough to measure, this heterogeneous collection contains 256 GB of data from at least 149,188 lithium-ion cells, 3,187 batteries, 115.6 MWh of energy storage capacity, and 3,040 unit-years of service. Critically, it also contains 6,465 labeled or reported faults, which provide greater visibility into what real battery performance and safety issues look like outside the lab. We release a public repository of metadata and code for each of the field datasets. For each dataset it provides a registry entry with sources, license and access links, checksums to confirm a download is complete, a metadata table, and code for how to load it. We also report where released datasets differ from their papers. We hope the collection can help serve researchers (human or otherwise) as a useful platform for improving battery field diagnostics, fault detection and health estimation. Better field evidence can also feed back to improve manufacturing quality, inform second-life screening and residual-value estimates, and broadly accelerate electrification.
comment: v3: 35 releases, matching v1.2.0 of the collection
♻ ★ Distributed Online Learning for Time-Critical Communication in 6G Industrial Subnetworks
Reliable and timely delivery of emergency alarms is critical for 6G industrial in-X subnetworks, particularly in dense and dynamic environments where multiple devices may simultaneously detect the same event and contend for limited radio resources. This scenario, known as medium access with a shared message, poses significant challenges for conventional random access protocols due to collisions and stringent trans- mission deadlines. To address this challenge, this paper proposes a distributed deep contextual-bandit-based random access pro- tocol that minimizes the probability of alarm delivery deadline violations. The proposed protocol enables each local access point (LAP) to autonomously learn its transmission strategy through a lightweight deep neural network (DNN), without requiring knowledge of the number of active LAPs or explicit inter-LAP coordination. A contention-signature acquisition mechanism is introduced to provide contextual information about network contention, enabling distributed and adaptive channel-access decisions. Moreover, an online learning mechanism allows LAPs to continuously refine their transmission strategies based on feedback from previous alarm transmissions. Simulation results demonstrate that the proposed protocol outperforms benchmark random access schemes, particularly under high-contention con- ditions, improving the probability of timely alarm delivery by at least 8% with 30 subnetworks and up to 21% with 60 subnetworks.
♻ ★ Accelerated Gradient Methods for Nonconvex Optimization: Escape Trajectories From Strict Saddle Points and Convergence to Local Minima
This paper considers the problem of understanding the behavior of a general class of accelerated gradient methods on smooth nonconvex functions. Motivated by some recent works that have proposed effective algorithms, based on Polyak's heavy ball method and the Nesterov accelerated gradient method, to achieve convergence to a local minimum of nonconvex functions, this work proposes a broad class of Nesterov-type accelerated methods and puts forth a rigorous study of these methods encompassing the escape from saddle points and convergence to local minima through both an asymptotic and a non-asymptotic analysis. In the asymptotic regime, this paper answers an open question of whether Nesterov's accelerated gradient method (NAG) with variable momentum parameter avoids strict saddle points almost surely. This work also develops two metrics of asymptotic rates of convergence and divergence, and evaluates these two metrics for several popular standard accelerated methods such as the NAG and Nesterov's accelerated gradient with constant momentum (NCM) near strict saddle points. In the non-asymptotic regime, this work provides an analysis that leads to the "linear" exit time estimates from strict saddle neighborhoods for trajectories of these accelerated methods as well the necessary conditions for the existence of such trajectories. Finally, this work studies a sub-class of accelerated methods that can converge in convex neighborhoods of nonconvex functions with a near optimal rate to a local minimum and at the same time this sub-class offers superior saddle-escape behavior compared to that of NAG.
comment: 124 pages, 9 figures; adds a short clarification to the proof of Theorem 7.7 and incorporates a proof-stage typo fix; includes additional corrections and clarifications to the version published in Foundations of Computational Mathematics, April 2026
♻ ★ Vector-field-guided constraint-following control for path following of uncertain mechanical systems
Achieving dynamics-level control is a critical yet challenging problem for accurate, high-speed geometric path-following of uncertain robotic systems. This paper proposes a vector-field-guided constraint-following control framework for geometric path following of uncertain mechanical systems. The key idea is to encode vector-field guidance as a velocity constraint, enabling a unified force/torque control design for a class of fully actuated and underactuated systems with desired paths that may be self-intersecting. A global, explicit error estimate connects constraint-following accuracy to path-following performance with convergence guarantees. An adaptive robust extension further addresses heterogeneous time-varying uncertainties with unknown bounds and ensures ultimately bounded path-following error. Applications to an underactuated aircraft system and a fully actuated manipulator system demonstrate the effectiveness and advantages of the framework.
♻ ★ Enabling Tidying Tasks in Robotic Vacuum Cleaners via a Geometry-Induced Self-Locking Deployable Linkage
Robotic vacuum cleaners must extend their functionality within tight limits on size, cost, and durability. Deployable linkages with few actuators meet these limits, but their backdrivability makes it difficult to hold a load-bearing posture without continuous motor torque. We introduce a two-stage deployable linkage that locks through its own geometry. At full deployment, an internal three-link subchain approaches a collinear, near-singular configuration in which the input-output transmission ratio tends to zero; plate loads therefore leave only a small residual back-driving torque, which drive-train friction and a lightweight magnetic detent absorb. A single actuator performs deployment and locking, a passive chassis-mounted trigger releases the lock for retraction, and a second actuator adjusts only the deployment height. We integrate the mechanism into a commercial robotic vacuum cleaner and validate it through a 100-cycle repeatability test, a quasi-static load test in which the locked plate held 50 N with the motor de-energized, and 140 pushing-type tidying trials on seven household objects against a commercial arm-equipped vacuum robot. We derive the analytical holding condition and discuss its limits as an intermediate design point between passive compliance and articulated manipulation.
comment: 8 pages, 6 figures, 1 table. v2: corrected author name, added minor revisions
♻ ★ Teleoperation Operational Design Domain based on Minimal Risk Maneuver Capability
This article discusses the concept of an Operational Design Domain (ODD) designed specifically for teleoperated road vehicles. For this purpose, the ODD concept designed for automated driving is adapted for teleoperation. As teleoperation becomes more common in regular traffic, the question arises under which operating conditions such vehicles are able and allowed to drive. Currently, these conditions are selected primarily based on network performance. From a safety perspective, it is difficult to base such a selection on a reliable connection because it is almost impossible to guarantee sufficient reliability. With this in mind, the ODD concept designed for automated driving is adapted for teleoperation: A concept is proposed for basing the ODD for a teleoperation system on the capability of the teleoperated vehicle to perform a minimal risk maneuver using a dedicated system designed solely for this purpose. This concept is then demonstrated using a use case example.
comment: This work has been submitted to Elsevier Reliability Engineering & System Safety for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible
♻ ★ A flexible language model-assisted electronic design automation framework
Large language models (LLMs) are transforming electronic design automation (EDA) by enhancing design stages such as schematic design, simulation, netlist synthesis, and place-and-route. Existing methods primarily focus these optimisations within isolated open-source EDA tools and often lack the flexibility to handle multiple domains, such as analogue, digital, and radio-frequency design. In contrast, modern systems require to interface with commercial EDA environments, adhere to tool-specific operation rules, and incorporate feedback from design outcomes while supporting diverse design flows. We propose a versatile framework that uses LLMs to generate files compatible with commercial EDA tools and optimise designs using power-performance-area reports. This is accomplished by guiding the LLMs with tool constraints and feedback from design outputs to meet tool requirements and user specifications. The framework is used to evaluate case studies on operational transconductance amplifiers, microstrip patch antennas, and FPGA circuits.
comment: 17 pages, 5 figures, 1 Supplementary (21 pages, 13 figures, 14 tables)
♻ ★ Residential Peak Load Reduction via Direct Load Control under Limited Information
Thermostatically controlled loads and electric vehicles offer flexibility to reduce power peaks in low-voltage distribution networks. This flexibility can be maximized if the devices are coordinated centrally, given some level of information about the controlled devices. In this paper, we propose novel optimization-based control schemes with prediction capabilities that utilize limited information from heat pumps, electric water heaters, and electric vehicles. The objective is to flatten the total load curve seen by the distribution transformer by restricting the times at which the available flexible loads are allowed to operate, subject to the flexibility constraints of the loads to preserve customers' comfort. The original scheme was tested in a real-world setup, considering both summer and winter days. The pilot results confirmed the technical feasibility but also informed the design of an improved version of the controller. Computer simulations using the adjusted controller show that, compared to the original formulation, the improved scheme achieves greater peak reductions in summer. Additionally, comparisons were made with an ideal controller, which assumes perfect knowledge of the inflexible load profile, the models of the controlled devices, the hot water and space heating demand, and future electric vehicle charging sessions. The proposed scheme with limited information achieves 43.3 % and 27.8 % of the potential average daily peak reduction that the ideal controller with perfect knowledge would achieve in summer and winter, respectively.
comment: Accepted for publication in IEEE Transactions on Smart Grid
♻ ★ Prospect Theoretic Approach to Pursuit-evasion Differential Games with Risk Aversion and Probability Sensitivity
This paper considers pursuit-evasion (PE) differential games in which the pursuer's and evader's irrational perceptions of environmental uncertainty lead to deviations from rational controllers. Firstly, the equivalent irrational behavioral models of risk aversion and probability sensitivity are established using Cumulative Prospect Theory (CPT). Secondly, sufficient conditions for capturability are established in terms of irrationality parameters. Finally, the CPT-Nash equilibrium is analyzed by Brouwer's fixed-point and self-consistency theorem. The new results reveal that irrational behaviors benefit the pursuer in some cases and the evader in others. Certain captures that are unachievable under rational behaviors can be achieved under irrational ones. By bridging irrational behavior with game-based control, a theoretical foundation is established for human-machine control systems.
♻ ★ Regularization in Data-driven Predictive Control: A Convex Relaxation Perspective
This paper explores the role of regularization in data-driven predictive control (DDPC) through the lens of convex relaxation. Using a bi-level optimization framework, we model system identification as an inner problem and predictive control as an outer problem. Within this framework, we show that several regularized DDPC formulations, including l1-norm penalties, projection-based regularizers, and a newly introduced causality-based regularizer, can be viewed as convex relaxations of their respective bi-level problems. This perspective clarifies the conceptual links between direct and indirect data-driven control and highlights how regularization implicitly enforces system identification. We further propose an optimality-based variant, A-DDPC, which approximately solves the inner problem with all identification constraints via an iterative algorithm. Numerical experiments demonstrate that A-DDPC outperforms existing regularized DDPC by reducing both bias and variance errors. These results indicate that further benefits may be obtained by applying system identification techniques to pre-process the trajectory library in nonlinear settings. Overall, our analysis contributes to a unified convex relaxation view of regularization in DDPC and sheds light on its strong empirical performance beyond linear time-invariant systems.
♻ ★ Adverse Effects of V2V Adoption on Road Safety
Vehicle-to-vehicle (V2V) communication is expected to improve road safety and reduce congestion. However, prior work shows that V2V information sharing under partial adoption may increase congestion and decrease safety. We study whether increasing V2V adoption itself affects road safety. We propose a corrected version of an existing model and analyze its behavior under varying adoption levels. We show that, in some cases, increased V2V adoption can increase accident probability. However, under an optimal signaling policy, the system can ensure that accident probability is non-increasing in the adoption level.
comment: 9 pages, 3 figures. Revised following peer review, with clarifications and presentation improvements. Extended version with complete proofs. An abbreviated version has been accepted for publication in the proceedings of IFAC CPHS 2026
♻ ★ AnyRank: Reliable Anytime Policy Ranking via Sequential Testing
Due to the high cost of hardware experiments, robot policies are predominantly evaluated by comparing their empirical performance on a limited number of hardware trials. However, finite-sample statistical uncertainty can lead to erroneous conclusions, especially when ranking multiple policies using a fixed batch of evaluation data. We propose AnyRank: a sequential method for multi-policy ranking that (i) adaptively allocates trials to hard-to-separate policies, (ii) provides (partial) rankings at any stopping time while bounding the probability of an incorrect ranking, and (iii) optionally exploits prior information to prioritize hardware evaluations more efficiently. This prior information could be informed from expert intuition, a small set of held-out hardware evaluations, or from policy rollouts in a simulator or an action-conditioned video world model. Notably, the prior can be imperfect; while a strong prior can provide efficiency gains on hardware, any inaccuracies in prior policy rankings do not invalidate the statistical assurances of AnyRank. We achieve the aforementioned properties through novel theoretical contributions that combine sequential testing with multiple-hypothesis graphical testing approaches. Experiments in simulation and real-world policy evaluation benchmarks show that AnyRank requires up to 50% fewer hardware evaluations compared to existing statistically rigorous approaches, while guarding against the incorrect rankings produced by heuristic methods used by state-of-the-art benchmarks.
comment: CoRL 2026
♻ ★ Safe Driving in Occluded Environments
Ensuring safe autonomous driving in the presence of occlusions poses a significant challenge in its policy design. While existing model-driven control techniques based on set invariance can handle visible risks, occlusions create latent risks in which safety-critical states are not observable. Data-driven techniques also struggle to handle latent risks because direct mappings from risk-critical objects in sensor inputs to safe actions cannot be learned without visible risk-critical objects. Motivated by these challenges, in this paper, we propose a probabilistic safety certificate for latent risk. Our key technical enabler is the application of probabilistic invariance: It relaxes the strict observability requirements imposed by set-invariance methods that demand the knowledge of risk-critical states. The proposed techniques provide linear action constraints that confine the latent risk probability within tolerance. Such constraints can be integrated into model predictive controllers or embedded in data-driven policies to mitigate latent risks. The proposed method is tested using the CARLA simulator and compared with a few existing techniques. The theoretical and empirical analysis jointly demonstrate that the proposed methods assure long-term safety in real-time control in occluded environments without being overly conservative and with transparency to exposed risks.
♻ ★ A Control Co-Design Framework to Optimize Sustainability with Application to Microgrid-Driven Data Centers
This work studies the optimization of physical plant characteristics and controller parameters for sustainability. Environmental sustainability is strongly correlated with the development and operation of energy systems, with data centers as the preeminent modern example. Data centers, and the grid technologies that provide their power, consume nonnegligible amounts of global energy and pose a risk to increase greenhouse gas (GHG) emissions and electronic waste. Addressing such issues requires improved plant design and control strategies, often approached through optimization-based methods. However, there are noticeable gaps regarding data center and microgrid design: (i) simultaneous optimization of plant and controller features is rarely explored, and (ii) sustainability criteria are not emphasized. This work addresses these gaps by establishing a generalized, sustainability-centric control co-design (CCD) framework for energy systems. Uniquely, the CCD framework defines and categorizes sustainability metrics into three lifecycle stages - manufacturing, operation, and disposal - supporting optimization and comparative analysis of the metrics for different design options. To exemplify its use, the CCD framework is applied to a microgrid-driven data center system, providing a family of sustainability metrics correlated to plant and controller parameters. The analysis enabled by the framework supports the determination of tradeoffs between different types of environmental sustainability metrics. The system designs identified by the proposed framework show substantial improvements to environmental sustainability categories as compared to designs identified through baseline procedures.
♻ ★ When is cumulative dose response monotonic? Analysis of incoherent feedforward motifs
We study the monotonicity of the cumulative dose response (cDR) for a class of incoherent feedforward motifs (IFFM) systems with linear intermediate dynamics and nonlinear output dynamics. While the instantaneous dose response (DR) may be nonmonotone with respect to the input, the cDR can still be monotone. To analyze this phenomenon, we derive an integral representation of the sensitivity of cDR with respect to the input and establish general sufficient conditions for both monotonicity and nonmonotonicity. These results reduce the problem to verifying qualitative sign properties along system trajectories. The structural conditions identify eight canonical IFFM models, four of which are analyzed in detail within the proposed framework. In particular, IFFM1 and IFFM3 exhibit monotone cDR despite potentially nonmonotone DR, while IFFM2 is monotone already at the level of DR, which implies monotonicity of cDR. In contrast, IFFM4 exhibits nonmonotone cDR. Numerical simulations indicate that these properties persist beyond the structured initial conditions used in the analysis. Overall, our results provide a unified framework for understanding how system structure shapes monotonicity in cumulative input--output responses.
comment: A shorter version of this work is to appear in the proceedings of the 2026 IEEE Conference on Decision and Control