AnchorQ maps movement directions to noise samples, refines them with critic scores, and decodes a movement to place bread on a plate.

A flow-matching policy turns a noise sample into a sequence of robot movements. AnchorQ runs that mapping in reverse to find inputs that correspond to a small set of reference movements in the current scene. A learned value function scores variations around each of those inputs and guides the choice of action. The pretrained policy stays frozen throughout training.

We measured both how often a policy finished a task and how much work it could do in an hour. Across the simulation tasks, AnchorQ averaged 294 completions per hour, compared with 149 for DSRL, a method that learns to steer the same policy directly in noise space.

Learning from 100 robot trials

Every time our Franka arm tried to hang a towel, someone had to score the attempt and reset the scene. We gave each reinforcement learning method 100 training attempts per task. AnchorQ could use those attempts to explore variations on movements such as shifting the gripper up or to the left.

We evaluated the trained policies on 20 held-out configurations each for stacking cups, toasting bread, and hanging a towel. AnchorQ outperformed both the original policy and DSRL on all three tasks. On towel hanging, AnchorQ was the only steering method that improved on the original policy.