Embodied AI Housework Robot
Built a mobile robot learning system with arm-based teleoperation and real-robot Diffusion Policy deployment. Achieved ~50% success on vision-only nut pick-and-place; now studying visual–tactile feedback.
Background
Research assistant at Huaxi Jingchuang Medical Technology (SenseTime–West China Hospital Joint Lab), building a mobile manipulation system from demonstration collection through policy training and real-robot execution, with a focus on visual–tactile feedback for fine assembly.
What I Did
Independently integrated a mobile base, 7-DoF arm, gripper, RGB fisheye and depth cameras, Jetson control node, and inference server. Designed a lerobot-Uarm arm with matching kinematics for leader–follower TeleOp data collection, then completed Diffusion Policy training, inference, and real-robot deployment. Achieved approximately 50% success with a vision-only policy on picking up a nut and placing it on top of a screw. Built bilateral 3×3 FSR tactile arrays to record contact pressure during grasping.
Challenges
The vision-only policy could locate the target but could not reliably detect nut slippage, unstable grasping, or small offsets after contact. Missing contact feedback became the main bottleneck for more stable fine assembly.
Takeaways
Completed the vision-only Diffusion Policy nut pick-and-place experiments. Research into tactile contact and slip detection and action correction is ongoing; the effects of adding tactile feedback are still being evaluated.
Gallery
