Research

2026

  1.
    MuscleMan: Humanoid Whole-Body Control for Heavy-Object Prehensile Interaction
    Haoyang He, Jay Karhade, Karan Khanna, Jinyao Zhou, Abhishek Iyer, Sebastian Scherer, Xue Bin Peng, Jay Patrikar, Dong-Ki Kim, and Shayegan Omidshafiei
    CoRL 2026 Workshop on Contact-Rich Loco-Manipulation

    Olympic-style clean-and-jerk with 45lbs+ (20.4kg) on G1 with Dex3 hands rated for 500g each.

    Website
  2. Retargeting Through Reasoning for Humanoid–Object Interaction
    Narayanan Palghat Parameswaran, Jay Karhade, Haoyang He, Neehar Peri, Jay Patrikar, Dong-Ki Kim, Shayegan Omidshafiei, Sebastian Scherer, and Deva Ramanan
    Project Blog

    An agentic approach to retargeting built around reasoning about how to perform the interaction and querying the environment for evidence to guide refinement.

    Website
  3. ProprioTrack3R: 3D Point Tracking with Humanoid Proprioception
    Jay Karhade, Haoyang He, Jinyao Zhou, Kabir Kedia, Guilherme Marcelo, Mahsa Raeisinezhad, Jay Patrikar, Dong-Ki Kim, Deva Ramanan, Sebastian Scherer, and Shayegan Omidshafiei
    Coming Soon

    Tracking 3D object motion through humanoid proprioception, even when objects are occluded or out of view.

2025

  1.
    GrndCtrl: Grounding World Models via Self-Supervised Reward Alignment
    Haoyang He, Jay Patrikar, Dong-Ki Kim, Max Smith, Daniel McGann, Ali-akbar Agha-mohammadi, Shayegan Omidshafiei, and Sebastian Scherer
    IROS 2026

    Bridge generative pretraining of World Models and grounded behavior with verifiable rewards, improving counterfactual generation.

    arXiv Website
  2. RayFronts: Open-Set Semantic Ray Frontiers for Online Scene Understanding and Exploration
    Omar Alama, Avigyan Bhattacharya, Haoyang He , Seungchan Kim, Yuheng Qiu, Wenshan Wang, Cherie Ho, Nikhil Keetha, and Sebastian Scherer
    IROS 2025

    Real-time semantic mapping system that enables fine-grained scene understanding within and beyond the depth perception range.

    arXiv Video Website