Projects

2026

  1. FieldAI Humanoid Project
    Demos and Deployments

    Training humanoid whole-body loco-manipulation for real-world use cases, and building safe policy deployment in uncertain environments.

2025

  1. AnyWill: A Human-Interactive Autonomous Wheelchair for Outdoor Urban Navigation
    MRSD Capstone Project 2025

    User-interactive autonomous wheelchair designed for end-to-end mobility on urban sidewalks while respecting individual preferences.

2024

  1. alfredanalysis.png
    Multimodal Analysis of Embodied Instruction Following on ALFRED
    Course Project: CMU 11-777 Multimodal Machine Learning

    Evaluating how updated foundation models like DepthAnything and MaskDINO affect embodied instruction following on ALFRED.

  2. scenegaussian.png
    SceneGaussian: Unconstrained Generation of 3D Gaussian Splatting Scenes
    Hanzhe Hu, Qin Han, and Haoyang He
    Course Project: CMU 16-825 Learning for 3D Vision

    Generating consistent 3D Gaussian splatting scenes from a single image along an arbitrary camera trajectory.

  3. naturalpianist.png
    Natural Dexterous Piano Playing at Scale With Video Hand Priors
    Jeff Tan , Yuanhao Wang, and Haoyang He
    Course Project: CMU 16-831 Introduction to Robot Learning

    Learning robotic piano playing from YouTube videos, using hand pose estimation and music transcription to label fingering.

2023

  1. offlinerl.png
    A Survey on Offline Model-Based Reinforcement Learning
    Haoyang He
    Course Project: CMU 18-813B Foundations of Reinforcement Learning

    Surveying recent offline model-based reinforcement learning methods and how they address distributional shift.

2022

  1. gasnext.png
    GAS-NeXt: Few-Shot Cross-Lingual Font Generator
    Haoyang He*, Xin Jin*, and Angela Chen*
    Course Project: CMU 11-785 Introduction to Deep Learning

    Few-shot cross-lingual font generation, translating font styles across languages such as English to Chinese.

  2. eyeinhand.png
    Automatic Eye-in-Hand Calibration using EKF
    Aditya Ramakrishnan*, Chinmay Garg*, Haoyang He*, Shravan Kumar Gulvadi*, and Sandeep Keshavegowda
    Course Project: CMU 16-833 Robot Localization and Mapping

    Automatically calibrating the camera-to-end-effector transform of an eye-in-hand robot arm with EKF-based SLAM.