Xiaoshan Wu 吴小山
I am a Visiting Scholar at Harvard University, working with Yilun Du and Mengyu Wang on 4D world models for robotic planning. I am in the fourth year of my Ph.D. at the University of Hong Kong, under the supervision of Xiaojuan Qi. I have been awarded the Hong Kong PhD Fellowship (HKPFS) and the HKU Presidential Scholarship (HKUPS). Previously, I received dual B.S. degrees in Computer Engineering from Zhejiang University and the University of Illinois Urbana-Champaign.
I am interested in world models for generalizable robotic planning: how robots can understand the physical world and use imagined futures to act in unfamiliar environments. My work makes geometry and motion explicit in these futures, connects them to robot actions, and develops the perception and memory needed to recover and retain useful evidence across observations. Looking ahead, I aim to explore agent–world model co-evolution, where interaction improves the world model, and the improved model supports better planning and policy learning.
News
- One paper accepted to NeurIPS 2026: UGG: Unified Generative Geometry Model.
- One paper accepted to CoRL 2026: OptiGeo.
- One paper accepted to ECCV 2026: FoundationGeo.
- I arrived at Harvard University as a Visiting Scholar, working with Yilun Du and Mengyu Wang.
- Our work on neural-field reconstruction using resistive memory was accepted by Nature.
- One paper accepted to CVPR 2026: DyFN for stable streaming video geometry.
- One paper accepted to ICLR 2026: LiFR-Seg.
- One paper accepted to NeurIPS 2025: EAG3R (Spotlight).
Research
View publications ↓What should a robot perceive, remember, and predict to act reliably in an unfamiliar environment?
4D world models for action
I develop models that generate visual futures together with corresponding robot actions. Making geometry and motion explicit lets the predicted future guide action generation and support consistency checks before execution.
Examples: S4NDBOX
Memory across views and time
Perception in changing environments
Publications
First- and Co-first-author Work
* Equal contribution · † Project lead.
Franka Research 3 · robot demo (10×) Verifiable 4D World–Action Model for Robust Robot Control
TL;DR: Explicit 4D futures connect visual prediction with robot control and support pre-execution geometric consistency checks.
Teaser GaMe: Geometry as Memory for Multi-View Spatial Reasoning
TL;DR: Geometric working memory brings multi-view spatial evidence into vision–language reasoning.
Teaser · Figure 1 EAG3R: Event-Augmented 3D Geometry Estimation for Dynamic and Extreme-Lighting Scenes
TL;DR: RGB–event fusion recovers scene geometry under fast motion and difficult lighting, including daytime-to-nighttime transfer.
Project overview VideoSSM: Autoregressive Long Video Generation with Hybrid State-Space Memory
TL;DR: Local attention and recurrent state-space memory retain recent detail and long-range context for autoregressive video generation.
Teaser LiFR-Seg: Anytime High-Frame-Rate Segmentation via Event-Guided Propagation
TL;DR: Event-guided propagation and temporal memory keep semantic predictions up to date between RGB frames, without requiring an image at each query time.
Method overview EVSplat: Uncertainty-Aware Gaussian Splatting for Dynamic Scenes with Event Camera
TL;DR: Uncertainty-aware Gaussian splatting reconstructs dynamic scenes from event-camera observations.
Teaser LiFR v2: Completion-Augmented Event Propagation for High-Rate Dense Prediction
TL;DR: Propagation reuses observed content, event-guided completion recovers missing source support, and memory retains inferred content for later queries.
Event-Based Depth Prediction With Deep Spiking Neural Network
TL;DR: A deep spiking neural network predicts scene depth from event-camera observations.
Collaborative Work
UGG: Unified Generative Geometry Model
TL;DR: A single generative model completes RGB video, depth, and camera geometry from partial observations.
Efficient and Accurate Neural-Field Reconstruction Using Resistive Memory
TL;DR: Compressed neural fields and resistive-memory computing enable efficient reconstruction from sparse observations.
OptiGeo: Efficient Monocular Geometry for Embodied Perception in Optically Challenging Scenes
TL;DR: Bias-aware monocular geometry estimation improves 3D perception of transparent and reflective objects.
FoundationGeo: Learning Spatial Pixel-Wise Fields for Monocular Metric Geometry
TL;DR: Pixel-wise spatial calibration turns relative geometry into metric 3D predictions across cameras and domains.
Stabilizing Streaming Video Geometry via Dynamic Feature Normalization
TL;DR: Dynamic feature normalization stabilizes scale and geometry in causal streaming-video reconstruction.
Continuous-Time Digital Twin with Analog Memristive Neural Ordinary Differential Equation Solver
TL;DR: An analog memristive neural ODE solver models continuous-time dynamics for efficient digital twins.
Topology Optimization of Random Memristors for Input-Aware Dynamic SNN
TL;DR: Topology optimization and input-aware computation make memristive spiking networks adaptive and energy-efficient.
Charge-Selective 2D Heterointerface-Driven Multifunctional Floating Gate Memory for In Situ Sensing-Memory-Computing
TL;DR: A charge-selective 2D heterointerface combines sensing, memory, and in-situ computing in one device.
Experience
-
Harvard University
Visiting Scholar (Embodied AI & 4D World Models)
Advisors: Yilun Du and Mengyu Wang.
Apr. 2026 – PresentCambridge, MA, USA
-
The University of Hong Kong (HKU)
Ph.D. in Electrical and Computer Engineering
Awards: Hong Kong PhD Fellowship (HKPFS, Top Tier), HKU Presidential Scholarship.
Advisor: Xiaojuan Qi. Expected graduation: Aug. 2027.
Sep. 2023 – PresentHong Kong
-
Zhejiang University (ZJU) & UIUC
Dual B.S. in Computer Engineering (ZJUI Institute)
Honors: Outstanding Graduate, Dean's List.
Sep. 2019 – Jun. 2023China & USA
Awards
- Hong Kong PhD Fellowship (HKPFS, 2023)
- HKU Presidential Scholarship (2023)
- Outstanding Graduate of Zhejiang University (2023)
- NeurIPS (2025) and ICLR (2026) Travel Awards
About me
The future I want to help build
Science fiction has fascinated me since childhood. Over the years, its films and novels have shaped how I imagine the future: the superintelligences of Hyperion, the robots in Asimov's Foundation universe, WALL-E, and TARS in Interstellar. I find it hard to imagine the future without thinking about the intelligences we might share it with. What draws me in is not only how capable they are, but how they live alongside people.
The imagined worlds of Ready Player One, Pantheon, and The Matrix raise another question for me: how does intelligence relate to the world it inhabits? These stories are part of why I am drawn to world models, and to the possibility of agents that can understand their surroundings, imagine what might happen, and learn through interaction.
I believe that LLM-based AGI may arrive soon. The part of that future I most want to help build is the physical agent: an intelligence that can do more than talk to us—it can act in the world with us. I am excited by the practical benefits this could bring, from help with everyday tasks to greater independence, better care, and companionship.
Joi in Blade Runner 2049 is especially close to this vision for me. As a holographic companion, she is present in K's daily life, shares his search for answers, and helps him interpret what his memories might mean. I am drawn to an agent that knows a person's history, helps connect scattered clues, and offers emotional support. But I keep imagining one step further: what could Joi do if she could also interact with the physical world?
In the future I imagine, such an agent would not be tied to a single body. At home, it could use smart appliances and a robot to cook and take care of chores. On the road, it could operate a vehicle. In a medical setting, it could bring relevant knowledge of a person's history to a surgical robot, assisting a clinical team under human supervision. Its physical form could change with the situation, while its memory and understanding of the person remain continuous.
This is why I want to work on physical agents. I want to understand how an intelligence can learn to use different bodies, anticipate the consequences of its actions, and become more useful through experience. My hope is to help turn the familiar science-fiction companion into something that can offer practical help—not only a presence on a screen, but a participant in the world we share.
Technical skills
Tools: Python, PyTorch (CUDA), Diffusers, Open3D, OpenCV.
Methods: World–action modeling; geometric action checks; sim-to-real evaluation on Franka Research 3.