EgoX: Announcement of an Egocentric Cross-Embodiment Manipulation Learning Framework
The EgoX framework has been announced to integrate diverse experiences from humans, real robots, and simulated agents.
Generalist manipulation policies can benefit from the complementary experience of humans, real robots, and simulated agents. However, differences in morphology, sensing, and control interfaces make it difficult to learn effectively from their combined demonstrations.
In response, EgoX introduces an egocentric cross-embodiment learning framework that organizes heterogeneous demonstrations through semantically corresponding observation and action modalities. EgoX pretrains a Modularized Cross-Embodiment Transformer with Embodiment Dreaming (MXT+) on this mixture and fine-tunes it on target-embodiment demonstrations.
This item is a summary. The full, checkable version is at the original source:
Read the original at Hugging Face Papers ↗