SPEAKER: Ted Xiao, Google Deepmind
TITLE: What’s Missing for Robotics-first Foundation Models?
ABSTRACT:
Intelligent robotics have seen tremendous progress in recent years. In this talk, I propose that trends in robot learning have historically almost exactly followed key trends in broader foundation modeling. After covering robotics projects which showcase the power of following such foundation modeling paradigms, I will focus on a few future-looking research directions which may suggest a unique future for how robot learning systems may develop differently from LLMs and VLMs.
Notes:
There are current pineapple pen techniques
VLM (frozen) -> LLM (frozen) -> Control Policy
Problems
Foundational models in robotics optimize model for all subtasks at once - vision, language, control
#1 positive transfer in scale