Alexander Sax

Alexander (Sasha) Sax

Comp Vision Research Scientist

FAIR, AI @ Meta

About me:

I am currently a Research Scientist in the Fundamental AI Research (FAIR) division of AI at Meta. My research focuses on building foundation models to enable Embodied AI that can perceive, act in, and communicate about the physical world.

A lot of my work is around large-scale pretraining for robust Embodied AI -- developing 3D datasets, simulators, and techniques for efficient training and finetuning. Some relevant keywords are pretraining, transfer learning, RL, sim2real.

I received my PhD at UC Berkeley, where I had the great pleasure of working with and being advised by Jitendra Malik and Amir Zamir (at EPFL). Before that I received a bachelors in Math and masters in CS from Stanford, where I was was fortunate to be advised by Silvio Savarese. My Erdős number is 3!

Selected Honors and Awards

  • CVPR 2020 Best Paper Award Nomination for X-Task Consistency.
  • CVPR 2018 Best Paper Award for Taskonomy.
  • NVIDIA Pioneering Research Award 2018 for Gibson Environment.
  • Stanford University Distinction in Research 2018 for Computational Evidence for Structure in the Space of Tasks.
  • Winner of CVPR19 Habitat Embodied Agents Challenge:
  • Outstanding Reviewer: ICLR CVPR .

Education

  • PhD in CS, 2023

    UC Berkeley

  • MSc in CS, 2018

    Stanford University

  • BSc in Math, 2018

    Stanford University

-->

Selected Publications

Updated version on Google scholar
OpenEQA: Embodied Question Answering in the Era of Foundation Models. In CVPR, 2024.  
Arjun Majumdar*, Anurag Ajay*, Xiaohan Zhang*, Pranav Putta, Sriram Yenamandra, Mikael Henaff, Sneha Silwal, Paul Mcvay, Oleksandr Maksymets, Sergio Arnaud, Karmesh Yadav, Qiyang Li, Ben Newman, Mohit Sharma, Vincent Berges, Shiqi Zhang, Pulkit Agrawal, Yonatan Bisk, Dhruv Batra, Mrinal Kalakrishnan, Franziska Meier, Chris Paxton, Alexander Sax, Aravind Rajeswaran (* Equal Contribution)

Project Site Code PDF
A benchmark of 1800 human annotated questions, and associated RGBD pose trajectories, with registered 3D scans. For categories like spatial understanding, no VLMs (Mar 2024) do better than a blind LLM that can't see the scene.

Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scans. In ICCV, 2021.  
Alexander Sax*, Ainaz Eftekhar*, Jitendra Malik, Amir Zamir.

Project Site Code PDF
A large diverse dataset of multiview posed RGBD images, 3D scans, and more -- of 1.9k scanned + artist-generated buildings + outdoor scenes.

Robustness via Cross-Domain Ensembles. In ICCV, 2021.  [Oral]
Teresa Yeo*, Oğuzhan Fatih Kar*, Alexander Sax, Amir Zamir.

PDF Code Project Site
Enforce consistency constraints between different tasks, to improve robustness to various domain shift.

Robust Policies via Mid-Level Visual Representations. In CoRL, 2020.  
Alexander Sax*, Bryan Chen*, Francis E. Lewis, Silvio Savarese, Jitendra Malik, Amir Zamir, Lerrel Pinto.

Project Site Code PDF
Using large pretrained visual representations improves performance and learning speed for various manipulation tasks with robotic arms, and with sim2real generalization for navigation.

Robust Learning Through Cross-Task Consistency. In CVPR, 2020.  [Best Paper Nominee] [Oral]
Alexander Sax*, Amir Zamir*, Teresa Yeo, Oğuzhan Faith Kar, Nikhil Cheerla, Rohan Suri, Zhangjie Cao, Jitendra Malik, Leonidas Guibas.

Project Site Code PDF
A strong way to enforce consistency constraints across multiple tasks in multi-task learning -- better generalization and ability to detect prediction failure.

Side-Tuning: A Baseline for Network Adaptation via Additive Side Networks. In CVPR, 2020.  [Oral]
Jeffrey O. Zhang, Alexander Sax, Amir Zamir, Leonidas Guibas, Jitendra Malik.

Project Site Code PDF
Simple method to adapt and control output of a larger network with a smaller "side" network.

Mid-Level Visual Priors Improve Generalization and Sample Efficiency for Learning Visuomotor Policies. In CoRL, in BayLearn, 2019.  [Oral]
Alexander Sax, Jeffrey O. Zhang, Amir Zamir, Silvio Savarese, Leonidas Guibas, Jitendra Malik.

Project Site Code PDF
Using large pretrained visual representations improves performance and learning speed for various navigation tasks.

See all publications

Press Coverage

CV

Education

Internships and Research Positions