OpenAI is apparently also ramping up its own robotics efforts. Last week, Caitlin Kalinowski, who previously led the development of virtual and augmented reality headsets at Meta, Announced on LinkedIn She was joining OpenAI to work on hardware, including robotics.
Lachie Groom, a friend of OpenAI CEO Sam Altman and investor and co-founder of Physical Intelligence, joins the team in the conference room to discuss the business side of the plan. The groom wears an expensive looking hoodie and looks remarkably young. He emphasized that physical intelligence has great potential to achieve success in robot learning. “I just called Kushner,” he said, in reference to Joshua Kushner, founder and managing partner of Thrive Capital, who led the startup's seed investment round. Of course, he's also the brother of Donald Trump's son-in-law, Jared Kushner.
Some other companies are now chasing similar success. Skilled, a company founded by roboticists at Carnegie Mellon University, raised $300 million in July. “Just as OpenAI created ChatGPT for language, we are building a general purpose brain for robots,” says Deepak PathakCEO of Skilled and adjunct professor at CMU.
Not everyone is convinced that this can be achieved in the same way as OpenAI cracked the language code of AI.
There is no Internet-scale repository of robot actions similar to text and image data available for training LLMs. Achieving breakthroughs in physical intelligence may require exponentially more data anyway.
“The words in the sequence, dimensionally speaking, are a small toy compared to all the motion and activity of objects in the physical world,” says Illah Nourbakhsh, a roboticist at CMU who is not affiliated with Schilder. “The degree of freedom we have in the physical world is greater than the letters of the alphabet.”
Ken Goldberg, an academic at UC Berkeley who works on applying AI to robots, warns that along with the data-driven robot revolution, there is growing excitement around the idea of humanoids that is reaching hype-like proportions. “To reach the expected performance levels, we will need 'good old-fashioned engineering,' modularity, algorithms, and metrics,” he says.
Russ TedrekeA computer scientist at the Massachusetts Institute of Technology and vice president of robotics research at the Toyota Research Institute, says the success of the LLM has prompted many roboticists, including himself, to reconsider their research priorities and pursue robotic learning further. Has inspired me to focus on finding ways. Ambitious scale. But he acknowledges that formidable challenges still remain.