OpenAI in Collaboration with Figure: Early Emergence of Embodied Intelligence
Eric Jang, a veteran robotics expert, recently predicted that intelligent robotics will appear overnight, just like ChatGPT did.
Maybe He is right.
OpenAI, which had never before demonstrated its robotic orientation capabilities, has now showcased its robotic intelligence for the first time in collaboration with a humanoid robot from a portfolio company.
Figure, the robotics company in which OpenAI invested, uploaded this video. In the video, Figure's humanoid robot can converse fluently with humans, understand human intent, and follow natural language commands to grasp and place objects.
The robot's intelligent brain, configured by OpenAI, enables these capabilities.
Figure emphasized that the video was shot in one take without any editing or acceleration. Simultaneously, the robot operates autonomously without any remote manipulation.
The robot's intelligence is impressive, and it is the result of OpenAI's recent partnership with Figure to advance humanoid robotics. The video was released only thirteen days after the announcement.
The intelligence powering Figure's humanoid robot is derived from end-to-end big-language-vision modelling, which is currently at the forefront of the field of embodied intelligence.
At the time, Google's robot model could only perform basic grasping tasks based on verbal commands. It was unable to communicate with humans or provide explanations for its actions. Google has extensive experience in robotics research, dating back to Everyday Robotics, which was founded over five years ago.
In contrast, Figure is a newer company, founded in 2022. Only 13 days have passed since OpenAI announced its collaboration, and today they are launching a robot capable of autonomous dialogue and decision-making.
The development of robotic intelligence is clearly accelerating.
Robots Are Approaching Human Speed
OpenAI enabled the breakthrough by providing visual reasoning and language understanding, while Figure's neural network facilitated fast, low-level, and dexterous robotic movements.
The robot's actions are driven by internalized capabilities, not teleoperation.
The researchers transcribed image input from the robot's camera and speech captured by the onboard microphone into a multimodal model (VLM) trained by OpenAI. The VLM understands both images and text and processes the entire conversation history to derive a verbal response, which is then communicated back to the human via text-to-speech.
The model is responsible for selecting the appropriate closed-loop behaviours to execute on the robot to accomplish specific commands. It loads specific neural network weights onto the GPU and enforces the policy.
The onboard camera of the Figure takes images at 10hz, and the neural network outputs 24 degrees of freedom of movement at 200hz.
The founders of Figure claim that this represents a significant improvement in the robot's speed, which is approaching human speed.
The modelling capabilities are so strong that it can also have short-term memory.
OpenAI Is Seeking to Challenge Intelligence
OpenAI terminated its exploration of the field of robotics in the summer of 2021 due to a lack of data needed to train robots to move and reason using AI.
However, the company remains focused on the field.
In March of this year, OpenAI participated in Figure's Series B funding round, along with other investors. This brought Figure's valuation to $2.6 billion, just two years after its founding.
OpenAI also announced its partnership with Figure after this funding round.
Although the partnership is going well, OpenAI does not seem to be placing its trust in a robotics company.
On 13th March, Physical Intelligence, a new robotics AI company founded by a group of researchers from Google's research team, UC Berkeley, Stanford professors, and others, was revealed by Bloomberg to have received funding from OpenAI.
The company is researching AI that could become a general-purpose robotic system in the future.
OpenAI's intentions in the field of robotics have attracted attention due to their multiple bets on the field and 13 days of cooperation to create a leading robotics model.
It is important to note that the future of intelligent humanoid robots is not solely dependent on Musk.
