- Microsoft has conducted research regarding using ChatGPT to interact and command robots and shared the results.
- The research shows that ChatGPT is already very capable of translating natural human language to lower-level robotics language.
- ChatGPT still needs some improvements in understanding the laws of physics to be able to fully control the robots.
ChatGPT quickly became the center of attention in the artificial intelligence world, which we just started exploring. Its exceptionally high capability of understanding natural human language to find -mostly- the right answers and submitting them again, in human language, can’t be compared to anything we have seen before. While many people are trying to find a good use for those kinds of artificial intelligence solutions, Microsoft comes out with an interesting idea.
Commanding the robots
Microsoft, one of the biggest investors in OpenAI, has conducted research regarding using ChatGPT to interact with robotics. As we mentioned before, ChatGPT is highly capable of understanding natural language. On the other hand, it can code as well. Improving and combining those capabilities might result in robotics being controlled with natural human language, with ChatGPT being in the middle.
Of course, with ChatGPT’s current state, it is not possible yet. According to Microsoft’s research, one of the biggest problems they need to solve is teaching the laws of physics so it can generate the code output for the robot accordingly.
« The key challenge here is teaching ChatGPT how to solve problems considering the laws of physics, the context of the operating environment, and how the robot’s physical actions can change the state of the world. »
The researchers state that ChatGPT can do a lot by itself, however, it still requires some assistance. Microsoft drops a technical paper that tells about the design principles for guiding language models toward solving robotics tasks. Those principles include special prompting structures, high-level APIs, and human feedback via text.
Robotics is difficult
Currently, programming robotics requires a lot of effort and very high skill. This is why we are getting shocked every time Boston Dynamics publishes a new video. We have seen their progress over the years; it really took many years for their robots to start moving more naturally.
While ChatGPT’s involvement will not automatically make robots have better acrobatics skills (or maybe they will, in the future), it will help with assigning tasks to robots.
There are a couple of video examples that Microsoft shared. One of the most interesting videos is the drone-controlling one. The researcher simply demands some tasks from the drone, through ChatGPT, and the drone completes the tasks:
ChatGPT also provides a drone-flying algorithm that avoids obstacles that can be used in a 3D world:
No wind physics, yet
If you have noticed, there is no wind in none of these situations. In its current state, ChatGPT will likely fail to provide a proper algorithm to the drone that will be able to handle winds. And this is one of the key obstacles that Microsoft is trying to overcome.