Robots have come a long way in recent years. From football-playing robots to school kids’ avatars, there’s no shortage of robot innovation. But Tidybot is the robot we’ve been wanting all along: not only can it clean your room, but it can clean it the way you like it.

A one-armed cleaning army

Household chores are annoying to many people. Shuran Song often asks her robotics students what task they’d want robots to carry out — and the answer is usually cleaning the house and doing the dishes. When it comes to washing dishes, we’ve already got specialized machines that do a pretty good job. But when it comes to cleaning your room, that’s proven to be much more difficult.

But Song and colleagues have a solution: meet TidyBot.

“We all experience it: our home or lab or office gets messy,” said Jeannette Bohg, assistant professor of computer science at Stanford and another of the paper’s authors. “Now imagine you just tell a robot, ‘In my place, this is where that thing goes, and that is where this thing goes.’ Then you leave and let the robot tidy up.”

TidyBot is the creation of a mixed group from ​​Princeton University, The Nueva School, Columbia University, and Google. Essentially, the robot is a seven-jointed arm that ends in a two-fingered grip. It can move around the house, open drawers, and arrange things. TidyBot navigates using a camera on the ceiling that enables it to see around obstacles in the room.

But that’s just the outside — on the inside, TidyBot features a smart language model that enables it to learn and clean up the room based on people’s personal preferences.

AI cleans your room

“A key challenge is determining the proper place to put each object, as people’s preferences can vary greatly depending on personal taste or cultural background. For instance, one person may prefer storing shirts in the drawer, while another may prefer them on the shelf,” the researchers write on TidyBot’s page.

The robot uses a large language model (LLM), somewhat similar to the one ChatGPT uses. Essentially, it’s been taught to classify different types of objects into groups such as clothes, food, or toys. Then you can just tell it where you want to put different objects. You can say things like “I want shirts on the second shelf, and pants on the first shelf.” You give it some examples, and then it will clean up as you ask it.

Well, for the most part.

TidyBot has a success rate of 85% in real-world scenarios. Meaning that in about 1 of 7 tasks, it makes an error. Sometimes, these errors are pretty funny. Bohg recalls a time when the robot tried to pick up one of the researchers who was in its space.

“He was visible in TidyBot’s ‘playing field,’ and TidyBot tried to pick him up,” she laughed.

Fortunately, TidyBot can’t really hurt people. It’s made for handling small and softer objects, like clothing or garbage.

The robot also struggles with new environments and new objects and gets better at its job the more it operates in one setting.

A multipurpose household robot?

TidyBot is far from perfect. There’s plenty of room for improvement and no doubt, the improvement of both language models and robotics can improve TidyBot. But already, the bot is very promising and opens up important research avenues. For instance, could you have multiple bots assembling furniture, or performing more difficult tasks together? Could you have a robot that cleans your house, cooks food, and moves the furniture?

The possibilities are truly exciting, especially if you’re not a fan of household chores (and let’s face it, who is). But Song says that, for now, she’d settle for a robot that cleans her room.

“Maybe in the long term we can think about a more general-purpose robot that’s also able to cook my meal, make my bed, do my laundry,” Song mused. But for now, she said, “If I have a robot that’s able to reliably put my things away, that’s a pretty good goal to achieve already.”

TidyBot represents a significant leap forward in the realm of household robotics. It’s not just a machine that performs tasks; it’s a machine that learns to perform tasks the way you want them done. While it may not be perfect, achieving an 85% success rate in real-world scenarios is no small feat. It’s a glimpse into a future where robots are not just tools, but partners in making our lives easier and more organized.

The group presented a paper on the robot and its performance at the International Conference on Intelligent Robots and Systems on October 2nd. An extended journal version of the paper will appear soon in Autonomous Robots in the special issue on robots and large language models.