A Problem That Keeps Warehouse Work From Being Fully Automated Has Just Been Solved

Amazon, the largest U.S. ecommerce company, has about 45,000 robots working in its warehouses, but so far, they don’t seem to be taking jobs from people. That’s partly because today’s commercially feasible robots are only capable of doing a small piece of the work involved in packaging an order.

But robots may soon take on a larger share of warehouse work.

A startup called RightHand robotics recently began piloting technology that automates a task robots have previously struggled to master: recognizing and picking up items from boxes. RightHand can’t say which companies are part of its pilot project and Amazon didn’t reply to a request for comment. But the new technology could help the ecommerce giant with a problem that has long vexed it.

Like robots elsewhere, Amazon’s robots retrieve entire shelves and transport them to humans who pick out items from them. They can find and move a shelf that holds a box of shirts, but they aren’t capable of removing the single shirt from that box to be packed into an order.

In order to pick items from boxes, robots need to master the more complex task of identifying a wide range of objects and adjusting their grips accordingly. Amazon has said this particular feat “remains a difficult challenge” and the company hosts an annual “picking challenge” in which robots compete to solve it.

The best robot yet is still too slow and too inaccurate to be commercially feasible. Last year, the robot that won the $50,000 prize moved items at a speed of 100 items per hour and failed to pick up and move the correct item around 17 percent of the time, according to TechRepublic.

RightHand robotics, which was started by a team of researchers from Harvard Biorobotics Lab, the Yale Grab Lab, and MIT, built a solution called RightPick that, according to co-founder Leif Jentoft, can pick items at a rate of 500 to 600 per hour–a speed on par with a human worker. It uses a machine learning background and a sensorized robot hand to recognize and handle thousands of items.

The robots still aren’t as good as humans. They get stumped by stuff wrapped in plastic or things partly obscured by other items, for instance. But RightPick is functional enough to be deployed in pilot programs at warehouses of unnamed companies, where the machines are picking “thousands” of orders.

“It’s just a matter of time [before the technology is good enough to be widely deployed],” says Bruce Welty, the founder of Locus Robotics, who started both a company that makes warehouse robots and a fulfillment company that ships more than $1 billion of ecommerce orders each year. “They’ll definitely get there. Whether it’s one year, two years, I don’t know.”

RightHand’s Jentoft said robots in the pilot programs are being used mostly to pick items from boxes. Packing items to be shipped requires more complex tasks like removing hangars and nestling items into boxes in a way that fits best. Which means for the time being, warehouses still will need to hire human warehouse workers, but fewer of them.

“You’re able to do much more with your people, that’s for sure,” Jentoft says. Amazon said in January it will create 100,000 full-time U.S. jobs—for people—within the next 18 months.

Like many executives in the fulfillment industry, Jentoft argues warehouses face a shortage of labor.

“The biggest challenge in the industry is trying to find the quality people,” he says. And it’s hard to mourn the loss of boring, low-paying, grueling jobs. But as ecommerce sucks more business from traditional retail, and more of the jobs involved in ecommerce get automated, it’s still not clear exactly where new jobs will be found.