People in Poor Countries Build AI Datasets To Train Self-Driving Cars For $1 An Hour

The people helping create the datasets for use in the AI that trains self-driving cars are often sub-contractors working for microtasking sites. 

They are frequently from poor countries like Venezuela and Kenya and paid as little as $1 an hour for work conducted under intense pressure ultimately benefiting big tech companies that will reap billions from their essential labor. JL

Vittoria Elliott reports in Rest of World:

Across the world, people have become part of a sophisticated new labor force training self-driving cars. Based from Kenya to the Philippines, these workers play a crucial but rarely acknowledged role in tech. Through microtasking platforms (they) identify objects that line the streets everywhere - trees, lampposts, pedestrians, stop signs - so autonomous vehicles learn to notice them. The client provides them with detailed instructions about how the job should be completed, but has no direct interaction with them. Paid $1 an hour, those doing microtasking need to maintain a high degree of accuracy. “I got a warning when my accuracy was 88%.”

Every day for over four years, Ramses woke up in his home in Barquisimeto, Venezuela, turned on his computer, and began labeling images that will help make self-driving cars ubiquitous one day. Through a microtasking platform called Remotasks, he would identify mundane objects that line the streets everywhere — trees, lampposts, pedestrians, stop signs — so that autonomous vehicles could learn to notice them, too.

Like many Venezuelans, Ramses turned to microtasking when his country plunged into economic turmoil. The gig gave him the opportunity to earn American dollars instead of the local currency, which is subject to extraordinarily high inflation. “I would work Sunday to Sunday,” Ramses, who asked to use only his first name for privacy reasons, told Rest of World over WhatsApp. “I never rested, but I made good money for 12-hour days, seven days a week.”

Across the world, people like Ramses, many in the Global South, have become part of a sophisticated new labor force training self-driving cars. Based everywhere from Kenya to the Philippines, these workers play a crucial but rarely acknowledged role in one of the most prominent parts of the tech industry. 

In the early 2010s, around the same time Venezuela’s economy began to collapse, companies started pouring enormous sums of money into self-driving cars. By 2015, Google’s parent company had already spent over $1 billion developing its autonomous vehicle program. A Brookings Institution report published in 2017 estimated that tech firms and car manufacturers had invested more than $80 billion in the technology, all in the hopes of being on what was then considered the “leading edge” of artificial intelligence. Many firms promised investors and consumers that it would be only a few years before driverless vehicles would be commonplace.

Self-driving cars rely on video cameras, radar sensors, lidar sensors, GPS antennas, and other tools to read street signs and continuously map their surroundings. To perform as well as a human driver, the cars must quickly process and respond to a constant stream of ever-changing information. A lost dog, the sudden onset of a rain shower, or a broken traffic light can all throw them off. To prepare for these and millions of other possibilities, the complex software and algorithms powering self-driving cars need immense amounts of highly accurate data — and an army of humans to feed it to them.

As the race to develop autonomous vehicles heated up, suddenly companies found themselves hungry for workers who could build training datasets, which typically contain hundreds of thousands of images and videos that self-driving cars captured during test drives. The workers are tasked with labeling what is depicted in them, so that a machine-learning algorithm can slowly learn to differentiate a tree from a stop sign. To complete all this tedious work, many companies turned to the existing global crowdsourcing industry, which allows people to make money online doing piecemeal tasks, like evaluating restaurant reviews or answering survey questions. 

“I would argue that the influx of the money from the car industry has actually substantially changed the crowdsourcing industry,” said Florian Alexander Schmidt, a professor at the Dresden University of Applied Sciences in Germany, who has studied the microtasking industry and autonomous vehicle training. Previously, companies primarily provided access to large pools of workers, who could answer surveys in bulk or complete lots of work quickly and cheaply. The problem was that the results weren’t necessarily very accurate. “A lot of [the data] was trash,” Schmidt explained. “That is not acceptable in the self-driving vehicle field.” 

Over the last few years, Schmidt said, many microtasking and third-party outsourcing firms have changed the way they operate. First, they introduced quality control measures to ensure jobs for autonomous vehicle clients come back with very few mistakes. There are now not only workers doing the actual labeling, but also other workers training them as well as checking and correcting completed tasks.

Companies also put more distance between their clients and workers, who are often unable to give companies feedback or even ask questions about the tasks they are assigned. According to nine workers who spoke to Rest of World, the client typically provides them with detailed instructions about how each type of job should be completed, but it has almost no direct interaction with them. In some cases, a representative will personally train certain taskers, who then teach their peers and double-check their work.

Before the pandemic, Marissa Zuniga, a Filipina administrative and finance professional, had spent almost 20 years living in Shenzhen as an overseas Filipino worker (OFW) and sent money home to her family in Quezon City. Like many Filipinos working in China, she used her time off during the Chinese New Year holiday in early 2020 to visit home. Soon after, both countries began limiting travel, as their governments scrambled to control the spread of the coronavirus. Zuniga found herself trapped in the Philippines with no work. 

She soon discovered Remotasks through a Facebook advertisement and now spends her days checking the work of other taskers on the platform to make sure it’s as close to perfect as possible. “My project now is making sure we are coloring everything the car can see on the road. You have to annotate everything,” said Zuniga. “It’s very large-scale.”

Like many other workers on the platform, Zuniga gets paid per task that she completes. A difficult job that could take three or four days might earn her between $20 and $30. When she finally returns to Shenzhen, Zuniga said she hopes to continue doing microtasking work to supplement her income. “If you break it down per hour, it’s not so much money,” she said. “But I’m enjoying the work I’m doing right now.”

Ramses, on the other hand, ultimately decided to stop working on Remotasks a few months ago, as pay rates fell in Venezuela. “When I first started, I could earn $200 a week,” he said. “But then rates decreased. Sometimes I was only making $20 or $30 a week.” He eventually decided to leave the country and look for work elsewhere.

Remotasks is owned by the San Francisco–based startup Scale AI, which was recently valued at over $7 billion and has raised more than $600 million from investors. In a statement, a spokesperson for the company said that “pay rates may vary over time as they are determined by factors including location, hours and the complexity of the assigned project.”

Julian Posada, a researcher at the University of Toronto who studies artificial intelligence and the outsourcing industry in Latin America, said his research found that people are earning less overall. “When these platforms first entered the market, they were trying to recruit people,” he said. “But now that they have a critical mass of taskers, they can start dropping their rates.”

Microtasking companies are not the only ones cashing in on the self-driving car rush. In 2019, after having a baby, Joy Olwande was looking to transition back into the workforce in Kenya. She saw that a company called CloudFactory, a business process outsourcing (BPO) firm with offices in Nairobi, was hiring workers for several artificial intelligence projects. For six months, Olwande worked two four-hour shifts per day, with a break in the middle. During one, she annotated lidar images — high-resolution pictures — for autonomous vehicles. 

But unlike Zuniga and Ramses, Olwande was working onsite with hundreds of other employees. She was paid around $1 an hour, and, like those doing labor on microtasking sites, needed to maintain a high degree of accuracy. “I got a warning once when my accuracy was at something like 88%,” she said. Though she liked the work and her colleagues, Olwande left after half a year to take a job with a shorter commute and higher pay.

In a category called “atmospherics,” workers may be asked to label each individual drop of water so the cars don’t mistake them for obstacles.

While the outsourcing industry may still be booming, the future promised by self-driving car companies — in which the streets would be filled with autonomous vehicles — has yet to arrive. In May, The New York Times reported that the cars are still unable to manage the multitude of scenarios that they may encounter while driving. Things like road flares or fog might be normal for humans but continue to befuddle machines. Perfecting the technology may require billions of dollars more in research and development. In the meantime, the evolving challenge is being reflected in the tasks workers are asked to perform.

Over time, Timm Ndirangu Gachanja, a former CloudFactory employee in Nairobi who now works at Remotasks, said he noticed the things he and his colleagues were being asked to identify had changed. “You find that they are introducing other, new labels,” he said. “For example, if it’s drizzling, all the cameras are so strong that they can capture the tiniest water drop in the atmosphere.” In a category called “atmospherics,” workers may be asked to label each individual drop of water so the cars don’t mistake them for obstacles.

Schmidt said that so far, the most important innovation to come out of this moment isn’t the autonomous cars themselves but the vast labor pool the industry accidentally helped create. Some of the workers who spoke to Rest of World said they also trained artificial intelligence for medical technology, smart home devices, and even garbage sorting.

Now, third-party contractors and microtasking platforms are trying to find ways to break tasks into smaller chunks or let people complete them on their phones, Schmidt said. That will help open up jobs to an even larger number of workers. “Think about how many more people will get access to good internet in poorer countries over the next few years,” Schmidt said. “This will really be a huge workforce for this global supply chain of labor.”