The Amazon Mechanical Turk "Master's Qualification" : How Do You Get It, And Is It Worth It?

                                                   "You got the Master's Qual!"

On Amazon Mechanical Turk, most jobs pay pennies. They're simple tasks, true, but between the actual work time, going through periodic CAPTCHA checks to verify there's still a human at the controls, and dealing with occasional non-payers and spurious task rejecters, the per-hour pay when all is said and done is usually below federal minimum wage for most people, even the diligent and dedicated.

But wait! There's a shining beacon promising to lift valiant Turkers out of poverty, if they only apply themselves hard enough! It's the "Master's Qualification", a legendary credential doled out by Amazon to their "top performers" on the site. With a Master's qual, you allegedly have exclusive access to a pool of higher-paying jobs, such that you can perhaps finally wring a living wage out of the infernal machine.

Does it actually work that way, though? And how are you supposed to get the thing if Amazon themselves won't even tell you how?

How To Get It

First, let's figure out how it's actually obtained. Then we'll assess the value.

Let's start by going directly to the horse's mouth for as much information as possible -- what is Amazon willing to tell us about this qualification?

Here's the extent of their information on the subject, as posted:

"What is a Mechanical Turk Master?
Masters are elite groups of Workers who have demonstrated accuracy on specific types of HITs on the Mechanical Turk marketplace. Workers achieve a Masters distinction by consistently completing HITs of a certain type with a high degree of accuracy across a variety of Requesters. Masters must continue to pass our statistical monitoring to remain Mechanical Turk Masters.
.....
Please note that Workers cannot apply for this status - it is a performance based distinction. The best thing a Worker can do to become a Master is to submit Assignments with accurate results across a wide variety of Requesters on the Mechanical Turk marketplace."

Well, that really doesn't tell us much. Like Google's search algorithm or the process of becoming a Jedi in the old game Star Wars Galaxies, it's apparently going to be kept opaque to the general public and leave everybody eternally guessing.

We can pick out a couple of concrete bits of info from Amazon's little blurb, though:

• "HITs of a certain type" -- so only certain types appear to matter, while others do not
• "High degree of accuracy" -- so this is definitely a factor in the qualification, though we don't know what the exact bar is to clear
• "across a wide variety of Requesters" -- so even if you're in the right category, lots of HITs for the same requester won't cut it

The most pertinent thing to know is, what categories matter? We can begin to get an answer to this by digging through public, shared information -- the best source of which is found at the MTurk Crowd forums. That thread is absolutely sprawling, though, so here's the key information about the important categories:

* Originally, there were two different Masters quals: one for Categorization and one for Photo Moderation. They were combined into the one qual a couple of years ago, but given how long they existed in that state it's safe to assume these are at least two of the task categories that really matter. Here's the description Amazon used to have up for these quals: "For instance, Photo Moderation Masters are the best choice if you want to have a photo reviewed to determine if it meets your site guidelines, or need specific types of content identified such as the number of people in a photo. Categorization Masters are the best choice if you need to have companies categorized by industry or identify the department for products in a retail catalog or characterize the sentiment of a tweet or blog post."

There's also a lot of additional helpful information -- here's the most important takeaways:

• Based on self-reporting on various forums by people who got it, only one or two new Masters seem to be confirmed each month. In some months there's been none. 
• Since 2012 Amazon has maintained a workforce of around 21,000 people with the qual, give or take. 
• People who have received the qual have always had at least 1,000 HITs completed and at least 90% overall accuracy, though from there the numbers are all over the map, and there's any number of people with 99% and tens of thousands of HITs done who don't have it. 
• The vast majority of Masters are in the United States

Is It Worth It?

On average, requesters are charged about 20% more to have access to Masters. So if you were making $5 per hour on average previously, with Masters status you could expect to be making as much as $6 on that basis.

However, just bumping your current average pay rate up by 20% might be underselling it a bit, as there's a couple of other factors in play. You have access to a pool of HITs that most people don't, so in theory they won't dry up as fast and you'll have less "dry spells" ... you also no longer have to stop for CAPTCHA checks, which speeds up your productivity a bit. All of that still moves in flux with how many requesters currently have jobs on offer, however.

Still, even if all the winds blow in your favor all of the time, I can't see this bumping up your expected hourly rate by more than $3 at a real generous maximum. Basically, moving you from "unreliable minimum wage" to "reliable minimum wage" status at best.

Final Word

Since Amazon is determined to keep their exact requirements a secret, all we can do is make educated guesses about how the qualification works based on what we know for certain.

Here's  my best guess: It's basically just a lottery.

This theory makes the most sense in keeping with the contradictions we are aware of (that someone with 50,000 HITs and 99% accuracy over four years may still not have it while someone with six months under their belt, 5,000 HITs and 93% accuracy does).

The actual qualifications aren't that rigorous, meaning that tens (if not hundreds) of thousands of people are eligible for it and don't know it. The problem is, Amazon only needs or wants to maintain a Masters workforce of roughly 21,000. So the qual has to get pulled from existing workers to be given to new ones -- thus only seeing one or two people per month get it. That's further controlled by requiring somewhere in the 90% range of those people to be United States-based workers, thus the extreme rarity of Masters from other countries.

They just pull someone from the tens/hundreds of thousands of qualified at random when they need to. It's that simple. Makes the most sense based on what we know for sure, anyway.

In my review of the site (posted way back in 2014), I gave it a "some potential" rating. Despite the consistently low pay that there's really no way to move up from, it's something that has a lot of requesters and is generally consistent though the pay is crappy. I haven't touched it in years now personally, because I do much better as a freelance writer, but I also recognize not everyone coming to this site is able to write / program / etc. They may well be in a position where making $5-6 an hour with sites like Turk is better than nothing, or at least better than having to turn up to Walmart or McDonalds every day.

The terms of the Masters Qual make it clear that you should never expect anything more than "maybe minimum wage on a good day" from it, however. It isn't some shining light that will someday turn the service into a "real job" for you if you just hang in long enough and work hard enough. Given Amazon's long history of unethical business practices it's not a stretch to think they keep this thing shrouded in secrecy intentionally to use it as a carrot to dangle over the heads of the naive and keep their underpaid labor force Turkin' And Twerkin'. As with everything in life, be armed with as much information as possible and know exactly what you're getting into.