How do you use Mechanical Turk?


I'm a big fan of mechanical turk and often leverage it with smartsheet. I've used mturk for basic writing, gathering content from multiple sources, and some minimal research. However, I don't think I've truly tapped into its potential.

How do you use Mechanical Turk to crowdsource effectively? What are some of the biggest problems or challenges you have solved with Mechanical Turk? Or are there failures you'd like to share?

Outsourcing Virtual Workforce

asked Oct 14 '09 at 11:58
1,057 points

3 Answers


I don't have the rep to add this as a comment. What is a mechanical turk?

answered Oct 16 '09 at 08:30
Coder Dennis
691 points

answered Oct 16 '09 at 23:36
171 points
  • Thanks for the great links! – Dlynton 14 years ago


You should look into Crowdflower. They automatically determine the accuracy of workers and correct for bad workers, something that has always been difficult with Mechanical Turks.

You should also see the empirical evaluation of Mechanical Turk called "Cheap and Fast - But is it Good? Evaluating Non-Expert Annotations for Natural Language Tasks " by my colleague Rion Snow. There is a paper and easier to read blog post.

answered Oct 17 '09 at 04:00
Joseph Turian
895 points
  • Thanks Joseph! Great suggestions. I've never heard of crowdspring, but it looks like an excellent resource for graphic design work. The quality appears a lot higher than 99designs. – Dlynton 14 years ago
  • Gah, my mistake, I meant Crowdflower, not Crowdspring. Check out Crowdflower. [editing original answer] – Joseph Turian 14 years ago

Your Answer

  • Bold
  • Italic
  • • Bullets
  • 1. Numbers
  • Quote
Not the answer you're looking for? Ask your own question or browse other questions in these topics:

Outsourcing Virtual Workforce