Amazon Mechanical Turk vs UserTesting

Need advice about which tool to choose?Ask the StackShare community!

Amazon Mechanical Turk

+ 1

+ 1
Add tool

Amazon Mechanical Turk vs UserTesting: What are the differences?


Below are the key differences between Amazon Mechanical Turk and UserTesting.

  1. Pricing and Payments: Amazon Mechanical Turk (AMT) offers a flexible pricing structure, allowing requesters to set their own prices for completed tasks. Requesters must pay fees to Amazon for using the platform, in addition to paying the workers. On the other hand, UserTesting has a fixed pricing model, with packages starting at a specific price per participant. Payment to workers on AMT is usually in the form of gift cards or cash, while UserTesting pays participants directly via PayPal.

  2. Task Variety and Complexity: AMT provides a wide range of tasks, including data validation, categorization, content moderation, surveys, and more. The tasks can vary in complexity, from simple tasks that take a few seconds to complete to more complex tasks that require specific skills or expertise. In contrast, UserTesting focuses primarily on user experience testing, including website and app usability testing, prototype testing, and UI/UX feedback. The tasks on UserTesting are generally focused on evaluating the user experience and usability aspects of a product.

  3. Worker Pool and Quality Control: AMT has a large pool of workers, often referred to as "turkers," who are available to complete tasks. The worker pool comprises a diverse range of individuals, including both experienced workers and those who are new to the platform. While AMT provides some tools for quality control, such as qualifications and reviews, maintaining consistent task quality can be a challenge due to the large worker pool and varying skill levels. UserTesting, on the other hand, has a curated panel of testers who are selected based on specific demographics and criteria. This allows UserTesting to ensure higher quality feedback and user testing results.

  4. Task Turnaround Time: Due to the large worker pool and potentially lower payment rates, tasks on AMT can be completed relatively quickly. Depending on the complexity of the task and the payment offered, results can be obtained within minutes or hours. UserTesting, on the other hand, typically requires a minimum of 24 hours to recruit and receive feedback from participants. This longer turnaround time may be necessary to ensure participants have sufficient time to complete the assigned tasks and provide comprehensive feedback.

  5. Interaction with Participants: AMT tasks are often designed as discrete tasks that do not typically require direct interaction between the requester and the worker. Communication between the requester and the worker is limited to specific instructions provided in the task. In contrast, UserTesting allows requesters to interact with participants through live conversations, interviews, and follow-up questions. This level of interaction enables UserTesting to gather more qualitative insights and clarifications directly from the participants.

  6. Expertise and Support: Amazon Mechanical Turk is a self-serve platform, offering limited support to both requesters and workers. While they provide documentation and resources, users are expected to have a certain level of expertise to effectively use the platform. UserTesting, on the other hand, provides more hands-on support to both requesters and participants. They offer guidance and assistance in setting up tests, recruiting participants, analyzing results, and conducting user research.

In summary, Amazon Mechanical Turk provides a broader range of tasks, a larger worker pool, and quicker task completion, but may have lower overall quality control. UserTesting focuses specifically on user experience testing, employs a curated panel of testers, offers more interaction options, and provides more support throughout the testing process.

Get Advice from developers at your company using StackShare Enterprise. Sign up for StackShare Enterprise.
Learn More
Pros of Amazon Mechanical Turk
Pros of UserTesting
    Be the first to leave a pro
    • 2
    • 1
      Detailed metrics

    Sign up to add or upvote prosMake informed product decisions

    What is Amazon Mechanical Turk?

    Amazon Mechanical Turk is a marketplace for work that requires human intelligence. The Mechanical Turk web service enables companies to programmatically access this marketplace and a diverse, on-demand workforce. Developers can leverage this service to build human intelligence directly into their applications.

    What is UserTesting?

    UserTesting provides on-demand usability testing. You create the test and we’ll get the testers. We let you “look over the shoulder” of your target audience while they use your website, so you can see and hear where users get stuck and why they leave.

    Need advice about which tool to choose?Ask the StackShare community!

    What companies use Amazon Mechanical Turk?
    What companies use UserTesting?
    See which teams inside your own company are using Amazon Mechanical Turk or UserTesting.
    Sign up for StackShare EnterpriseLearn More

    Sign up to get full access to all the companiesMake informed product decisions

    What tools integrate with Amazon Mechanical Turk?
    What tools integrate with UserTesting?
      No integrations found
      What are some alternatives to Amazon Mechanical Turk and UserTesting?
      CrowdFlower is the world's leading crowdsourcing service, with over 800 million tasks submitted by over four million contributors. We specialize in microtasking: distributing small, discrete tasks to many online contributors, assembly-line fashion - for instance, using people to check hundreds of thousands of photos every day for obscene content.
      JavaScript is most known as the scripting language for Web pages, but used in many non-browser environments as well such as node.js or Apache CouchDB. It is a prototype-based, multi-paradigm scripting language that is dynamic,and supports object-oriented, imperative, and functional programming styles.
      Git is a free and open source distributed version control system designed to handle everything from small to very large projects with speed and efficiency.
      GitHub is the best place to share code with friends, co-workers, classmates, and complete strangers. Over three million people use GitHub to build amazing things together.
      Python is a general purpose programming language created by Guido Van Rossum. Python is most praised for its elegant syntax and readable code, if you are just beginning your programming career python suits you best.
      See all alternatives