Amazon S3 vs Hadoop vs Pachyderm

Need advice about which tool to choose?Ask the StackShare community!

Amazon S3

49.1K
36.5K
+ 1
2K
Hadoop

2.1K
2.2K
+ 1
56
Pachyderm

21
88
+ 1
5
Decisions about Amazon S3, Hadoop, and Pachyderm
Gabriel Pa

We offer our customer HIPAA compliant storage. After analyzing the market, we decided to go with Google Storage. The Nodejs API is ok, still not ES6 and can be very confusing to use. For each new customer, we created a different bucket so they can have individual data and not have to worry about data loss. After 1000+ customers we started seeing many problems with the creation of new buckets, with saving or retrieving a new file. Many false positive: the Promise returned ok, but in reality, it failed.

That's why we switched to S3 that just works.

See more
Get Advice from developers at your company using StackShare Enterprise. Sign up for StackShare Enterprise.
Learn More
Pros of Amazon S3
Pros of Hadoop
Pros of Pachyderm
  • 592
    Reliable
  • 493
    Scalable
  • 458
    Cheap
  • 329
    Simple & easy
  • 83
    Many sdks
  • 30
    Logical
  • 13
    Easy Setup
  • 11
    1000+ POPs
  • 11
    REST API
  • 6
    Secure
  • 4
    Easy
  • 4
    Plug and play
  • 3
    Web UI for uploading files
  • 2
    Flexible
  • 2
    Faster on response
  • 2
    GDPR ready
  • 1
    Easy integration with CloudFront
  • 1
    Easy to use
  • 1
    Plug-gable
  • 39
    Great ecosystem
  • 11
    One stack to rule them all
  • 4
    Great load balancer
  • 1
    Amazon aws
  • 1
    Java syntax
  • 3
    Containers
  • 1
    Versioning
  • 1
    Can run on GCP or AWS

Sign up to add or upvote prosMake informed product decisions

Cons of Amazon S3
Cons of Hadoop
Cons of Pachyderm
  • 7
    Permissions take some time to get right
  • 6
    Takes time/work to organize buckets & folders properly
  • 5
    Requires a credit card
  • 3
    Complex to set up
    Be the first to leave a con
      Be the first to leave a con

      Sign up to add or upvote consMake informed product decisions

      - No public GitHub repository available -
      - No public GitHub repository available -

      What is Amazon S3?

      Amazon Simple Storage Service provides a fully redundant data storage infrastructure for storing and retrieving any amount of data, at any time, from anywhere on the web

      What is Hadoop?

      The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage.

      What is Pachyderm?

      Pachyderm is an open source MapReduce engine that uses Docker containers for distributed computations.

      Need advice about which tool to choose?Ask the StackShare community!

      What companies use Amazon S3?
      What companies use Hadoop?
      What companies use Pachyderm?

      Sign up to get full access to all the companiesMake informed product decisions

      What tools integrate with Amazon S3?
      What tools integrate with Hadoop?
      What tools integrate with Pachyderm?

      Sign up to get full access to all the tool integrationsMake informed product decisions

      What are some alternatives to Amazon S3, Hadoop, and Pachyderm?
      Amazon Glacier
      In order to keep costs low, Amazon Glacier is optimized for data that is infrequently accessed and for which retrieval times of several hours are suitable. With Amazon Glacier, customers can reliably store large or small amounts of data for as little as $0.01 per gigabyte per month, a significant savings compared to on-premises solutions.
      Amazon EBS
      Amazon EBS volumes are network-attached, and persist independently from the life of an instance. Amazon EBS provides highly available, highly reliable, predictable storage volumes that can be attached to a running Amazon EC2 instance and exposed as a device within the instance. Amazon EBS is particularly suited for applications that require a database, file system, or access to raw block level storage.
      Amazon EC2
      It is a web service that provides resizable compute capacity in the cloud. It is designed to make web-scale computing easier for developers.
      Google Drive
      Keep photos, stories, designs, drawings, recordings, videos, and more. Your first 15 GB of storage are free with a Google Account. Your files in Drive can be reached from any smartphone, tablet, or computer.
      Microsoft Azure
      Azure is an open and flexible cloud platform that enables you to quickly build, deploy and manage applications across a global network of Microsoft-managed datacenters. You can build applications using any language, tool or framework. And you can integrate your public cloud applications with your existing IT environment.
      See all alternatives