Need advice about which tool to choose?Ask the StackShare community!
Decisions about Amazon S3, Hadoop, and Pachyderm
Gabriel Pa
CEO at NaoLogic Inc · | 2 upvotes · 256K views
We offer our customer HIPAA compliant storage. After analyzing the market, we decided to go with Google Storage. The Nodejs API is ok, still not ES6 and can be very confusing to use. For each new customer, we created a different bucket so they can have individual data and not have to worry about data loss. After 1000+ customers we started seeing many problems with the creation of new buckets, with saving or retrieving a new file. Many false positive: the Promise returned ok, but in reality, it failed.
That's why we switched to S3 that just works.
Get Advice from developers at your company using StackShare Enterprise. Sign up for StackShare Enterprise.
Learn MorePros of Amazon S3
Pros of Hadoop
Pros of Pachyderm
Pros of Amazon S3
- Reliable592
- Scalable493
- Cheap458
- Simple & easy329
- Many sdks83
- Logical30
- Easy Setup13
- 1000+ POPs11
- REST API11
- Secure6
- Easy4
- Plug and play4
- Web UI for uploading files3
- Flexible2
- Faster on response2
- GDPR ready2
- Easy integration with CloudFront1
- Easy to use1
- Plug-gable1
Pros of Hadoop
- Great ecosystem39
- One stack to rule them all11
- Great load balancer4
- Amazon aws1
- Java syntax1
Pros of Pachyderm
- Containers3
- Versioning1
- Can run on GCP or AWS1
Sign up to add or upvote prosMake informed product decisions
Cons of Amazon S3
Cons of Hadoop
Cons of Pachyderm
Cons of Amazon S3
- Permissions take some time to get right7
- Takes time/work to organize buckets & folders properly6
- Requires a credit card5
- Complex to set up3
Cons of Hadoop
Be the first to leave a con
Cons of Pachyderm
Be the first to leave a con
Sign up to add or upvote consMake informed product decisions
- No public GitHub repository available -
- No public GitHub repository available -
What is Amazon S3?
Amazon Simple Storage Service provides a fully redundant data storage infrastructure for storing and retrieving any amount of data, at any time, from anywhere on the web
What is Hadoop?
The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage.
What is Pachyderm?
Pachyderm is an open source MapReduce engine that uses Docker containers for distributed computations.
Need advice about which tool to choose?Ask the StackShare community!
Jobs that mention Amazon S3, Hadoop, and Pachyderm as a desired skillset
What companies use Amazon S3?
What companies use Hadoop?
What companies use Pachyderm?
What companies use Pachyderm?
Sign up to get full access to all the companiesMake informed product decisions
What tools integrate with Amazon S3?
What tools integrate with Hadoop?
What tools integrate with Pachyderm?
What tools integrate with Amazon S3?
What tools integrate with Hadoop?
What tools integrate with Pachyderm?
Sign up to get full access to all the tool integrationsMake informed product decisions
Blog Posts
+4 2
1200
+5 3
901
+5 8
1506
+7 5
1367
+6 4
567
+4 2
1181
+4 4
1091
+17 6
9072
What are some alternatives to Amazon S3, Hadoop, and Pachyderm?
Amazon Glacier
In order to keep costs low, Amazon Glacier is optimized for data that is infrequently accessed and for which retrieval times of several hours are suitable. With Amazon Glacier, customers can reliably store large or small amounts of data for as little as $0.01 per gigabyte per month, a significant savings compared to on-premises solutions.
Amazon EBS
Amazon EBS volumes are network-attached, and persist independently from the life of an instance. Amazon EBS provides highly available, highly reliable, predictable storage volumes that can be attached to a running Amazon EC2 instance and exposed as a device within the instance. Amazon EBS is particularly suited for applications that require a database, file system, or access to raw block level storage.
Amazon EC2
It is a web service that provides resizable compute capacity in the cloud. It is designed to make web-scale computing easier for developers.
Google Drive
Keep photos, stories, designs, drawings, recordings, videos, and more. Your first 15 GB of storage are free with a Google Account. Your files in Drive can be reached from any smartphone, tablet, or computer.
Microsoft Azure
Azure is an open and flexible cloud platform that enables you to quickly build, deploy and manage applications across a global network of Microsoft-managed datacenters. You can build applications using any language, tool or framework. And you can integrate your public cloud applications with your existing IT environment.