StackShareStackShare
Follow on
StackShare

Discover and share technology stacks from companies around the world.

Follow on

© 2025 StackShare. All rights reserved.

Product

  • Stacks
  • Tools
  • Feed

Company

  • About
  • Contact

Legal

  • Privacy Policy
  • Terms of Service
  1. Stackups
  2. Application & Data
  3. Databases
  4. Databases
  5. AWS Lambda vs Hadoop

AWS Lambda vs Hadoop

OverviewDecisionsComparisonAlternatives

Overview

Hadoop
Hadoop
Stacks2.7K
Followers2.3K
Votes56
GitHub Stars15.3K
Forks9.1K
AWS Lambda
AWS Lambda
Stacks26.0K
Followers18.8K
Votes432

AWS Lambda vs Hadoop: What are the differences?

Introduction

This markdown code provides a comparison between AWS Lambda and Hadoop, highlighting their key differences.

  1. Scalability: AWS Lambda is a serverless computing platform that automatically scales based on demand. It can handle individual requests or events in parallel and can scale up or down as needed. On the other hand, Hadoop is a distributed computing framework that allows for the storage and processing of large datasets across a cluster of computers. It provides parallel processing and fault tolerance for data-intensive applications.

  2. Execution Model: AWS Lambda follows an event-driven execution model, where functions are triggered by events and run in response to those events. It supports a wide range of event sources, including data changes, API calls, or scheduled events. In contrast, Hadoop runs on a batch processing model, where data is processed in a batch fashion. It requires explicit submission and processing of jobs.

  3. Managed Service vs Distributed Framework: AWS Lambda is a fully managed service provided by Amazon Web Services. It abstracts away server management and infrastructure concerns, allowing developers to focus on writing code. Hadoop, on the other hand, is a distributed framework that requires manual setup and configuration of a cluster of machines to operate.

  4. Use Cases: AWS Lambda is often used for serverless application development, event-driven computing, and building serverless microservices. It provides a flexible and cost-effective solution for executing small, isolated tasks in response to events. Hadoop, on the other hand, is designed for big data processing, including tasks such as data ingestion, data preparation, and data analysis. It excels in processing large volumes of data in parallel.

  5. Programming Models: AWS Lambda supports multiple programming languages, including Node.js, Python, Java, and C#. Developers can choose the language they are most comfortable with to write their functions. Hadoop, on the other hand, primarily supports Java for writing MapReduce jobs. It also provides support for other programming languages through high-level interfaces like Pig Latin and Hive Query Language.

  6. Data Storage: AWS Lambda does not provide built-in data storage. It is designed to integrate with other AWS services like S3, DynamoDB, and RDS to store and retrieve data. Hadoop, on the other hand, comes with its own distributed file system called Hadoop Distributed File System (HDFS). HDFS provides fault-tolerant storage for large datasets across a cluster of machines.

In summary, AWS Lambda is a managed service that provides serverless computing capabilities with an event-driven execution model, while Hadoop is a distributed computing framework suitable for big data processing with a batch processing model.

Share your Stack

Help developers discover the tools you use. Get visibility for your team's tech choices and contribute to the community's knowledge.

View Docs
CLI (Node.js)
or
Manual

Advice on Hadoop, AWS Lambda

Tim
Tim

CTO at Checkly Inc.

Sep 18, 2019

Needs adviceonHerokuHerokuAWS LambdaAWS Lambda

When adding a new feature to Checkly rearchitecting some older piece, I tend to pick Heroku for rolling it out. But not always, because sometimes I pick AWS Lambda . The short story:

  • Developer Experience trumps everything.
  • AWS Lambda is cheap. Up to a limit though. This impact not only your wallet.
  • If you need geographic spread, AWS is lonely at the top.

The setup

Recently, I was doing a brainstorm at a startup here in Berlin on the future of their infrastructure. They were ready to move on from their initial, almost 100% Ec2 + Chef based setup. Everything was on the table. But we crossed out a lot quite quickly:

  • Pure, uncut, self hosted Kubernetes — way too much complexity
  • Managed Kubernetes in various flavors — still too much complexity
  • Zeit — Maybe, but no Docker support
  • Elastic Beanstalk — Maybe, bit old but does the job
  • Heroku
  • Lambda

It became clear a mix of PaaS and FaaS was the way to go. What a surprise! That is exactly what I use for Checkly! But when do you pick which model?

I chopped that question up into the following categories:

  • Developer Experience / DX 🤓
  • Ops Experience / OX 🐂 (?)
  • Cost 💵
  • Lock in 🔐

Read the full post linked below for all details

357k views357k
Comments
Mark
Mark

Nov 2, 2020

Needs adviceonMicrosoft AzureMicrosoft Azure

Need advice on what platform, systems and tools to use.

Evaluating whether to start a new digital business for which we will need to build a website that handles all traffic. Website only right now. May add smartphone apps later. No desktop app will ever be added. Website to serve various countries and languages. B2B and B2C type customers. Need to handle heavy traffic, be low cost, and scale well.

We are open to either build it on AWS or on Microsoft Azure.

Apologies if I'm leaving out some info. My first post. :) Thanks in advance!

133k views133k
Comments
pionell
pionell

Sep 16, 2020

Needs adviceonMariaDBMariaDB

I have a lot of data that's currently sitting in a MariaDB database, a lot of tables that weigh 200gb with indexes. Most of the large tables have a date column which is always filtered, but there are usually 4-6 additional columns that are filtered and used for statistics. I'm trying to figure out the best tool for storing and analyzing large amounts of data. Preferably self-hosted or a cheap solution. The current problem I'm running into is speed. Even with pretty good indexes, if I'm trying to load a large dataset, it's pretty slow.

159k views159k
Comments

Detailed Comparison

Hadoop
Hadoop
AWS Lambda
AWS Lambda

The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage.

AWS Lambda is a compute service that runs your code in response to events and automatically manages the underlying compute resources for you. You can use AWS Lambda to extend other AWS services with custom logic, or create your own back-end services that operate at AWS scale, performance, and security.

-
Extend other AWS services with custom logic;Build custom back-end services;Completely Automated Administration;Built-in Fault Tolerance;Automatic Scaling;Integrated Security Model;Bring Your Own Code;Pay Per Use;Flexible Resource Model
Statistics
GitHub Stars
15.3K
GitHub Stars
-
GitHub Forks
9.1K
GitHub Forks
-
Stacks
2.7K
Stacks
26.0K
Followers
2.3K
Followers
18.8K
Votes
56
Votes
432
Pros & Cons
Pros
  • 39
    Great ecosystem
  • 11
    One stack to rule them all
  • 4
    Great load balancer
  • 1
    Java syntax
  • 1
    Amazon aws
Pros
  • 129
    No infrastructure
  • 83
    Cheap
  • 70
    Quick
  • 59
    Stateless
  • 47
    No deploy, no server, great sleep
Cons
  • 7
    Cant execute ruby or go
  • 3
    Compute time limited
  • 1
    Can't execute PHP w/o significant effort

What are some alternatives to Hadoop, AWS Lambda?

MongoDB

MongoDB

MongoDB stores data in JSON-like documents that can vary in structure, offering a dynamic, flexible schema. MongoDB was also designed for high availability and scalability, with built-in replication and auto-sharding.

MySQL

MySQL

The MySQL software delivers a very fast, multi-threaded, multi-user, and robust SQL (Structured Query Language) database server. MySQL Server is intended for mission-critical, heavy-load production systems as well as for embedding into mass-deployed software.

PostgreSQL

PostgreSQL

PostgreSQL is an advanced object-relational database management system that supports an extended subset of the SQL standard, including transactions, foreign keys, subqueries, triggers, user-defined types and functions.

Microsoft SQL Server

Microsoft SQL Server

Microsoft® SQL Server is a database management and analysis system for e-commerce, line-of-business, and data warehousing solutions.

SQLite

SQLite

SQLite is an embedded SQL database engine. Unlike most other SQL databases, SQLite does not have a separate server process. SQLite reads and writes directly to ordinary disk files. A complete SQL database with multiple tables, indices, triggers, and views, is contained in a single disk file.

Cassandra

Cassandra

Partitioning means that Cassandra can distribute your data across multiple machines in an application-transparent matter. Cassandra will automatically repartition as machines are added and removed from the cluster. Row store means that like relational databases, Cassandra organizes data by rows and columns. The Cassandra Query Language (CQL) is a close relative of SQL.

Memcached

Memcached

Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering.

MariaDB

MariaDB

Started by core members of the original MySQL team, MariaDB actively works with outside developers to deliver the most featureful, stable, and sanely licensed open SQL server in the industry. MariaDB is designed as a drop-in replacement of MySQL(R) with more features, new storage engines, fewer bugs, and better performance.

RethinkDB

RethinkDB

RethinkDB is built to store JSON documents, and scale to multiple machines with very little effort. It has a pleasant query language that supports really useful queries like table joins and group by, and is easy to setup and learn.

ArangoDB

ArangoDB

A distributed free and open-source database with a flexible data model for documents, graphs, and key-values. Build high performance applications using a convenient SQL-like query language or JavaScript extensions.

Related Comparisons

Bootstrap
Materialize

Bootstrap vs Materialize

Laravel
Django

Django vs Laravel vs Node.js

Bootstrap
Foundation

Bootstrap vs Foundation vs Material UI

Node.js
Spring Boot

Node.js vs Spring-Boot

Liquibase
Flyway

Flyway vs Liquibase