Delta Lake vs Apache Flink vs Apache Spark

Need advice about which tool to choose?Ask the StackShare community!

Delta Lake

99
314
+ 1
0
Apache Flink

526
871
+ 1
38
Apache Spark

3K
3.5K
+ 1
140
Manage your open source components, licenses, and vulnerabilities
Learn More
Pros of Delta Lake
Pros of Apache Flink
Pros of Apache Spark
    Be the first to leave a pro
    • 16
      Unified batch and stream processing
    • 8
      Easy to use streaming apis
    • 8
      Out-of-the box connector to kinesis,s3,hdfs
    • 4
      Open Source
    • 2
      Low latency
    • 61
      Open-source
    • 48
      Fast and Flexible
    • 8
      One platform for every big data problem
    • 8
      Great for distributed SQL like applications
    • 6
      Easy to install and to use
    • 3
      Works well for most Datascience usecases
    • 2
      Interactive Query
    • 2
      Machine learning libratimery, Streaming in real
    • 2
      In memory Computation

    Sign up to add or upvote prosMake informed product decisions

    Cons of Delta Lake
    Cons of Apache Flink
    Cons of Apache Spark
      Be the first to leave a con
        Be the first to leave a con
        • 4
          Speed

        Sign up to add or upvote consMake informed product decisions

        What is Delta Lake?

        An open-source storage layer that brings ACID transactions to Apache Spark™ and big data workloads.

        What is Apache Flink?

        Apache Flink is an open source system for fast and versatile data analytics in clusters. Flink supports batch and streaming analytics, in one system. Analytical programs can be written in concise and elegant APIs in Java and Scala.

        What is Apache Spark?

        Spark is a fast and general processing engine compatible with Hadoop data. It can run in Hadoop clusters through YARN or Spark's standalone mode, and it can process data in HDFS, HBase, Cassandra, Hive, and any Hadoop InputFormat. It is designed to perform both batch processing (similar to MapReduce) and new workloads like streaming, interactive queries, and machine learning.

        Need advice about which tool to choose?Ask the StackShare community!

        What companies use Delta Lake?
        What companies use Apache Flink?
        What companies use Apache Spark?

        Sign up to get full access to all the companiesMake informed product decisions

        What tools integrate with Delta Lake?
        What tools integrate with Apache Flink?
        What tools integrate with Apache Spark?

        Sign up to get full access to all the tool integrationsMake informed product decisions

        Blog Posts

        What are some alternatives to Delta Lake, Apache Flink, and Apache Spark?
        Snowflake
        Snowflake eliminates the administration and management demands of traditional data warehouses and big data platforms. Snowflake is a true data warehouse as a service running on Amazon Web Services (AWS)—no infrastructure to manage and no knobs to turn.
        MySQL
        The MySQL software delivers a very fast, multi-threaded, multi-user, and robust SQL (Structured Query Language) database server. MySQL Server is intended for mission-critical, heavy-load production systems as well as for embedding into mass-deployed software.
        PostgreSQL
        PostgreSQL is an advanced object-relational database management system that supports an extended subset of the SQL standard, including transactions, foreign keys, subqueries, triggers, user-defined types and functions.
        MongoDB
        MongoDB stores data in JSON-like documents that can vary in structure, offering a dynamic, flexible schema. MongoDB was also designed for high availability and scalability, with built-in replication and auto-sharding.
        Redis
        Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache, and message broker. Redis provides data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes, and streams.
        See all alternatives