Alternatives to Redpanda logo

Alternatives to Redpanda

Kafka Streams, Apache NiFi, Confluent, Apache Storm, and KSQL are the most popular alternatives and competitors to Redpanda.
10
11
+ 1
0

What is Redpanda and what are its top alternatives?

It is a streaming platform for mission critical workloads. Kafka® compatible, No Zookeeper®, no JVM, and no code changes required. Use all your favorite open source tooling - 10x faster.
Redpanda is a tool in the Stream Processing category of a tech stack.
Redpanda is an open source tool with GitHub stars and GitHub forks. Here’s a link to Redpanda's open source repository on GitHub

Top Alternatives to Redpanda

  • Kafka Streams
    Kafka Streams

    It is a client library for building applications and microservices, where the input and output data are stored in Kafka clusters. It combines the simplicity of writing and deploying standard Java and Scala applications on the client side with the benefits of Kafka's server-side cluster technology. ...

  • Apache NiFi
    Apache NiFi

    An easy to use, powerful, and reliable system to process and distribute data. It supports powerful and scalable directed graphs of data routing, transformation, and system mediation logic. ...

  • Confluent
    Confluent

    It is a data streaming platform based on Apache Kafka: a full-scale streaming platform, capable of not only publish-and-subscribe, but also the storage and processing of data within the stream ...

  • Apache Storm
    Apache Storm

    Apache Storm is a free and open source distributed realtime computation system. Storm makes it easy to reliably process unbounded streams of data, doing for realtime processing what Hadoop did for batch processing. Storm has many use cases: realtime analytics, online machine learning, continuous computation, distributed RPC, ETL, and more. Storm is fast: a benchmark clocked it at over a million tuples processed per second per node. It is scalable, fault-tolerant, guarantees your data will be processed, and is easy to set up and operate. ...

  • KSQL
    KSQL

    KSQL is an open source streaming SQL engine for Apache Kafka. It provides a simple and completely interactive SQL interface for stream processing on Kafka; no need to write code in a programming language such as Java or Python. KSQL is open-source (Apache 2.0 licensed), distributed, scalable, reliable, and real-time. ...

  • Kapacitor
    Kapacitor

    It is a native data processing engine for InfluxDB 1.x and is an integrated component in the InfluxDB 2.0 platform. It can process both stream and batch data from InfluxDB, acting on this data in real-time via its programming language TICKscript. ...

  • Heron
    Heron

    Heron is realtime analytics platform developed by Twitter. It is the direct successor of Apache Storm, built to be backwards compatible with Storm's topology API but with a wide array of architectural improvements. ...

  • Faust
    Faust

    It is a stream processing library, porting the ideas from Kafka Streams to Python. It provides both stream processing and event processing, sharing similarity with tools such as Kafka Streams, Apache Spark/Storm/Samza/Flink. ...

Redpanda alternatives & related posts

Kafka Streams logo

Kafka Streams

321
389
0
A client library for building applications and microservices
321
389
+ 1
0
PROS OF KAFKA STREAMS
    Be the first to leave a pro
    CONS OF KAFKA STREAMS
      Be the first to leave a con

      related Kafka Streams posts

      I have recently started using Confluent/Kafka cloud. We want to do some stream processing. As I was going through Kafka I came across Kafka Streams and KSQL. Both seem to be A good fit for stream processing. But I could not understand which one should be used and one has any advantage over another. We will be using Confluent/Kafka Managed Cloud Instance. In near future, our Producers and Consumers are running on premise and we will be interacting with Confluent Cloud.

      Also, Confluent Cloud Kafka has a primitive interface; is there any better UI interface to manage Kafka Cloud Cluster?

      See more
      Shared insights
      on
      Apache FlinkApache FlinkKafka StreamsKafka Streams

      We currently have 2 Kafka Streams topics that have records coming in continuously. We're looking into joining the 2 streams based on a key with a window of 5 minutes based on their timestamp.

      Should I consider kStream - kStream join or Apache Flink window joins? Or is there any other better way to achieve this?

      See more
      Apache NiFi logo

      Apache NiFi

      289
      577
      62
      A reliable system to process and distribute data
      289
      577
      + 1
      62
      PROS OF APACHE NIFI
      • 15
        Visual Data Flows using Directed Acyclic Graphs (DAGs)
      • 8
        Free (Open Source)
      • 7
        Simple-to-use
      • 5
        Reactive with back-pressure
      • 5
        Scalable horizontally as well as vertically
      • 4
        Fast prototyping
      • 3
        Bi-directional channels
      • 2
        Data provenance
      • 2
        Built-in graphical user interface
      • 2
        End-to-end security between all nodes
      • 2
        Can handle messages up to gigabytes in size
      • 1
        Hbase support
      • 1
        Kudu support
      • 1
        Hive support
      • 1
        Slack integration
      • 1
        Support for custom Processor in Java
      • 1
        Lot of articles
      • 1
        Lots of documentation
      CONS OF APACHE NIFI
      • 2
        HA support is not full fledge
      • 2
        Memory-intensive

      related Apache NiFi posts

      I am looking for the best tool to orchestrate #ETL workflows in non-Hadoop environments, mainly for regression testing use cases. Would Airflow or Apache NiFi be a good fit for this purpose?

      For example, I want to run an Informatica ETL job and then run an SQL task as a dependency, followed by another task from Jira. What tool is best suited to set up such a pipeline?

      See more
      Confluent logo

      Confluent

      194
      177
      13
      A stream data platform to help companies harness their high volume real-time data streams
      194
      177
      + 1
      13
      PROS OF CONFLUENT
      • 3
        No hypercloud lock-in
      • 3
        Dashboard for kafka insight
      • 3
        Free for casual use
      • 2
        Easily scalable
      • 2
        Zero devops
      CONS OF CONFLUENT
      • 1
        Proprietary

      related Confluent posts

      I have recently started using Confluent/Kafka cloud. We want to do some stream processing. As I was going through Kafka I came across Kafka Streams and KSQL. Both seem to be A good fit for stream processing. But I could not understand which one should be used and one has any advantage over another. We will be using Confluent/Kafka Managed Cloud Instance. In near future, our Producers and Consumers are running on premise and we will be interacting with Confluent Cloud.

      Also, Confluent Cloud Kafka has a primitive interface; is there any better UI interface to manage Kafka Cloud Cluster?

      See more
      Apache Storm logo

      Apache Storm

      186
      268
      24
      Distributed and fault-tolerant realtime computation
      186
      268
      + 1
      24
      PROS OF APACHE STORM
      • 10
        Flexible
      • 6
        Easy setup
      • 3
        Clojure
      • 3
        Event Processing
      • 2
        Real Time
      CONS OF APACHE STORM
        Be the first to leave a con

        related Apache Storm posts

        Marc Bollinger
        Infra & Data Eng Manager at Thumbtack · | 5 upvotes · 505.4K views

        Lumosity is home to the world's largest cognitive training database, a responsibility we take seriously. For most of the company's history, our analysis of user behavior and training data has been powered by an event stream--first a simple Node.js pub/sub app, then a heavyweight Ruby app with stronger durability. Both supported decent throughput and latency, but they lacked some major features supported by existing open-source alternatives: replaying existing messages (also lacking in most message queue-based solutions), scaling out many different readers for the same stream, the ability to leverage existing solutions for reading and writing, and possibly most importantly: the ability to hire someone externally who already had expertise.

        We ultimately migrated to Kafka in early- to mid-2016, citing both industry trends in companies we'd talked to with similar durability and throughput needs, the extremely strong documentation and community. We pored over Kyle Kingsbury's Jepsen post (https://aphyr.com/posts/293-jepsen-Kafka), as well as Jay Kreps' follow-up (http://blog.empathybox.com/post/62279088548/a-few-notes-on-kafka-and-jepsen), talked at length with Confluent folks and community members, and still wound up running parallel systems for quite a long time, but ultimately, we've been very, very happy. Understanding the internals and proper levers takes some commitment, but it's taken very little maintenance once configured. Since then, the Confluent Platform community has grown and grown; we've gone from doing most development using custom Scala consumers and producers to being 60/40 Kafka Streams/Connects.

        We originally looked into Storm / Heron , and we'd moved on from Redis pub/sub. Heron looks great, but we already had a programming model across services that was more akin to consuming a message consumers than required a topology of bolts, etc. Heron also had just come out while we were starting to migrate things, and the community momentum and direction of Kafka felt more substantial than the older Storm. If we were to start the process over again today, we might check out Pulsar , although the ecosystem is much younger.

        To find out more, read our 2017 engineering blog post about the migration!

        See more
        KSQL logo

        KSQL

        51
        103
        5
        Open source streaming SQL for Apache Kafka
        51
        103
        + 1
        5
        PROS OF KSQL
        • 3
          Streamprocessing on Kafka
        • 2
          SQL syntax with windowing functions over streams
        • 0
          Easy transistion for SQL Devs
        CONS OF KSQL
          Be the first to leave a con

          related KSQL posts

          I have recently started using Confluent/Kafka cloud. We want to do some stream processing. As I was going through Kafka I came across Kafka Streams and KSQL. Both seem to be A good fit for stream processing. But I could not understand which one should be used and one has any advantage over another. We will be using Confluent/Kafka Managed Cloud Instance. In near future, our Producers and Consumers are running on premise and we will be interacting with Confluent Cloud.

          Also, Confluent Cloud Kafka has a primitive interface; is there any better UI interface to manage Kafka Cloud Cluster?

          See more
          Kapacitor logo

          Kapacitor

          39
          48
          0
          A real-time streaming data processing engine
          39
          48
          + 1
          0
          PROS OF KAPACITOR
            Be the first to leave a pro
            CONS OF KAPACITOR
              Be the first to leave a con

              related Kapacitor posts

              Heron logo

              Heron

              22
              58
              4
              Realtime, distributed, fault-tolerant stream processing engine from Twitter
              22
              58
              + 1
              4
              PROS OF HERON
              • 1
                Highly Customizable
              • 1
                Support most popular container environment
              • 1
                Operation friendly
              • 1
                Realtime Stream Processing
              CONS OF HERON
                Be the first to leave a con

                related Heron posts

                Marc Bollinger
                Infra & Data Eng Manager at Thumbtack · | 5 upvotes · 505.4K views

                Lumosity is home to the world's largest cognitive training database, a responsibility we take seriously. For most of the company's history, our analysis of user behavior and training data has been powered by an event stream--first a simple Node.js pub/sub app, then a heavyweight Ruby app with stronger durability. Both supported decent throughput and latency, but they lacked some major features supported by existing open-source alternatives: replaying existing messages (also lacking in most message queue-based solutions), scaling out many different readers for the same stream, the ability to leverage existing solutions for reading and writing, and possibly most importantly: the ability to hire someone externally who already had expertise.

                We ultimately migrated to Kafka in early- to mid-2016, citing both industry trends in companies we'd talked to with similar durability and throughput needs, the extremely strong documentation and community. We pored over Kyle Kingsbury's Jepsen post (https://aphyr.com/posts/293-jepsen-Kafka), as well as Jay Kreps' follow-up (http://blog.empathybox.com/post/62279088548/a-few-notes-on-kafka-and-jepsen), talked at length with Confluent folks and community members, and still wound up running parallel systems for quite a long time, but ultimately, we've been very, very happy. Understanding the internals and proper levers takes some commitment, but it's taken very little maintenance once configured. Since then, the Confluent Platform community has grown and grown; we've gone from doing most development using custom Scala consumers and producers to being 60/40 Kafka Streams/Connects.

                We originally looked into Storm / Heron , and we'd moved on from Redis pub/sub. Heron looks great, but we already had a programming model across services that was more akin to consuming a message consumers than required a topology of bolts, etc. Heron also had just come out while we were starting to migrate things, and the community momentum and direction of Kafka felt more substantial than the older Storm. If we were to start the process over again today, we might check out Pulsar , although the ecosystem is much younger.

                To find out more, read our 2017 engineering blog post about the migration!

                See more
                Faust logo

                Faust

                21
                61
                0
                A library for building streaming applications in Python
                21
                61
                + 1
                0
                PROS OF FAUST
                  Be the first to leave a pro
                  CONS OF FAUST
                    Be the first to leave a con

                    related Faust posts