Alternatives to Apache Kudu logo

Alternatives to Apache Kudu

Cassandra, HBase, Apache Spark, Apache Impala, and Hadoop are the most popular alternatives and competitors to Apache Kudu.
66
238
+ 1
10

What is Apache Kudu and what are its top alternatives?

A new addition to the open source Apache Hadoop ecosystem, Kudu completes Hadoop's storage layer to enable fast analytics on fast data.
Apache Kudu is a tool in the Big Data Tools category of a tech stack.
Apache Kudu is an open source tool with 828 GitHub stars and 282 GitHub forks. Here’s a link to Apache Kudu's open source repository on GitHub

Top Alternatives to Apache Kudu

  • Cassandra
    Cassandra

    Partitioning means that Cassandra can distribute your data across multiple machines in an application-transparent matter. Cassandra will automatically repartition as machines are added and removed from the cluster. Row store means that like relational databases, Cassandra organizes data by rows and columns. The Cassandra Query Language (CQL) is a close relative of SQL. ...

  • HBase
    HBase

    Apache HBase is an open-source, distributed, versioned, column-oriented store modeled after Google' Bigtable: A Distributed Storage System for Structured Data by Chang et al. Just as Bigtable leverages the distributed data storage provided by the Google File System, HBase provides Bigtable-like capabilities on top of Apache Hadoop. ...

  • Apache Spark
    Apache Spark

    Spark is a fast and general processing engine compatible with Hadoop data. It can run in Hadoop clusters through YARN or Spark's standalone mode, and it can process data in HDFS, HBase, Cassandra, Hive, and any Hadoop InputFormat. It is designed to perform both batch processing (similar to MapReduce) and new workloads like streaming, interactive queries, and machine learning. ...

  • Apache Impala
    Apache Impala

    Impala is a modern, open source, MPP SQL query engine for Apache Hadoop. Impala is shipped by Cloudera, MapR, and Amazon. With Impala, you can query data, whether stored in HDFS or Apache HBase – including SELECT, JOIN, and aggregate functions – in real time. ...

  • Hadoop
    Hadoop

    The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage. ...

  • Druid
    Druid

    Druid is a distributed, column-oriented, real-time analytics data store that is commonly used to power exploratory dashboards in multi-tenant environments. Druid excels as a data warehousing solution for fast aggregate queries on petabyte sized data sets. Druid supports a variety of flexible filters, exact calculations, approximate algorithms, and other useful calculations. ...

  • Apache Ignite
    Apache Ignite

    It is a memory-centric distributed database, caching, and processing platform for transactional, analytical, and streaming workloads delivering in-memory speeds at petabyte scale ...

  • Clickhouse
    Clickhouse

    It allows analysis of data that is updated in real time. It offers instant results in most cases: the data is processed faster than it takes to create a query. ...

Apache Kudu alternatives & related posts

Cassandra logo

Cassandra

3.3K
3.4K
500
A partitioned row store. Rows are organized into tables with a required primary key.
3.3K
3.4K
+ 1
500
PROS OF CASSANDRA
  • 116
    Distributed
  • 97
    High performance
  • 81
    High availability
  • 74
    Easy scalability
  • 52
    Replication
  • 26
    Reliable
  • 26
    Multi datacenter deployments
  • 9
    OLTP
  • 8
    Schema optional
  • 8
    Open source
  • 2
    Workload separation (via MDC)
  • 1
    Fast
CONS OF CASSANDRA
  • 3
    Reliability of replication
  • 1
    Size
  • 1
    Updates

related Cassandra posts

Thierry Schellenbach
Shared insights
on
RedisRedisCassandraCassandraRocksDBRocksDB
at

1.0 of Stream leveraged Cassandra for storing the feed. Cassandra is a common choice for building feeds. Instagram, for instance started, out with Redis but eventually switched to Cassandra to handle their rapid usage growth. Cassandra can handle write heavy workloads very efficiently.

Cassandra is a great tool that allows you to scale write capacity simply by adding more nodes, though it is also very complex. This complexity made it hard to diagnose performance fluctuations. Even though we had years of experience with running Cassandra, it still felt like a bit of a black box. When building Stream 2.0 we decided to go for a different approach and build Keevo. Keevo is our in-house key-value store built upon RocksDB, gRPC and Raft.

RocksDB is a highly performant embeddable database library developed and maintained by Facebook’s data engineering team. RocksDB started as a fork of Google’s LevelDB that introduced several performance improvements for SSD. Nowadays RocksDB is a project on its own and is under active development. It is written in C++ and it’s fast. Have a look at how this benchmark handles 7 million QPS. In terms of technology it’s much more simple than Cassandra.

This translates into reduced maintenance overhead, improved performance and, most importantly, more consistent performance. It’s interesting to note that LinkedIn also uses RocksDB for their feed.

#InMemoryDatabases #DataStores #Databases

See more
Umair Iftikhar
Technical Architect at ERP Studio · | 3 upvotes · 299.6K views

Developing a solution that collects Telemetry Data from different devices, nearly 1000 devices minimum and maximum 12000. Each device is sending 2 packets in 1 second. This is time-series data, and this data definition and different reports are saved on PostgreSQL. Like Building information, maintenance records, etc. I want to know about the best solution. This data is required for Math and ML to run different algorithms. Also, data is raw without definitions and information stored in PostgreSQL. Initially, I went with TimescaleDB due to PostgreSQL support, but to increase in sites, I started facing many issues with timescale DB in terms of flexibility of storing data.

My major requirement is also the replication of the database for reporting and different purposes. You may also suggest other options other than Druid and Cassandra. But an open source solution is appreciated.

See more
HBase logo

HBase

374
462
15
The Hadoop database, a distributed, scalable, big data store
374
462
+ 1
15
PROS OF HBASE
  • 9
    Performance
  • 5
    OLTP
  • 1
    Fast Point Queries
CONS OF HBASE
    Be the first to leave a con

    related HBase posts

    I am researching different querying solutions to handle ~1 trillion records of data (in the realm of a petabyte). The data is mostly textual. I have identified a few options: Milvus, HBase, RocksDB, and Elasticsearch. I was wondering if there is a good way to compare the performance of these options (or if anyone has already done something like this). I want to be able to compare the speed of ingesting and querying textual data from these tools. Does anyone have information on this or know where I can find some? Thanks in advance!

    See more

    Hi, I'm building a machine learning pipelines to store image bytes and image vectors in the backend.

    So, when users query for the random access image data (key), we return the image bytes and perform machine learning model operations on it.

    I'm currently considering going with Amazon S3 (in the future, maybe add Redis caching layer) as the backend system to store the information (s3 buckets with sharded prefixes).

    As the latency of S3 is 100-200ms (get/put) and it has a high throughput of 3500 puts/sec and 5500 gets/sec for a given bucker/prefix. In the future I need to reduce the latency, I can add Redis cache.

    Also, s3 costs are way fewer than HBase (on Amazon EC2 instances with 3x replication factor)

    I have not personally used HBase before, so can someone help me if I'm making the right choice here? I'm not aware of Hbase latencies and I have learned that the MOB feature on Hbase has to be turned on if we have store image bytes on of the column families as the avg image bytes are 240Kb.

    See more
    Apache Spark logo

    Apache Spark

    2.7K
    3.2K
    139
    Fast and general engine for large-scale data processing
    2.7K
    3.2K
    + 1
    139
    PROS OF APACHE SPARK
    • 60
      Open-source
    • 48
      Fast and Flexible
    • 8
      Great for distributed SQL like applications
    • 8
      One platform for every big data problem
    • 6
      Easy to install and to use
    • 3
      Works well for most Datascience usecases
    • 2
      Interactive Query
    • 2
      In memory Computation
    • 2
      Machine learning libratimery, Streaming in real
    CONS OF APACHE SPARK
    • 3
      Speed

    related Apache Spark posts

    Eric Colson
    Chief Algorithms Officer at Stitch Fix · | 21 upvotes · 2.6M views

    The algorithms and data infrastructure at Stitch Fix is housed in #AWS. Data acquisition is split between events flowing through Kafka, and periodic snapshots of PostgreSQL DBs. We store data in an Amazon S3 based data warehouse. Apache Spark on Yarn is our tool of choice for data movement and #ETL. Because our storage layer (s3) is decoupled from our processing layer, we are able to scale our compute environment very elastically. We have several semi-permanent, autoscaling Yarn clusters running to serve our data processing needs. While the bulk of our compute infrastructure is dedicated to algorithmic processing, we also implemented Presto for adhoc queries and dashboards.

    Beyond data movement and ETL, most #ML centric jobs (e.g. model training and execution) run in a similarly elastic environment as containers running Python and R code on Amazon EC2 Container Service clusters. The execution of batch jobs on top of ECS is managed by Flotilla, a service we built in house and open sourced (see https://github.com/stitchfix/flotilla-os).

    At Stitch Fix, algorithmic integrations are pervasive across the business. We have dozens of data products actively integrated systems. That requires serving layer that is robust, agile, flexible, and allows for self-service. Models produced on Flotilla are packaged for deployment in production using Khan, another framework we've developed internally. Khan provides our data scientists the ability to quickly productionize those models they've developed with open source frameworks in Python 3 (e.g. PyTorch, sklearn), by automatically packaging them as Docker containers and deploying to Amazon ECS. This provides our data scientist a one-click method of getting from their algorithms to production. We then integrate those deployments into a service mesh, which allows us to A/B test various implementations in our product.

    For more info:

    #DataScience #DataStack #Data

    See more
    Conor Myhrvold
    Tech Brand Mgr, Office of CTO at Uber · | 7 upvotes · 1.2M views

    Why we built Marmaray, an open source generic data ingestion and dispersal framework and library for Apache Hadoop :

    Built and designed by our Hadoop Platform team, Marmaray is a plug-in-based framework built on top of the Hadoop ecosystem. Users can add support to ingest data from any source and disperse to any sink leveraging the use of Apache Spark . The name, Marmaray, comes from a tunnel in Turkey connecting Europe and Asia. Similarly, we envisioned Marmaray within Uber as a pipeline connecting data from any source to any sink depending on customer preference:

    https://eng.uber.com/marmaray-hadoop-ingestion-open-source/

    (Direct GitHub repo: https://github.com/uber/marmaray Kafka Kafka Manager )

    See more
    Apache Impala logo

    Apache Impala

    129
    280
    18
    Real-time Query for Hadoop
    129
    280
    + 1
    18
    PROS OF APACHE IMPALA
    • 11
      Super fast
    • 1
      Load Balancing
    • 1
      Replication
    • 1
      Scalability
    • 1
      Distributed
    • 1
      High Performance
    • 1
      Massively Parallel Processing
    • 1
      Open Sourse
    CONS OF APACHE IMPALA
      Be the first to leave a con

      related Apache Impala posts

      I have been working on a Java application to demonstrate the latency for the select/insert/update operations on KUDU storage using Apache Kudu API - Java based client. I have a few queries about using Apache Kudu API

      1. Do we have JDBC wrapper to use Apache Kudu API for getting connection to Kudu masters with connection pool mechanism and all DB operations?

      2. Does Apache KuduAPI supports order by, group by, and aggregate functions? if yes, how to implement these functions using Kudu APIs.

      3. How can we add kudu predicates to Kudu update operation? if yes, how?

      4. Does Apache Kudu API supports batch insertion (execute the Kudu Insert for multiple rows at one go instead of row by row)? (like Kudusession.apply(List);)

      5. Does Apache Kudu API support join on tables?

      6. which tool is preferred over others (Apache Impala /Kudu API) for read and update/insert DB operations?

      See more
      Hadoop logo

      Hadoop

      2.1K
      2.2K
      56
      Open-source software for reliable, scalable, distributed computing
      2.1K
      2.2K
      + 1
      56
      PROS OF HADOOP
      • 39
        Great ecosystem
      • 11
        One stack to rule them all
      • 4
        Great load balancer
      • 1
        Amazon aws
      • 1
        Java syntax
      CONS OF HADOOP
        Be the first to leave a con

        related Hadoop posts

        Shared insights
        on
        KafkaKafkaHadoopHadoop
        at

        The early data ingestion pipeline at Pinterest used Kafka as the central message transporter, with the app servers writing messages directly to Kafka, which then uploaded log files to S3.

        For databases, a custom Hadoop streamer pulled database data and wrote it to S3.

        Challenges cited for this infrastructure included high operational overhead, as well as potential data loss occurring when Kafka broker outages led to an overflow of in-memory message buffering.

        See more
        Conor Myhrvold
        Tech Brand Mgr, Office of CTO at Uber · | 7 upvotes · 1.2M views

        Why we built Marmaray, an open source generic data ingestion and dispersal framework and library for Apache Hadoop :

        Built and designed by our Hadoop Platform team, Marmaray is a plug-in-based framework built on top of the Hadoop ecosystem. Users can add support to ingest data from any source and disperse to any sink leveraging the use of Apache Spark . The name, Marmaray, comes from a tunnel in Turkey connecting Europe and Asia. Similarly, we envisioned Marmaray within Uber as a pipeline connecting data from any source to any sink depending on customer preference:

        https://eng.uber.com/marmaray-hadoop-ingestion-open-source/

        (Direct GitHub repo: https://github.com/uber/marmaray Kafka Kafka Manager )

        See more
        Druid logo

        Druid

        353
        796
        30
        Fast column-oriented distributed data store
        353
        796
        + 1
        30
        PROS OF DRUID
        • 14
          Real Time Aggregations
        • 6
          Batch and Real-Time Ingestion
        • 4
          OLAP
        • 3
          OLAP + OLTP
        • 2
          Combining stream and historical analytics
        • 1
          OLTP
        CONS OF DRUID
        • 3
          Limited sql support
        • 2
          Joins are not supported well
        • 1
          Complexity

        related Druid posts

        Umair Iftikhar
        Technical Architect at ERP Studio · | 3 upvotes · 299.6K views

        Developing a solution that collects Telemetry Data from different devices, nearly 1000 devices minimum and maximum 12000. Each device is sending 2 packets in 1 second. This is time-series data, and this data definition and different reports are saved on PostgreSQL. Like Building information, maintenance records, etc. I want to know about the best solution. This data is required for Math and ML to run different algorithms. Also, data is raw without definitions and information stored in PostgreSQL. Initially, I went with TimescaleDB due to PostgreSQL support, but to increase in sites, I started facing many issues with timescale DB in terms of flexibility of storing data.

        My major requirement is also the replication of the database for reporting and different purposes. You may also suggest other options other than Druid and Cassandra. But an open source solution is appreciated.

        See more
        Apache Ignite logo

        Apache Ignite

        83
        146
        31
        An open-source distributed database, caching and processing platform
        83
        146
        + 1
        31
        PROS OF APACHE IGNITE
        • 4
          Free
        • 4
          Written in java. runs on jvm
        • 3
          Load balancing
        • 3
          High Avaliability
        • 3
          Multiple client language support
        • 3
          Sql query support in cluster wide
        • 3
          Rest interface
        • 2
          Easy to use
        • 2
          Better Documentation
        • 1
          Distributed Locking
        • 1
          gtj
        • 1
          Distributed compute
        • 1
          Benfica
        CONS OF APACHE IGNITE
          Be the first to leave a con

          related Apache Ignite posts

          Clickhouse logo

          Clickhouse

          310
          438
          76
          A column-oriented database management system
          310
          438
          + 1
          76
          PROS OF CLICKHOUSE
          • 19
            Fast, very very fast
          • 11
            Good compression ratio
          • 6
            Horizontally scalable
          • 5
            Utilizes all CPU resources
          • 5
            RESTful
          • 5
            Great CLI
          • 4
            Open-source
          • 4
            Great number of SQL functions
          • 3
            Buggy
          • 3
            Has no transactions
          • 2
            Flexible compression options
          • 2
            Flexible connection options
          • 2
            ODBC
          • 2
            Server crashes its normal :(
          • 2
            Highly available
          • 1
            In IDEA data import via HTTP interface not working
          CONS OF CLICKHOUSE
          • 5
            Slow insert operations

          related Clickhouse posts