Need advice about which tool to choose?Ask the StackShare community!

Apache Kylin

61
236
+ 1
24
Druid

380
867
+ 1
32
Add tool

Apache Kylin vs Druid: What are the differences?

Introduction

In this Markdown code, I will present the key differences between Apache Kylin and Druid, two popular open-source projects for big data processing and analytics.

  1. Data Processing Approach: Apache Kylin is an online analytical processing (OLAP) engine that uses columnar storage to accelerate query performance. It builds and maintains pre-calculated cubes to provide fast query responses. On the other hand, Druid is a distributed, real-time analytics data store designed to process high volumes of event-driven data in real-time. It organizes data in memory for fast data ingestion and query execution.

  2. Query Capabilities: Apache Kylin supports complex OLAP queries with advanced features like group-by, distinct count, and top-N. It offers dimensional modeling and allows users to explore multi-dimensional data sets efficiently. Druid, on the contrary, focuses on ad-hoc querying and provides sub-second query response times for real-time data exploration. It excels at filtering, aggregating, and slicing and dicing data based on time-based dimensions.

  3. Data Ingestion and Storage: Apache Kylin primarily relies on Apache Hadoop and HBase for data ingestion and storage. It leverages the distributed file system for storing and processing large volumes of data. In contrast, Druid has its own data ingestion engine that supports a wide range of data sources, including streaming platforms like Apache Kafka. Druid stores data in a specialized in-memory columnar format for fast queries.

  4. Scalability and Performance: Apache Kylin offers high scalability and can handle large data volumes efficiently. It uses distributed processing to parallelize query execution and achieve high performance. However, it requires additional hardware resources to support high throughput and quick response times. Druid, on the other hand, is designed to scale horizontally, with the ability to handle petabytes of data and thousands of nodes. It can deliver near real-time analytics even at massive scale.

  5. Data Model Flexibility: Apache Kylin supports traditional star and snowflake schemas commonly used in OLAP systems. It enables users to define and build data cubes that optimize query performance for specific use cases. In contrast, Druid follows a denormalized, flat-table data model. It focuses on real-time analytics and provides flexible schemas that suit ad-hoc querying and multidimensional analysis.

  6. Ecosystem Integration: Apache Kylin integrates well with the Apache Hadoop ecosystem and other big data tools like Hive, HBase, and Spark. It leverages the benefits of these technologies for data processing and storage. On the other hand, Druid has extensive integrations with various data sources, including Kafka, Hadoop, and cloud storage systems like Amazon S3. It also provides connectors for popular analytics and visualization tools like Apache Superset and Tableau.

In summary, Apache Kylin is an OLAP engine that focuses on complex OLAP queries and dimensional modeling, while Druid is a real-time analytics data store that excels at ad-hoc querying and real-time data exploration. Kylin leverages Hadoop and HBase for data processing and storage, while Druid has its own ingestion engine and relies on in-memory columnar storage. Both projects offer high scalability and performance but differ in data model flexibility and ecosystem integrations.

Get Advice from developers at your company using StackShare Enterprise. Sign up for StackShare Enterprise.
Learn More
Pros of Apache Kylin
Pros of Druid
  • 7
    Star schema and snowflake schema support
  • 5
    Seamless BI integration
  • 4
    OLAP on Hadoop
  • 3
    Easy install
  • 3
    Sub-second latency on extreme large dataset
  • 2
    ANSI-SQL
  • 15
    Real Time Aggregations
  • 6
    Batch and Real-Time Ingestion
  • 5
    OLAP
  • 3
    OLAP + OLTP
  • 2
    Combining stream and historical analytics
  • 1
    OLTP

Sign up to add or upvote prosMake informed product decisions

Cons of Apache Kylin
Cons of Druid
    Be the first to leave a con
    • 3
      Limited sql support
    • 2
      Joins are not supported well
    • 1
      Complexity

    Sign up to add or upvote consMake informed product decisions

    - No public GitHub repository available -

    What is Apache Kylin?

    Apache Kylin™ is an open source Distributed Analytics Engine designed to provide SQL interface and multi-dimensional analysis (OLAP) on Hadoop/Spark supporting extremely large datasets, originally contributed from eBay Inc.

    What is Druid?

    Druid is a distributed, column-oriented, real-time analytics data store that is commonly used to power exploratory dashboards in multi-tenant environments. Druid excels as a data warehousing solution for fast aggregate queries on petabyte sized data sets. Druid supports a variety of flexible filters, exact calculations, approximate algorithms, and other useful calculations.

    Need advice about which tool to choose?Ask the StackShare community!

    What companies use Apache Kylin?
    What companies use Druid?
    See which teams inside your own company are using Apache Kylin or Druid.
    Sign up for StackShare EnterpriseLearn More

    Sign up to get full access to all the companiesMake informed product decisions

    What tools integrate with Apache Kylin?
    What tools integrate with Druid?

    Sign up to get full access to all the tool integrationsMake informed product decisions

    Blog Posts

    Dec 22 2021 at 5:41AM

    Pinterest

    MySQLKafkaDruid+3
    3
    585
    MySQLKafkaApache Spark+6
    4
    2026
    What are some alternatives to Apache Kylin and Druid?
    Apache Spark
    Spark is a fast and general processing engine compatible with Hadoop data. It can run in Hadoop clusters through YARN or Spark's standalone mode, and it can process data in HDFS, HBase, Cassandra, Hive, and any Hadoop InputFormat. It is designed to perform both batch processing (similar to MapReduce) and new workloads like streaming, interactive queries, and machine learning.
    Presto
    Distributed SQL Query Engine for Big Data
    Apache Impala
    Impala is a modern, open source, MPP SQL query engine for Apache Hadoop. Impala is shipped by Cloudera, MapR, and Amazon. With Impala, you can query data, whether stored in HDFS or Apache HBase – including SELECT, JOIN, and aggregate functions – in real time.
    AtScale
    Its Virtual Data Warehouse delivers performance, security and agility to exceed the demands of modern-day operational analytics.
    Clickhouse
    It allows analysis of data that is updated in real time. It offers instant results in most cases: the data is processed faster than it takes to create a query.
    See all alternatives