Need advice about which tool to choose?Ask the StackShare community!

Druid

379
866
+ 1
32
Prometheus

4.2K
3.8K
+ 1
239
Add tool

Druid vs Prometheus: What are the differences?

Key Differences between Druid and Prometheus

Druid and Prometheus are both open-source data storage and analytics platforms, but they have some key differences in terms of architecture, query capabilities, data model, and scaling. The following are the main differences between Druid and Prometheus:

  1. Data Model: Druid is designed for OLAP (Online Analytical Processing) workloads and stores data in a columnar format optimized for fast analytics. It uses a segmented architecture, where data is divided into time-based chunks for efficient queries. Prometheus, on the other hand, is designed for real-time monitoring and stores data in a time-series format.

  2. Query Capabilities: Druid supports complex analytical queries, such as filtering, aggregations, grouping, and joins, making it suitable for interactive exploratory analysis. It also provides sub-second query response times. Prometheus, on the other hand, focuses on monitoring and alerting, providing querying capabilities for time-series data, including range queries and aggregations.

  3. Data Ingestion: Druid provides a scalable, distributed real-time ingestion framework called Tranquility, which allows streaming data to be ingested into Druid clusters. It can handle high-throughput data ingestion from various sources. Prometheus is primarily designed for pull-based scraping, where it regularly fetches metrics from target systems using HTTP. It supports a range of integrations with different monitoring systems.

  4. Scalability: Druid is built to scale horizontally and can handle large volumes of data by distributing it across multiple nodes. It supports automatic data sharding and replication for high availability. Prometheus, on the other hand, can be vertically scaled by adding more resources to a single instance or federated with multiple Prometheus servers for distributed monitoring.

  5. Data Retention: Druid provides configurable data retention policies, allowing users to define how long data should be retained in the system. It also supports data compaction to reduce storage requirements. Prometheus has a fixed retention period for metrics data, which can be configured but is typically limited to a few weeks or months.

  6. Ecosystem Integration: Druid integrates well with other big data ecosystem tools such as Apache Hadoop, Apache Spark, and Apache Kafka, making it suitable for building end-to-end data pipelines. Prometheus, on the other hand, has a rich ecosystem of exporters, plugins, and alerting integrations that make it easy to integrate with various monitoring systems and tools.

In summary, Druid is optimized for OLAP analytics, supports complex queries, and provides scalable ingestion and data retention capabilities. Prometheus, on the other hand, is focused on real-time monitoring, offers flexible querying for time-series data, and has a rich ecosystem of integrations for monitoring and alerting.

Advice on Druid and Prometheus
Susmita Meher
Senior SRE at African Bank · | 4 upvotes · 800.7K views
Needs advice
on
GrafanaGrafanaGraphiteGraphite
and
PrometheusPrometheus

Looking for a tool which can be used for mainly dashboard purposes, but here are the main requirements:

  • Must be able to get custom data from AS400,
  • Able to display automation test results,
  • System monitoring / Nginx API,
  • Able to get data from 3rd parties DB.

Grafana is almost solving all the problems, except AS400 and no database to get automation test results.

See more
Replies (1)
Sakti Behera
Technical Specialist, Software Engineering at AT&T · | 3 upvotes · 586.1K views
Recommends
on
GrafanaGrafanaPrometheusPrometheus

You can look out for Prometheus Instrumentation (https://prometheus.io/docs/practices/instrumentation/) Client Library available in various languages https://prometheus.io/docs/instrumenting/clientlibs/ to create the custom metric you need for AS4000 and then Grafana can query the newly instrumented metric to show on the dashboard.

See more
Sunil Chaudhari
Needs advice
on
MetricbeatMetricbeat
and
PrometheusPrometheus

Hi, We have a situation, where we are using Prometheus to get system metrics from PCF (Pivotal Cloud Foundry) platform. We send that as time-series data to Cortex via a Prometheus server and built a dashboard using Grafana. There is another pipeline where we need to read metrics from a Linux server using Metricbeat, CPU, memory, and Disk. That will be sent to Elasticsearch and Grafana will pull and show the data in a dashboard.

Is it OK to use Metricbeat for Linux server or can we use Prometheus?

What is the difference in system metrics sent by Metricbeat and Prometheus node exporters?

Regards, Sunil.

See more
Replies (2)
Matthew Rothstein
Recommends
on
PrometheusPrometheus

If you're already using Prometheus for your system metrics, then it seems like standing up Elasticsearch just for Linux host monitoring is excessive. The node_exporter is probably sufficient if you'e looking for standard system metrics.

Another thing to consider is that Metricbeat / ELK use a push model for metrics delivery, whereas Prometheus pulls metrics from each node it is monitoring. Depending on how you manage your network security, opting for one solution over two may make things simpler.

See more
Recommends
on
InstanaInstana

Hi Sunil! Unfortunately, I don´t have much experience with Metricbeat so I can´t advise on the diffs with Prometheus...for Linux server, I encourage you to use Prometheus node exporter and for PCF, I would recommend using the instana tile (https://www.instana.com/supported-technologies/pivotal-cloud-foundry/). Let me know if you have further questions! Regards Jose

See more
Mat Jovanovic
Head of Cloud at Mats Cloud · | 3 upvotes · 729.5K views
Needs advice
on
DatadogDatadogGrafanaGrafana
and
PrometheusPrometheus

We're looking for a Monitoring and Logging tool. It has to support AWS (mostly 100% serverless, Lambdas, SNS, SQS, API GW, CloudFront, Autora, etc.), as well as Azure and GCP (for now mostly used as pure IaaS, with a lot of cognitive services, and mostly managed DB). Hopefully, something not as expensive as Datadog or New relic, as our SRE team could support the tool inhouse. At the moment, we primarily use CloudWatch for AWS and Pandora for most on-prem.

See more
Replies (2)
Recommends
on
DatadogDatadog

I worked with Datadog at least one year and my position is that commercial tools like Datadog are the best option to consolidate and analyze your metrics. Obviously, if you can't pay the tool, the best free options are the mix of Prometheus with their Alert Manager and Grafana to visualize (that are complementary not substitutable). But I think that no use a good tool it's finally more expensive that use a not really good implementation of free tools and you will pay also to maintain its.

See more
Lucas Rincon
Recommends
on
InstanaInstana

this is quite affordable and provides what you seem to be looking for. you can see a whole thing about the APM space here https://www.apmexperts.com/observability/ranking-the-observability-offerings/

See more
Get Advice from developers at your company using StackShare Enterprise. Sign up for StackShare Enterprise.
Learn More
Pros of Druid
Pros of Prometheus
  • 15
    Real Time Aggregations
  • 6
    Batch and Real-Time Ingestion
  • 5
    OLAP
  • 3
    OLAP + OLTP
  • 2
    Combining stream and historical analytics
  • 1
    OLTP
  • 47
    Powerful easy to use monitoring
  • 38
    Flexible query language
  • 32
    Dimensional data model
  • 27
    Alerts
  • 23
    Active and responsive community
  • 22
    Extensive integrations
  • 19
    Easy to setup
  • 12
    Beautiful Model and Query language
  • 7
    Easy to extend
  • 6
    Nice
  • 3
    Written in Go
  • 2
    Good for experimentation
  • 1
    Easy for monitoring

Sign up to add or upvote prosMake informed product decisions

Cons of Druid
Cons of Prometheus
  • 3
    Limited sql support
  • 2
    Joins are not supported well
  • 1
    Complexity
  • 12
    Just for metrics
  • 6
    Bad UI
  • 6
    Needs monitoring to access metrics endpoints
  • 4
    Not easy to configure and use
  • 3
    Supports only active agents
  • 2
    Written in Go
  • 2
    TLS is quite difficult to understand
  • 2
    Requires multiple applications and tools
  • 1
    Single point of failure

Sign up to add or upvote consMake informed product decisions

- No public GitHub repository available -

What is Druid?

Druid is a distributed, column-oriented, real-time analytics data store that is commonly used to power exploratory dashboards in multi-tenant environments. Druid excels as a data warehousing solution for fast aggregate queries on petabyte sized data sets. Druid supports a variety of flexible filters, exact calculations, approximate algorithms, and other useful calculations.

What is Prometheus?

Prometheus is a systems and service monitoring system. It collects metrics from configured targets at given intervals, evaluates rule expressions, displays the results, and can trigger alerts if some condition is observed to be true.

Need advice about which tool to choose?Ask the StackShare community!

What companies use Druid?
What companies use Prometheus?
See which teams inside your own company are using Druid or Prometheus.
Sign up for StackShare EnterpriseLearn More

Sign up to get full access to all the companiesMake informed product decisions

What tools integrate with Druid?
What tools integrate with Prometheus?

Sign up to get full access to all the tool integrationsMake informed product decisions

Blog Posts

Dec 22 2021 at 5:41AM

Pinterest

MySQLKafkaDruid+3
3
575
Dec 8 2020 at 5:50PM

DigitalOcean

GitHubMySQLPostgreSQL+11
2
2372
May 21 2020 at 12:02AM

Rancher Labs

KubernetesAmazon EC2Grafana+12
5
1500
MySQLKafkaApache Spark+6
2
2018
PythonDockerKubernetes+14
12
2615
Node.jsnpmKubernetes+6
1
1440
What are some alternatives to Druid and Prometheus?
HBase
Apache HBase is an open-source, distributed, versioned, column-oriented store modeled after Google' Bigtable: A Distributed Storage System for Structured Data by Chang et al. Just as Bigtable leverages the distributed data storage provided by the Google File System, HBase provides Bigtable-like capabilities on top of Apache Hadoop.
MongoDB
MongoDB stores data in JSON-like documents that can vary in structure, offering a dynamic, flexible schema. MongoDB was also designed for high availability and scalability, with built-in replication and auto-sharding.
Cassandra
Partitioning means that Cassandra can distribute your data across multiple machines in an application-transparent matter. Cassandra will automatically repartition as machines are added and removed from the cluster. Row store means that like relational databases, Cassandra organizes data by rows and columns. The Cassandra Query Language (CQL) is a close relative of SQL.
Elasticsearch
Elasticsearch is a distributed, RESTful search and analytics engine capable of storing data and searching it in near real time. Elasticsearch, Kibana, Beats and Logstash are the Elastic Stack (sometimes called the ELK Stack).
Clickhouse
It allows analysis of data that is updated in real time. It offers instant results in most cases: the data is processed faster than it takes to create a query.
See all alternatives