Need advice about which tool to choose?Ask the StackShare community!
Druid vs Prometheus: What are the differences?
Key Differences between Druid and Prometheus
Druid and Prometheus are both open-source data storage and analytics platforms, but they have some key differences in terms of architecture, query capabilities, data model, and scaling. The following are the main differences between Druid and Prometheus:
Data Model: Druid is designed for OLAP (Online Analytical Processing) workloads and stores data in a columnar format optimized for fast analytics. It uses a segmented architecture, where data is divided into time-based chunks for efficient queries. Prometheus, on the other hand, is designed for real-time monitoring and stores data in a time-series format.
Query Capabilities: Druid supports complex analytical queries, such as filtering, aggregations, grouping, and joins, making it suitable for interactive exploratory analysis. It also provides sub-second query response times. Prometheus, on the other hand, focuses on monitoring and alerting, providing querying capabilities for time-series data, including range queries and aggregations.
Data Ingestion: Druid provides a scalable, distributed real-time ingestion framework called Tranquility, which allows streaming data to be ingested into Druid clusters. It can handle high-throughput data ingestion from various sources. Prometheus is primarily designed for pull-based scraping, where it regularly fetches metrics from target systems using HTTP. It supports a range of integrations with different monitoring systems.
Scalability: Druid is built to scale horizontally and can handle large volumes of data by distributing it across multiple nodes. It supports automatic data sharding and replication for high availability. Prometheus, on the other hand, can be vertically scaled by adding more resources to a single instance or federated with multiple Prometheus servers for distributed monitoring.
Data Retention: Druid provides configurable data retention policies, allowing users to define how long data should be retained in the system. It also supports data compaction to reduce storage requirements. Prometheus has a fixed retention period for metrics data, which can be configured but is typically limited to a few weeks or months.
Ecosystem Integration: Druid integrates well with other big data ecosystem tools such as Apache Hadoop, Apache Spark, and Apache Kafka, making it suitable for building end-to-end data pipelines. Prometheus, on the other hand, has a rich ecosystem of exporters, plugins, and alerting integrations that make it easy to integrate with various monitoring systems and tools.
In summary, Druid is optimized for OLAP analytics, supports complex queries, and provides scalable ingestion and data retention capabilities. Prometheus, on the other hand, is focused on real-time monitoring, offers flexible querying for time-series data, and has a rich ecosystem of integrations for monitoring and alerting.
Looking for a tool which can be used for mainly dashboard purposes, but here are the main requirements:
- Must be able to get custom data from AS400,
- Able to display automation test results,
- System monitoring / Nginx API,
- Able to get data from 3rd parties DB.
Grafana is almost solving all the problems, except AS400 and no database to get automation test results.
You can look out for Prometheus Instrumentation (https://prometheus.io/docs/practices/instrumentation/) Client Library available in various languages https://prometheus.io/docs/instrumenting/clientlibs/ to create the custom metric you need for AS4000 and then Grafana can query the newly instrumented metric to show on the dashboard.
Hi, We have a situation, where we are using Prometheus to get system metrics from PCF (Pivotal Cloud Foundry) platform. We send that as time-series data to Cortex via a Prometheus server and built a dashboard using Grafana. There is another pipeline where we need to read metrics from a Linux server using Metricbeat, CPU, memory, and Disk. That will be sent to Elasticsearch and Grafana will pull and show the data in a dashboard.
Is it OK to use Metricbeat for Linux server or can we use Prometheus?
What is the difference in system metrics sent by Metricbeat and Prometheus node exporters?
Regards, Sunil.
If you're already using Prometheus for your system metrics, then it seems like standing up Elasticsearch just for Linux host monitoring is excessive. The node_exporter is probably sufficient if you'e looking for standard system metrics.
Another thing to consider is that Metricbeat / ELK use a push model for metrics delivery, whereas Prometheus pulls metrics from each node it is monitoring. Depending on how you manage your network security, opting for one solution over two may make things simpler.
Hi Sunil! Unfortunately, I don´t have much experience with Metricbeat so I can´t advise on the diffs with Prometheus...for Linux server, I encourage you to use Prometheus node exporter and for PCF, I would recommend using the instana tile (https://www.instana.com/supported-technologies/pivotal-cloud-foundry/). Let me know if you have further questions! Regards Jose
We're looking for a Monitoring and Logging tool. It has to support AWS (mostly 100% serverless, Lambdas, SNS, SQS, API GW, CloudFront, Autora, etc.), as well as Azure and GCP (for now mostly used as pure IaaS, with a lot of cognitive services, and mostly managed DB). Hopefully, something not as expensive as Datadog or New relic, as our SRE team could support the tool inhouse. At the moment, we primarily use CloudWatch for AWS and Pandora for most on-prem.
this is quite affordable and provides what you seem to be looking for. you can see a whole thing about the APM space here https://www.apmexperts.com/observability/ranking-the-observability-offerings/
I worked with Datadog at least one year and my position is that commercial tools like Datadog are the best option to consolidate and analyze your metrics. Obviously, if you can't pay the tool, the best free options are the mix of Prometheus with their Alert Manager and Grafana to visualize (that are complementary not substitutable). But I think that no use a good tool it's finally more expensive that use a not really good implementation of free tools and you will pay also to maintain its.
Pros of Druid
- Real Time Aggregations15
- Batch and Real-Time Ingestion6
- OLAP5
- OLAP + OLTP3
- Combining stream and historical analytics2
- OLTP1
Pros of Prometheus
- Powerful easy to use monitoring47
- Flexible query language38
- Dimensional data model32
- Alerts27
- Active and responsive community23
- Extensive integrations22
- Easy to setup19
- Beautiful Model and Query language12
- Easy to extend7
- Nice6
- Written in Go3
- Good for experimentation2
- Easy for monitoring1
Sign up to add or upvote prosMake informed product decisions
Cons of Druid
- Limited sql support3
- Joins are not supported well2
- Complexity1
Cons of Prometheus
- Just for metrics12
- Bad UI6
- Needs monitoring to access metrics endpoints6
- Not easy to configure and use4
- Supports only active agents3
- Written in Go2
- TLS is quite difficult to understand2
- Requires multiple applications and tools2
- Single point of failure1
Sign up to add or upvote consMake informed product decisions
What is Druid?
What is Prometheus?
Need advice about which tool to choose?Ask the StackShare community!
What companies use Druid?
What companies use Prometheus?
Sign up to get full access to all the companiesMake informed product decisions
What tools integrate with Prometheus?
Sign up to get full access to all the tool integrationsMake informed product decisions
Blog Posts
+3
+4
+11
+12
+6
+14
+6
+12