Logstash vs Prometheus

Need advice about which tool to choose?Ask the StackShare community!

Logstash

11.3K
8.7K
+ 1
103
Prometheus

4.3K
3.8K
+ 1
239
Add tool

Logstash vs Prometheus: What are the differences?

Introduction

Logstash and Prometheus are both popular tools used for monitoring and analyzing data in different environments. While they may have some similarities, there are several key differences between the two.

  1. Data Collection and Processing: Logstash is a part of the ELK stack and is specifically designed for collecting, parsing, and transforming data from various sources, including logs, databases, and APIs. It provides a wide range of input, filter, and output plugins to handle different data formats and destinations. On the other hand, Prometheus is a standalone monitoring and alerting toolkit that focuses on time-series data collection and storage. It comes with its own query language (PromQL) and is best suited for monitoring application and infrastructure metrics.

  2. Data Storage: Logstash does not have built-in storage capabilities and is primarily used as a data transport pipeline. It typically sends processed data to Elasticsearch for indexing and storage. In contrast, Prometheus has its own built-in time-series database that stores collected metrics. By default, Prometheus uses an on-disk, single-node storage format. However, it also supports remote storage integration with other databases like Thanos for scalable, long-term storage.

  3. Metrics Discovery: Logstash relies on explicit configuration to determine which data sources to collect from. It requires users to define input plugins for specific data sources and formats. Prometheus, on the other hand, follows a pull-based model where it scrapes data from instrumented applications and infrastructures based on configured endpoint targets. It uses service discovery mechanisms such as DNS, Kubernetes, or file-based static configs to automatically discover available targets.

  4. Alerting and Monitoring: Logstash has limited built-in monitoring and alerting capabilities. It is primarily focused on data transformation and transportation. However, it can integrate with other monitoring and alerting tools like Kibana and Elasticsearch to provide a comprehensive monitoring solution. Prometheus, on the other hand, has extensive alerting and monitoring features. It allows users to define alerting rules based on Prometheus metrics and send alerts to various notification channels like email, Slack, or PagerDuty.

  5. Scalability and Performance: Logstash can scale horizontally by distributing the workload across multiple instances, but it still relies on the capacity of the underlying Elasticsearch cluster for high throughput. On the other hand, Prometheus is designed to be highly scalable and can handle large amounts of time-series data. It supports federation, allowing multiple Prometheus servers to scrape and aggregate metrics from different sources. This enables distributed monitoring and improved performance.

  6. Community and Ecosystem: Logstash is part of the larger ELK stack, which includes Elasticsearch and Kibana. It has a vibrant community and extensive ecosystem of plugins, add-ons, and integrations. Prometheus, on the other hand, has gained popularity in the cloud-native space and is widely adopted by organizations using Kubernetes for container orchestration. It has a growing community and supports integration with popular frameworks like Grafana for visualization.

In summary, Logstash is primarily focused on data collection and transportation with limited monitoring capabilities, while Prometheus is a dedicated monitoring and alerting tool with a built-in time-series database. Logstash relies on manual configuration for data sources, while Prometheus uses a pull-based model with automatic target discovery. Prometheus has stronger alerting and monitoring features and is highly scalable, especially in cloud-native environments.

Advice on Logstash and Prometheus
Susmita Meher
Senior SRE at African Bank · | 4 upvotes · 822.1K views
Needs advice
on
GrafanaGrafanaGraphiteGraphite
and
PrometheusPrometheus

Looking for a tool which can be used for mainly dashboard purposes, but here are the main requirements:

  • Must be able to get custom data from AS400,
  • Able to display automation test results,
  • System monitoring / Nginx API,
  • Able to get data from 3rd parties DB.

Grafana is almost solving all the problems, except AS400 and no database to get automation test results.

See more
Replies (1)
Sakti Behera
Technical Specialist, Software Engineering at AT&T · | 3 upvotes · 607.5K views
Recommends
on
GrafanaGrafanaPrometheusPrometheus

You can look out for Prometheus Instrumentation (https://prometheus.io/docs/practices/instrumentation/) Client Library available in various languages https://prometheus.io/docs/instrumenting/clientlibs/ to create the custom metric you need for AS4000 and then Grafana can query the newly instrumented metric to show on the dashboard.

See more
Sunil Chaudhari
Needs advice
on
MetricbeatMetricbeat
and
PrometheusPrometheus

Hi, We have a situation, where we are using Prometheus to get system metrics from PCF (Pivotal Cloud Foundry) platform. We send that as time-series data to Cortex via a Prometheus server and built a dashboard using Grafana. There is another pipeline where we need to read metrics from a Linux server using Metricbeat, CPU, memory, and Disk. That will be sent to Elasticsearch and Grafana will pull and show the data in a dashboard.

Is it OK to use Metricbeat for Linux server or can we use Prometheus?

What is the difference in system metrics sent by Metricbeat and Prometheus node exporters?

Regards, Sunil.

See more
Replies (2)
Matthew Rothstein
Recommends
on
PrometheusPrometheus

If you're already using Prometheus for your system metrics, then it seems like standing up Elasticsearch just for Linux host monitoring is excessive. The node_exporter is probably sufficient if you'e looking for standard system metrics.

Another thing to consider is that Metricbeat / ELK use a push model for metrics delivery, whereas Prometheus pulls metrics from each node it is monitoring. Depending on how you manage your network security, opting for one solution over two may make things simpler.

See more
Recommends
on
InstanaInstana

Hi Sunil! Unfortunately, I don´t have much experience with Metricbeat so I can´t advise on the diffs with Prometheus...for Linux server, I encourage you to use Prometheus node exporter and for PCF, I would recommend using the instana tile (https://www.instana.com/supported-technologies/pivotal-cloud-foundry/). Let me know if you have further questions! Regards Jose

See more
Mat Jovanovic
Head of Cloud at Mats Cloud · | 3 upvotes · 750.2K views
Needs advice
on
DatadogDatadogGrafanaGrafana
and
PrometheusPrometheus

We're looking for a Monitoring and Logging tool. It has to support AWS (mostly 100% serverless, Lambdas, SNS, SQS, API GW, CloudFront, Autora, etc.), as well as Azure and GCP (for now mostly used as pure IaaS, with a lot of cognitive services, and mostly managed DB). Hopefully, something not as expensive as Datadog or New relic, as our SRE team could support the tool inhouse. At the moment, we primarily use CloudWatch for AWS and Pandora for most on-prem.

See more
Replies (2)
Recommends
on
DatadogDatadog

I worked with Datadog at least one year and my position is that commercial tools like Datadog are the best option to consolidate and analyze your metrics. Obviously, if you can't pay the tool, the best free options are the mix of Prometheus with their Alert Manager and Grafana to visualize (that are complementary not substitutable). But I think that no use a good tool it's finally more expensive that use a not really good implementation of free tools and you will pay also to maintain its.

See more
Lucas Rincon
Recommends
on
InstanaInstana

this is quite affordable and provides what you seem to be looking for. you can see a whole thing about the APM space here https://www.apmexperts.com/observability/ranking-the-observability-offerings/

See more
Manage your open source components, licenses, and vulnerabilities
Learn More
Pros of Logstash
Pros of Prometheus
  • 69
    Free
  • 18
    Easy but powerful filtering
  • 12
    Scalable
  • 2
    Kibana provides machine learning based analytics to log
  • 1
    Great to meet GDPR goals
  • 1
    Well Documented
  • 47
    Powerful easy to use monitoring
  • 38
    Flexible query language
  • 32
    Dimensional data model
  • 27
    Alerts
  • 23
    Active and responsive community
  • 22
    Extensive integrations
  • 19
    Easy to setup
  • 12
    Beautiful Model and Query language
  • 7
    Easy to extend
  • 6
    Nice
  • 3
    Written in Go
  • 2
    Good for experimentation
  • 1
    Easy for monitoring

Sign up to add or upvote prosMake informed product decisions

Cons of Logstash
Cons of Prometheus
  • 4
    Memory-intensive
  • 1
    Documentation difficult to use
  • 12
    Just for metrics
  • 6
    Bad UI
  • 6
    Needs monitoring to access metrics endpoints
  • 4
    Not easy to configure and use
  • 3
    Supports only active agents
  • 2
    Written in Go
  • 2
    TLS is quite difficult to understand
  • 2
    Requires multiple applications and tools
  • 1
    Single point of failure

Sign up to add or upvote consMake informed product decisions

What is Logstash?

Logstash is a tool for managing events and logs. You can use it to collect logs, parse them, and store them for later use (like, for searching). If you store them in Elasticsearch, you can view and analyze them with Kibana.

What is Prometheus?

Prometheus is a systems and service monitoring system. It collects metrics from configured targets at given intervals, evaluates rule expressions, displays the results, and can trigger alerts if some condition is observed to be true.

Need advice about which tool to choose?Ask the StackShare community!

What companies use Logstash?
What companies use Prometheus?
Manage your open source components, licenses, and vulnerabilities
Learn More

Sign up to get full access to all the companiesMake informed product decisions

What tools integrate with Logstash?
What tools integrate with Prometheus?

Sign up to get full access to all the tool integrationsMake informed product decisions

Blog Posts

Dec 8 2020 at 5:50PM

DigitalOcean

GitHubMySQLPostgreSQL+11
3
2423
May 21 2020 at 12:02AM

Rancher Labs

KubernetesAmazon EC2Grafana+12
6
1522
PythonDockerKubernetes+14
12
2642
Node.jsnpmKubernetes+6
1
1467
May 21 2019 at 12:20AM

Elastic

ElasticsearchKibanaLogstash+4
12
5256
GitHubPythonReact+42
49
40862
What are some alternatives to Logstash and Prometheus?
Fluentd
Fluentd collects events from various data sources and writes them to files, RDBMS, NoSQL, IaaS, SaaS, Hadoop and so on. Fluentd helps you unify your logging infrastructure.
Splunk
It provides the leading platform for Operational Intelligence. Customers use it to search, monitor, analyze and visualize machine data.
Kafka
Kafka is a distributed, partitioned, replicated commit log service. It provides the functionality of a messaging system, but with a unique design.
Beats
Beats is the platform for single-purpose data shippers. They send data from hundreds or thousands of machines and systems to Logstash or Elasticsearch.
Graylog
Centralize and aggregate all your log files for 100% visibility. Use our powerful query language to search through terabytes of log data to discover and analyze important information.
See all alternatives