Needs advice
on
LogstashLogstashGrafanaGrafana
and
GoGo

Hi everyone. I'm trying to create my personal syslog monitoring.

  1. To get the logs, I have uncertainty to choose the way: 1.1 Use Logstash like a TCP server. 1.2 Implement a Go TCP server.

  2. To store and plot data. 2.1 Use Elasticsearch tools. 2.2 Use InfluxDB and Grafana.

I would like to know... Which is a cheaper and scalable solution?

Or even if there is a better way to do it.

READ LESS
9 upvotes·8.1K views
Replies (3)
Recommends
Loki
Grafana

Hi Juan

A very simple and cheap (resource usage) option here would be to use promtail to send syslog data to Loki and visualise Loki with Grafana using the native Grafana Loki data source. I have recently put together this set up and promtail and Loki are less resource intensive than Logstash/ES and it is a simple set up and configuration and works very nicely.

READ MORE
4 upvotes·2 comments·1.4K views
Sunil Chaudhari
Sunil Chaudhari
·
October 27th 2021 at 1:23AM

Hi,

Does promtel available for PCF?

·
Reply
Gary Wilson
Gary Wilson
·
October 27th 2021 at 1:38PM

Hi @sunilmchaudhari I do not know. I assume by PCF you are refering to Pivot Cloud Foundry, which I have no knowledge of sorry. Promtail is a go binary so if you can add log data to a syslog, then you can process it with Promtail.

·
Reply
Team Lead at XYZ·

For Syslog, you can certainly use TCP Input. Really interested to know what is your syslog client( which will ship logs to logstash). Anyways you can check that and see if that client has capability to configure multiple logstash host ports so that it works as a load balancer. This will increase throughput. Also check pipeline-to-pipeline communcation of logstash: https://www.elastic.co/guide/en/logstash/current/pipeline-to-pipeline.html This helps to implement distributor pattern of pipeline where multiple type of data is coming to same input and you may want to route filtering and processing based on types. It increases parallelism. About Elasticsearch: Its a native component and perfectly fits with logstash so you can use elasticsearch for storage and search. Its one of the datasource of grafana.

READ MORE
3 upvotes·700 views
View all (3)
Avatar of Juan Felipe

Juan Felipe

Developer