kaffarell's Stack Decisions

Nov 28, 2021

Recommends

I don't exactly know how much and which data you are collecting from the machine but React seems kinda overkill. If you only plan on showing some graphs and keywords, it would be better to fall back on plain simple javascript (with maybe bootstrap for styling and chart.js for the graphs). If you create a react application it quickly gets heavy and weights multiple GB, furthermore React is made for reactive websites, and I don't know if simply showing some stats is considered reactive?

On a sidenote, if you already have implemented all the analytics in Java, that's ok. But if you are about to implement them, I would look around for another programming language because Java really is really not optimal for this usecase. Java works in the jvm (Basically a virtual machine) and that's why filesystem access is tedious and slow.

If you are familiar with javascript (which I think so, because you are trying to use react) then I would recommend to use node.js on the analytics/local part and the show the results in the browser, on a web-interface written in html, javascript (with maybe chart.js).

10 upvotes·1 comment·13.9K views

Samarth Gupta

February 22nd 2022 at 5:27AM

Hi @kaffarell thanks a lot for the detailed insights. Since we are creating the graphs which will be interactive and the user will be able to drag, drop, edit them therefore we finally chose React to move ahead. Also in React we found this Amazing Library ReactFlow which we are using for drawing the graphs and is quite powerful.

Needs advice

and

We are currently moving to a microservice architecture and are debating about the different options there are to handle communication between services. We are currently considering Kafka, Redis or RabbitMQ as a message broker. As RabbitMQ is a little bit older, we thought that it may be outdated. Is that true? Can RabbitMQ hold up to more modern tools like Redis and Kafka?

4 upvotes·172.4K views

Replies (4)

Tarun Batra

Senior Software Developer at Okta·Dec 4, 2021

Recommends

Kafka

RabbitMQ

My Stack

We have faced the same question some time ago. Before I begin, DO NOT use Redis as a message broker. It is fast and easy to set up in the beginning but it does not scale. It is not made to be reliable in scale and that is mentioned in the official docs. This analysis of our problems with Redis may help you.

We have used Kafka and RabbitMQ both in scale. We concluded that RabbitMQ is a really good general purpose message broker (for our case) and Kafka is really fast but limited in features. That’s the trade off that we understood from using it. In-fact I blogged about the trade offs between Kafka and RabbitMQ to document it. I hope it helps you in choosing the best pub-sub layer for your use case.

Tag: message-queue (tarunbatra.com)

10 upvotes·138.4K views

rizwansaeedshaikh

Nov 29, 2021

It depends on your requirements like number of messages to be processed per second, real time messages vs delayed, number of servers available for your cluster, whether you need streaming, etc.. Kafka works for most use cases. Not related to answer but would like to add no matter whatever broker you chose, for connecting to the broker always go for the library provided by the broker rather than Spring kafka or Spring AMQP. If you use Spring, then you will be stuck with specific Spring versions. In case you find bugs in spring then difficult because you will have to upgrade entire application to use a later Spring core version. In general, use as minimum libraries as possible to get rid of nuisance of upgrading them when they are outdated or bugs are found with them.

5 upvotes·3 comments·139.1K views

kaffarell

December 1st 2021 at 12:16PM

Thanks for the insight! A fast message broker would be important, persistency isn't. We also plan to deploy the message broker as a docker container to our cluster. I read somewhere online that kafka is not meant to be deployed as a container... Is that true? (What also confused me is that there isn't a official docker image for kafka).

Makarand Joshi

December 2nd 2021 at 11:03AM

Radis is bit different compared to Rabbit MQand Kafka so use Redis only if its for non critical message flow. Between Rabbit MQ and Kafka , our experience as been for large message processing application Rabbit becomes really unstable and even have encountered corrupt data so we switched to Kafka which is more reliable

Bipul Agarwal

December 16th 2021 at 8:37PM

Hi Makarand, how easy was your journey from RabbitMQ to Kafka? Is it okay to ask if you have had any specific challenges as RabbitMQ sends to consumers while in Kafka Consumer needs to read. + Message structure is a bit different too?

Also, It would be nice to know if you migrated to Managed Kafka service or self hosted? (I am trying to understand how tough would it be to manage our own Kafka as we are almost finalising going with Kafka) :)

Thanks

View all (4)