Kafka vs Redis To Go

Overview

Redis To Go

Stacks51

Followers119

Votes18

Kafka

Stacks24.2K

Followers22.3K

Votes607

GitHub Stars31.2K

Forks14.8K

Kafka vs Redis To Go: What are the differences?

Scalability and Use Cases: Kafka is designed for scalable event streaming and messaging, making it ideal for use cases such as data processing pipelines, real-time analytics, and log aggregation. In contrast, Redis To Go is a fully managed Redis service that primarily focuses on caching, session storage, real-time analytics, and message brokering.
Data Persistence: Kafka persists data to disk, ensuring data durability and fault tolerance even in the face of node failures. On the other hand, Redis To Go stores data in-memory, providing fast read and write operations but posing a risk of data loss upon node failure unless a persistence option like RDB or AOF is configured.
Querying and Processing Capabilities: Kafka is optimized for high-throughput message processing, supporting complex event stream processing with tools like Kafka Streams and KSQL. Redis To Go, on the other hand, offers a wide range of data structures and functionalities such as sorted sets, counters, and bitmaps for efficient data manipulation and retrieval.
Data Structure Management: Kafka organizes data into topics and partitions for efficient distribution and parallel processing of messages. In comparison, Redis To Go uses key-value pairs within distinct databases for logical data separation and efficient retrieval based on keys.
Messaging Paradigm: Kafka follows a publish-subscribe messaging paradigm where producers publish messages to topics, and consumers subscribe to those topics to receive messages. Redis To Go, on the other hand, supports both pub/sub messaging and direct requests through commands to access and manipulate data stored in the cache.
Consistency and Durability: Kafka guarantees strong ordering guarantees and fault tolerance through replication and partitioning, ensuring data consistency in the event of failures. While Redis To Go can achieve eventual consistency through replication and persistence options, it may lack the same level of durability as Kafka in certain high-availability scenarios.

In Summary, Kafka excels in scalable event streaming and analytics, while Redis To Go focuses on managed caching and storage with different data persistence and querying capabilities.

Share your Stack

Help developers discover the tools you use. Get visibility for your team's tech choices and contribute to the community's knowledge.

View Docs

CLI (Node.js)

Manual

Advice on Redis To Go, Kafka

viradiya

Apr 12, 2020

Needs adviceon

AngularJS

ASP.NET Core

MSSQL

We are going to develop a microservices-based application. It consists of AngularJS, ASP.NET Core, and MSSQL.

We have 3 types of microservices. Emailservice, Filemanagementservice, Filevalidationservice

I am a beginner in microservices. But I have read about RabbitMQ, but come to know that there are Redis and Kafka also in the market. So, I want to know which is best.

933k views933k

Comments

Ishfaq

Feb 28, 2020

Needs advice

Our backend application is sending some external messages to a third party application at the end of each backend (CRUD) API call (from UI) and these external messages take too much extra time (message building, processing, then sent to the third party and log success/failure), UI application has no concern to these extra third party messages.

So currently we are sending these third party messages by creating a new child thread at end of each REST API call so UI application doesn't wait for these extra third party API calls.

I want to integrate Apache Kafka for these extra third party API calls, so I can also retry on failover third party API calls in a queue(currently third party messages are sending from multiple threads at the same time which uses too much processing and resources) and logging, etc.

Question 1: Is this a use case of a message broker?

Question 2: If it is then Kafka vs RabitMQ which is the better?

804k views804k

Comments

Roman

Senior Back-End Developer, Software Architect

Feb 12, 2019

Reviewon

Kafka

I use Kafka because it has almost infinite scaleability in terms of processing events (could be scaled to process hundreds of thousands of events), great monitoring (all sorts of metrics are exposed via JMX).

Downsides of using Kafka are:

you have to deal with Zookeeper
you have to implement advanced routing yourself (compared to RabbitMQ it has no advanced routing)

10.8k views10.8k

Comments

Detailed Comparison

Redis To Go	Kafka
Redis To Go was created to make the managing Redis instances easier, whether it is just one instance or serveral. Deploying a new instance of Redis is dead simple, whether for production or development.	Kafka is a distributed, partitioned, replicated commit log service. It provides the functionality of a messaging system, but with a unique design.
-	Written at LinkedIn in Scala;Used by LinkedIn to offload processing of all page and other views;Defaults to using persistence, uses OS disk cache for hot data (has higher throughput then any of the above having persistence enabled);Supports both on-line as off-line processing
Statistics
GitHub Stars -	GitHub Stars 31.2K
GitHub Forks -	GitHub Forks 14.8K
Stacks 51	Stacks 24.2K
Followers 119	Followers 22.3K
Votes 18	Votes 607
Pros & Cons
Pros 5 Heroku Add-on 3 Easy setup 3 Always up 3 Affordable 3 Pub-Sub	Pros 126 High-throughput 119 Distributed 92 Scalable 86 High-Performance 66 Durable Cons 32 Non-Java clients are second-class citizens 29 Needs Zookeeper 9 Operational difficulties 5 Terrible Packaging
Integrations
Nodejitsu Heroku Engine Yard Cloud AppHarbor	No integrations available

What are some alternatives to Redis To Go, Kafka?

RabbitMQ

RabbitMQ gives your applications a common platform to send and receive messages, and your messages a safe place to live until received.

Celery

Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.

Amazon SQS

Transmit any volume of data, at any level of throughput, without losing messages or requiring other services to be always available. With SQS, you can offload the administrative burden of operating and scaling a highly available messaging cluster, while paying a low price for only what you use.

NSQ

NSQ is a realtime distributed messaging platform designed to operate at scale, handling billions of messages per day. It promotes distributed and decentralized topologies without single points of failure, enabling fault tolerance and high availability coupled with a reliable message delivery guarantee. See features & guarantees.

ActiveMQ

Apache ActiveMQ is fast, supports many Cross Language Clients and Protocols, comes with easy to use Enterprise Integration Patterns and many advanced features while fully supporting JMS 1.1 and J2EE 1.4. Apache ActiveMQ is released under the Apache 2.0 License.

ZeroMQ

The 0MQ lightweight messaging kernel is a library which extends the standard socket interfaces with features traditionally provided by specialised messaging middleware products. 0MQ sockets provide an abstraction of asynchronous message queues, multiple messaging patterns, message filtering (subscriptions), seamless access to multiple transport protocols and more.

Apache NiFi

An easy to use, powerful, and reliable system to process and distribute data. It supports powerful and scalable directed graphs of data routing, transformation, and system mediation logic.

Gearman

Gearman allows you to do work in parallel, to load balance processing, and to call functions between languages. It can be used in a variety of applications, from high-availability web sites to the transport of database replication events.

Memphis

Highly scalable and effortless data streaming platform. Made to enable developers and data teams to collaborate and build real-time and streaming apps fast.

IronMQ

An easy-to-use highly available message queuing service. Built for distributed cloud applications with critical messaging needs. Provides on-demand message queuing with advanced features and cloud-optimized performance.

Related Comparisons

Kafka vs Redis To Go: What are the differences?

Scalability and Use Cases: Kafka is designed for scalable event streaming and messaging, making it ideal for use cases such as data processing pipelines, real-time analytics, and log aggregation. In contrast, Redis To Go is a fully managed Redis service that primarily focuses on caching, session storage, real-time analytics, and message brokering.
Data Persistence: Kafka persists data to disk, ensuring data durability and fault tolerance even in the face of node failures. On the other hand, Redis To Go stores data in-memory, providing fast read and write operations but posing a risk of data loss upon node failure unless a persistence option like RDB or AOF is configured.
Querying and Processing Capabilities: Kafka is optimized for high-throughput message processing, supporting complex event stream processing with tools like Kafka Streams and KSQL. Redis To Go, on the other hand, offers a wide range of data structures and functionalities such as sorted sets, counters, and bitmaps for efficient data manipulation and retrieval.
Data Structure Management: Kafka organizes data into topics and partitions for efficient distribution and parallel processing of messages. In comparison, Redis To Go uses key-value pairs within distinct databases for logical data separation and efficient retrieval based on keys.
Messaging Paradigm: Kafka follows a publish-subscribe messaging paradigm where producers publish messages to topics, and consumers subscribe to those topics to receive messages. Redis To Go, on the other hand, supports both pub/sub messaging and direct requests through commands to access and manipulate data stored in the cache.
Consistency and Durability: Kafka guarantees strong ordering guarantees and fault tolerance through replication and partitioning, ensuring data consistency in the event of failures. While Redis To Go can achieve eventual consistency through replication and persistence options, it may lack the same level of durability as Kafka in certain high-availability scenarios.

In Summary, Kafka excels in scalable event streaming and analytics, while Redis To Go focuses on managed caching and storage with different data persistence and querying capabilities.

Kafka vs Redis To Go

Overview