Need advice about which tool to choose?Ask the StackShare community!

Amazon SQS

2.2K
2K
+ 1
171
Confluent

240
235
+ 1
14
Add tool

Amazon SQS vs Confluent: What are the differences?

Introduction

In this article, we will explore the key differences between Amazon SQS (Simple Queue Service) and Confluent. Both Amazon SQS and Confluent are messaging systems that provide reliable communication between distributed components in modern software architectures. However, there are distinct differences that set them apart. Let's delve into these differences below.

  1. Focus: Amazon SQS primarily focuses on providing a fully managed message queuing service, while Confluent is a streaming platform built around Apache Kafka. While SQS focuses on message queuing and delivery, Confluent provides a broader set of capabilities for building real-time streaming applications.

  2. Data Model: In Amazon SQS, messages are stored as discrete entities and the system guarantees at-least-once delivery. On the other hand, Confluent uses Kafka, a distributed streaming platform, where messages are stored in distributed commit logs called topics. Kafka provides strong durability guarantees as it persists messages to disk, ensuring high availability and fault tolerance.

  3. Scalability: Amazon SQS is fully managed and can automatically scale to handle large amounts of message traffic. It allows you to create an unlimited number of queues and supports a high number of concurrent readers and writers. Confluent also offers scalability, but it requires manual configuration and tuning of Kafka brokers, topics, and partitions to achieve desired throughput and latency.

  4. Latency: While Amazon SQS guarantees high message durability, it may introduce some additional latency due to its fully managed nature. In contrast, Confluent Kafka offers exceptionally low end-to-end latency, often in the millisecond range, making it well-suited for real-time event streaming and high-throughput environments.

  5. Ecosystem: Amazon SQS is tightly integrated with other AWS services, making it easy to build serverless architectures and leverage other AWS capabilities like AWS Lambda for message processing. Confluent, being built around Kafka, benefits from a rich ecosystem that includes connectors, stream processing frameworks like Apache Flink and Apache Samza, and various tooling for data integration, monitoring, and management.

  6. Community and Support: Being a key offering of Amazon Web Services, Amazon SQS has a large user community and benefits from the support and documentation provided by AWS. Confluent also has a growing community around Apache Kafka, with extensive documentation, online resources, and a vibrant ecosystem that actively contributes to its development and maintenance.

In summary, the key differences between Amazon SQS and Confluent lie in their focus, data model, scalability, latency, ecosystem, and community and support. While Amazon SQS is a managed message queuing service, Confluent is a broader streaming platform built around Kafka, offering high scalability, low latency, and a rich ecosystem for real-time event streaming applications.

Advice on Amazon SQS and Confluent
MITHIRIDI PRASANTH
Software Engineer at LightMetrics · | 4 upvotes · 277.6K views
Needs advice
on
Amazon MQAmazon MQ
and
Amazon SQSAmazon SQS
in

I want to schedule a message. Amazon SQS provides a delay of 15 minutes, but I want it in some hours.

Example: Let's say a Message1 is consumed by a consumer A but somehow it failed inside the consumer. I would want to put it in a queue and retry after 4hrs. Can I do this in Amazon MQ? I have seen in some Amazon MQ videos saying scheduling messages can be done. But, I'm not sure how.

See more
Replies (1)
Andres Paredes
Lead Senior Software Engineer at InTouch Technology · | 1 upvotes · 212.3K views
Recommends
on
Amazon SQSAmazon SQS

Mithiridi, I believe you are talking about two different things. 1. If you need to process messages with delays of more 15m or at specific times, it's not a good idea to use queues, independently of tool SQM, Rabbit or Amazon MQ. you should considerer another approach using a scheduled job. 2. For dead queues and policy retries RabbitMQ, for example, doesn't support your use case. https://medium.com/@kiennguyen88/rabbitmq-delay-retry-schedule-with-dead-letter-exchange-31fb25a440fc I'm not sure if that is possible SNS/SQS support, they have a maximum delay for delivery (maxDelayTarget) in seconds but it's not clear the number. You can check this out: https://docs.aws.amazon.com/sns/latest/dg/sns-message-delivery-retries.html

See more
Get Advice from developers at your company using StackShare Enterprise. Sign up for StackShare Enterprise.
Learn More
Pros of Amazon SQS
Pros of Confluent
  • 62
    Easy to use, reliable
  • 40
    Low cost
  • 28
    Simple
  • 14
    Doesn't need to maintain it
  • 8
    It is Serverless
  • 4
    Has a max message size (currently 256K)
  • 3
    Triggers Lambda
  • 3
    Easy to configure with Terraform
  • 3
    Delayed delivery upto 15 mins only
  • 3
    Delayed delivery upto 12 hours
  • 1
    JMS compliant
  • 1
    Support for retry and dead letter queue
  • 1
    D
  • 4
    Free for casual use
  • 3
    No hypercloud lock-in
  • 3
    Dashboard for kafka insight
  • 2
    Easily scalable
  • 2
    Zero devops

Sign up to add or upvote prosMake informed product decisions

Cons of Amazon SQS
Cons of Confluent
  • 2
    Has a max message size (currently 256K)
  • 2
    Proprietary
  • 2
    Difficult to configure
  • 1
    Has a maximum 15 minutes of delayed messages only
  • 1
    Proprietary

Sign up to add or upvote consMake informed product decisions

What is Amazon SQS?

Transmit any volume of data, at any level of throughput, without losing messages or requiring other services to be always available. With SQS, you can offload the administrative burden of operating and scaling a highly available messaging cluster, while paying a low price for only what you use.

What is Confluent?

It is a data streaming platform based on Apache Kafka: a full-scale streaming platform, capable of not only publish-and-subscribe, but also the storage and processing of data within the stream

Need advice about which tool to choose?Ask the StackShare community!

What companies use Amazon SQS?
What companies use Confluent?
See which teams inside your own company are using Amazon SQS or Confluent.
Sign up for StackShare EnterpriseLearn More

Sign up to get full access to all the companiesMake informed product decisions

What tools integrate with Amazon SQS?
What tools integrate with Confluent?

Sign up to get full access to all the tool integrationsMake informed product decisions

Blog Posts

GitHubPythonNode.js+47
55
72496
GitGitHubSlack+30
27
18471
GitHubDockerAmazon EC2+23
12
6584
GitHubPythonSlack+25
7
3184
What are some alternatives to Amazon SQS and Confluent?
Amazon MQ
Amazon MQ is a managed message broker service for Apache ActiveMQ that makes it easy to set up and operate message brokers in the cloud.
Kafka
Kafka is a distributed, partitioned, replicated commit log service. It provides the functionality of a messaging system, but with a unique design.
Redis
Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache, and message broker. Redis provides data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes, and streams.
ActiveMQ
Apache ActiveMQ is fast, supports many Cross Language Clients and Protocols, comes with easy to use Enterprise Integration Patterns and many advanced features while fully supporting JMS 1.1 and J2EE 1.4. Apache ActiveMQ is released under the Apache 2.0 License.
Amazon SNS
Amazon Simple Notification Service makes it simple and cost-effective to push to mobile devices such as iPhone, iPad, Android, Kindle Fire, and internet connected smart devices, as well as pushing to other distributed services. Besides pushing cloud notifications directly to mobile devices, SNS can also deliver notifications by SMS text message or email, to Simple Queue Service (SQS) queues, or to any HTTP endpoint.
See all alternatives