Alternatives to Amazon SQS logo

Alternatives to Amazon SQS

Amazon MQ, Kafka, Redis, ActiveMQ, and Amazon SNS are the most popular alternatives and competitors to Amazon SQS.
2.2K
2K
+ 1
171

What is Amazon SQS and what are its top alternatives?

Transmit any volume of data, at any level of throughput, without losing messages or requiring other services to be always available. With SQS, you can offload the administrative burden of operating and scaling a highly available messaging cluster, while paying a low price for only what you use.
Amazon SQS is a tool in the Message Queue category of a tech stack.

Top Alternatives to Amazon SQS

  • Amazon MQ
    Amazon MQ

    Amazon MQ is a managed message broker service for Apache ActiveMQ that makes it easy to set up and operate message brokers in the cloud. ...

  • Kafka
    Kafka

    Kafka is a distributed, partitioned, replicated commit log service. It provides the functionality of a messaging system, but with a unique design. ...

  • Redis
    Redis

    Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache, and message broker. Redis provides data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes, and streams. ...

  • ActiveMQ
    ActiveMQ

    Apache ActiveMQ is fast, supports many Cross Language Clients and Protocols, comes with easy to use Enterprise Integration Patterns and many advanced features while fully supporting JMS 1.1 and J2EE 1.4. Apache ActiveMQ is released under the Apache 2.0 License. ...

  • Amazon SNS
    Amazon SNS

    Amazon Simple Notification Service makes it simple and cost-effective to push to mobile devices such as iPhone, iPad, Android, Kindle Fire, and internet connected smart devices, as well as pushing to other distributed services. Besides pushing cloud notifications directly to mobile devices, SNS can also deliver notifications by SMS text message or email, to Simple Queue Service (SQS) queues, or to any HTTP endpoint. ...

  • RabbitMQ
    RabbitMQ

    RabbitMQ gives your applications a common platform to send and receive messages, and your messages a safe place to live until received. ...

  • Amazon Kinesis
    Amazon Kinesis

    Amazon Kinesis can collect and process hundreds of gigabytes of data per second from hundreds of thousands of sources, allowing you to easily write applications that process information in real-time, from sources such as web site click-streams, marketing and financial information, manufacturing instrumentation and social media, and operational logs and metering data. ...

  • Azure Service Bus
    Azure Service Bus

    It is a cloud messaging system for connecting apps and devices across public and private clouds. You can depend on it when you need highly-reliable cloud messaging service between applications and services, even when one or more is offline. ...

Amazon SQS alternatives & related posts

Amazon MQ logo

Amazon MQ

54
325
12
Managed Message Broker Service for ActiveMQ
54
325
+ 1
12
PROS OF AMAZON MQ
  • 7
    Supports low IQ developers
  • 3
    Supports existing protocols (JMS, NMS, AMQP, STOMP, …)
  • 2
    Easy to migrate existing messaging service
CONS OF AMAZON MQ
  • 4
    Slow AF

related Amazon MQ posts

MITHIRIDI PRASANTH
Software Engineer at LightMetrics · | 4 upvotes · 283.5K views
Shared insights
on
Amazon MQAmazon MQAmazon SQSAmazon SQS

I want to schedule a message. Amazon SQS provides a delay of 15 minutes, but I want it in some hours.

Example: Let's say a Message1 is consumed by a consumer A but somehow it failed inside the consumer. I would want to put it in a queue and retry after 4hrs. Can I do this in Amazon MQ? I have seen in some Amazon MQ videos saying scheduling messages can be done. But, I'm not sure how.

See more
Kavitha Padmakumar
devops engineer at Harman · | 3 upvotes · 12.2K views
Shared insights
on
Amazon MQAmazon MQCloudAMQPCloudAMQP

How is CloudAMQP different from Amazon MQ?

which is the better one to use?

See more
Kafka logo

Kafka

23.4K
21.9K
607
Distributed, fault tolerant, high throughput pub-sub messaging system
23.4K
21.9K
+ 1
607
PROS OF KAFKA
  • 126
    High-throughput
  • 119
    Distributed
  • 92
    Scalable
  • 86
    High-Performance
  • 66
    Durable
  • 38
    Publish-Subscribe
  • 19
    Simple-to-use
  • 18
    Open source
  • 12
    Written in Scala and java. Runs on JVM
  • 9
    Message broker + Streaming system
  • 4
    KSQL
  • 4
    Avro schema integration
  • 4
    Robust
  • 3
    Suport Multiple clients
  • 2
    Extremely good parallelism constructs
  • 2
    Partioned, replayable log
  • 1
    Simple publisher / multi-subscriber model
  • 1
    Fun
  • 1
    Flexible
CONS OF KAFKA
  • 32
    Non-Java clients are second-class citizens
  • 29
    Needs Zookeeper
  • 9
    Operational difficulties
  • 5
    Terrible Packaging

related Kafka posts

Nick Rockwell
SVP, Engineering at Fastly · | 46 upvotes · 3.8M views

When I joined NYT there was already broad dissatisfaction with the LAMP (Linux Apache HTTP Server MySQL PHP) Stack and the front end framework, in particular. So, I wasn't passing judgment on it. I mean, LAMP's fine, you can do good work in LAMP. It's a little dated at this point, but it's not ... I didn't want to rip it out for its own sake, but everyone else was like, "We don't like this, it's really inflexible." And I remember from being outside the company when that was called MIT FIVE when it had launched. And been observing it from the outside, and I was like, you guys took so long to do that and you did it so carefully, and yet you're not happy with your decisions. Why is that? That was more the impetus. If we're going to do this again, how are we going to do it in a way that we're gonna get a better result?

So we're moving quickly away from LAMP, I would say. So, right now, the new front end is React based and using Apollo. And we've been in a long, protracted, gradual rollout of the core experiences.

React is now talking to GraphQL as a primary API. There's a Node.js back end, to the front end, which is mainly for server-side rendering, as well.

Behind there, the main repository for the GraphQL server is a big table repository, that we call Bodega because it's a convenience store. And that reads off of a Kafka pipeline.

See more
Ashish Singh
Tech Lead, Big Data Platform at Pinterest · | 38 upvotes · 3.1M views

To provide employees with the critical need of interactive querying, we’ve worked with Presto, an open-source distributed SQL query engine, over the years. Operating Presto at Pinterest’s scale has involved resolving quite a few challenges like, supporting deeply nested and huge thrift schemas, slow/ bad worker detection and remediation, auto-scaling cluster, graceful cluster shutdown and impersonation support for ldap authenticator.

Our infrastructure is built on top of Amazon EC2 and we leverage Amazon S3 for storing our data. This separates compute and storage layers, and allows multiple compute clusters to share the S3 data.

We have hundreds of petabytes of data and tens of thousands of Apache Hive tables. Our Presto clusters are comprised of a fleet of 450 r4.8xl EC2 instances. Presto clusters together have over 100 TBs of memory and 14K vcpu cores. Within Pinterest, we have close to more than 1,000 monthly active users (out of total 1,600+ Pinterest employees) using Presto, who run about 400K queries on these clusters per month.

Each query submitted to Presto cluster is logged to a Kafka topic via Singer. Singer is a logging agent built at Pinterest and we talked about it in a previous post. Each query is logged when it is submitted and when it finishes. When a Presto cluster crashes, we will have query submitted events without corresponding query finished events. These events enable us to capture the effect of cluster crashes over time.

Each Presto cluster at Pinterest has workers on a mix of dedicated AWS EC2 instances and Kubernetes pods. Kubernetes platform provides us with the capability to add and remove workers from a Presto cluster very quickly. The best-case latency on bringing up a new worker on Kubernetes is less than a minute. However, when the Kubernetes cluster itself is out of resources and needs to scale up, it can take up to ten minutes. Some other advantages of deploying on Kubernetes platform is that our Presto deployment becomes agnostic of cloud vendor, instance types, OS, etc.

#BigData #AWS #DataScience #DataEngineering

See more
Redis logo

Redis

59.1K
45.5K
3.9K
Open source (BSD licensed), in-memory data structure store
59.1K
45.5K
+ 1
3.9K
PROS OF REDIS
  • 886
    Performance
  • 542
    Super fast
  • 513
    Ease of use
  • 444
    In-memory cache
  • 324
    Advanced key-value cache
  • 194
    Open source
  • 182
    Easy to deploy
  • 164
    Stable
  • 155
    Free
  • 121
    Fast
  • 42
    High-Performance
  • 40
    High Availability
  • 35
    Data Structures
  • 32
    Very Scalable
  • 24
    Replication
  • 22
    Great community
  • 22
    Pub/Sub
  • 19
    "NoSQL" key-value data store
  • 16
    Hashes
  • 13
    Sets
  • 11
    Sorted Sets
  • 10
    NoSQL
  • 10
    Lists
  • 9
    Async replication
  • 9
    BSD licensed
  • 8
    Bitmaps
  • 8
    Integrates super easy with Sidekiq for Rails background
  • 7
    Keys with a limited time-to-live
  • 7
    Open Source
  • 6
    Lua scripting
  • 6
    Strings
  • 5
    Awesomeness for Free
  • 5
    Hyperloglogs
  • 4
    Transactions
  • 4
    Outstanding performance
  • 4
    Runs server side LUA
  • 4
    LRU eviction of keys
  • 4
    Feature Rich
  • 4
    Written in ANSI C
  • 4
    Networked
  • 3
    Data structure server
  • 3
    Performance & ease of use
  • 2
    Dont save data if no subscribers are found
  • 2
    Automatic failover
  • 2
    Easy to use
  • 2
    Temporarily kept on disk
  • 2
    Scalable
  • 2
    Existing Laravel Integration
  • 2
    Channels concept
  • 2
    Object [key/value] size each 500 MB
  • 2
    Simple
CONS OF REDIS
  • 15
    Cannot query objects directly
  • 3
    No secondary indexes for non-numeric data types
  • 1
    No WAL

related Redis posts

Russel Werner
Lead Engineer at StackShare · | 32 upvotes · 2.6M views

StackShare Feed is built entirely with React, Glamorous, and Apollo. One of our objectives with the public launch of the Feed was to enable a Server-side rendered (SSR) experience for our organic search traffic. When you visit the StackShare Feed, and you aren't logged in, you are delivered the Trending feed experience. We use an in-house Node.js rendering microservice to generate this HTML. This microservice needs to run and serve requests independent of our Rails web app. Up until recently, we had a mono-repo with our Rails and React code living happily together and all served from the same web process. In order to deploy our SSR app into a Heroku environment, we needed to split out our front-end application into a separate repo in GitHub. The driving factor in this decision was mostly due to limitations imposed by Heroku specifically with how processes can't communicate with each other. A new SSR app was created in Heroku and linked directly to the frontend repo so it stays in-sync with changes.

Related to this, we need a way to "deploy" our frontend changes to various server environments without building & releasing the entire Ruby application. We built a hybrid Amazon S3 Amazon CloudFront solution to host our Webpack bundles. A new CircleCI script builds the bundles and uploads them to S3. The final step in our rollout is to update some keys in Redis so our Rails app knows which bundles to serve. The result of these efforts were significant. Our frontend team now moves independently of our backend team, our build & release process takes only a few minutes, we are now using an edge CDN to serve JS assets, and we have pre-rendered React pages!

#StackDecisionsLaunch #SSR #Microservices #FrontEndRepoSplit

See more
Simon Reymann
Senior Fullstack Developer at QUANTUSflow Software GmbH · | 30 upvotes · 10.5M views

Our whole DevOps stack consists of the following tools:

  • GitHub (incl. GitHub Pages/Markdown for Documentation, GettingStarted and HowTo's) for collaborative review and code management tool
  • Respectively Git as revision control system
  • SourceTree as Git GUI
  • Visual Studio Code as IDE
  • CircleCI for continuous integration (automatize development process)
  • Prettier / TSLint / ESLint as code linter
  • SonarQube as quality gate
  • Docker as container management (incl. Docker Compose for multi-container application management)
  • VirtualBox for operating system simulation tests
  • Kubernetes as cluster management for docker containers
  • Heroku for deploying in test environments
  • nginx as web server (preferably used as facade server in production environment)
  • SSLMate (using OpenSSL) for certificate management
  • Amazon EC2 (incl. Amazon S3) for deploying in stage (production-like) and production environments
  • PostgreSQL as preferred database system
  • Redis as preferred in-memory database/store (great for caching)

The main reason we have chosen Kubernetes over Docker Swarm is related to the following artifacts:

  • Key features: Easy and flexible installation, Clear dashboard, Great scaling operations, Monitoring is an integral part, Great load balancing concepts, Monitors the condition and ensures compensation in the event of failure.
  • Applications: An application can be deployed using a combination of pods, deployments, and services (or micro-services).
  • Functionality: Kubernetes as a complex installation and setup process, but it not as limited as Docker Swarm.
  • Monitoring: It supports multiple versions of logging and monitoring when the services are deployed within the cluster (Elasticsearch/Kibana (ELK), Heapster/Grafana, Sysdig cloud integration).
  • Scalability: All-in-one framework for distributed systems.
  • Other Benefits: Kubernetes is backed by the Cloud Native Computing Foundation (CNCF), huge community among container orchestration tools, it is an open source and modular tool that works with any OS.
See more
ActiveMQ logo

ActiveMQ

609
1.3K
77
A message broker written in Java together with a full JMS client
609
1.3K
+ 1
77
PROS OF ACTIVEMQ
  • 18
    Easy to use
  • 14
    Open source
  • 13
    Efficient
  • 10
    JMS compliant
  • 6
    High Availability
  • 5
    Scalable
  • 3
    Distributed Network of brokers
  • 3
    Persistence
  • 3
    Support XA (distributed transactions)
  • 1
    Docker delievery
  • 1
    Highly configurable
  • 0
    RabbitMQ
CONS OF ACTIVEMQ
  • 1
    ONLY Vertically Scalable
  • 1
    Support
  • 1
    Low resilience to exceptions and interruptions
  • 1
    Difficult to scale

related ActiveMQ posts

I want to choose Message Queue with the following features - Highly Available, Distributed, Scalable, Monitoring. I have RabbitMQ, ActiveMQ, Kafka and Apache RocketMQ in mind. But I am confused which one to choose.

See more
Naushad Warsi
software developer at klingelnberg · | 1 upvote · 783.5K views
Shared insights
on
ActiveMQActiveMQRabbitMQRabbitMQ

I use ActiveMQ because RabbitMQ have stopped giving the support for AMQP 1.0 or above version and the earlier version of AMQP doesn't give the functionality to support OAuth.

If OAuth is not required and we can go with AMQP 0.9 then i still recommend rabbitMq.

See more
Amazon SNS logo

Amazon SNS

1.4K
1.2K
18
Fully managed push messaging service
1.4K
1.2K
+ 1
18
PROS OF AMAZON SNS
  • 12
    Low cost
  • 6
    Supports multi subscribers
CONS OF AMAZON SNS
    Be the first to leave a con

    related Amazon SNS posts

    Praveen Mooli
    Engineering Manager at Taylor and Francis · | 19 upvotes · 4M views

    We are in the process of building a modern content platform to deliver our content through various channels. We decided to go with Microservices architecture as we wanted scale. Microservice architecture style is an approach to developing an application as a suite of small independently deployable services built around specific business capabilities. You can gain modularity, extensive parallelism and cost-effective scaling by deploying services across many distributed servers. Microservices modularity facilitates independent updates/deployments, and helps to avoid single point of failure, which can help prevent large-scale outages. We also decided to use Event Driven Architecture pattern which is a popular distributed asynchronous architecture pattern used to produce highly scalable applications. The event-driven architecture is made up of highly decoupled, single-purpose event processing components that asynchronously receive and process events.

    To build our #Backend capabilities we decided to use the following: 1. #Microservices - Java with Spring Boot , Node.js with ExpressJS and Python with Flask 2. #Eventsourcingframework - Amazon Kinesis , Amazon Kinesis Firehose , Amazon SNS , Amazon SQS, AWS Lambda 3. #Data - Amazon RDS , Amazon DynamoDB , Amazon S3 , MongoDB Atlas

    To build #Webapps we decided to use Angular 2 with RxJS

    #Devops - GitHub , Travis CI , Terraform , Docker , Serverless

    See more
    Tim Specht
    ‎Co-Founder and CTO at Dubsmash · | 14 upvotes · 26.1K views
    Shared insights
    on
    AWS LambdaAWS LambdaAmazon SNSAmazon SNS
    at

    Whenever we need to notify a user of something happening on our platform, whether it’s a personal push notification from one user to another, a new Dub, or a notification going out to millions of users at the same time that new content is available, we rely on AWS Lambda to do this task for us. When we started implementing this feature 2 years ago we were luckily able to get early access to the Lambda Beta and are still happy with the way things are running on there, especially given all the easy to set up integrations with other AWS services.

    Lambda enables us to quickly send out million of pushes within a couple of minutes by acting as a multiplexer in front of Amazon SNS. We simply call a first Lambda function with a batch of up to 300 push notifications to be sent, which then calls a subsequent Lambda function with 20 pushes each, which then does the call to SNS to actually send out the push notifications.

    This multi-tier process of sending push notifications enables us to quickly adjust our sending volume while keeping costs & maintenance overhead, on our side, to a bare minimum.

    #ApplicationHosting

    See more
    RabbitMQ logo

    RabbitMQ

    21.2K
    18.6K
    557
    Open source multiprotocol messaging broker
    21.2K
    18.6K
    + 1
    557
    PROS OF RABBITMQ
    • 235
      It's fast and it works with good metrics/monitoring
    • 80
      Ease of configuration
    • 60
      I like the admin interface
    • 52
      Easy to set-up and start with
    • 22
      Durable
    • 19
      Standard protocols
    • 19
      Intuitive work through python
    • 11
      Written primarily in Erlang
    • 9
      Simply superb
    • 7
      Completeness of messaging patterns
    • 4
      Reliable
    • 4
      Scales to 1 million messages per second
    • 3
      Better than most traditional queue based message broker
    • 3
      Distributed
    • 3
      Supports MQTT
    • 3
      Supports AMQP
    • 2
      Clear documentation with different scripting language
    • 2
      Better routing system
    • 2
      Inubit Integration
    • 2
      Great ui
    • 2
      High performance
    • 2
      Reliability
    • 2
      Open-source
    • 2
      Runs on Open Telecom Platform
    • 2
      Clusterable
    • 2
      Delayed messages
    • 1
      Supports Streams
    • 1
      Supports STOMP
    • 1
      Supports JMS
    CONS OF RABBITMQ
    • 9
      Too complicated cluster/HA config and management
    • 6
      Needs Erlang runtime. Need ops good with Erlang runtime
    • 5
      Configuration must be done first, not by your code
    • 4
      Slow

    related RabbitMQ posts

    James Cunningham
    Operations Engineer at Sentry · | 18 upvotes · 1.7M views
    Shared insights
    on
    CeleryCeleryRabbitMQRabbitMQ
    at

    As Sentry runs throughout the day, there are about 50 different offline tasks that we execute—anything from “process this event, pretty please” to “send all of these cool people some emails.” There are some that we execute once a day and some that execute thousands per second.

    Managing this variety requires a reliably high-throughput message-passing technology. We use Celery's RabbitMQ implementation, and we stumbled upon a great feature called Federation that allows us to partition our task queue across any number of RabbitMQ servers and gives us the confidence that, if any single server gets backlogged, others will pitch in and distribute some of the backlogged tasks to their consumers.

    #MessageQueue

    See more

    Around the time of their Series A, Pinterest’s stack included Python and Django, with Tornado and Node.js as web servers. Memcached / Membase and Redis handled caching, with RabbitMQ handling queueing. Nginx, HAproxy and Varnish managed static-delivery and load-balancing, with persistent data storage handled by MySQL.

    See more
    Amazon Kinesis logo

    Amazon Kinesis

    723
    600
    9
    Store and process terabytes of data each hour from hundreds of thousands of sources
    723
    600
    + 1
    9
    PROS OF AMAZON KINESIS
    • 9
      Scalable
    CONS OF AMAZON KINESIS
    • 3
      Cost

    related Amazon Kinesis posts

    Praveen Mooli
    Engineering Manager at Taylor and Francis · | 19 upvotes · 4M views

    We are in the process of building a modern content platform to deliver our content through various channels. We decided to go with Microservices architecture as we wanted scale. Microservice architecture style is an approach to developing an application as a suite of small independently deployable services built around specific business capabilities. You can gain modularity, extensive parallelism and cost-effective scaling by deploying services across many distributed servers. Microservices modularity facilitates independent updates/deployments, and helps to avoid single point of failure, which can help prevent large-scale outages. We also decided to use Event Driven Architecture pattern which is a popular distributed asynchronous architecture pattern used to produce highly scalable applications. The event-driven architecture is made up of highly decoupled, single-purpose event processing components that asynchronously receive and process events.

    To build our #Backend capabilities we decided to use the following: 1. #Microservices - Java with Spring Boot , Node.js with ExpressJS and Python with Flask 2. #Eventsourcingframework - Amazon Kinesis , Amazon Kinesis Firehose , Amazon SNS , Amazon SQS, AWS Lambda 3. #Data - Amazon RDS , Amazon DynamoDB , Amazon S3 , MongoDB Atlas

    To build #Webapps we decided to use Angular 2 with RxJS

    #Devops - GitHub , Travis CI , Terraform , Docker , Serverless

    See more
    John Kodumal

    As we've evolved or added additional infrastructure to our stack, we've biased towards managed services. Most new backing stores are Amazon RDS instances now. We do use self-managed PostgreSQL with TimescaleDB for time-series data—this is made HA with the use of Patroni and Consul.

    We also use managed Amazon ElastiCache instances instead of spinning up Amazon EC2 instances to run Redis workloads, as well as shifting to Amazon Kinesis instead of Kafka.

    See more
    Azure Service Bus logo

    Azure Service Bus

    272
    529
    7
    Reliable cloud messaging as a service (MaaS)
    272
    529
    + 1
    7
    PROS OF AZURE SERVICE BUS
    • 4
      Easy Integration with .Net
    • 2
      Cloud Native
    • 1
      Use while high messaging need
    CONS OF AZURE SERVICE BUS
    • 1
      Limited features in Basic tier
    • 1
      Skills can only be used in Azure - vendor lock-in
    • 1
      Lacking in JMS support
    • 1
      Observability of messages in the queue is lacking

    related Azure Service Bus posts

    Shared insights
    on
    Azure Service BusAzure Service BusIBM MQIBM MQ

    Want to get the differences in features and enhancement, pros and cons, and also how to Migrate from IBM MQ to Azure Service Bus.

    See more