What is NATS and what are its top alternatives?
Top Alternatives to NATS
- Kafka
Kafka is a distributed, partitioned, replicated commit log service. It provides the functionality of a messaging system, but with a unique design. ...
- gRPC
gRPC is a modern open source high performance RPC framework that can run in any environment. It can efficiently connect services in and across data centers with pluggable support for load balancing, tracing, health checking... ...
- MQTT
It was designed as an extremely lightweight publish/subscribe messaging transport. It is useful for connections with remote locations where a small code footprint is required and/or network bandwidth is at a premium. ...
- NSQ
NSQ is a realtime distributed messaging platform designed to operate at scale, handling billions of messages per day. It promotes distributed and decentralized topologies without single points of failure, enabling fault tolerance and high availability coupled with a reliable message delivery guarantee. See features & guarantees. ...
- RabbitMQ
RabbitMQ gives your applications a common platform to send and receive messages, and your messages a safe place to live until received. ...
- Mosquitto
It is lightweight and is suitable for use on all devices from low power single board computers to full servers.. The MQTT protocol provides a lightweight method of carrying out messaging using a publish/subscribe model. This makes it suitable for Internet of Things messaging such as with low power sensors or mobile devices such as phones, embedded computers or microcontrollers. ...
- Firebase
Firebase is a cloud service designed to power real-time, collaborative applications. Simply add the Firebase library to your application to gain access to a shared data structure; any changes you make to that data are automatically synchronized with the Firebase cloud and with other clients within milliseconds. ...
- Socket.IO
It enables real-time bidirectional event-based communication. It works on every platform, browser or device, focusing equally on reliability and speed. ...
NATS alternatives & related posts
Kafka
- High-throughput126
- Distributed119
- Scalable92
- High-Performance86
- Durable66
- Publish-Subscribe38
- Simple-to-use19
- Open source18
- Written in Scala and java. Runs on JVM12
- Message broker + Streaming system9
- KSQL4
- Avro schema integration4
- Robust4
- Suport Multiple clients3
- Extremely good parallelism constructs2
- Partioned, replayable log2
- Simple publisher / multi-subscriber model1
- Fun1
- Flexible1
- Non-Java clients are second-class citizens32
- Needs Zookeeper29
- Operational difficulties9
- Terrible Packaging5
related Kafka posts
The algorithms and data infrastructure at Stitch Fix is housed in #AWS. Data acquisition is split between events flowing through Kafka, and periodic snapshots of PostgreSQL DBs. We store data in an Amazon S3 based data warehouse. Apache Spark on Yarn is our tool of choice for data movement and #ETL. Because our storage layer (s3) is decoupled from our processing layer, we are able to scale our compute environment very elastically. We have several semi-permanent, autoscaling Yarn clusters running to serve our data processing needs. While the bulk of our compute infrastructure is dedicated to algorithmic processing, we also implemented Presto for adhoc queries and dashboards.
Beyond data movement and ETL, most #ML centric jobs (e.g. model training and execution) run in a similarly elastic environment as containers running Python and R code on Amazon EC2 Container Service clusters. The execution of batch jobs on top of ECS is managed by Flotilla, a service we built in house and open sourced (see https://github.com/stitchfix/flotilla-os).
At Stitch Fix, algorithmic integrations are pervasive across the business. We have dozens of data products actively integrated systems. That requires serving layer that is robust, agile, flexible, and allows for self-service. Models produced on Flotilla are packaged for deployment in production using Khan, another framework we've developed internally. Khan provides our data scientists the ability to quickly productionize those models they've developed with open source frameworks in Python 3 (e.g. PyTorch, sklearn), by automatically packaging them as Docker containers and deploying to Amazon ECS. This provides our data scientist a one-click method of getting from their algorithms to production. We then integrate those deployments into a service mesh, which allows us to A/B test various implementations in our product.
For more info:
- Our Algorithms Tour: https://algorithms-tour.stitchfix.com/
- Our blog: https://multithreaded.stitchfix.com/blog/
- Careers: https://multithreaded.stitchfix.com/careers/
#DataScience #DataStack #Data
As we've evolved or added additional infrastructure to our stack, we've biased towards managed services. Most new backing stores are Amazon RDS instances now. We do use self-managed PostgreSQL with TimescaleDB for time-series data—this is made HA with the use of Patroni and Consul.
We also use managed Amazon ElastiCache instances instead of spinning up Amazon EC2 instances to run Redis workloads, as well as shifting to Amazon Kinesis instead of Kafka.
- Higth performance24
- The future of API15
- Easy setup13
- Contract-based5
- Polyglot4
- Garbage2
related gRPC posts
I used GraphQL extensively at a previous employer a few years ago and really appreciated the data-driven schema etc alongside the many other benefits it provided. At that time, it seemed like it was set to replace RESTful APIs and many companies were adopting it.
However, as of late, it seems like interest has been waning for GraphQL as opposed to increasing as I had assumed it would. Am I missing something here? What is the current perspective regarding this technology?
Currently, I'm working with gRPC and was curious as to the state of everything now.
We need to interact from several different Web applications (remote) to a client-side application (.exe in .NET Framework, Windows.Console under our controlled environment). From the web applications, we need to send and receive data and invoke methods to client-side .exe on javascript events like users onclick. SignalR is one of the .Net alternatives to do that, but it adds overhead for what we need. Is it better to add SignalR at both client-side application and remote web application, or use gRPC as it sounds lightest and is multilingual?
SignalR or gRPC are always sending and receiving data on the client-side (from browser to .exe and back to browser). And web application is used for graphical visualization of data to the user. There is no need for local .exe to send or interact with remote web API. Which architecture or framework do you suggest to use in this case?
- Varying levels of Quality of Service to fit a range of3
- Lightweight with a relatively small data footprint2
- Very easy to configure and use with open source tools2
- Easy to configure in an unsecure manner1
related MQTT posts
Kindly suggest the best tool for generating 10Mn+ concurrent user load. The tool must support MQTT traffic, REST API, support to interfaces such as Kafka, websockets, persistence HTTP connection, auth type support to assess the support /coverage.
The tool can be integrated into CI pipelines like Azure Pipelines, GitHub, and Jenkins.
Hi Marc,
For the com part, depending of more details not provided, i'd use SSE, OR i'd run either Mosquitto or RabbitMQ running on Amazon EC2 instances and leverage MQTT or amqp 's subscribe/publish features with my users running mqtt or amqp clients (tcp or websockets) somehow. (publisher too.. you don't say how and who gets to update the document(s).
I find "a ton of end users", depending on how you define a ton (1k users ;) ?) and how frequent document updates are, that can mean a ton of ressources, can't cut it at some point, even using SSE
how many, how big, how persistant do the document(s) have to be ? Db-wise,can't say for lack of details and context, yeah could also be Redis, any RDBMS or nosql or even static json files stored on an Amazon S3 bucket .. anything really
Good luck!
- It's in golang29
- Lightweight20
- Distributed19
- Easy setup18
- High throughput16
- Publish-Subscribe10
- Save data if no subscribers are found7
- Scalable7
- Open source6
- Temporarily kept on disk5
- Simple-to use2
- Load balanced1
- Free1
- Primarily in-memory1
- Topics and channels concept1
- Long term persistence1
- Get NSQ behavior out of Kafka but not inverse1
- HA1
related NSQ posts
- It's fast and it works with good metrics/monitoring234
- Ease of configuration79
- I like the admin interface59
- Easy to set-up and start with50
- Durable21
- Intuitive work through python18
- Standard protocols18
- Written primarily in Erlang10
- Simply superb8
- Completeness of messaging patterns6
- Scales to 1 million messages per second3
- Reliable3
- Distributed2
- Supports MQTT2
- Better than most traditional queue based message broker2
- Supports AMQP2
- Clusterable1
- Clear documentation with different scripting language1
- Great ui1
- Inubit Integration1
- Better routing system1
- High performance1
- Runs on Open Telecom Platform1
- Delayed messages1
- Reliability1
- Open-source1
- Too complicated cluster/HA config and management9
- Needs Erlang runtime. Need ops good with Erlang runtime6
- Configuration must be done first, not by your code5
- Slow4
related RabbitMQ posts
As Sentry runs throughout the day, there are about 50 different offline tasks that we execute—anything from “process this event, pretty please” to “send all of these cool people some emails.” There are some that we execute once a day and some that execute thousands per second.
Managing this variety requires a reliably high-throughput message-passing technology. We use Celery's RabbitMQ implementation, and we stumbled upon a great feature called Federation that allows us to partition our task queue across any number of RabbitMQ servers and gives us the confidence that, if any single server gets backlogged, others will pitch in and distribute some of the backlogged tasks to their consumers.
#MessageQueue
Around the time of their Series A, Pinterest’s stack included Python and Django, with Tornado and Node.js as web servers. Memcached / Membase and Redis handled caching, with RabbitMQ handling queueing. Nginx, HAproxy and Varnish managed static-delivery and load-balancing, with persistent data storage handled by MySQL.
- Simple and light10
- Performance4
related Mosquitto posts
Hi Marc,
For the com part, depending of more details not provided, i'd use SSE, OR i'd run either Mosquitto or RabbitMQ running on Amazon EC2 instances and leverage MQTT or amqp 's subscribe/publish features with my users running mqtt or amqp clients (tcp or websockets) somehow. (publisher too.. you don't say how and who gets to update the document(s).
I find "a ton of end users", depending on how you define a ton (1k users ;) ?) and how frequent document updates are, that can mean a ton of ressources, can't cut it at some point, even using SSE
how many, how big, how persistant do the document(s) have to be ? Db-wise,can't say for lack of details and context, yeah could also be Redis, any RDBMS or nosql or even static json files stored on an Amazon S3 bucket .. anything really
Good luck!
- Realtime backend made easy371
- Fast and responsive270
- Easy setup242
- Real-time215
- JSON191
- Free134
- Backed by google128
- Angular adaptor83
- Reliable68
- Great customer support36
- Great documentation32
- Real-time synchronization25
- Mobile friendly21
- Rapid prototyping18
- Great security14
- Automatic scaling12
- Freakingly awesome11
- Super fast development8
- Angularfire is an amazing addition!8
- Chat8
- Built in user auth/oauth6
- Ios adaptor6
- Awesome next-gen backend6
- Firebase hosting6
- Speed of light4
- Very easy to use4
- Great3
- It's made development super fast3
- Brilliant for startups3
- The concurrent updates create a great experience2
- Push notification2
- .net2
- Cloud functions2
- Free hosting2
- Free authentication solution2
- JS Offline and Sync suport2
- Low battery consumption2
- I can quickly create static web apps with no backend2
- Great all-round functionality2
- Large1
- Easy to use1
- Free SSL1
- Faster workflow1
- Google's support1
- CDN & cache out of the box1
- Easy Reactjs integration1
- Simple and easy1
- Good Free Limits1
- Serverless1
- Can become expensive31
- No open source, you depend on external company16
- Scalability is not infinite15
- Not Flexible Enough9
- Cant filter queries7
- Very unstable server3
- No Relational Data3
- Too many errors2
- No offline sync2
related Firebase posts
Hi Otensia! I'd definitely recommend using the skills you've already got and building with JavaScript is a smart way to go these days. Most platform services have JavaScript/Node SDKs or NPM packages, many serverless platforms support Node in case you need to write any backend logic, and JavaScript is incredibly popular - meaning it will be easy to hire for, should you ever need to.
My advice would be "don't reinvent the wheel". If you already have a skill set that will work well to solve the problem at hand, and you don't need it for any other projects, don't spend the time jumping into a new language. If you're looking for an excuse to learn something new, it would be better to invest that time in learning a new platform/tool that compliments your knowledge of JavaScript. For this project, I might recommend using Netlify, Vercel, or Google Firebase to quickly and easily deploy your web app. If you need to add user authentication, there are great examples out there for Firebase Authentication, Auth0, or even Magic (a newcomer on the Auth scene, but very user friendly). All of these services work very well with a JavaScript-based application.
This is my stack in Application & Data
JavaScript PHP HTML5 jQuery Redis Amazon EC2 Ubuntu Sass Vue.js Firebase Laravel Lumen Amazon RDS GraphQL MariaDB
My Utilities Tools
Google Analytics Postman Elasticsearch
My Devops Tools
Git GitHub GitLab npm Visual Studio Code Kibana Sentry BrowserStack
My Business Tools
Slack
- Real-time219
- Node.js142
- Event-based communication141
- Open source102
- WebSockets102
- Binary streaming26
- No internet dependency21
- Large community10
- Fallback to polling if WebSockets not supported9
- Push notification6
- Ease of access and setup5
- Test1
- Bad documentation12
- Githubs that complement it are mostly deprecated4
- Doesn't work on React Native3
- Small community2
- Websocket Errors2
related Socket.IO posts
I use Socket.IO because the application has 2 frontend clients, which need to communicate in real-time. The backend-server handles the communication between these two clients via websockets. Socket.io is very easy to set up in Node.js and ExpressJS.
In the research project, the 1st client shows panoramic videos in a so called cave system (it is the VR setup of our research lab, which consists of three big screens, which are specially arranged, so the user experience the videos more immersive), the 2nd client controls the videos/locations of the 1st client.
We are starting to work on a web-based platform aiming to connect artists (clients) and professional freelancers (service providers). In-app, timeline-based, real-time communication between users (& storing it), file transfers, and push notifications are essential core features. We are considering using Node.js, ExpressJS, React, MongoDB stack with Socket.IO & Apollo, or maybe using Real-Time Database and functionalities of Firebase.