Need advice about which tool to choose?Ask the StackShare community!
ActiveMQ vs Celery: What are the differences?
Developers describe ActiveMQ as "A message broker written in Java together with a full JMS client". Apache ActiveMQ is fast, supports many Cross Language Clients and Protocols, comes with easy to use Enterprise Integration Patterns and many advanced features while fully supporting JMS 1.1 and J2EE 1.4. Apache ActiveMQ is released under the Apache 2.0 License. On the other hand, Celery is detailed as "Distributed task queue". Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.
ActiveMQ and Celery belong to "Message Queue" category of the tech stack.
"Open source" is the primary reason why developers consider ActiveMQ over the competitors, whereas "Task queue" was stated as the key factor in picking Celery.
ActiveMQ and Celery are both open source tools. It seems that Celery with 12.9K GitHub stars and 3.33K forks on GitHub has more adoption than ActiveMQ with 1.5K GitHub stars and 1.05K GitHub forks.
According to the StackShare community, Celery has a broader approval, being mentioned in 272 company stacks & 77 developers stacks; compared to ActiveMQ, which is listed in 33 company stacks and 17 developer stacks.
I am just a beginner at these two technologies.
Problem statement: I am getting lakh of users from the sequel server for whom I need to create caches in MongoDB by making different REST API requests.
Here these users can be treated as messages. Each REST API request is a task.
I am confused about whether I should go for RabbitMQ alone or Celery.
If I have to go with RabbitMQ, I prefer to use python with Pika module. But the challenge with Pika is, it is not thread-safe. So I am not finding a way to execute a lakh of API requests in parallel using multiple threads using Pika.
If I have to go with Celery, I don't know how I can achieve better scalability in executing these API requests in parallel.
For large amounts of small tasks and caches I have had good luck with Redis and RQ. I have not personally used celery but I am fairly sure it would scale well, and I have not used RabbitMQ for anything besides communication between services. If you prefer python my suggestions should feel comfortable.
Sorry I do not have a more information