Operations Engineer at Sentry·
Shared insights
on
CeleryCeleryRabbitMQRabbitMQ
at

As Sentry runs throughout the day, there are about 50 different offline tasks that we execute—anything from “process this event, pretty please” to “send all of these cool people some emails.” There are some that we execute once a day and some that execute thousands per second.

Managing this variety requires a reliably high-throughput message-passing technology. We use Celery's RabbitMQ implementation, and we stumbled upon a great feature called Federation that allows us to partition our task queue across any number of RabbitMQ servers and gives us the confidence that, if any single server gets backlogged, others will pitch in and distribute some of the backlogged tasks to their consumers.

#MessageQueue

READ LESS
How Sentry Receives 20 Billion Events Per Month While Preparing to Handle Twice That - Sentry Tech Stack | StackShare (stackshare.io)
18 upvotes·1.8M views
Avatar of James Cunningham

James Cunningham

Operations Engineer at Sentry