Sidekiq logo

Sidekiq

Simple, efficient background processing for Ruby
1.1K
629
+ 1
408

What is Sidekiq?

Sidekiq uses threads to handle many jobs at the same time in the same process. It does not require Rails but will integrate tightly with Rails 3/4 to make background processing dead simple.
Sidekiq is a tool in the Background Processing category of a tech stack.
Sidekiq is an open source tool with GitHub stars and GitHub forks. Here’s a link to Sidekiq's open source repository on GitHub

Who uses Sidekiq?

Companies
512 companies reportedly use Sidekiq in their tech stacks, including StackShare, Accenture, and GitLab.

Developers
520 developers on StackShare have stated that they use Sidekiq.

Sidekiq Integrations

Pros of Sidekiq
124
Simple
99
Efficient background processing
60
Scalability
37
Better then resque
26
Great documentation
15
Admin tool
14
Great community
8
Integrates with redis automatically, with zero config
7
Stupidly simple to integrate and run on Rails/Heroku
7
Great support
3
Ruby
3
Freeium
2
Pro version
1
Dashboard w/live polling
1
Great ecosystem of addons
1
Fast
Decisions about Sidekiq

Here are some stack decisions, common use cases and reviews by companies and developers who chose Sidekiq in their tech stack.

Needs advice
on
AirflowAirflow
and
SidekiqSidekiq

Need to use one of them, we have already a server running with Sidekiq. But we have a task to trigger scheduled jobs to sync data with airbyte.

See more
Cyril Duchon-Doris
Shared insights
on
RollbarRollbar
at

After splitting our monolith into a Rails API + a React Redux.js frontend app, it became a necessity to monitor frontend errors. Our frontend application is not your typical website, and features a lot of interesting SPA mechanics that need to be followed closely (many async flows, redux-saga , etc.) in addition to regular browser incompatibility issues. Rollbar kicks in so that we can monitor every bug that happens on our frontend, and aggregate this with almost 0 work. The number of occurrences and affected browsers on each occurence helps us understand the priority and severity of bugs even when our users don't tell us about them, so we can decide whether we need to fix this bug that was encountered by 1k users in less than a few days days VERSUS telling this SINGLE user to switch browsers because he's using a very outdated version that no one else uses. Now we also use Rollbar with Rails, Sidekiq and even AWS Lambda errors since the interface is quite convenient.

See more
Simon Bettison
Managing Director at Bettison.org Limited · | 8 upvotes · 771K views
Shared insights
at

In 2012 we made the very difficult decision to entirely re-engineer our existing monolithic LAMP application from the ground up in order to address some growing concerns about it's long term viability as a platform.

Full application re-write is almost always never the answer, because of the risks involved. However the situation warranted drastic action as it was clear that the existing product was going to face severe scaling issues. We felt it better address these sooner rather than later and also take the opportunity to improve the international architecture and also to refactor the database in. order that it better matched the changes in core functionality.

PostgreSQL was chosen for its reputation as being solid ACID compliant database backend, it was available as an offering AWS RDS service which reduced the management overhead of us having to configure it ourselves. In order to reduce read load on the primary database we implemented an Elasticsearch layer for fast and scalable search operations. Synchronisation of these indexes was to be achieved through the use of Sidekiq's Redis based background workers on Amazon ElastiCache. Again the AWS solution here looked to be an easy way to keep our involvement in managing this part of the platform at a minimum. Allowing us to focus on our core business.

Rails ls was chosen for its ability to quickly get core functionality up and running, its MVC architecture and also its focus on Test Driven Development using RSpec and Selenium with Travis CI providing continual integration. We also liked Ruby for its terse, clean and elegant syntax. Though YMMV on that one!

Unicorn was chosen for its continual deployment and reputation as a reliable application server, nginx for its reputation as a fast and stable reverse-proxy. We also took advantage of the Amazon CloudFront CDN here to further improve performance by caching static assets globally.

We tried to strike a balance between having control over management and configuration of our core application with the convenience of being able to leverage AWS hosted services for ancillary functions (Amazon SES , Amazon SQS Amazon Route 53 all hosted securely inside Amazon VPC of course!).

Whilst there is some compromise here with potential vendor lock in, the tasks being performed by these ancillary services are no particularly specialised which should mitigate this risk. Furthermore we have already containerised the stack in our development using Docker environment, and looking to how best to bring this into production - potentially using Amazon EC2 Container Service

See more

Blog Posts

Jun 6 2019 at 5:11PM

AppSignal

RedisRubyKafka+9
15
1651
GitHubMySQLSlack+44
109
50679

Sidekiq Alternatives & Comparisons

What are some alternatives to Sidekiq?
Resque
Background jobs can be any Ruby class or module that responds to perform. Your existing classes can easily be converted to background jobs or you can create new classes specifically to do work. Or, you can do both.
Celery
Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.
RabbitMQ
RabbitMQ gives your applications a common platform to send and receive messages, and your messages a safe place to live until received.
delayed_job
Delayed_job (or DJ) encapsulates the common pattern of asynchronously executing longer tasks in the background. It is a direct extraction from Shopify where the job table is responsible for a multitude of core tasks.
Kafka
Kafka is a distributed, partitioned, replicated commit log service. It provides the functionality of a messaging system, but with a unique design.
See all alternatives

Sidekiq's Followers
629 developers follow Sidekiq to keep up with related blogs and decisions.