Sidekiq Style Guide

This document outlines various guidelines that should be followed when adding or modifying Sidekiq workers.

ApplicationWorker

All workers should include ApplicationWorker instead of Sidekiq::Worker, which adds some convenience methods and automatically sets the queue based on the worker's name.

Dedicated Queues

All workers should use their own queue, which is automatically set based on the worker class name. For a worker named ProcessSomethingWorker, the queue name would be process_something. If you're not sure what queue a worker uses, you can find it using SomeWorker.queue. There is almost never a reason to manually override the queue name using sidekiq_options queue: :some_queue.

You must always add any new queues to app/workers/all_queues.yml or ee/app/workers/all_queues.yml otherwise your worker will not run.

Queue Namespaces

While different workers cannot share a queue, they can share a queue namespace.

Defining a queue namespace for a worker makes it possible to start a Sidekiq process that automatically handles jobs for all workers in that namespace, without needing to explicitly list all their queue names. If, for example, all workers that are managed by sidekiq-cron use the cronjob queue namespace, we can spin up a Sidekiq process specifically for these kinds of scheduled jobs. If a new worker using the cronjob namespace is added later on, the Sidekiq process will automatically pick up jobs for that worker too (after having been restarted), without the need to change any configuration.

A queue namespace can be set using the queue_namespace DSL class method:

class SomeScheduledTaskWorker
  include ApplicationWorker

  queue_namespace :cronjob

  # ...
end

Behind the scenes, this will set SomeScheduledTaskWorker.queue to cronjob:some_scheduled_task. Commonly used namespaces will have their own concern module that can easily be included into the worker class, and that may set other Sidekiq options besides the queue namespace. CronjobQueue, for example, sets the namespace, but also disables retries.

bundle exec sidekiq is namespace-aware, and will automatically listen on all queues in a namespace (technically: all queues prefixed with the namespace name) when a namespace is provided instead of a simple queue name in the --queue (-q) option, or in the :queues: section in config/sidekiq_queues.yml.

Note that adding a worker to an existing namespace should be done with care, as the extra jobs will take resources away from jobs from workers that were already there, if the resources available to the Sidekiq process handling the namespace are not adjusted appropriately.

Tests

Each Sidekiq worker must be tested using RSpec, just like any other class. These tests should be placed in spec/workers.

Removing or renaming queues

Try to avoid renaming or removing workers and their queues in minor and patch releases. During online update instance can have pending jobs and removing the queue can lead to those jobs being stuck forever. If you can't write migration for those Sidekiq jobs, please consider doing rename or remove queue in major release only.

3.1 KiB Raw Blame History