> [!METADATA]+ > **Type**:: [[Conference Talk]] > **Conference**:: [[Sin City Ruby 2022]] > **Authors**:: [[@Kelly Sutton]] > **Links**:: ## Notes - [[@Kelly Sutton]] - [[Seattle]] - [[Gusto]] - payroll and HR in the US - Turned [[Sidekiq]] into managed service - use latency based queues - sharing to increase capacity - read-only queues for high throughput workflows - Managed service - One team is responsible for the healthy operation of a piece of the infrastructure - Teams use products or features - Tells product engineer about it’s constraints - Create leverage for engineering orgs - frees up times and salary for the company - Focus on what you do best, pay for what you don’t - Latency-based queue - Uses job latency to set a worst-case expectation for when a job will be picked up - latency != runtime - Latency: how long has this job been wating - A queue’s latency is the time till it’s first job - Queue names: `within_thirty_seconds`, `within_five_minutes`, `within_one_hour` - Infra engineers make changes that affect latency - Product engineers “how long does a job take to run”, “when must a job be run by” - Contract between product and infra engineers - organizing jobs by explicitly latency makes life easier for on-call engineers - What is a shard - Write-heavy workloads at a high concurrency are a risk for traditional databases - to keep the db healthy, we reduce or cap the number of total workers - When one queue gets too much work, create a shard - `within_five_minutes_shard_0` - Shards increase capacity while minimizing the risk of crushing the db - What are read-only queues - let us safely run at high concurrency - Need a replica database - Work consumes off the replica - Can’t write to database since it’s a replica so route a job trying to write to the db to the primary - Much easier with Rails 6+ multiple ```ruby class ReadOnlyMiddleware def call(worker, job, queue, &block) ActiveRecord::Base.connected_to(role: :reading, &block) rescue ActiveRecord::ReadOnlyError enqueue_onto_read_write_queue(job) end end ```