Sizing Laravel queue workers with Horizon
More workers is not automatically faster. Each one is a PHP process with its own memory and database connections.
A queue that backs up looks like a worker shortage and is often a slow job or a missing index. Horizon gives you the numbers to tell the difference: wait time, throughput and runtime per job.
Read the metrics before adding workers
If wait time is high but job runtime is short, add workers or processes. If runtime is 30 seconds, more workers just move the backlog around and multiply memory use. Horizon's job metrics show both.
Set timeouts and retries deliberately
The worker timeout must be slightly longer than the job timeout, and retry_after must be longer than the longest job or the same job gets processed twice. These three numbers are the source of most duplicate-processing bugs.
Separate queues by importance
Emails and reports should not wait behind an image import. Give latency-sensitive work its own queue and its own worker, and prioritise with --queue=high,default rather than hoping.
Watch memory
A worker that grows over hours will eventually hit the memory limit and restart mid-job. Set maxJobs and maxTime so workers recycle cleanly, and investigate jobs that retain large payloads.
Database connections
Each worker holds a connection. Twenty workers against a database with a 20-connection limit will fail under load. Either pool connections or keep the worker count within what the database can serve.
Frequently asked questions
How many processes per queue?
Start with one or two, measure wait time, and increase until the queue keeps up without saturating CPU or connections. There is no universal number.
Should failed jobs retry forever?
No. Use a bounded retry with backoff, and send persistent failures to a dead-letter queue or an alert so a broken integration does not retry silently for a week.
More on this topic: Performance & SEO articles.
Admin
Related articles
Internal linking: the cheapest SEO work you are not doing
Every important page should be linked from somewhere with descriptive anchor text. It costs nothing and it compounds.
Getting a new site indexed by Google: a checklist that works
Verify Search Console, submit a sitemap, make sure nothing is blocking crawl, and then be patient. Most indexing problems are self...
How to speed up a slow WordPress site without rebuilding it
Hosting, page cache, images and plugins — in that order. Most slow WordPress sites improve dramatically before any theme work.