Dev Skills
Go·Concurrency

Worker pools in Go: bounded concurrency, built step by step

A fixed number of goroutines pulling jobs from one channel, so 10,000 tasks never turn into 10,000 open connections.

Topic 44 of 101 · 20 min to read · 20 min to build

By devskills editorial team · Last updated

A worker pool is N goroutines that all read jobs from one channel. The producer sends jobs, the workers process them, and at most N jobs are in progress at once. That limit is the whole point.

Goroutines are cheap, so "why not one goroutine per task?" is a fair question. Often that is fine. The goroutine is not the problem; what it holds is. Ten thousand goroutines that each open a database connection, call a rate-limited API, or hold a 5 MB image in memory will exhaust the connection pool, get you throttled, or run the process out of memory. A pool caps that at a number you chose.

This page builds the pattern in depth. For the overview of all the concurrency patterns (sync.Once, errgroup, leaks), see Concurrency patterns.

ApproachHow it bounds workBest forWatch out for
Goroutine per taskIt does notSmall, known number of cheap tasksUnbounded memory and connections under load
Worker poolN long-lived goroutines read one jobs channelLong streams of similar jobs; reusing a per-worker resourceWho closes which channel; draining results
Semaphore (buffered channel)Acquire a slot before starting each goroutineA loop over a slice where each item is independentAcquiring inside the goroutine gives no backpressure
errgroup with SetLimitg.Go blocks until a slot is freeA batch where the first error should cancel the restIt is in golang.org/x/sync, not the standard library

A minimal worker pool, with results in order

Three channels of responsibility. The producer sends every job and then closes jobs. Each worker ranges over jobs, so its loop ends when jobs is closed and empty. A separate goroutine waits for all workers and then closes results, so the reader's range ends too.

Results arrive in whatever order workers finish. To get them back in input order, send the index with each job and write each result into its own slot of a slice. Each slot has exactly one writer, so there is no race.

pool.go
package main
import (
	"fmt"
	"strings"
	"sync"
)
type job struct {
	index int
	word  string
}
type result struct {
	index int
	out   string
}
func main() {
	words := []string{"alpha", "bravo", "charlie", "delta", "echo", "foxtrot"}
	jobs := make(chan job)
	results := make(chan result)
	const workers = 3
	var wg sync.WaitGroup
	for w := 1; w <= workers; w++ {
		wg.Add(1) // Add before go, never inside the goroutine
		go func() {
			defer wg.Done()
			for j := range jobs { // ends when jobs is closed and drained
				results <- result{j.index, strings.ToUpper(j.word)}
			}
		}()
	}
	go func() { // producer: send everything, then close
		for i, w := range words {
			jobs <- job{i, w}
		}
		close(jobs)
	}()
	go func() { // closer: results is closed only after every worker returned
		wg.Wait()
		close(results)
	}()
	out := make([]string, len(words))
	for r := range results {
		out[r.index] = r.out // one writer per slot: no race
	}
	fmt.Println(out)
}
Always prints the words in input order, whichever worker finished first.

Who closes which channel?

Almost every worker pool bug is a channel closed by the wrong goroutine, or not closed at all. The rule is that only the side that knows no more values are coming may close a channel.

  • `jobs` is closed by the producer, once, after the last send. A worker must never close it: the producer would panic on its next send.
  • `results` is closed by a goroutine that waits on the WaitGroup. Not by a worker (the others are still sending), and not by main before it reads (that is a deadlock: workers block on the send, wg.Wait never returns).
  • The reader must drain `results`. If it stops reading early, every worker blocks on its next send forever. If you need to stop early, cancel a context (next section) instead of walking away.
  • `wg.Add(1)` goes before `go`. Inside the goroutine, Wait can run before Add and return too early.

Stop on the first error, or on a timeout, with a context

Real jobs fail. When one fails you usually want the rest to stop, and when the caller gives up (an HTTP request is cancelled, a deadline passes) the whole pool should stop. A context.Context carries that signal to every goroutine.

Three changes make the pool cancellable. Workers select on ctx.Done() so they stop picking up jobs. The producer also selects on ctx.Done(), otherwise it blocks forever on a send nobody will receive, which is a leak. And the first error cancels the context with a cause, using context.WithCancelCause (Go 1.20), so everyone can see why they were stopped.

cancel.go
package main
import (
	"context"
	"errors"
	"fmt"
	"sync"
	"time"
)
// process pretends to do I/O. Job 4 fails.
func process(ctx context.Context, n int) error {
	select {
	case <-time.After(20 * time.Millisecond):
	case <-ctx.Done():
		return ctx.Err()
	}
	if n == 4 {
		return fmt.Errorf("job %d: %w", n, errors.New("bad input"))
	}
	return nil
}
func run(parent context.Context, jobsToDo []int, workers int) error {
	ctx, cancel := context.WithCancelCause(parent)
	defer cancel(nil)
	jobs := make(chan int)
	var wg sync.WaitGroup
	for range workers {
		wg.Add(1)
		go func() {
			defer wg.Done()
			for {
				select {
				case <-ctx.Done():
					return
				case n, ok := <-jobs:
					if !ok {
						return
					}
					if err := process(ctx, n); err != nil {
						cancel(err) // the first cause wins; later calls do nothing
						return
					}
				}
			}
		}()
	}
	// the producer gives up too, or it would block on a send forever
	send:
	for _, n := range jobsToDo {
		select {
		case jobs <- n:
		case <-ctx.Done():
			break send
		}
	}
	close(jobs)
	wg.Wait() // every worker has returned before run does
	if err := context.Cause(ctx); err != nil && !errors.Is(err, context.Canceled) {
		return err
	}
	return nil
}
func main() {
	jobs := []int{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
	fmt.Println("error:", run(context.Background(), jobs, 3))
	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Millisecond)
	defer cancel()
	fmt.Println("timeout:", run(ctx, []int{1, 2, 3, 5, 6, 7, 8, 9}, 2))
}
The first run stops at job 4 and returns its wrapped error. The second hits the 30 ms deadline and returns `context.DeadlineExceeded`. In both, every goroutine has returned before `run` does.

The same pool with errgroup.SetLimit

When the jobs are a slice you already have, golang.org/x/sync/errgroup does all of the above in fewer lines. SetLimit(n) makes g.Go block until fewer than n functions are running, WithContext cancels the shared context on the first error, and Wait returns that first error. It is maintained by the Go team but lives outside the standard library, so the Playground cannot run it.

Reach for the hand-written pool instead when jobs arrive as a stream with no known end, or when each worker owns something long-lived, like its own connection or a reusable buffer.

errgroup.go — needs golang.org/x/sync
func resizeAll(ctx context.Context, paths []string) error {
	g, ctx := errgroup.WithContext(ctx)
	g.SetLimit(8) // at most 8 resizes at once
	for _, p := range paths {
		g.Go(func() error { // blocks here while 8 are running
			return resize(ctx, p) // must return early when ctx is done
		})
	}
	return g.Wait() // the first error, after every goroutine returned
}
Because `g.Go` blocks, the loop itself is throttled: there are never more than 8 goroutines, not 10,000 parked ones.

A semaphore instead of a pool

A buffered channel of empty structs is a counting semaphore. Sending takes a slot, receiving gives it back, and a send blocks while all slots are taken. It gives the same bound with a goroutine per task, which reads more naturally when you are looping over a slice.

Acquire the slot before the go statement. Then the loop waits for a free slot, which is backpressure. If you acquire inside the goroutine, all 10,000 goroutines start at once and park on the channel: concurrency is bounded but memory is not. For slots of different sizes (one job needs 3 units of a budget, another 1) use golang.org/x/sync/semaphore, whose Weighted.Acquire takes a context and a weight.

semaphore.go
package main
import (
	"fmt"
	"sync"
	"sync/atomic"
	"time"
)
func main() {
	sem := make(chan struct{}, 3) // 3 slots
	var wg sync.WaitGroup
	var running, peak atomic.Int32
	for range 12 {
		sem <- struct{}{} // take a slot BEFORE starting: the loop waits here
		wg.Add(1)
		go func() {
			defer wg.Done()
			defer func() { <-sem }() // give the slot back
			n := running.Add(1)
			for {
				p := peak.Load()
				if n <= p || peak.CompareAndSwap(p, n) {
					break
				}
			}
			time.Sleep(10 * time.Millisecond) // the work
			running.Add(-1)
		}()
	}
	wg.Wait()
	fmt.Println("tasks: 12, most running at once:", peak.Load())
}
Prints 3. Move `sem <- struct{}{}` inside the goroutine and it still prints 3, but all 12 goroutines exist from the start.

How many workers should a Go worker pool have?

It depends on what the job waits for. CPU-bound work (hashing, image decoding, compression) cannot go faster than the cores you have, so start at runtime.GOMAXPROCS(0) workers. More only adds scheduling overhead.

I/O-bound work (HTTP calls, database queries) spends most of its time waiting, so you need more workers than cores. Little's law gives a starting number: workers ≈ target throughput × time per job. To make 200 API calls a second that each take 250 ms, you need about 200 × 0.25 = 50 in flight. Then cap it by the limit downstream: if the database pool allows 25 connections, 50 workers means 25 of them are always queued inside database/sql, adding latency for nothing.

The program below runs 20 jobs that each wait 50 ms with 1, 4 and 20 workers. For waiting work, time drops in proportion to workers until every job has its own.

how-many.go
package main
import (
	"fmt"
	"sync"
	"time"
)
func runPool(workers, jobs int, work time.Duration) time.Duration {
	start := time.Now()
	ch := make(chan int)
	var wg sync.WaitGroup
	for range workers {
		wg.Add(1)
		go func() {
			defer wg.Done()
			for range ch {
				time.Sleep(work) // stands in for a network call
			}
		}()
	}
	for i := range jobs {
		ch <- i
	}
	close(ch)
	wg.Wait()
	return time.Since(start).Round(10 * time.Millisecond)
}
func main() {
	for _, w := range []int{1, 4, 20} {
		fmt.Printf("%2d workers: %v\n", w, runPool(w, 20, 50*time.Millisecond))
	}
}
About 1s, 250ms and 50ms. With CPU-bound jobs the curve flattens at the number of cores instead, which is why the two cases get different worker counts.

Backpressure: what happens when jobs arrive faster than workers finish?

With an unbuffered jobs channel the producer blocks until a worker is free. That is backpressure, and it is usually what you want: a slow pool slows the producer down instead of piling work up in memory.

A buffered channel (make(chan job, 100)) adds slack for short bursts. It does not add capacity: if jobs arrive faster than workers finish for long enough, the buffer fills and the producer blocks anyway. When blocking is not acceptable, for example in an HTTP handler, reject instead and let the client retry.

submit.go — reject instead of waiting
// TrySubmit hands a job to the pool, or reports that the pool is full.
func (p *Pool) TrySubmit(j Job) bool {
	select {
	case p.jobs <- j:
		return true
	default: // every worker busy and the buffer full
		return false // the handler returns 503 with Retry-After
	}
}
This is load shedding. The same idea at system scale is on the [message queues](/concepts/message-queues) page.

Common worker pool bugs

  • "all goroutines are asleep - deadlock!" Usually main ranges over results but nobody closes it, or main calls wg.Wait() before reading results, so workers block on the send and the wait never ends.
  • "send on closed channel". A worker, or a second producer, closed jobs or results. With several producers, give them their own WaitGroup and close jobs after it.
  • A leaked producer. Workers stop on an error, but the producer is still blocked on jobs <- n. Give the producer a select with ctx.Done().
  • One panic kills the program. A panic in any goroutine crashes the whole process, not just that worker. If a job can panic on bad input, recover inside the per-job function and turn it into an error.
  • Results lost to a shared slice with `append`. Several workers calling append on one slice is a data race. Write to out[index], send on a channel, or lock.
  • Run your tests with `go test -race`. The race detector catches the last bug and most of its relatives.

Worker pool interview questions

  • Why use a worker pool if goroutines are cheap? To bound what the goroutines hold or call: connections, API quota, memory. The pool is a concurrency limit, not a goroutine-saving device.
  • How do you return results in the input order? Send the index with the job and write the result into that index of a preallocated slice.
  • How do you stop all workers when one fails? Cancel a shared context on the first error (errgroup.WithContext does this), have workers and the producer select on ctx.Done(), and still wait for all of them before returning.
  • Worker pool or semaphore? Equivalent bounds. A pool keeps N goroutines alive and suits a stream of jobs or a per-worker resource. A semaphore starts a goroutine per task and suits looping over a slice.
  • How many workers? Cores for CPU-bound work. For I/O, throughput times latency, capped by the smallest downstream limit.
Check you can now
  • Write a worker pool that returns results in input order and never leaks a goroutine.
  • Stop every worker on the first error or a timeout, using a context.
  • Choose between a worker pool, a semaphore and errgroup.SetLimit, and pick a worker count with a number behind it.
The short version
  • A worker pool bounds what runs at once. Goroutines are cheap; the connections and memory they hold are not.
  • The producer closes jobs; a goroutine that waits on the WaitGroup closes results; the reader drains results.
  • Send an index with each job to get results back in order.
  • Cancel a context on the first error or timeout, and make the producer select on it too.
  • errgroup.SetLimit or a buffered-channel semaphore gives the same bound with less code for a known list of jobs.
  • Workers: cores for CPU work; throughput × latency for I/O, capped by the downstream limit.
Remember: Bound the work, not the goroutines: N workers, one jobs channel, and every channel closed by the side that knows it is finished.
Then try these in your own editor
  1. 1In pool.go, move close(results) to straight after the producer goroutine is started, in main. What happens, and why?
  2. 2Change cancel.go so that a failing job does not stop the others, and run returns all the errors together.
  3. 3Your pool calls an API that allows 100 requests a second, and each call takes about 400 ms. How many workers, and what else do you need?
Official documentation
Check yourselfTake the Concurrency test