Scale workers without overwhelming the database

Start with arrival rate, service time, acceptable queue age or request latency, warm-up time, and the safe capacity of every downstream dependency. The largest queue number is not enough by itself.

For queue workers, raw queue depth is not proportional to capacity: 10,000 messages means something different with 10 versus 100 workers. Backlog per worker and oldest-message age relate demand to capacity and the SLO. For HTTP fleets, request count per target or a correlated utilization metric can drive target tracking. CPU is useful only when it moves with demand.

Select target tracking for a stable desired metric, step scaling for severity bands, scheduled scaling for recurring events, and predictive scaling only for forecastable patterns. Include minimum, maximum, warm-up, scale-in protection, draining, and failure headroom. Elasticity is bounded by quotas, subnet IPs, startup, registration, and downstream connections. A correct policy states what happens while capacity catches up: queueing, backpressure, or controlled rejection.

A durable queue allows producers to accept work while consumers process at another rate. It also makes waiting work measurable through backlog and age signals. Worker ASGs, ECS services, or EKS deployments can scale independently from an API. Lambda scales environments for concurrent invocations, but account and function concurrency plus downstream limits still apply.

Reserved concurrency reserves and caps a function; provisioned concurrency prepares environments to reduce startup latency. Neither makes a database accept more connections. Containers do not dictate one mechanism: ECS and EKS maintain desired tasks or pods, while the capacity layer must have room. Fargate removes host scaling; EC2-backed clusters need instance capacity. Serverless fits invocation or task boundaries, but long-lived services can use autoscaled containers.

Messaging is the demand buffer and signal; delivery guarantees are separate. Scale each tier against its constraint and preserve a hard ceiling before the fragile dependency.

Elastic compute should stop growing before it overloads the finite database, API, or external service underneath it. If workers process 40 jobs per second each while a database safely accepts 400 writes per second, ten workers are the safe ceiling even if compute can launch 100. Target tracking can add workers until ten while queue-age alarms expose an SLO breach.

To process more, improve or partition the database path; raising compute maximum turns backlog into connection storms. Use bounded retries with jitter, batch where safe, and defer noncritical work. Reserve capacity for critical functions sharing an account, but do not confuse reservation with latency preparation. Observe desired, actual, healthy, and effective capacity; queue age; throttles; downstream connections; and completed rate.

Test a burst and AZ failure. Scale to zero only when startup meets the SLO, and keep minimum multi-AZ capacity when immediate availability is required. Admission control belongs before the dependency that can be exhausted. For a synchronous API, API Gateway throttling can shape accepted request rate and return an overload response, but its token-bucket targets are best-effort rather than a substitute for backend protection.

For asynchronous work, SQS accepts a durable backlog while worker maximums enforce the database-safe processing rate. These branches are observable: synchronous excess appears as throttled requests, queued excess appears as rising oldest-message age, and a missing consumer cap appears as database saturation. Decide deliberately which outcome callers can tolerate.

A payment lookup may reject quickly so clients retry with jitter; a render job may wait in a queue; a low-priority enrichment may be dropped. Preserve tenant isolation with per-client authorization and, where appropriate, usage controls so one noisy caller cannot consume every worker.

Compare the cost of retained backlog, idle minimums, and retries against the business cost of rejection, while never buying compute that merely accelerates failure at the database.