Scale the work without losing the state

Imagine one storefront server disappears during checkout. What must still exist when its replacement starts? The customer’s session, uploaded files, cart, and completed payment cannot depend only on that server’s memory or local disk. Those items are state: information that must survive beyond one running process.

Compute is stateless when another healthy unit can handle the next request without needing hidden local history. Put durable state in a shared store that every replacement can reach. A load balancer can then send requests only to healthy targets, and an Auto Scaling group can replace or add targets.

Load-balancer stickiness keeps one customer returning to one target while it is healthy. It does not copy the session elsewhere. If that target fails, its local session still disappears. External state—not stickiness—is what makes the compute safely replaceable.

A tier is a part of the request path with its own job. A simple storefront might have a load balancer, an application tier, a worker tier, and a database. Separating tiers lets the application add capacity without pretending the database also became larger.

Some work must finish before the caller receives an answer. Checkout authorization is usually synchronous: the customer needs a result now. Other work can finish later. Image resizing can be accepted first and placed in Amazon SQS, where workers process retained messages at their own pace.

Use one SQS queue when one competing worker pool should process each job. Use Amazon SNS with a separate queue per subscriber when billing, analytics, and resizing each need their own retained copy. Use Amazon EventBridge when matching event content to several targets is the deciding requirement.

Microservices are not required for this shape. A well-separated modular application can remain one deployment and still use shared state, multiple targets, and a queue. Split services only when independent ownership, release, scaling, or failure isolation pays for the extra operational work.

Vertical scaling makes one unit larger. It is the smallest change for software that cannot distribute work, but the unit still has a maximum size and may need a restart. Horizontal scaling adds interchangeable units. It works only when requests can be distributed and state is not trapped inside one unit.

Choose a signal that changes with usable capacity. Requests per target can guide an HTTP tier. Backlog per worker or oldest-message age can guide workers. Scheduled scaling can prepare for a known promotion, while target tracking reacts to observed demand. Neither removes startup time, so leave room for warming capacity.

The whole request path is limited by its narrowest tier. Ten application targets are not safe if they can send 1,800 requests per second to a database rated for 1,000. Limit concurrency at the dependency, queue work where delay is acceptable, and recalculate after losing one Availability Zone.