Throttling, quotas, and cost-safe demand shaping

Throttling design starts at the narrowest safe resource, not at the largest published service quota. Record incoming steady rate, burst size and duration, accepted-request latency, tenant fairness unit, API integration capacity, Lambda concurrency, queue age objective, database connection and transaction limits, and budget. A quota is an AWS account or service ceiling; a throttle is an admission policy; downstream safe capacity is the rate the complete path can sustain. These values can differ. If the database safely commits fewer transactions than the API can admit, raising the API ceiling increases failure and cost rather than throughput. Decide which work is synchronous and must receive an immediate result, which can be accepted asynchronously, and which should be rejected. Authenticate before enforcing tenant fairness so callers cannot evade per-tenant limits by changing an untrusted identifier. Reserve capacity for critical operations where the business contract requires it, and make rejection behavior explicit with stable errors and retry guidance.

API Gateway uses token-bucket throttling concepts for rate and burst control under documented precedence and best-effort behavior. Apply an account, stage, method, usage-plan, or client boundary only after mapping it to fairness and downstream protection; current details belong to the direct throttling source. A queue smooths asynchronous bursts by converting request spikes into backlog, but it is valid only if retention and oldest-message age remain inside the SLO. Lambda reserved concurrency can cap a consumer and protect other functions or a database, while autoscaling adds workers after delay. Neither should be allowed to exceed downstream-safe concurrency. Caching removes repeated safe reads when keys, TTL, and authorization permit; it does not solve unique writes. Client retries must be bounded, use exponential backoff with jitter, honor retry guidance where available, and preserve idempotency. Immediate synchronized retries multiply offered load exactly when the system is least able to absorb it. Protective throttling should shed excess work before an expensive dependency, not after that dependency has already performed most of the work.

Use the `saa-viz-network-path-lab` throttling preset to predict the first failed gate before reveal. Trace client, DNS, API entry, configured rate and burst admission, application, and downstream capacity in order. In the preset, routing and authorization can pass while the application-response-after-routing gate fails because offered rate exceeds the safe admitted rate; later database overload is a secondary consequence, not the first explanation. Repair the earliest gate by lowering or partitioning admission, queueing eligible writes, capping consumer concurrency, or increasing the actual bottleneck if budget and architecture allow. Then replay with bounded jittered retries and verify that retry traffic does not amplify the burst. Monitor admitted and throttled counts, 429 responses, integration latency, Lambda concurrency and throttles, queue depth and oldest age, database connections and TPS, retry attempts, and cost per successful request. A sustained backlog above its age target indicates permanent undercapacity or excessive admission, not a transient burst. Throttling loses when it rejects required throughput despite available safe capacity; quota increase becomes correct only when the AWS ceiling, rather than downstream or budget, is the measured bottleneck.