A Rails app I worked on at a day job sent its email in the middle of handling the web request. One day the mail server got slow.
Look at what that costs. The app can handle a fixed number of requests at once, say ten. If the mail server takes five seconds to answer, a request that sends an email sits there for five seconds, doing nothing, occupying one of those ten slots.
Ten slots all parked on a slow mail server means nobody is left to serve anything else. Pages that never sent an email went down too. Visitors got a 503.
The alert said the web tier was unhealthy. The web tier was fine. It was queued behind a mail server.
That’s back-pressure, and this was the accidental kind. Work arrives faster than one slow part can absorb it, and because nobody decided where the overflow should go, it went everywhere. The failure showed up in a part that was working fine. Mine was a mail server. Yours might be Postgres, a cache, a payment API, anything with a limit and a line in front of it.
The trap is the obvious next move. The alert named the web tier, so you add capacity there. Now twenty requests wait on the same mail server instead of ten, the mail server is no faster, the bill is bigger, and the pages are still down. You scaled the innocent part, and the extra capacity made the accident bigger.
Making the slow part faster does work, the way paying for peak capacity always works. You pay for what your worst five minutes need, every minute of the month.
The cheap fix is to stop doing slow work inside the request. Put the email on a job queue, return the page, and let a background worker send it when it can. Sidekiq, a jobs table, whatever you already have. The request never waits on the mail server again, and a slow mail server makes email late instead of taking the site down.
A queue only helps if it has a limit. Let it grow without one and the pile-up moves somewhere you can’t see, and the emails go out later and later instead of the pages going down. Sometimes the right answer is a 503. A queue that’s full should say so, fast, so the caller can back off and try again. That’s the deliberate kind. You decided where the overflow goes and who hears about it. The 503 in my story was the right signal, sent by accident from the wrong layer.
That’s the difference between pushing and pulling. Pushing is how you get the accidental kind. The request shoved email onto the mail server whether it had room or not, and the overflow landed on visitors. Pulling is how you get the deliberate kind. A background worker takes the next job when it’s free, so the mail server only gets what it can handle, and the overflow waits in a queue you can see.
Anything a request waits on that you don’t control is a place back-pressure can show up by accident. A queue with a limit, and a worker to drain it, is usually all it takes to make it deliberate.