Whew this is quite a fun article: How Dropbox Saved Millions Of Dollars By Writing Their Own Load Balancer
-
Whew this is quite a fun article: How Dropbox Saved Millions Of Dollars By Writing Their Own Load Balancer
How Dropbox Saved Millions of Dollars by Building a Load Balancer
Dropbox saved resources by creating a superior version of a tool everyone uses
(newsletter.betterstack.com)
-
@thisismissem I found the use of a pid controller interesting! At a past job I advocated for their use (to another team, or I would have done it myself) for exactly this purpose, but they chose to continue using a combination of average and I think p95.
Also I wonder if they integrate other metrics beyond cpu into the calculation at this point. -
-
@thisismissem ah no, I'm pretty sure they were referring to this technique: en.m.wikipedia.org/wiki/Proportional%E2%80%93integral%E2%80%93derivative_controller - it's a way of smoothly ramping up or down a process with various tuning parameters.
-
T [email protected] shared this topic
-
-
@thisismissem @amy pro tip: you can do away with what the article calls "the proxy" in a design like this by realizing that the server can include information, like the current load*, in the response headers. The LBS can use that in the routing decisions, and strip those headers from the response that goes to the client.
* Doesn't have to mean the load average. Could be e.g. weighted summed costs of recent queries.
-
@thisismissem @amy that does require that you control or can modify most of the tech stack to do that, which might not be feasible for some deployments.
And there are some wrinkles around correctly accounting for error responses and aging out the data to ensure decisions aren't being made with stale info.
-
R [email protected] shared this topic
-
@nikclayton @thisismissem @amy
I guess in direct-server-return you can't use response headers but that's kinda cool
