I'm not sure how Varnish would work with memcached, that I used to have running, but I think those two, along with nginx, is going to be part of my future scaling efforts. I want to work with that MogileFS soon as well - sounds v.cool. If anything uses distributed nodes that can become the master upon another nodes failure, I'm all about it. Along those lines, I was thinking of a way I could use Bittorrent to do permanent file stores this way too, globally distributed, all mirrored automatically, all are trackers, seeders, clients and servers at the same time.

Thanks for the info!

fak3r

I am actually not necessarily going to implement a reverse caching proxy. I just need load balancing, which I am using nginx for as well at the moment. I am also using it to gzip my content, and I hope if I need SSL, it would be a bonus if I can do it on the proxy server.

The things lacking in nginx for load balancing: healthchecks (what if the file store is not mounted? You'll get a 404... which means you have to set proxy_next_upstream http_404 as well; but if the file is truly a 404, then you're wasting a bunch of resources having it cycle through each server. It needs some sort of healthcheck with an expected response. That's outside of the scope though of the core nginx server though, so I'm not really complaining.

I am thinking about looking at HAproxy, or possibly just going back to LVS+ldirectord. It has the healthchecking and notification when a server drops out that I am looking for, but it does not have any proxy_next_upstream behavior internally. That's where I am hoping HAproxy might be the best of both worlds for a load balancer...

I could do my own sanity check, have an include file of servers that nginx reads for upstreams, and if one fails, remove it and then reload nginx. But that seems kinda messy.