Does anybody know of software that could be used to build a mirroring reverse proxy? I am looking for something to cache and mirror files for distributions and other archives of mostly static files.
To clients, it looks like a copy of the archive. If a file does not exist, or is too old, it is fetched from the master servers and saved to disk. The files can be populated from CDs or mirrored through rsync to get complete copies. Ideally, it would load balance from multiple mirror servers.
It is possible to set something up with Apache2, mod_proxy, and mod_disk_cache but that does not allow load balancing between servers and stores files in an opaque cache.
Would Perlbal be usable for this? Could it be extended? I am leaning toward writing a mod_perl module.