Mirroring 2.0

Welcome to the rubygems-mirror wiki!

Discussing ideas, implementations, and more for fixing the mirroring pain we have in the rubygems ecosystem here.

Motivation

1. The current "gem mirror" system sucks.
2. Failover is a concern if the main gem repo dies/disappears
3. There are countless mirror organizations that will help us with hosting

Problems

1. There's a lot of little files (over 200,000 gems and gemspecs)
2. Gemcutter has spoiled us, we want our gems to be quickly pushed and installed. Waiting hours/days for gems to sync to mirrors is not acceptable.
3. Pull/round robin mirroring has failed us before (RubyForge used to work this way)
4. "Picking" a mirror sucks and this should be done for you. (Use a sane default, maybe opt for geoip)

Ways to fix it

• Simply pointing all downloads at a CloudFront distribution, like cloudfront.rubygems.org, which has the original rubygems.org server set as its custom origin. This way, CloudFront essentially acts as a super-fast cache, and it obviates the need for mirror-picking. The only downside is cost (~$.12/GB). How much traffic is RubyGems serving each month at the moment?

Other things that have been tried

@qrush tried to set up MirrorBrain and it was really, really painful. Didn't get far.