Google Fiber Beyond CPE

Ever since I saw the first Google Fiber hut I wondered what equipment was running everything and how it was all setup. I'm going to give you what my best guess is and maybe some day I'll get a chance to figure out if I was right or not. (So far Google has been very secretive about exactly how their network is built.)

Since all the CPE equipment was custom designed my first thought was that maybe all the equipment in the fiber huts was also custom designed. That doesn't make a lot of sense though, the only reason the CPE equipment was custom designed was that nothing existed that met their needs at the right price point. There is equipment that would meet Google's needs for the fiber huts though. At least there's hardware that could meet their needs. That got me thinking, what hardware would Google choose for the job? Who makes the best access router that could handle the amount of bandwidth they plan on having?
That's when I found Juniper's MX2010 and MX2020.



If you've never heard of either of those access routers take a look at the spec sheet for the MX2020.

It's a monster sized router, designed to eventually scale to 80 Terabits per chassis! With room for 960 Ten Gigabit ports and plenty of bandwidth left to send all of that traffic upstream. If I had a way to know how many waves are on each fiber going back to the fiber hut then I would be able to guess the modules in use for customers (remember Google Fiber uses WDM-PON). It's either 10GE split into 8 waves or 40GE split into 32 waves. I'm leaning toward 10GE split into 8 waves, which would give one MX2020 the capacity to serve 7680 Gigabit customers. I'm guessing the links from the huts to the rest of their network are aggregates of 100GE links. Maybe a couple 4x100GE to start with. That would give an over-subscription of less than 10:1 for a fully loaded MX2020.

I also wondered about what the network might look like further upstream from the huts. Through a lot of trace-routes this is what I think the network looks like.
There appears to be five sites in KC (I'm assuming these are a layer up from the fiber huts since there should be a lot more sites if each one was a fiber hut.) MCI101, MCI102, MCI103, MCI104 and MCI105. MCI101 and MCI102 both have a single access router in them. While MCI103 has two distribution routers and one peering router. MCI104 and MCI105 both have a broadband network gateway (these are what actually connect back to the fiber huts). I attempted to map KC and the sites used for most upstream connectivity. (There's a lot more to map but I'll finish creating it later.)