Google Fiber - Large increase in IPv4 address space

Google Fiber is gearing up to provide access for many more users.

Before October 6th, 2015 Google Fiber owned eight IPv4 address blocks totaling around ~147 thousand IPv4 addresses. On October 6th, 2015 Google Fiber acquired the /11 IPv4 address block 136.32.0.0/11. Google Fiber first began announcing this new address block to the global BGP table on November 24th, 2015.
This /11 address block adds an additional ~2.1 million IPv4 addresses.

Google Fiber - EdgeRouter Remote VPN Setup

Google Fiber's Network Box currently doesn't have support for remote access VPNs but the Ubiquiti EdgeRouter does. I'll walk through setting up remote access via L2TP over IPsec.

The firewall rule changes can be done through the GUI but everything else must be done through the CLI or the config tree.

Enable hardware offload for IPsec:

set system offload ipsec enable

Reorder a rule so there's space to add two new rules:

edit firewall
edit name WAN_LOCAL
rename rule 3 to rule 6
top

Add firewall rules for L2TP:

set firewall name WAN_LOCAL rule 4 action accept
set firewall name WAN_LOCAL rule 4 description 'Allow L2TP'
set firewall name WAN_LOCAL rule 4 destination port 500,1701,4500
set firewall name WAN_LOCAL rule 4 log disable
set firewall name WAN_LOCAL rule 4 protocol udp
set firewall name WAN_LOCAL rule 5 action accept
set firewall name WAN_LOCAL rule 5 description 'Allow ESP'
set firewall name WAN_LOCAL rule 5 log disable
set firewall name WAN_LOCAL rule 5 protocol esp

Setup IPsec:

set vpn ipsec auto-firewall-nat-exclude disable
set vpn ipsec ipsec-interfaces interface eth1.2
set vpn ipsec nat-networks allowed-network 10.0.0.0/8
set vpn ipsec nat-networks allowed-network 172.16.0.0/12
set vpn ipsec nat-networks allowed-network 192.168.0.0/16
set vpn ipsec nat-traversal enable

Setup L2TP:
#Change the username and password

set vpn l2tp remote-access authentication local-users username testuser password 'testpassword'
set vpn l2tp remote-access authentication mode local
set vpn l2tp remote-access client-ip-pool start 192.168.100.101
set vpn l2tp remote-access client-ip-pool stop 192.168.100.110
set vpn l2tp remote-access dhcp-interface eth1.2
set vpn l2tp remote-access dns-servers server-1 8.8.8.8
set vpn l2tp remote-access dns-servers server-2 8.8.4.4
set vpn l2tp remote-access ipsec-settings authentication mode pre-shared-secret

#Change the L2TP Shared Secret

set vpn l2tp remote-access ipsec-settings authentication pre-shared-secret testsecret
set vpn l2tp remote-access ipsec-settings ike-lifetime 3600
set vpn l2tp remote-access mtu 1492

Commit and Save:

commit
save

Additional Resources:
EdgeMax - L2TP Server Setup
Android Client Setup

Google Fiber - Dynamic DNS Setup on an EdgeRouter

Google Fiber's Network Box supports several dynamic DNS providers but I wanted to use one they don't currently support called afraid.org. Now that I have a router that can supports almost any dynamic DNS provider I can configure it!

Here's how to set it up on a Ubiquiti EdgeRouter.

From WebUI:

Go to the Service Tab
Then go to the DNS Tab
Press the "Add DDNS Interface" button
Select interface eth1.2 (assuming eth1.2 is your WAN interface)
Select afraid for the service
Set your hostname, login and password
Set the server to freedns.afraid.org
Press the Apply button



From CLI:

configure
set service dns dynamic interface eth1.2 service afraid host-name mydomain.mooo.com
set service dns dynamic interface eth1.2 service afraid login dyndns
set service dns dynamic interface eth1.2 service afraid password 123456
set service dns dynamic interface eth1.2 service afraid server freedns.afraid.org
commit
save

Testing the new configuration

From CLI:

Run this command to force an update:

update dns dynamic interface eth1.2

Run this command to see the status information for dynamic DNS.

show dns dynamic status

interface    : eth1.2
ip address   : 1.2.3.4
host-name    : mydomain.mooo.com
last update  : Sat Oct 17 10:53:36 2015
update-status: good

Notes:
If you have any uppercase letters in your afriad.org username, convert them to lowercase when setting your username on the EdgeRouter. (The afraid.org site is converting your username to lowercase when you login and the API calls the dynamic DNS client tries to make will fail if your username doesn't match exactly.)

Additional Resources:
Guide from Ubiquiti's Help Center on Setting up Dynamic DNS

Google Fiber - EdgeRouter IGMP Proxy for TV Service

In my first blog post about switching to the Uqituiti EdgeRouter I noted that users with TV service would have issues running Atlantisman's TV script as is after running the modified script I posted.

I've modified the TV script he created so it will work with the IPv6 script I posted before.
Disclaimer: I don't have TV service, so I have no way to verify that this script actually works.

Modified TV Script

If you have the first gen Google Fiber equipment, attach your Storage Box (GFMS100) to your LAN and your TV boxes should work again.

If you have the second gen Google Fiber equipment things are more complicated because you have a combined Network and Storage Box in the Network+ box (GFRG200, GFRG210).

There are two options to get TV working with the second gen Network+ boxes:

First Option - 

• Plug a switch into the Fiber Jack and connect both the EdgeRouter and Network+ box to the switch. 

There are a couple problems with this option though:
First problem, by connecting two routers to the Fiber Jack you'll be assigned two public IPv4 addresses. While Google Fiber currently allows this, in the future it will probably be restricted to only one IPv4 address and this option will no longer work.

Second problem, Google Fiber already restricts IPv6 address assignment and prefix delegation to one MAC address or DUID (DHCP Unique Identifier) per Fiber Jack. What this means is the Network+ Box will almost always win the race to get an IPv6 address and prefix before the EdgeRouter and the EdgeRouter will be stuck with only IPv4 connectivity and the CPU at 100% as the DHCPv6-PD process tries in vain to get an IPv6 address and prefix. By powering on the EdgeRouter first waiting a few minutes then powering on the Network+ box you can get around this issue.

Second Option - 

• Login to fiber.google.com and in the Advanced drop down for your network box change the "Router LAN IPv4 address", "DHCP start address", and "DHCP end address" to be on a different network than the LAN side of your EdgeRouter. (e.g. Change 192.168.1.1 to 192.168.3.1, etc)
• Plug the Network+ box into the LAN side of the EdgeRouter

Advantages of this option:

With this option you won't have to rely on a second IPv4 address that may go away or need another switch or vlan connected to the Fiber Jack. 

You can continue to use the wireless from the Network+ until your real access point arrives. :)

Current issues with this option:

Devices plugged into the Network+ won't receive an IPv6 address.

Devices plugged into the Network+ box will be going through a double IPv4 NAT.

Other Important Notes:

If you have a switch connected to the LAN side of the EdgeRouter and plan to connect your own wireless access point later, make sure your switch supports IGMP snooping and that you have it enabled/configured. The TV channels are multicast to the Network+ box/Storage Box and a switch without IGMP snooping will flood the traffic out all ports on the same vlan. (All ports if it's an unmanaged switch.) You can also put your WAP on the third EdgeRouter port and leave IGMP proxy disabled on it to get around this issue.

Google Fiber - Hardware Change for Basic Tier

Google Fiber's Basic Tier is their 5 Mb/s down, 1 Mb/s up tier that's free for seven years after installation.

Currently customers signing up for the Basic Tier have been getting the first generation Network Boxes (GFRG100 or GFRG110) from the stockpile that was originally manufactured. Sometime in the near future Basic Tier customers will be getting a different Network Box, the Mini Network Box (GFMN100).

Unofficial Specs:
One WAN port (With PoE to power a newer Fiber Jack GFLT110)
One LAN port
2.4 Ghz 2x2 WiFi (802.11n)

Google Fiber - Next Steps with the EdgeRouter

Ubiquiti's EdgeRouter is amazingly versatile for the price point. There's also a lot of documentation on their support and help center that covers almost anything you would want to configure.

Here's the link to the main documentation page:
EdgeMAX - Ubiquiti Networks Support and Help Center

I've put together a list of stuff I want to configure and I'll be creating a blog post for each one as I configure them.

IGMP Proxy (For TV service)

Port Forwarding

Dynamic DNS

Remote Access VPN

If you want to play with the latest and greatest features on your EdgeRouter, you can also sign up to join the version 1.8 alpha/beta testing group.

Google Fiber with a 3rd Party Router/Firewall

So you've had Google Fiber for a while and played with the advanced settings but you still can't do what you want?

Maybe you want to port forward to multiple devices that all listen on the same port, monitor bandwidth usage with SNMP, allow traffic to an IPv6 address in your home network, change DHCP server settings, use static routes, or even something far more advanced.

What options are available you might ask.

For non-power users, the easiest way to get the feature you want added is to create an idea or vote on an existing idea here on the Google Fiber Customer Ideas page. New features can take some time to get added to the product though and some features may never be added.

What if you are a power user and don't want to wait for the feature(s) you want?
In general there are two options that are popular so far. Replacing the Network Box with a computer running pfSense or replacing the Network box with a Ubiquiti EdgeRouter Lite.

pfSense has more features but will require more expensive hardware for gigabit throughput than the EdgeRouter Lite. I decided on the EdgeRouter Lite since I've never used Ubiquiti products before and wanted try one out.

If you decide to get an EdgeRouter, here's how to set it up.

After receiving your EdgeRouter you'll want to follow Ubiquiti's Guide to upgrading the firmware. The download link for EdgeRouter firmware is here.

A user named Atlantisman already has a great guide on setting up an EdgeRouter here.
Since he created that guide though, Ubiquiti has released EdgeOS 1.7, which offers many new features including full support for DHCPv6-PD (Prefix-Delegation, used by Google Fiber to assign IPv6 addresses). I've created an updated version of the script based off of the IPv6 config from a user named TK. I recommend following Atlantisman's guide but substituting the script linked below for his if you want IPv6.

GF-ERL-Commands - Update version of Edge-Setup-Interfacesv2
Note: I created this script off of the completed config, so it may need a little tweaking.

You'll probably want to change at least line 132 for the hostname and line 141 for the time zone.
Don't forget to change the password for the ubnt user (either through cli or the gui).

GF-ERL - Full Config - Easier to read if you only want to see how things are setup

Important Note:
When switching from the Network Box to the EdgeRouter you must wait for the DHCPv6 prefix the Network Box had to expire or the CPU on the EdgeRouter will be at 100% until it does. This is because of the way the EdgeRouter handles what are called unsolicited DHCPv6-PD advertisements. The EdgeRouter spawns a new DHCPv6-PD responder for every unsolicited advertisement but is never able to receive a prefix or address, this causes high CPU on the EdgeRouter which will cause slow throughput and no IPv6 connectivity.
What I did was unplug the Network Box before I went to bed and plugged in the EdgeRouter went I woke up. The lease time is one hour, so you don't really need to wait as long as I did.

Important Note about TV Service:
Since I don't have TV service I didn't create an updated version of the TV script yet. Do NOT run the TV script after running the updated one I linked, it won't work without changing a few things in the TV script. If you have gen 2 gear (e.g. GFRG2XX written on the bottom of your network box) then you have a combined Network and Storage box that can't just be removed or your TV service will stop working.

Powering the Fiber Jack with PoE (Power over Ethernet):
If you have a newer fiber jack (model GFLT110) you can power it with any 802.3af compliant power injector. I'm running mine with the TP-LINK TL-PoE150S.

Screenshots:
This is from a program named Cacti. It's monitoring the EdgeRouter via SNMP. Cacti is easy to setup on a Linux distribution like Ubuntu.

Monitoring CPU, interface discards/errors, and interface bandwidth. (There's an extra interface that's not in the script/config for guest WiFi - eth2.)







EdgeRouter WebUI:

Dashboard

The Config Tree view that can be used to configure anything that the CLI can.

Ubiquiti is still working on adding all the features in the main web UI but most of the basic stuff is there.






Update 10-13-2015: Added DHCPv6 lease time comment.

Phase III Pre-Registration Continues

West Johnson County, KS signups will open on October 1st, 2015 at 9am and close on November 19th.

Stop by an event to learn more:

Shawnee Great Grillers
September 25th
5 p.m. - 10 p.m. CST, Shawnee Town Hall
and
September 26th
10 a.m. - 2 p.m. CST, Shawnee Town Hall

Shawnee Oktoberfest
October 2nd
5:30 p.m. - 10 p.m. CST, Shawnee Civic Centre

Lenexa Enchanted Forest
October 22nd
6:30 p.m. - 8:30 p.m. CST, Sar-Ko-Par Trails Park

Phase III Pre-Registration Continues

Olathe East sign-ups opened today August 11th and will close on September 24th.

All fiberhoods in Olathe East have met their goals!

Google Fiber Expansion

On August 5th Google Fiber announced one more metro area: San Antonio.

Here's the link to follow progress with each city:
https://fiber.google.com/newcities/

Google Fiber's blog post about Salt Lake City:
http://googlefiberblog.blogspot.com/2015/08/san-antonio-fiber.html

Google Fiber Routing and Latency

When I first got Google Fiber I noticed the latency to some locations was higher than expected. After poking around with traceroute it was easy to see why. The path traffic was taking out of Google Fiber's network was much longer than it needed to be. Going to a location in Kansas City from my KC Google Fiber, it would route through San Jose. Other cities around the US had similar odd routing, except for locations near San Jose.

Early in the morning on March 19th, 2015 Google Fiber made routing changes that have now significantly improved latency for all the test locations I'd been monitoring that were higher than they should've been.

Note: All of these monitors are to Hurricane Electric core routers, so the loss statistics aren't necessarily accurate. (ICMP is low priority to respond to.)

The jump in latency in KC is the most staggering. From 33.3 ms average to 8.6ms.

To the person at Google Fiber made this change: I want to buy you a beer for making my latency in games that much better!

Google Fiber Expansion

On March 24th Google Fiber announced one more metro area: Salt Lake City.

Here's the link to follow progress with each city:
https://fiber.google.com/newcities/

Google Fiber's blog post about Salt Lake City:
http://googlefiberblog.blogspot.com/2015/03/google-fiber-is-coming-to-salt-lake-city.html

Areas Opened for Sign-ups Indefinitely in Kansas City

As of today Kansas City, KS, KC Central, MO, and the South Kansas City, MO areas have been opened for sign-ups indefinitely. Only fiberhoods that met their goals during one of the previous rallies in those areas are eligible to signup though.

This change may have been because of this letter that Rep. Emanuel Cleaver (D-Mo.) sent to Larry Page (CEO/cofounder of Google) on February 5th, 2015.

http://cleaver.house.gov/sites/cleaver.house.gov/files/Letter%20-%20Close%20Digital%20Divide.pdf

This news site made the link that these two events might be related:
http://thehill.com/policy/technology/237472-after-dem-reps-request-google-reopens-local-fiber-signup

Phase III Pre-Registration Continues

Lee's Summit sign-ups opened yesterday March 10th and will close on April 30th.

If you have a contract with your current provider, think about signing up for the free plan until your current contract is over. If your fiberhood doesn't meet it's signup goal then no one will be getting Google Fiber in your neighborhood until signups reopen again.

Updated 5-5-2015:

All but four fiberhoods met their signup goals.

Google Fiber Expansion

On January 27th Google Fiber announced four more metro areas: Atlanta, Charlotte, Nashville, and Raleigh-Durham.

Still waiting on word for the other five metro areas that were interested in.

Here's the link to follow progress with each city:
https://fiber.google.com/newcities/

Google Fiber's blog post about the announcement:
http://googlefiberblog.blogspot.com/2015/01/google-fiber-is-coming-to-atlanta.html