Point-to-Point Layer 3 GRE tunnel interfaces (gre) example

Host X ---- Host A ------------ tunnel ------------ Cisco D ---- Host E
               \                                      /
                \                                    /
                 +------ Host B ------ Host C ------+

On Host A (OpenBSD):

# route add default B
# ifconfig greN create
# ifconfig greN A D netmask 0xffffffff up
# ifconfig greN tunnel A D
# route add E D

On Host D (Cisco):

Interface TunnelX
 ip unnumbered D   ! e.g. address from Ethernet interface
 tunnel source D   ! e.g. address from Ethernet interface
 tunnel destination A
ip route C <some interface and mask>
ip route A mask C
ip route X mask tunnelX

OR

On Host D (OpenBSD):

# route add default C
# ifconfig greN create
# ifconfig greN D A
# ifconfig greN tunnel D A

To reach Host A over the tunnel (from Host D), there has to be an alias on Host A for the Ethernet interface:

and on the Cisco:

gre keepalive packets may be enabled with ifconfig(8) like this:

# ifconfig greN keepalive period count

This will send a keepalive packet every period seconds. If no response is received in count * period seconds, the link is considered down. To return keepalives, the remote host must be configured to forward packets:

# sysctl net.inet.ip.forwarding=1

If pf(4) is enabled then it is necessary to add a pass rule specific for the keepalive packets. The rule must use no state because the keepalive packet is entering the network stack multiple times. In most cases the following should work:

pass quick on gre proto gre no state

Point-to-Point Ethernet over GRE tunnel interfaces (egre) example

For example, a pair of routers separated by the internet could bridge several Ethernet networks using egre and bridge(4).

In this example the first router has a public IP of 192.0.2.1, and the second router has 203.0.113.2. They are connecting the Ethernet networks on two vlan(4) interfaces over the internet. A separate egre tunnel is created for each VLAN and given different virtual network identifiers so the routers can tell which network the encapsulated traffic is for. The egre interfaces are explicitley configured to provide the same MTU as the vlan(4) interfaces (1500 bytes) with fragmentation enabled so they can be carried over the internet, which has the same or lower MTU.

At the first site:

# ifconfig vlan0 vnetid 100
# ifconfig egre0 create
# ifconfig egre0 tunnel 192.0.2.1 203.0.113.2
# ifconfig egre0 vnetid 100
# ifconfig egre0 mtu 1500 -tunneldf
# ifconfig egre0 up
# ifconfig bridge0 add vlan0 add egre0 up
# ifconfig vlan1 vnetid 200
# ifconfig egre1 create
# ifconfig egre1 tunnel 192.0.2.1 203.0.113.2
# ifconfig egre1 vnetid 200
# ifconfig egre1 mtu 1500 -tunneldf
# ifconfig egre1 up
# ifconfig bridge1 add vlan1 add egre1 up

At the second site:

# ifconfig vlan0 vnetid 100
# ifconfig egre0 create
# ifconfig egre0 tunnel 203.0.113.2 192.0.2.1
# ifconfig egre0 vnetid 100
# ifconfig egre0 mtu 1500 -tunneldf
# ifconfig egre0 up
# ifconfig bridge0 add vlan0 add egre0 up
# ifconfig vlan1 vnetid 200
# ifconfig egre1 create
# ifconfig egre1 tunnel 203.0.113.2 192.0.2.1
# ifconfig egre1 vnetid 200
# ifconfig egre1 mtu 1500 -tunneldf
# ifconfig egre1 up
# ifconfig bridge1 add vlan1 add egre1 up

Network Virtualization using GRE interfaces (nvgre) example

The following shows a basic nvgre configuration and an equivalent vlan(4) configuration. In the examples, 192.168.0.1/24 will be the network configured on the relevent virtual interfaces. The NVGRE underlay network will be configured on 100.64.10.0/24, and will use 239.1.1.100 as the multicast group address.

The vlan(4) interface only relies on Ethernet, it does not rely on IP configuration on the parent interface:

# ifconfig em0 up
# ifconfig vlan0 create
# ifconfig vlan0 parent em0
# ifconfig vlan0 vnetid 10
# ifconfig vlan0 inet 192.168.0.1/24
# ifconfig vlan0 up

nvgre relies on IP configuration on the parent interface, and an MTU large enough to carry the encapsulated traffic:

# ifconfig em0 mtu 1600
# ifconfig em0 inet 100.64.10.1/24
# ifconfig em0 up
# ifconfig nvgre0 create
# ifconfig nvgre0 parent em0 tunnel 100.64.10.1 239.1.1.100
# ifconfig nvgre0 vnetid 10010
# ifconfig nvgre0 inet 192.168.0.1/24
# ifconfig nvgre0 up

NVGRE is intended for use in a multitenant datacentre environment to provide each customer with distinct Ethernet networks as needed, but without running into the limit on the number of VLAN tags, and without requiring reconfiguration of the underlying Ethernet infrastructure. Another way to look at it is NVGRE can be used to construct multipoint Ethernet VPNs across an IP core.

For example, if a customer has multiple virtual machines running in vmm(4) on distinct physical hosts, nvgre and bridge(4) can be used to provide network connectivity between the tap(4) interfaces connected to the virtual machines. If there are 3 virtual machines, all using tap0 on each hosts, and those hosts are connected to the same network described above, nvgre with a distinct virtual network identifier and multicast group can be created for them. The following assumes nvgre1 and bridge1 have already been created on each host, and em0 has had the MTU raised:

On physical host 1:

# ifconfig em0 inet 100.64.10.10/24
# ifconfig nvgre1 parent em0 tunnel 100.64.10.10 239.1.1.111
# ifconfig nvgre1 vnetid 10011
# ifconfig bridge1 add nvgre1 add tap0 up

On physical host 2:

# ifconfig em0 inet 100.64.10.11/24
# ifconfig nvgre1 parent em0 tunnel 100.64.10.11 239.1.1.111
# ifconfig nvgre1 vnetid 10011
# ifconfig bridge1 add nvgre1 add tap0 up

On physical host 3:

# ifconfig em0 inet 100.64.10.12/24
# ifconfig nvgre1 parent em0 tunnel 100.64.10.12 239.1.1.111
# ifconfig nvgre1 vnetid 10011
# ifconfig bridge1 add nvgre1 add tap0 up

Being able to carry working multicast and jumbo frames over the public internet is unlikely, which makes it difficult to use NVGRE to extended Ethernet VPNs between different sites. nvgre and egre can be bridged together to provide such connectivity. See the egre section for an example.