LACP, VLANs, always stay connected.
I was bored last weekend, so I configured a two-port LACP bonded trunk from my FreeBSD-running NAS connected to my HP Procurve switch. Why?
- I could?
- I had all these spare Ethernet ports on my NAS, and they seemed bored.
- More seriously: high availability. One interface serving all my storage traffic just seemed ripe for failure. Imagine serving all your VMs over NFS to a VM server across the network over one NIC, and that one dies. Bad news bears.
- I also wanted to set up VLANs on top of the trunk. Why? So if I wanted to add a network segment for my NAS on another Layer 3 domain, I don't have to walk down to the basement to patch another cable.
On to the configuration. First, I configured the NAS box. Relevant /etc/rc.conf configuration:
# LACP Group
ifconfig_igb0="up” ifconfig_igb1="up” cloned_interfaces="lagg0 vlan12” ifconfig_lagg0="laggproto lacp laggport igb0 laggport igb1”
vlans_lagg0="12” ifconfig_lagg0_12="inet 10.10.2.31/24”
In the first stanza, define the interfaces (igb0, igb1) that comprise the LAGG (link aggregation) interface, simply up'ing them. Then configure your LAGG interface. It's important to specify the LAGG proto (LACP in my case). Since I'll be assigning IPs to the tagged VLAN interface, I don't assign an IP to the raw trunk interface. If I wanted to handle untagged traffic on the trunk, I would specify an IP address. In the second stanza, configuring the tagged VLAN interface. “vlans_lagg0” specifies a list of VLANs which traverse the lagg0 interface. Then, configure the IP address which will be tagged with VLAN 12 traffic, riding on lagg0. Ifconfig output should look like the following:
nas1:~ % ifconfig lagg0 lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=6403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6> ether 0c:c0:7a:54:84:12 nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> media: Ethernet autoselect status: active groups: lagg laggproto lacp lagghash l2,l3,l4 laggport: igb0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> laggport: igb1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
nas1:~ % ifconfig lagg0.12 lagg0.12: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=303<RXCSUM,TXCSUM,TSO4,TSO6> ether 0c:c0:7a:54:84:12 inet 10.10.2.31 netmask 0xffffff00 broadcast 10.10.2.255 nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> media: Ethernet autoselect status: active vlan: 12 vlanpcp: 0 parent interface: lagg0 groups: vlan
With the NAS's ethernet cables disconnected from the switch, we configure the static LACP trunk. To note, I run an HP Procurve 2910al-24G switch, so my commands will not work for instance on a Cisco switch running IOS. This is the point in the exercise where I wish I had command history on the switch. I can't remember the commands and in what order I did them. It probably went something not far from the following:
switch1# config t
switch1(config)# trunk ethernet 14,16 trk1 lacp
switch1(config)# vlan 12
switch1(vlan-12)# tagged trk1
What does this do? Creates a two port, static LACP trunk on ports 14 and 16. We then create VLAN 12, and assigning lagg0 to this trunk group. Now plug those Ethernet cables in and wait a minute for LACP neogtiation to take place.
switch1# show trunk
Port | Name Type | Group Type —- + ——————————– 14 | nas1-trunk-1 100/1000T | Trk1 LACP 16 | nas1-trunk-2 100/1000T | Trk1 LACP
switch1# show lacp
LACP Trunk Port LACP Port Enabled Group Status Partner Status
14 Active Trk1 Up Yes Success 16 Active Trk1 Up Yes Success
Hooray. A working, two-port Ethernet LACP trunk. I did some testing, removing Ethernet cables from the two-trunk pair seeing how many packets were dropped, or what the logs of the switch looked like.
64 bytes from 10.10.2.31: icmp_seq=204 ttl=63 time=5.204 ms Request timeout for icmp_seq 207 64 bytes from 10.10.2.31: icmp_seq=208 ttl=63 time=5.223 ms …. 64 bytes from 10.10.2.31: icmp_seq=224 ttl=63 time=4.502 ms Request timeout for icmp_seq 225 Request timeout for icmp_seq 226 Request timeout for icmp_seq 227 64 bytes from 10.10.2.31: icmp_seq=228 ttl=63 time=7.078 ms
Four packets dropped twice. Not bad! What does this look like from the switch?
switch1# log Trk1 Keys: W=Warning I=Information M=Major D=Debug E=Error —- Event Log listing: Events Since Boot —- I 10/22/16 14:00:51 00078 ports: trunk Trk1 is now active I 10/22/16 14:00:53 00076 ports: port 14 in Trk1 is now on-line I 10/22/16 14:00:59 00079 ports: trunk Trk1 is now inactive I 10/22/16 14:00:59 00078 ports: trunk Trk1 is now active I 10/22/16 14:01:02 00076 ports: port 16 in Trk1 is now on-line I 10/22/16 14:02:01 00077 ports: port 14 in Trk1 is now off-line I 10/22/16 14:02:20 00076 ports: port 14 in Trk1 is now on-line I 10/22/16 14:04:50 00077 ports: port 16 in Trk1 is now off-line I 10/22/16 14:05:02 00076 ports: port 16 in Trk1 is now on-line I 10/22/16 14:05:09 00077 ports: port 14 in Trk1 is now off-line I 10/22/16 14:05:21 00076 ports: port 14 in Trk1 is now on-line
Hopefully this end-to-end configuration example helps others who might find it useful. References: