How to connect the Meraki MX to MS switches

Connecting the Meraki MX to an internal switched network? Sounds easy and if the network is build without any redundancy, it is very easy indeed:

It can get a little bit problematic if redundancy is added. If you come from the Cisco ASA, you have tools like

• routed interfaces
• Port-Channel

But on the MX we have

• no port-channels
• switched interfaces
• no spanning-tree

In addition to that, we have Meraki switches that “only” support RSTP and no per-VLAN RSTP.

In the redundant setup, we have basically two options to connect the redundant MX firewalls to the redundant MS switches:

Option 1:

Based on Meraki documentation and best practice, both MX are connected to both switches and for this discussion, it is not relevant if the switches are standalone or stacked switches:

But from a Spanning-Tree view, the topology looks like this:

With the MX not participating in Spanning-Tree, we need to make sure that the switches can block one interface of each of these connections. For this we need to make sure that STP BPDUs can flow freely through the MX. When coming from an ASA background, you are probably used to have sub interfaces on the ASA, but not to configure the main-interface with a nameif, security-level and IP-address. Lets assume we configure something similar on the MX:

Although we only want to use the two tagged networks CORE-TRANSFER and DMZ, by dropping untagged traffic on the MX, we also drop BPDUs and Spanning-Tree can not block the two ports on the switches and the network is likely to be severely impacted.

If configured correctly, the ports that connect the two MX look like this:

The upper switch has two STP designated ports, both are forwarding and the one with the white arrow indicates the connection to the active/primary MX. The second switch has two ports in the STP blocking state.

The switch ports connecting the MX have RSTP enabled, but no STP guard active:

What is the corresponding MX configuration? There are multiple ways to implement it, each with its own pros and cons. The easy way is the following (ignoring the best practice not to use VLAN1):

VLAN 1 is used as the native VLAN on the MX and is used for the management-ports of the connected Meraki MS switches. The default port- and management-vlan-configuration on the MS switches matches the MX config and no adjustment is needed. But special caution is needed to make sure no user-port is configured for VLAN 1 in this setup. Probably extra Firewall rules for VLAN 1 should allow Meraki Dashboard communication but nothing else in case a user is placed in VLAN 1 by mistake.

Alternatively we can use a different VLAN as the native VLAN:

Now we need to make sure that the switches use this VLAN as the management-VLAN and also have this VLAN native on the uplinks. For this setup, the management VLAN was not configured individually on the “LAN IP”, it was set on the “Switch settings” page for all switches.

The main benefit is that users are not likely to be assigned to this VLAN by mistake.

Option 2:

In option 1 we rely on Spanning-Tree to block ports. Probably every network-admin in the world already faced problems with Spanning-Tree and/or caused unplanned downtimes by misconfiguring Spanning-Tree. For this reason a different approach could be used that has less redundancy, but can be more stable:

Each MX has only one connection to the switched network. Spanning-Tree is not needed any more to block ports, both switch ports that connect the MX firewalls are in the forwarding state. In this scenario we don’t need a native VLAN on the MX and we can drop untagged traffic without causing any problem.

Although this setup can be more stable, there is at least one major drawback (in addition to the reduced redundancy):

When the link between the two switches fail, we run into a split-brain situation (Anyone else already had a non-working stack-connection after a reboot?). VRRP between both MX can’t reach each other and both Firewalls become active:

For sure, this is a bad situation and you want to get out of it as fast as possible. But with option 2 it can even get worse. Both firewalls want to use the virtual IP but only one can do that. The part of the network that had the previous “Current master” likely has uninterrrupted internet connectivity. But the part of the network with the previous “passive, ready” will have limited connectivity.

When facing the problem some time ago in a “real” network, one switch completely lost connectivity to the dashboard which can make troubleshooting problematic if you are not onsite. While doing some more tests recently, the dashboard connectivity was delayed, but I was able to manage both switches and a PCAP showed that the spare MX fell back to its interface IP.

Recommendations?

I have no recommendation to generally use Option 1 or Option 2. Most of the time I prefer the oficially suggested way which is Option 1. Just be aware of how the network behaves and choose wisely!