Mac Address Flapping between ports because of Multiple Checkpoint Cluster on Same Switch and Same Vlan

Our log system shows there are port flapping in one of our switches. Actually it was experienced before when configured ClusterXL. It seems there is another new Checkpoint cluster connecting to same switch and not considering Magic Mac Address Conflicting. Here are the steps to resolve this issue.

1. Symptoms.

Switch Logs: 
655033: .Oct  1 11:48:27.091 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 0000.0000.fe00 in vlan 3 is flapping between port Gi0/6 and port Po1
655034: .Oct  1 11:48:41.615 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 0000.0000.fe01 in vlan 3 is flapping between port Gi0/6 and port Gi0/5
655035: .Oct  1 11:48:42.134 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 0000.0000.fe00 in vlan 3 is flapping between port Gi0/6 and port Po1
655036: .Oct  1 11:48:57.030 EDT: %SW_MATM-4-MACFLAP_NOTIF: Host 0000.0000.fe01 in vlan 3 is flapping between port Gi0/6 and port Gi0/5

2. Solutions

Basically it is caused by multiple Checkpoint ClusterXL firewalls connecting to same switch and same vlan. It is well described at SK25977. Based on your ClusterXL configuration, either Load Sharing or High Availability mode, either Mulitcast or BroadCast mode, you will have to change magic Source Mac Address or Destination Mac Address. In my case, it is using High Avaliability ClusterXL mode, Magic Source Mac Address change will resolve this issue.

a. Verify the magic mac addresses on both cluster members

[Expert@CP1:0]# fw ctl get int fwha_mac_magic
fwha_mac_magic = 254
[Expert@CP1:0]# fw ctl get int fwha_mac_forward_magic
fwha_mac_forward_magic = 253

[Expert@cp2:0]# fw ctl get int fwha_mac_magic
fwha_mac_magic = 254
[Expert@cp2:0]# fw ctl get int fwha_mac_forward_magic
fwha_mac_forward_magic = 253

b. Change the magic mac addresses on both cluster members

[Expert@CP1:0]# fw ctl set int fwha_mac_magic 40
[Expert@CP1:0]# fw ctl set int fwha_mac_forward_magic 41

[Expert@cp2:0]# fw ctl set int fwha_mac_magic 40
[Expert@cp2:0]# fw ctl set int fwha_mac_forward_magic 41

c. Make a permanent change on booting configuration file

Previous changes are on the fly and taking effect right away, but not able to survive a reboot. To make those change saved to survive reboot, booting configuration file has to be changed with following instructions.

Edit the $FWDIR/boot/modules/fwkern.conf file in Vi editor:

[Expert@CP1:0]# cd /opt/CPsuite-R77/fw1/boot/modules/
[Expert@CP1:0]# vi fwkern.conf

Add the following line (spaces are not allowed):

fwha_mac_magic=40
fwha_mac_forward_magic=41

Reboot each cluster member one by one.

3. Reference:

• sk25977 Connecting multiple clusters to the same network segment (same VLAN, same switch)
• sk26202 Changing the kernel global parameters for Check Point Security Gateway