Helpful ReplyHot!Atypical HA config with two ISPs

Author
SigniVain
New Member
  • Total Posts : 13
  • Scores: 0
  • Reward points: 0
  • Joined: 2014/02/26 09:10:56
  • Status: offline
2020/06/04 08:18:20 (permalink)
0

Atypical HA config with two ISPs

Howdy,
Perhaps you can shed some light on the following.  We have two Fortigate 300Ds (v6.2.3) in an Active-Passive HA cluster.  Up to now, only the Primary unit has had the "outside" interface (let's call it WAN1) plugged in; we don't have a switch between the Fortigate and the ISP (ISP1) in order to have WAN1 plugged in on both Primary and Slave.
Now, we have a second internet pipe (ISP2).  I know the typical deployment would have a switch between each Fortigate in the HA cluster and the ISP:
  • Primary WAN1 = ISP1
  • Primary WAN2 = ISP2
  • Secondary WAN1 = ISP1
  • Secondary WAN2 = ISP2
The above would be ideal, but I need to make things work without the upstream switches.
 
Here are the requirements:
If ISP1 is having issues, which is plugged into Primary WAN1, HA fails over to Secondary which has ISP2 plugged into WAN2.
 
If I keep ISP1 plugged into Primary WAN1 (Secondary WAN1 has nothing plugged in), and plug ISP2 into Secondary WAN2, is it as easy as setting up link monitoring, adding the default route, and adding the WAN2 interface of the HA cluster to the existing WAN1 policies?  Any issues with keeping HA as Active-Passive?
Here's the kicker, we're advertising a /24 to ISP1 via BGP.  I won't be able to set the secondary IP address of WAN2 to anything in the /24 advertised by WAN1.  This might be a whole different topic, but in order to achieve all of the above *AND* advertise a /24 via BGP, would creating an SD-WAN interface be the way to go (add both WAN1 and WAN2 to the SD-WAN interface)?
 
Thank, in advance, you for your guidance.
#1
James_G
Gold Member
  • Total Posts : 250
  • Scores: 11
  • Reward points: 0
  • Joined: 2016/02/28 02:55:47
  • Status: offline
Re: Atypical HA config with two ISPs 2020/06/04 08:34:07 (permalink)
0
Whats stopping you putting a switch between WAN1 ports and ISP
#2
SigniVain
New Member
  • Total Posts : 13
  • Scores: 0
  • Reward points: 0
  • Joined: 2014/02/26 09:10:56
  • Status: offline
Re: Atypical HA config with two ISPs 2020/06/04 08:36:05 (permalink)
0
Rack space, cost, and the powers that be. :)
#3
James_G
Gold Member
  • Total Posts : 250
  • Scores: 11
  • Reward points: 0
  • Joined: 2016/02/28 02:55:47
  • Status: offline
Re: Atypical HA config with two ISPs 2020/06/04 09:14:59 (permalink)
0
I don't think you would be able to have an ISP failure trigger an HA event, you would be better with switches, even if they were $20 5 port jobs.
 
You would have a 5 port switch for each ISP, so still no SPOF, worst that happens on switch failure is it fails to secondary ISP.
#4
brycemd
Silver Member
  • Total Posts : 118
  • Scores: 6
  • Reward points: 0
  • Joined: 2016/12/03 11:24:30
  • Status: offline
Re: Atypical HA config with two ISPs 2020/06/04 09:31:44 (permalink) ☄ Helpfulby SigniVain 2020/06/04 10:36:54
0
If you setup a link monitor to down the port(wan1) rather than just remove the route it might failover to secondary as connected ports is the main criteria for primary HA selection. But, even if it does work, I do believe a WAN switch is by far the way to go.
#5
emnoc
Expert Member
  • Total Posts : 5863
  • Scores: 387
  • Reward points: 0
  • Joined: 2008/03/20 13:30:33
  • Location: AUSTIN TX AREA
  • Status: offline
Re: Atypical HA config with two ISPs 2020/06/04 09:38:39 (permalink)
0
Seriously you are doing 2x ISP upstream with BGP and you're balking over cost a single switch, 1U of rack space,.... and you rather to  in some wacked-hackup-job  of  HA deployment in place with 2x FGT300D that cost approx 900 usd each before subscription bundle. Seriously? That make no sense form my standpoint.
 
If the FGT300D are in a HA cluster how did you connect the internal LAN connectivity ? A switch ???
 
Ken Felix

PCNSE 
NSE 
StrongSwan  
#6
SigniVain
New Member
  • Total Posts : 13
  • Scores: 0
  • Reward points: 0
  • Joined: 2014/02/26 09:10:56
  • Status: offline
Re: Atypical HA config with two ISPs 2020/06/04 09:53:54 (permalink)
0
I ask that you take into account that I am not a decision maker.  This directive was bestowed upon me, and I'm trying to figure out a solution within its confines.  Any and all help is appreciated.  I agree switches/a switch upstream of the HA cluster is "best practice."  It's the correct way, it's the easiest way, and offers the most redundancy.  I agree with all of it.  Alas, I'm still in the same position, and am reaching out to the community for guidance.
 
Perhaps changing from Active-Passive to Active-Active will help?  Sure, the WAN1 and WAN2 interfaces won't be redundant between the Primary and Secondary, but, in theory, it seems like ISP redundancy can be achieved.  May not even need an SD-WAN virtual interface.  Input is greatly appreciated.
#7
James_G
Gold Member
  • Total Posts : 250
  • Scores: 11
  • Reward points: 0
  • Joined: 2016/02/28 02:55:47
  • Status: offline
Re: Atypical HA config with two ISPs 2020/06/04 10:16:48 (permalink)
5 (1)
I think the guidance you need to feedback to management is that it's impossible to automate fail over in the suggested configuration.
 
Management either accept the risk, or fix it. Let them decide.
#8
lobstercreed
Platinum Member
  • Total Posts : 345
  • Scores: 43
  • Reward points: 0
  • Joined: 2018/11/28 14:57:58
  • Location: Sedalia, MO
  • Status: offline
Re: Atypical HA config with two ISPs 2020/06/04 10:37:27 (permalink)
0
I don't think you answered Ken's question about what is on the inside of the HA cluster?  If you have a switch there, just use a dedicated VLAN on 3 ports for each ISP.  Yeah it consumes 6 switch ports, which if you're down to your last $20 as someone else pointed out, might be too much, but...
 
I agree with James.  Tell management it can't be done.  Make them accept the risk for the want of a few bucks.
#9
SigniVain
New Member
  • Total Posts : 13
  • Scores: 0
  • Reward points: 0
  • Joined: 2014/02/26 09:10:56
  • Status: offline
Re: Atypical HA config with two ISPs 2020/06/04 10:52:02 (permalink)
0
Sorry, I figured that was a rhetorical question.  There is a router on the inside of the HA cluster.  This device must stay "clean," and not be directly connected to the public.
#10
emnoc
Expert Member
  • Total Posts : 5863
  • Scores: 387
  • Reward points: 0
  • Joined: 2008/03/20 13:30:33
  • Location: AUSTIN TX AREA
  • Status: offline
Re: Atypical HA config with two ISPs 2020/06/04 11:09:26 (permalink)
0
Yes this is more interesting. What do you mean "stay clean"? Following FTNT BCP you suppose to connect all interfaces in a HA cluster to the lan. HA -HB is going to be a issues if enable and interface unitA can see unitB for example
 
FWIW, I would not substitute a poor design due to a few dollars. A simple L2-switch can be had for 25-30 dollars on ebay and you would only need to set 2x vlans ( VLAN-ISP1 VLAN-ISP2 ) and cable the HA cluster wan1/wan2 to those vlans.
 
Ken Felix
 

PCNSE 
NSE 
StrongSwan  
#11
SigniVain
New Member
  • Total Posts : 13
  • Scores: 0
  • Reward points: 0
  • Joined: 2014/02/26 09:10:56
  • Status: offline
Re: Atypical HA config with two ISPs 2020/06/04 11:29:33 (permalink)
0
Clean, a device that does not touch the public WAN directly.  A dirty device would straddle both private LAN and public WAN.
 
If I have the following, in an Active-Passive HA cluster:
  • Primary WAN1 connected to ISP1
  • Primary WAN2 not connected to anything
  • Secondary WAN1 not connected to anything
  • Secondary WAN2 connected to ISP2
My questions are:
  1. Is it technically impossible for this scenario to work in Active-Passive?
  2. What about Active-Active, in conjunction with the following KB article?
#12
James_G
Gold Member
  • Total Posts : 250
  • Scores: 11
  • Reward points: 0
  • Joined: 2016/02/28 02:55:47
  • Status: offline
Re: Atypical HA config with two ISPs 2020/06/04 12:01:05 (permalink) ☄ Helpfulby lobstercreed 2020/06/04 12:02:41
0
What I believe is completely impossible is to automate failover on ISP failure
#13
lobstercreed
Platinum Member
  • Total Posts : 345
  • Scores: 43
  • Reward points: 0
  • Joined: 2018/11/28 14:57:58
  • Location: Sedalia, MO
  • Status: offline
Re: Atypical HA config with two ISPs 2020/06/04 12:05:04 (permalink)
0
You need to get rid of HA.  Keep the second box as a cold spare and put it into place if needed.
 
This will not work without symmetrical connectivity.  The conditional BGP article is assuming both ISPs are connected at all times.
#14
SigniVain
New Member
  • Total Posts : 13
  • Scores: 0
  • Reward points: 0
  • Joined: 2014/02/26 09:10:56
  • Status: offline
Re: Atypical HA config with two ISPs 2020/06/04 12:14:19 (permalink)
0
In an Active-Active HA cluster, the failover would be at the route level not the HA member level.  In theory, accomplishing the original goal?
 
In an Active-Active HA cluster, in the config described in my previous post (#12), would it not basically be as follows?
  • All traffic routing to ISP1 via Primary on WAN1
  • By ping monitor, Primary ISP1 is becomes unreachable on WAN1
    • Primary WAN2 has nothing plugged into it
  • By ping monitor, Secondary ISP2 is reachable on WAN2
    • Secondary WAN1 has nothing plugged into it
  • Route all traffic to ISP2 via Secondary WAN2 interface
#15
James_G
Gold Member
  • Total Posts : 250
  • Scores: 11
  • Reward points: 0
  • Joined: 2016/02/28 02:55:47
  • Status: offline
Re: Atypical HA config with two ISPs 2020/06/04 12:24:14 (permalink)
0
Active-active does not work like that, even in active-active all the active IP addresses are only on the primary unit, it just transfers processing of some UTM to the secondary node.
 
I wonder if something could be cooked up by using vdoms and virtual clustering with affinity to physical units, each WAN port being a seperate vdom, then some cross over cables to a central vdom then then in turn connects to internal LAN.
#16
James_G
Gold Member
  • Total Posts : 250
  • Scores: 11
  • Reward points: 0
  • Joined: 2016/02/28 02:55:47
  • Status: offline
Re: Atypical HA config with two ISPs 2020/06/04 12:53:52 (permalink)
0
I was thinking the below - then remembered you have a 300D and run our of ports. Read on if you want to get an idea of where I was going.
 
VDOM "WAN1" - has ports WAN1 and ports 3 and 4 in a hardware switch - virtual cluster with affinity to node A
VDOM "WAN2" - has ports WAN2 and ports 5 and 6 in a hardware switch - virtual cluster with affinity to node B
VDOM "INTERNAL" - has port 7 on each node cabled to port 3 and 4 on node A - port 8 on each node cabled to port 5 and 6 on node B. On the internal vdom, create interface monitors on port 7 and 8.
 
Run out of ports for internal :(
#17
emnoc
Expert Member
  • Total Posts : 5863
  • Scores: 387
  • Reward points: 0
  • Joined: 2008/03/20 13:30:33
  • Location: AUSTIN TX AREA
  • Status: offline
Re: Atypical HA config with two ISPs 2020/06/04 15:23:33 (permalink)
0
That could work if you use physical port between vclusters-vdom, but man your over complicating the network design for a meer cost of a switch and add more policy, more work, more break points and defeating the concept of HA-cluster to begin with.
 
Ebay has WS-2960 for 60 or less dollar or a EX4200 for 80 or less dollars. I just gave away like 8x 2960 to church, I could have donated you a switch or two ;)
 
 
Ken Felix

PCNSE 
NSE 
StrongSwan  
#18
compuchris
New Member
  • Total Posts : 1
  • Scores: 0
  • Reward points: 0
  • Joined: 2020/06/09 12:56:11
  • Status: offline
Re: Atypical HA config with two ISPs 2020/06/09 13:04:51 (permalink)
0
Please take into considerations that if you are using CAPWAP for wireless all wireless will drop when the gates fail-over.  If you are running bridged then you should be OK.
 
To solve this problem, we utilize inexpensive Netgear 5-port gig switches to split the ISP to each gate.
 
#19
Jump to:
© 2020 APG vNext Commercial Version 5.5