How configure more than two ISPs

[aufg]

Hi, I have two FG 100D configured inside an active/pasive cluster (HA). I have two ISP configured and I´m making load balancing throught static routing, therefore I have two default static routes with the same metric and priority. I have also configured keepaplive at both lines to make that FG remove this routes if there is any problem witch each ISP. This configuration is working properly several months ago. Now, my customer wants to add another two ISPs. I've extracted two ports from FG LAN Switch to connect the new ISPs, I have added another two static routes (same metric and priority), Iv'e configured the keepalive and I've added this ports to the policies. Right now I have an issue, all sessions (except 2 or 3) are using the old two ISPs (even If i make al full reboot of both FGs to restart all sessions), if I put out of service one of the old ISPs, all session bascul to the other old ISP and no to the new two ISPs....... What is wrong in my configuration? The new lines are operative throght FG, because when I make a Policy routing and force that a network go out to internet throught new lines there isn't any problem and users have internet. Thank you!!!

[alhashem]

the new default routes should have the same distance and priority. did you set the same value for administrative distance in your new default routes? because you didn't refer to the distance in your post.

and type the below command in CLI to see the routing table

get router info routing-table all

do you see your new default routes under the 0.0.0.0 destination network?

sorry for my poor English :)

[aufg]

Hi,

Thank you for your help. Yes, all static routes have the same administrative distance and same priority, I've checked the phisical interfaces and also have the same administrative distance.

I have the four ISPs connected and failover is working fine, If I disconnect two ISPs (1 and 2) all sessions are balanced to the other two ISPs (3 and 4). It is strange, if I make a query to the routing table FG only has two 0.0.0.0/0 static routes (ISPs 1 and 2) but I have four configured (1,2,3 and 4).... All sessions are balanced between ISPs 1 and 2, If I disconnect one of this (1 for example), all sessions bascul to the other one (2), I only get sessions over ISPs 3 and 4 when I disconnect the ISPs 1 and 2........ Then, routing table erases ISPs 1 and 2 routes and adds ISPs 3 and 4 routes......

I have not found the way to make the load balancing between four ISPs throught static routing. Could you help me please?

[alhashem]

run the below command:

get router info routing table database

do you see the static route of ISP 3 & 4 as a inactive route?

what kind of internet connections do you have?? 

what is the framware version

[aufg]

alhashem wrote:

run the below command:

get router info routing table database

do you see the static route of ISP 3 & 4 as a inactive route?

what kind of internet connections do you have?? 

what is the framware version

Hi, thank you four your help, you can find below several commands refered to static-routing configuration: get router info routing-table database S       0.0.0.0/0 [10/0] via 10.10.0.1, WAN3                   [10/0] via 10.10.1.1, WAN4                   [10/0] via 192.168.1.1, WAN2 inactive                   [10/0] via 192.168.1.1, WAN1 inactive S    *> 0.0.0.0/0 [5/0] via 192.168.1.1, ppp1      *>           [5/0] via 192.168.1.1, ppp2 Fortinet (static) # show     edit 2         set gateway 192.168.1.1         set device "WAN2"     next     edit 5         set gateway 192.168.1.1         set device "WAN1"     next     edit 4         set gateway 10.10.0.1         set device "WAN3"     next     edit 6         set gateway 10.10.1.1         set device "WAN4"     next end Fortinet (static) # get 2 seq-num             : 2 dst                 : 0.0.0.0 0.0.0.0 gateway             : 192.168.1.1 distance            : 10 weight              : 0 priority            : 0 device              : WAN2 comment             : blackhole           : disable dynamic-gateway     : disable virtual-wan-link    : disable   Fortinet-B (static) # get 5 seq-num             : 5 dst                 : 0.0.0.0 0.0.0.0 gateway             : 192.168.1.1 distance            : 10 weight              : 0 priority            : 0 device              : WAN1 comment             : blackhole           : disable dynamic-gateway     : disable virtual-wan-link    : disable   Fortinet-B (static) # get 4 seq-num             : 4 dst                 : 0.0.0.0 0.0.0.0 gateway             : 10.10.0.1 distance            : 10 weight              : 0 priority            : 0 device              : WAN3 comment             : blackhole           : disable dynamic-gateway     : disable virtual-wan-link    : disable Fortinet-B (static) # get 6 seq-num             : 6 dst                 : 0.0.0.0 0.0.0.0 gateway             : 10.10.1.1 distance            : 10 weight              : 0 priority            : 0 device              : WAN4 comment             : blackhole           : disable dynamic-gateway     : disable virtual-wan-link    : disable

[aufg]

With this configuration, right now I have:

WAN1-> 798 sessions -> 183.51kb/s WAN2-> 548 sessions -> 614.4kb/s WAN3-> 2 sessions -> 36b/s WAN4-> 115 sessions -> 167b/s

I look forward your response alhashem; thank you.

[Toshi_Esumi]

You're getting lower distance default routes over PPPoE from old ISPs [5/0] against all your static default routes [10/0]. I suggest you "set defaultgw disable" on wan1 and wan2, which would stop pulling the [5/0] default routes. Then you have to correct your default static routes with "set dyamic-gateway enable" instead of specifying GW IP statically. It would be pulled via PPPoE and you would see like below instead in routing database:

S    *> 0.0.0.0/0 [10/0] via 192.168.1.1, ppp1

S    *> 0.0.0.0/0 [10/0] via 192.168.1.1, ppp2

I thought having the same gw IP on both wan1 and wan2 would cause some problems but it seemed to be ok because you said that part had been working fine.

Below is my home primary INET (vlan) interface config (masked some info), which gets IP and GW over PPPoE. I have a secondary INET interface in addition to it. So doing the same thing to use static default routes I configured.

config system interface     edit "mainINET"         set vdom "root"         set mode pppoe         set allowaccess ping          set role wan         set username "xxxxxxxxxxxxx"         set password ENC <ENCRYPTED_PASSWORD>         set defaultgw disable       <--         set interface "wan1"         set vlanid xxx     next end config router static     edit 8         set device "mainINET"         set dynamic-gateway enable     <--     next end

fg50e # get router info routing-t database | grep 0.0.0.0 S    *> 0.0.0.0/0 [10/0] via xxx.xxx.xxx.xxx, ppp1     <-- notice the interface is not "mainINET" but "ppp1"