Hot!VIP doesn't work with my Second ISP WAN Link

Author
mahmoudmaaiteh
New Member
  • Total Posts : 4
  • Scores: 0
  • Reward points: 0
  • Joined: 2019/05/16 12:08:45
  • Status: offline
2019/05/16 13:04:40 (permalink)
0

VIP doesn't work with my Second ISP WAN Link

Hello everybody,
 
I have been struggling to get this done for about three days with no result,
 
i have Fortigate 501E, Firmware v5.6.0.
 
i have 2 ISP connections each one with different public IP ranges, ISP A with public subnet X.X.X.X/X is connected directly from fortigate port 1 to the ISP MPLS, the fortigate interface port 1 configuration is straight through, physical interface with static IP address X.X.X.2/X and its role is WAN. for this interface everything is working fine and virtual IPs from the same subnet (X.X.X.X/X) are reachable via this interface.
 
my second WAN link Y.Y.Y.Y/Y is somehow complicated, FortiGate is connected to an Outside Switch, the fortigate interface port 8 which is connected to the outside switch has two VLAN interfaces configured under port 8, the physical port doesn't have any configuration nor IP Address, the two vlan interfaces configuration is listed bellow:
    edit "Main Link"
        set vdom "root"
        set ip 10.100.100.2 255.255.255.248
        set allowaccess ping
        set role wan
        set snmp-index 29
        set interface "port8"
        set vlanid 1000
    next
 
edit "Backup Link"
        set vdom "root"
        set ip 10.100.200.2 255.255.255.248
        set allowaccess ping
        set role wan
        set snmp-index 30
        set interface "port8"
        set vlanid 1001
    next
 
the Outside switch interface (connected to FortiGate) is configured as trunk with both vlan 1000,1001 allowed. 
 
Also the switch has two interfaces each is connected to a different MPLS ( both MPLSs are owned by the same ISP) one interface is access vlan 1000 with the opposite interface on the MPLS has an IP Address of 10.100.100.1, and the other interface is access vlan 1001 with MPLS interface 10.100.200.1. those links are for the same IP Range and the same WAN Link ( active/ standby) with SLA configured from ISP side on both of the links and a link monitor from fortigate side. 
 
im planning to use the second link Y.Y.Y.Y/Y to publish a number of web servers. 
 
i created a virtual IP with configuration as bellow 
config firewall vip
    edit "Mahmoud-TEST"
        set id 0
        set uuid 34e526f8-77ff-51e9-4207-cda8d9aafcdf
        set comment ''
        set type static-nat
        set extip Y.Y.Y.187
        set extintf "Main Link"
        set arp-reply enable
        set nat-source-vip disable
        set portforward disable
        set gratuitous-arp-interval 0
        set color 0
        set mappedip "172.16.16.69"
    next
 
the server is reachable from the FG , and can reach the FG, the WAN link works fine as i tried to browse the internet from the server using the link ( created a policy and NATed with dynamic pool from the same public subnet Y.Y.Y.Y/Y ) and everything works fine.
 
i also created a policy to access the server from the wan ( later i allowed All sources with All services ) but still can't reach it, i did a sniffing on the Main Link interface and find out that the packets arrives The fortigate interface but it didn't route any of them, also the PING doesn't show on the traffic log.
 
i tried to publish the server using the First link X.X.X.X/X and everything works fine.
 
Please Advise.
#1
Toshi Esumi
Expert Member
  • Total Posts : 1538
  • Scores: 131
  • Reward points: 0
  • Joined: 2014/11/06 09:56:42
  • Status: offline
Re: VIP doesn't work with my Second ISP WAN Link 2019/05/16 14:21:51 (permalink)
0
Technically you have three wan links to choose from to go outside (internet, I'm assuming). What kind of default route(s) do you have in the FGT? If the outside sources are on the internet you need to have a default route toward the interface to send returning packets back to the source. If don't, and if you run "debug flow", you would the common error  "iprope_in_check() check failed, drop" as described in below because the returning route is pointing to another interface, which would cause asymmetric routing.
https://kb.fortinet.com/kb/viewContent.do?externalId=FD31702
You need to choose from either complete equal distance (load share) or equal distance but different priorities to set all default routes.
#2
mahmoudmaaiteh
New Member
  • Total Posts : 4
  • Scores: 0
  • Reward points: 0
  • Joined: 2019/05/16 12:08:45
  • Status: offline
Re: VIP doesn't work with my Second ISP WAN Link 2019/05/17 11:36:58 (permalink)
0
well, my default route will point to the first link (x.x.x.x) which is 0.0.0.0/0 => X.X.X.1/X. 
 
After running debug flow, i found that the error was  "reverse path check failed, drop". after alot of digging and reading KB articles i found out that Fortigate only checks for the reverse path in the directly connected networks, static routes and dynamic routes (BGP,OSPF, etc.). and in my case the reverse path for the packets are known via policy route. so from the Reverse path check point of view, this will cause asymmetric routing which is disabled by default.
 
My question now is, is it fine from a security point of view to enable asymmetric routing ? because i just read that it has smth to do with IP spoofing and other security consedrations ? 
 
A note here, im not using asymmetric routing, because the policy route i added will make sure that the packets will leave from the same interface it arrived from.
 
and how can the packets bypass the path check if the reverse path is known via a policy route, not a static route ?
 
Thanks again,
#3
Toshi Esumi
Expert Member
  • Total Posts : 1538
  • Scores: 131
  • Reward points: 0
  • Joined: 2014/11/06 09:56:42
  • Status: offline
Re: VIP doesn't work with my Second ISP WAN Link 2019/05/18 21:08:57 (permalink)
0
You could eanble asym routing by "set asymroute enable" if you want to use your FGT as just a router. You would lose most of firewall features. I don't recommend it. You need to question yourself "Why did I get a firewall instead of a router???"
As in the last part of my comment above, instead you just need to have multiple default routes, in your case, with different priorities. That's all you need. Below is my home FG50E static default routes, the main one to a vlan from my ISP (C-Link) and another for a 4G Modem/router, which I use as a backup.
config router static
    edit 8
        set device "ClinkVl201"
        set dynamic-gateway enable      <-- because pppoe
    next
    edit 9
        set priority 10              <--- by default the priority is 0 (highest), so set this higher than C-link
        set device "wan2"
        set dynamic-gateway enable    <-- because DHCP
    next
end
 
As the result, both show up in routing table "get router info routing-t all":
S*      0.0.0.0/0 [10/0] via 63.231.10.69, ppp1
                      [10/0] via 100.107.137.57, wan2, [10/0]

So the key is the last "10/0". "10" is the priority I set above. So all traffic/session initiated inside of this 50E always use "ppp1" interface toward C-link, of which default route has priority 0. But if any traffic from ouside comes to 100.107.137.57, like pinging from outide, the returning traffic, like ping reply, goes out of wan2 because the route is in the table. If I set VPN or VIP on wan2, it should work in the same way without making asym routing.



#4
mahmoudmaaiteh
New Member
  • Total Posts : 4
  • Scores: 0
  • Reward points: 0
  • Joined: 2019/05/16 12:08:45
  • Status: offline
Re: VIP doesn't work with my Second ISP WAN Link 2019/05/19 04:34:01 (permalink)
0
Well, 
 
I created a second static route as follows 
 
config router static
edit 15
set status enable
set dst 0.0.0.0 0.0.0.0
set gateway 10.100.100.1
set distance 20
set weight 0
set priority 0
set device "Main Link"  
set comment ''
set blackhole disable
set dynamic-gateway disable
set virtual-wan-link disable
set dstaddr ''
unset internet-service
set internet-service-custom ''
set link-monitor-exempt disable
next
end
 
but, it doesn't show in the routing table as yours does,
 
get router info routing-table all
 
S* 0.0.0.0/0 [10/0] via X.X.X.1, port1


there is no second entry as your case 
#5
Toshi Esumi
Expert Member
  • Total Posts : 1538
  • Scores: 131
  • Reward points: 0
  • Joined: 2014/11/06 09:56:42
  • Status: offline
Re: VIP doesn't work with my Second ISP WAN Link 2019/05/19 09:13:54 (permalink)
0
Look at my config and compare it with yours. I never set "distance" but "priority". The priority is the one to make the secondary, tertiary route show up as well as the primary because the distance for those routes are the same.
#6
mahmoudmaaiteh
New Member
  • Total Posts : 4
  • Scores: 0
  • Reward points: 0
  • Joined: 2019/05/16 12:08:45
  • Status: offline
Re: VIP doesn't work with my Second ISP WAN Link 2019/05/19 16:34:16 (permalink)
0
Well, you really saved my day now with the three default routes with different priorities and same distance , everything works fine and i can sleep well

Thank you sir.

But im little bit disappointed because reverse path check never looks in the policy routes to determine the reverse path,,,

But my issue now has been resolved
#7
Toshi Esumi
Expert Member
  • Total Posts : 1538
  • Scores: 131
  • Reward points: 0
  • Joined: 2014/11/06 09:56:42
  • Status: offline
Re: VIP doesn't work with my Second ISP WAN Link 2019/05/19 21:45:54 (permalink)
0
To make a policy route work, there needs to be a proper route for the traffic through the interface. You never had route before through that interface that's why it failed. It's by design, not a problem.
#8
Toshi Esumi
Expert Member
  • Total Posts : 1538
  • Scores: 131
  • Reward points: 0
  • Joined: 2014/11/06 09:56:42
  • Status: offline
Re: VIP doesn't work with my Second ISP WAN Link 2019/05/19 23:32:25 (permalink)
0
I probably misstated. Once a session has been established, I believe it would try sending returning packets back to the interface the original packet came in regardless what kind of policy route has been set. But there was no route then it dropped before a session is established. Simply needed a route to send returning packets to.
#9
Jump to:
© 2019 APG vNext Commercial Version 5.5