AnsweredHot!IPSec Tunnel not passing traffic after link drop

Author
alex199
New Member
  • Total Posts : 6
  • Scores: 0
  • Reward points: 0
  • Joined: 2019/06/27 18:07:23
  • Status: offline
2019/06/27 21:35:23 (permalink)
0

IPSec Tunnel not passing traffic after link drop

Hi,
 
New here so forgive me if I've not posted this in the correct spot or if it has been asked before (couldnt find it anywhere).
 
I have an IPSec Tunnel configured with a Fortigate 201E at the local end and a Cisco Meraki MX appliance at the other end.
At the other end, we have frequent ISP drop outs (another issue we are working to fix) but it usually comes back up quite quickly. 
The problem for us is that obviously when the link drops, the tunnel drops, but the link usually comes up within a minute or so and I can see the tunnel coming back online on the Fortigate but there is no traffic passing through. I have to manually take down the tunnel on the Fortigate, and it then immediately comes back up and traffic starts passing through.
 
I cant for the life of me work out why traffic does not resume when the tunnel reconnects. 
 
Any ideas?
 
Thanks
Alex
 
#1
Grave_Rose
Bronze Member
  • Total Posts : 25
  • Scores: 4
  • Reward points: 0
  • Joined: 2017/08/11 10:54:59
  • Status: offline
Re: IPSec Tunnel not passing traffic after link drop 2019/06/28 06:42:55 (permalink) ☄ Helpfulby alex199 2019/07/14 18:49:02
0
Hey Alex199,
 
Without getting into logs and debugs, it seems like there's a mismatch on the SAs between the devices when the link flaps where one of them is holding on to an old SA and another is expecting a new one.
 
Do you have Dead-Peer Detection configured inside of Phase-1 on the FortiGate? If not, try turning that on to "On-Demand" which may help recover the session. If you want to get really crazy you could create an automation stitch to send a trigger which can be processed by another box which can then make API calls to reset the tunnel... But try DPD first if it's not already set. :)
 
Hope this helps,
 
Sean (Gr@ve_Rose)
#2
rwpatterson
Expert Member
  • Total Posts : 8404
  • Scores: 195
  • Reward points: 0
  • Joined: 2006/08/08 10:08:18
  • Location: Long Island, New York, USA
  • Status: online
Re: IPSec Tunnel not passing traffic after link drop 2019/06/28 08:39:07 (permalink) ☄ Helpfulby alex199 2019/07/14 18:49:13
0
In the tunnel phase1 (may be phase2, I can't recall) setting, you should be able to 'set autonegotiate enable' to bring the tunnel up when both sides see each other again.

-Bob - self proclaimed posting junkie!
See my Fortigate related scripts at: http://fortigate.camerabob.com

-4.3.19-b0694
FWF60B
FWF80CM (4)
FWF81CM (2)
 
#3
alex199
New Member
  • Total Posts : 6
  • Scores: 0
  • Reward points: 0
  • Joined: 2019/06/27 18:07:23
  • Status: offline
Re: IPSec Tunnel not passing traffic after link drop 2019/06/28 21:37:18 (permalink)
0
Hi Sean,
 
Thanks for the response. We do have Dead-Peer Detection set to On-Demand at the moment but it doesn't seem to help. After doing a bit of reading on the SA side of things, this could definitely be the issue. Ill need to investigate this one a bit further and see if I can see what happens when the link goes down.
 
Thanks
Alex
#4
alex199
New Member
  • Total Posts : 6
  • Scores: 0
  • Reward points: 0
  • Joined: 2019/06/27 18:07:23
  • Status: offline
Re: IPSec Tunnel not passing traffic after link drop 2019/06/28 21:38:04 (permalink)
0
Hi Bob,
 
Autonegotiate is already enabled. Unfortunately that isnt helping us either!
 
Thanks
Alex
 
#5
alex199
New Member
  • Total Posts : 6
  • Scores: 0
  • Reward points: 0
  • Joined: 2019/06/27 18:07:23
  • Status: offline
Re: IPSec Tunnel not passing traffic after link drop 2019/07/14 18:47:03 (permalink)
0
Just an update on this.
 
We've actually added in a backup service on the Meraki side with an additional tunnel on the Fortigate side.
 
On the Fortigate we have set the backup tunnel with a higher Administrative Distance to monitor the Primary and it takes over when the backup fails.
 
Now when the Primary comes back up, it fails back seamlessly. 
 
I am not sure why is wasnt working before but everything is working as expected now. 
 
Thanks for the help guys.
#6
sw2090
Gold Member
  • Total Posts : 374
  • Scores: 21
  • Reward points: 0
  • Joined: 2017/06/14 01:27:25
  • Location: Regensburg
  • Status: offline
Re: IPSec Tunnel not passing traffic after link drop 2019/07/15 01:29:00 (permalink)
0
DPD and autonegotioan are all in IPSec itself.
I encountered similar issues...tunnel was still there or came back asap when online again but no traffic.
WHat solved it here was to turn on NAT-T on the tunnel. This will send keepalives on the ip layer where your traffic flows over the tunnel.
Since I enabeld NAT-T the issue is gone...
#7
ede_pfau
Expert Member
  • Total Posts : 6025
  • Scores: 480
  • Reward points: 0
  • Joined: 2004/03/09 01:20:18
  • Location: Heidelberg, Germany
  • Status: offline
Re: IPSec Tunnel not passing traffic after link drop 2019/07/15 02:19:20 (permalink) ☼ Best Answerby alex199 2019/07/29 15:48:55
0
FWIW:
For all others encountering this issue, there is an explanations and an easy fix.
 
When a tunnel drops, it's route is dropped as well, along with all affected sessions. Consequently, the outgoing traffic to the remote private network is sent out along the default route, usually through the WAN interface.
That alone is not especially bad, the next router will drop traffic to RFC 1918 private networks.
 
But, the FGT will establish a session for it, as there is a valid policy from LAN to WAN, destination ALL.
Now when the tunnel comes back up, there is already a current session which has to time out first before a new session through the tunnel can be established. This causes a major delay in the data flow.
 
There is a fix for this:
Create blackhole routes for traffic to RFC 1918 subnets, that is, 192.168.0.0/24, 172.16.0.0/12, 10.0.0.0/8 among others. These bh routes need to have a distance of 254 (not 255!) in order to kick in when there is no better route available. The bh route will be used when the tunnel goes down and traffic will be discarded; NO session is established.
When the tunnel comes up again, a new session can be built right away, without any delay.
 
I've posted that 4 years ago along with a batch command file to download. Just import it (System>Advanced>batch...) to create the bh routes. This will not harm existing routes at all as they are the least attractive routes of all: https://forum.fortinet.com/FindPost/120872

Ede

" Kernel panic: Aiee, killing interrupt handler!"
#8
rwpatterson
Expert Member
  • Total Posts : 8404
  • Scores: 195
  • Reward points: 0
  • Joined: 2006/08/08 10:08:18
  • Location: Long Island, New York, USA
  • Status: online
Re: IPSec Tunnel not passing traffic after link drop 2019/07/15 05:38:00 (permalink)
0
Scary, but I remember the bogons posts...

-Bob - self proclaimed posting junkie!
See my Fortigate related scripts at: http://fortigate.camerabob.com

-4.3.19-b0694
FWF60B
FWF80CM (4)
FWF81CM (2)
 
#9
alex199
New Member
  • Total Posts : 6
  • Scores: 0
  • Reward points: 0
  • Joined: 2019/06/27 18:07:23
  • Status: offline
Re: IPSec Tunnel not passing traffic after link drop 2019/07/15 19:27:31 (permalink)
0
Awesome, thanks Ede, we'll do some testing with this and report back!
 
 
#10
alex199
New Member
  • Total Posts : 6
  • Scores: 0
  • Reward points: 0
  • Joined: 2019/06/27 18:07:23
  • Status: offline
Re: IPSec Tunnel not passing traffic after link drop 2019/07/29 15:48:49 (permalink)
0
Just reporting in to let you know that Ede's response worked like a charm! thanks for all the help, tunnel is coming back online automatically now. 
 
Thanks all.
#11
Jump to:
© 2019 APG vNext Commercial Version 5.5