Routing / ICMP gets partially lost

[echo]

I have seen a strange issue in the past years with Win7, Win10 (have never used Win8), Win2008 and Win2012. When I ping something, like a router or switch or some other device that I restart remotely, then after it gets up, the ping won't recover in many cases. When I ping some other address in that network that I didn't ping before, then that replies to pings. After a while, the device starts replying to pings again. I have also noticed it happens not only over the tunnel but also in the same lan. On the other hand, there are many cases where our monitoring software pings a device over the tunnel, the device gets restarted or the tunnel just goes down (because the VPN device in between gets restarted) and the ping won't recover after that. When I go to the monitoring server then I see the same situation: I can ping other devices in the same network and even, in case of pinging a server, the shares open, but ping doesn't reply and traceroute goes to internet. If I paused pinging for 5 minutes, the ping started replying again.

So far I see that the lowest common denominator of all these is a Fortigate firewall/router and newer Windows operating systems. Such behaviour is really unexpected for me and disturbs and leads to unnecessary problems.

Has anybody else seen that?

I have one theory: the slight disruption of reachability of the device leads Fortinet device to think and record in some buffer that from now on that device is not reachable over the tunnel anymore but in internet. That's the most likely cause that I can make so far considering all such cases.

In one case the thing became really crucial because I noticed such scenario: IP-phones registered themselves over the tunnel to a voip-server, 3AM ISP restarts some of their routers, Fortigate starts thinking that that particular IP is no more accessible over the tunnel and routes to internet as the next hop based on routing table, and stays in such state forever, until the traffic is stopped somehow, eg restarting the router. I resolved that case when I set up specific black hole routes for the voip server with distances between the tunnel-routing distance and default route distance. It's just another hint that Fortigate gets confused with routing in some circumstances. There were no other anomalies related to internet access in that case.

Adding black hole routes in any case seems more like a workaround to me although I can start using this in tunnel configurations.