Hot!Fortigate 60E - dropping internet connection

Author
pgregor@orsia.cz
New Member
  • Total Posts : 4
  • Scores: 0
  • Reward points: 0
  • Joined: 2019/10/09 06:57:16
  • Status: offline
2019/10/09 07:16:27 (permalink)
0

Fortigate 60E - dropping internet connection

Hi, we have Fortigate 60E.
Last two months we have problem with unexpected breaking of 60E operation.
It breaks all traffic for few minutes and after that it starts operating (no admin action is required).
 
We have small office with up to 15 PCs and up to 20 virtual servers within our intranet. 
 
We have 2 internet connections and there is no problem with ISPs.
Both lines are stable with no outages.
The first internet line is about 20Mbit the second one is 5Mbit. 
The problem with Fortigate occures only if we use first (20Mbit) line.
 
If I disconnect WAN during the time of Fortigate outage and immediatelly I connect it back to Fortigate,
the Fortigate outgage is solved.
 
Thanks for any idea.
 
Petr
 
 
#1

7 Replies Related Threads

    Dave Hall
    Expert Member
    • Total Posts : 1548
    • Scores: 167
    • Reward points: 0
    • Joined: 2012/05/11 07:55:58
    • Location: Canada
    • Status: offline
    Re: Fortigate 60E - dropping internet connection 2019/10/09 11:30:26 (permalink)
    0
    Have you configured the two ISP connections under SD-WAN?  Have you set the ingress/egress values on both ISP connections? Does the Bandwidth history graph for the 20Mbit line show the connection being maxed out (what about the 5 Mbit line)?  What does FortiView shows which devices are using up most of the bandwidth?   Have you checked for duplex/speed mismatch or line cable issues?  (e.g. perform diag hardware deviceinfo nic <interface name> on the CLI and check for errors - perform the diag test again in a few mins if there are errors and see if the counters increase.)
     
     

    NSE4/FMG-VM64/FortiAnalyzer-VM/5.4/6.0 (FWF40C/FW92D/FGT200D/FGT101E)/ FAP220B/221C
    #2
    emnoc
    Expert Member
    • Total Posts : 5397
    • Scores: 355
    • Reward points: 0
    • Joined: 2008/03/20 13:30:33
    • Location: AUSTIN TX AREA
    • Status: offline
    Re: Fortigate 60E - dropping internet connection 2019/10/09 15:25:53 (permalink)
    0
    Also double check speed/duplex, just had this issue in a 60D where the ISP changed hardware and the FGT-nic would reset like every 9-12 mins. I had to lock the speed/duplex
     
    Ken Felix

    PCNSE 
    NSE 
    StrongSwan  
    #3
    pgregor@orsia.cz
    New Member
    • Total Posts : 4
    • Scores: 0
    • Reward points: 0
    • Joined: 2019/10/09 06:57:16
    • Status: offline
    Re: Fortigate 60E - dropping internet connection 2019/10/10 02:07:51 (permalink)
    0
    The ISPs are under SD-WAN. The main is 20Mbit line. When it is down, then 5Mbit line is switched (and back).
    There is no error in diag commands (wan1 is 5Mbit, wan2 20Mbit).
     
    We used the second ISP because of the main line was often down.
    When 5Mbit is used alone then no problem occurres.
     
    How to lock the speed/duplex? On interface wan?
    We have set Traffic Shapers only.
     
    Petr
     
    Info:
    FG-Orsia # dia hardware deviceinfo nic wan1
    Description :FortiASIC NP6LITE Adapter
    Driver Name :FortiASIC NP6LITE Driver
    Board :60E
    lif id :0
    lif oid :64
    netdev oid :64
    tx group :1
    Current_HWaddr e8:1c:ba:75:f7:d2
    Permanent_HWaddr e8:1c:ba:75:f7:d2
    ========== Link Status ==========
    Admin :up
    netdev status :up
    autonego_setting:1
    link_setting :1
    speed_setting :10
    duplex_setting :0
    Speed :1000
    Duplex :Full
    link_status :Up
    ============ Counters ===========
    Rx Pkts :16466947
    Rx Bytes :14658704761
    Tx Pkts :12558736
    Tx Bytes :2655143521
    Host Rx Pkts :7674970
    Host Rx Bytes :6527220133
    Host Tx Pkts :5883525
    Host Tx Bytes :663989880
    Host Tx dropped :0

    FG-Orsia # dia hardware deviceinfo nic wan2
    Description :FortiASIC NP6LITE Adapter
    Driver Name :FortiASIC NP6LITE Driver
    Board :60E
    lif id :1
    lif oid :65
    netdev oid :65
    tx group :2
    Current_HWaddr e8:1c:ba:75:f7:d3
    Permanent_HWaddr e8:1c:ba:75:f7:d3
    ========== Link Status ==========
    Admin :up
    netdev status :up
    autonego_setting:1
    link_setting :1
    speed_setting :10
    duplex_setting :0
    Speed :1000
    Duplex :Full
    link_status :Up
    ============ Counters ===========
    Rx Pkts :12498696
    Rx Bytes :10747982776
    Tx Pkts :9288245
    Tx Bytes :1628554518
    Host Rx Pkts :6966142
    Host Rx Bytes :5388007857
    Host Tx Pkts :4686829
    Host Tx Bytes :759243061
    Host Tx dropped :0
    #4
    Dave Hall
    Expert Member
    • Total Posts : 1548
    • Scores: 167
    • Reward points: 0
    • Joined: 2012/05/11 07:55:58
    • Location: Canada
    • Status: offline
    Re: Fortigate 60E - dropping internet connection 2019/10/10 13:38:22 (permalink)
    0
    Unless someone can say otherwise, I do not think you need to worry about the duplex/speed as an issue the diag tests does not show there being a problem with them - otherwise  you would get various rx/tx counter errors, that would increase over time.  That said, if you want to set/force the duplex/speed on an interface, you can do this via the CLI:
     
    config system interface
        edit <interface name>
            set speed ?
        next
    end

    where ? is:
    auto        Automatically adjust speed.
    10full      10M full-duplex.
    10half      10M half-duplex.
    100full     100M full-duplex.
    100half     100M half-duplex.
    1000full    1000M full-duplex.

    What I mean by setting the ingress/egress values on both ISP connections is to set values for "Estimated Bandwidth" on each Interface. 
     
    Later fgt firmware versions come with some nice SD-WAN settings/monitoring tools.  I would make sure that the all WAN interfaces have the proper default route, distance/metric, and you have setup the load-balancing (aka SD-WAN Rules).  The SD-WAN monitor will tell you how many sessions are open/going out which ISP connection. 
     
    If you do not have a bandwidth history graph on the main dashboard, I suggest adding two (one for each ISP connection). I would monitor the bandwidth usage, and CPU, memory, and sessions.  The fgt will (should) go into conserve mode should memory usage go near/over 80%. 
     
    If you have direct access to the ISP gateway devices, I would log into each device and check for any log or events.  Sometimes one side of that WAN connection may look fine, but the other side may tell a different story.
     
    If you have ping watch guard settings enabled (under Performance SLA) you will likely want to confirm they are working as expected.  If you are using Google's DNS there is rate limits set on how often you can ping their DNS servers.
     
    And of course you should check the System Events/Router Events (under Log & Report) for issues.
     

    Attached Image(s)


    NSE4/FMG-VM64/FortiAnalyzer-VM/5.4/6.0 (FWF40C/FW92D/FGT200D/FGT101E)/ FAP220B/221C
    #5
    pgregor@orsia.cz
    New Member
    • Total Posts : 4
    • Scores: 0
    • Reward points: 0
    • Joined: 2019/10/09 06:57:16
    • Status: offline
    Re: Fortigate 60E - dropping internet connection 2019/10/14 06:51:23 (permalink)
    0
    I set Estimated Bandwidth, but nothing changed.
    Static Route is set for interface SD-WAN and Dynamic Gateway is enabled. Routing monitor shows correct settings.
    I can not monitor the main ISP, I only ping to it.
    I have enabled a bandwidth history graph on the main dashboard and sessions and so on, but everything looks fine.
    System Events:
    • when wan2 is down
    The member2(wan2) link is unreachable or miss threshold. Stop forwarding traffic.
    Service1(VLAN-wan2) will failover to other available interface(s).
    • when wan2 is up
    The member2(wan2) link is available. Start forwarding traffic.
    Service1(VLAN-wan2) prioritized by latency will be redirected in seq-num order 2(wan2).
    Current SD-WAN settings:
    config system virtual-wan-link
        set status enable
        set load-balance-mode weight-based
        config members
            edit 2
                set interface "wan2"
                set gateway 212.158.144.193
            next
            edit 4
                set interface "wan1"
                set gateway 192.168.8.1
            next
        end
        config health-check
            edit "Google"
                set server "8.8.8.8"
                set interval 10
                set update-static-route disable
                set members 2 4
                config sla
                    edit 1
                    next
                end
            next
            edit "Quad9"
                set server "9.9.9.9"
                set interval 10
                set update-static-route disable
                set members 2 4
                config sla
                    edit 1
                    next
                end
            next
        end
        config service
            edit 1
                set name "VLAN-wan2"
                set mode priority
                set dst "all"
                set src "ORSIA-VLAN102-VoIP" "ORSIA-VLAN103-Guest" "ORSIA-VLAN104-DMZ" "ORSIA-VLAN199-MGMT" "ORSIA-VLAN101-LAN"
                set health-check "Google"
                set priority-members 2
            next
        end
    end

     
    The problem is on wan2 only. It does not matter additional wan1 is connected or not. Wan1 alone worked fine too.
    Wan2 is down for minutes (last time yesterday 16 minutes, 27 minutes, 30 minutes), sometimes only for a while.
    If I restart Fortigate, wan2 goes up.

    Attached Image(s)

    #6
    Dave Hall
    Expert Member
    • Total Posts : 1548
    • Scores: 167
    • Reward points: 0
    • Joined: 2012/05/11 07:55:58
    • Location: Canada
    • Status: offline
    Re: Fortigate 60E - dropping internet connection 2019/10/21 07:25:11 (permalink)
    0
    If WAN2 goes down are you able to ping WAN2's GW address (or the ISP's modem/route device) from/through WAN1 connection and/or from another location on the Internet? (e.g. http://www.kloth.net/services/).  I am speculating the ping server settings in the health-check section may need to be tweaked (i.e. perhaps set the interval higher than 10.)
     
     
     

    NSE4/FMG-VM64/FortiAnalyzer-VM/5.4/6.0 (FWF40C/FW92D/FGT200D/FGT101E)/ FAP220B/221C
    #7
    pgregor@orsia.cz
    New Member
    • Total Posts : 4
    • Scores: 0
    • Reward points: 0
    • Joined: 2019/10/09 06:57:16
    • Status: offline
    Re: Fortigate 60E - dropping internet connection 2019/10/21 11:57:18 (permalink)
    0
    If WAN2 goes down, WAN2's GW ping is not accessible (from intranet). From the Internet is not accessible our public IP which is set on WAN2.
    I've already tried to disable ping or set it to 10, no change.
    I think it could be caused by our ISP. For the past few days we have had twice the speed of wan2 (better line?) and there are no downs yet. We will see.
    #8
    Jump to:
    © 2019 APG vNext Commercial Version 5.5