Helpful ReplyHot!Internet Connection Speed

Author
Brent
New Member
  • Total Posts : 14
  • Scores: 0
  • Reward points: 0
  • Joined: 2017/11/26 18:13:46
  • Status: offline
2017/11/26 18:32:24 (permalink)
0

Internet Connection Speed

Hi, 
 
First time poster here, so hopefully I got the most appropriate thread.
 
I have just purchased a Fortigate 600c firewall to route my home office network to the internet.  I have two ISPs and am on a 1Gb plan with both of them.
 
When I connect the basic routers that were provided from each ISP I get close to what is promised 900 Mb+.  However, when I run my connections through the Fortigate, I am only getting circa 500 Mb.  (As tested using Speedtest.net)
 
I previously had a Fortigate 200b, which gave me the same speeds, and while investigating I noted that the CPU of the fortigate was maxing out.  Actually, initially I was getting slower speeds but after turning off logging it increased to around the 500 mark.
 
I thought okay, the CPU isn't powerful enough to handle what I wanted, so I kept an eye out for a second hand later model when I found the 600c being sold locally.  Checking the specs on the 600c and seeing that it has 2.5 Gbps IPS I thought this would be powerful enough to give me closer to the 1 Gbps speeds.
 
Is there any way I can tweak the fortigate to get better performance, or does anyone have any ideas as to why I am not getting the speeds as advertised?
 
I'm running FortiOS 5.4, not using WAN LLB (as I require VPN).  Other than this the Fortigate is functioning as I require;
* Seperate Network for myself and my flatmates (I can't have them accessing the VPN to work)
* Traffic routed through specific ISP for specific tasks (Mail through one, other traffic through another)
* Reverse Proxy
 
Thanks
 
Brent
 
#1
packetpusher
Silver Member
  • Total Posts : 85
  • Scores: 0
  • Reward points: 0
  • Joined: 2016/07/18 08:44:14
  • Status: offline
Re: Internet Connection Speed 2017/11/28 08:46:10 (permalink)
0
You can start with setting a baseline. 
 
"Is there any way I can tweak the fortigate to get better performance, or does anyone have any ideas as to why I am not getting the speeds as advertised?"
There is way, we just need to know what is the root cause for the described issue. For example, CPU & high memory utilization.
#2
Brent
New Member
  • Total Posts : 14
  • Scores: 0
  • Reward points: 0
  • Joined: 2017/11/26 18:13:46
  • Status: offline
Re: Internet Connection Speed 2017/11/28 18:42:37 (permalink)
0
Thanks for your reply.
 
I'm not sure what you mean by setting a baseline.  I know what a baseline is, but I'm not sure how it applies here, can you give me more information about what you mean?
 
Regarding the CPU and Memory utilization, I did forget to mention that once I switched to the 600c the memory and cpu utilization were not excessive.
 
CPU Usage: 12%
Memory Usage : 19 %
 
#3
packetpusher
Silver Member
  • Total Posts : 85
  • Scores: 0
  • Reward points: 0
  • Joined: 2016/07/18 08:44:14
  • Status: offline
Re: Internet Connection Speed 2017/11/29 09:10:14 (permalink)
0
I wanted to ask you to start recording the values of each test and the corresponding CPU and Memory utilization. 
#4
Brent
New Member
  • Total Posts : 14
  • Scores: 0
  • Reward points: 0
  • Joined: 2017/11/26 18:13:46
  • Status: offline
Re: Internet Connection Speed 2017/11/29 14:51:48 (permalink)
0
Okay, 
 
So I have done multiple tests, including running a test on the same device multiple times, on multiple devices at the same time.  All using speedtest.net to the ISP's Speedtest server.  On average I get between 500 and 650 Mbs download, CPU Usage doesn't exceed 10-13%, and Memory Usage remains constant at 20%.
 
Does that help?
 
#5
packetpusher
Silver Member
  • Total Posts : 85
  • Scores: 0
  • Reward points: 0
  • Joined: 2016/07/18 08:44:14
  • Status: offline
Re: Internet Connection Speed 2017/11/30 11:10:36 (permalink)
0
Did you check the speed & duplex? Any errors? Cables are Cat5e and above?
#6
packetpusher
Silver Member
  • Total Posts : 85
  • Scores: 0
  • Reward points: 0
  • Joined: 2016/07/18 08:44:14
  • Status: offline
Re: Internet Connection Speed 2017/11/30 11:12:21 (permalink)
0
BTW, is this a broadband connection? Also, is the firewall in production whereas during the test there is a inbound/outbound  traffic originating from the WAN or LAN?
 
#7
Brent
New Member
  • Total Posts : 14
  • Scores: 0
  • Reward points: 0
  • Joined: 2017/11/26 18:13:46
  • Status: offline
Re: Internet Connection Speed 2017/11/30 14:31:04 (permalink)
0
mstoyanoff
Did you check the speed & duplex? Any errors? Cables are Cat5e and above?



As per the original post, if I swap the fortigate with the ISP supplied router, I get speeds in excess of 900Mbs.  This is using the same infrastructure (cables, switches, devices).  Are there any speed and duplex settings on the Fortigate? I can't see any.
 
mstoyanoff
BTW, is this a broadband connection? Also, is the firewall in production whereas during the test there is a inbound/outbound  traffic originating from the WAN or LAN?

 
This is a Fibre connection.  No, this is not in production, there is no other traffic originating from the WAN or LAN during testing.
 
Thanks
#8
rwpatterson
Expert Member
  • Total Posts : 8078
  • Scores: 163
  • Reward points: 0
  • Joined: 2006/08/08 10:08:18
  • Location: Long Island, New York, USA
  • Status: online
Re: Internet Connection Speed 2017/11/30 20:07:26 (permalink)
0
Have you tried a simple PING test with various packet sizes? May be a fragmentation issue.

-Bob - self proclaimed posting junkie!
See my Fortigate related scripts at: http://fortigate.camerabob.com

-4.3.18-b0689
FGT60B
FWF60B
FWF80CM (2)
FWF81CM
 
#9
Brent
New Member
  • Total Posts : 14
  • Scores: 0
  • Reward points: 0
  • Joined: 2017/11/26 18:13:46
  • Status: offline
Re: Internet Connection Speed 2017/11/30 22:49:39 (permalink)
0
rwpatterson
Have you tried a simple PING test with various packet sizes? May be a fragmentation issue.




How would I go about doing that?  I'm afraid I'm not a network expert and have only ever used ping to confirm connection.
#10
rwpatterson
Expert Member
  • Total Posts : 8078
  • Scores: 163
  • Reward points: 0
  • Joined: 2006/08/08 10:08:18
  • Location: Long Island, New York, USA
  • Status: online
Re: Internet Connection Speed 2017/12/01 08:45:10 (permalink) ☄ Helpfulby Brent 2017/12/02 15:02:05
0
From a Windows work station:
Get to the command prompt ("CMD" from the start box/globe thing)
In the open window, type:
C:\windows\system32> ping -f -l <packet size> <IP address>
 
The Ethernet packet size on the WAN maxes out at 1500, so start there and decrease until you get a valid response. The "-f" tells the network to not fragment the packets. This will prevent the ping from happening until the entire PING packet can be transmitted. When you get this number, set this on your WAN interface in the GUI. This will then enable communications over the Internet without either end having to break down and reassemble the traffic to traverse the Internet.
 
Hope that helps.

-Bob - self proclaimed posting junkie!
See my Fortigate related scripts at: http://fortigate.camerabob.com

-4.3.18-b0689
FGT60B
FWF60B
FWF80CM (2)
FWF81CM
 
#11
tanr
Gold Member
  • Total Posts : 409
  • Scores: 14
  • Reward points: 0
  • Joined: 2016/05/09 17:09:43
  • Status: offline
Re: Internet Connection Speed 2017/12/01 21:20:34 (permalink)
0
Also, what UTM features do you have turned on, and are you running in flow-based or proxy-based mode?
 
Just want to make sure that you're not killing your perf with certain UTM features.  For example, the 600C specs only show 400 Mbps throughput for proxy mode antivirus, probably without the extended database.
#12
Brent
New Member
  • Total Posts : 14
  • Scores: 0
  • Reward points: 0
  • Joined: 2017/11/26 18:13:46
  • Status: offline
Re: Internet Connection Speed 2017/12/02 15:01:53 (permalink)
0
rwpatterson
From a Windows work station:
Get to the command prompt ("CMD" from the start box/globe thing)
In the open window, type:
C:\windows\system32> ping -f -l <packet size> <IP address>
 
The Ethernet packet size on the WAN maxes out at 1500, so start there and decrease until you get a valid response. The "-f" tells the network to not fragment the packets. This will prevent the ping from happening until the entire PING packet can be transmitted. When you get this number, set this on your WAN interface in the GUI. This will then enable communications over the Internet without either end having to break down and reassemble the traffic to traverse the Internet.
 
Hope that helps.




I'm not 100% sure that has resolved the issue, but it certainly has done something as I'm not consistently getting around 500Mbps, I'm now getting closer to 700Mbps when I test after first opening a browser window, then on subsequent tests I get higher, I've even seen 900, but I assume that is some sort of caching in the browser as I don't see that on the first test.
 
Any other tips - I liked that one, but would still like to see it closer to what I'm paying for.
 
PS. I couldn't find the MTU settings in the GUI.  So I set it via the CLI.  How do you set it via the GUI?
 
Thanks
#13
Brent
New Member
  • Total Posts : 14
  • Scores: 0
  • Reward points: 0
  • Joined: 2017/11/26 18:13:46
  • Status: offline
Re: Internet Connection Speed 2017/12/02 15:20:25 (permalink)
0
tanr
Also, what UTM features do you have turned on, and are you running in flow-based or proxy-based mode?
 
Just want to make sure that you're not killing your perf with certain UTM features.  For example, the 600C specs only show 400 Mbps throughput for proxy mode antivirus, probably without the extended database.




I'm in proxy-based mode, though I have tried it in both as I'm not sure I understand the difference.
 
I don't have any UTM features enabled as the Fortigate is second hand and the licenses have expired.  However, that being said, I had the web filter disabled in the interface, but the feature was still enabled.  After disabling the feature, the speed seemed to improve slightly.  Is that something you would expect?
 
 
#14
tanr
Gold Member
  • Total Posts : 409
  • Scores: 14
  • Reward points: 0
  • Joined: 2016/05/09 17:09:43
  • Status: offline
Re: Internet Connection Speed 2017/12/02 16:57:53 (permalink)
0
In general, flow mode is supposed to be a little faster than proxy mode.  It really depends on what sort of traffic you have.  Useful overview is in Life of a Packet https://docs.fortinet.com/d/fortigate-life-of-a-packet-5.4
 
If you're not using any UTM features in your security policies I don't think turning the features on or off will a big difference.  Others with more complete knowledge may know better than me, though.
 
BTW, before you spend too much time testing, is your ISP guaranteeing the 1Gbps speed?  I know in my area residential 1Gbps is usually nowhere near its advertised speed and can vary wildly, whereas my 1Gbps fiber business line has (supposedly) guaranteed levels of bandwidth and uptime.  And even with that I see more variation than I'd like, especially outside of normal business hours.
#15
Brent
New Member
  • Total Posts : 14
  • Scores: 0
  • Reward points: 0
  • Joined: 2017/11/26 18:13:46
  • Status: offline
Re: Internet Connection Speed 2017/12/03 00:33:55 (permalink)
0
tanr
 
BTW, before you spend too much time testing, is your ISP guaranteeing the 1Gbps speed?  I know in my area residential 1Gbps is usually nowhere near its advertised speed and can vary wildly, whereas my 1Gbps fiber business line has (supposedly) guaranteed levels of bandwidth and uptime.  And even with that I see more variation than I'd like, especially outside of normal business hours.




No, they don't guarantee it, but when a cheap little give-away router that the ISP provided can give a Speedtest result of 900+ I would have thought a unit such as the Fortigate 600c should be able to match it.  I am testing at different times during the day to eliminate shared usage on the line.
 
I'd be happy to consistently see 900+ to my ISP's speedtest server, I'm now (after the suggestions above) reasonably constant around 800+ which is up from circa 500.  So I'm not unhappy.
 
#16
ede_pfau
Expert Member
  • Total Posts : 5315
  • Scores: 339
  • Reward points: 0
  • Joined: 2004/03/09 01:20:18
  • Location: Heidelberg, Germany
  • Status: offline
Re: Internet Connection Speed 2017/12/03 04:47:51 (permalink) ☄ Helpfulby Brent 2017/12/07 15:06:03
0
I would have checked first if this phenomenon is connected to the WAN port and/or protocols.
Please run a performance test between 2 hosts on your LAN, both with GbE ports, AV disabled. I usually use iperf for this as the same exe-file contains the server and the client.
You should see wirespeed here.
 
If you don't there is a problem within FortiOS such as the traffic offloading to the network ASIC (NP) being disabled. Have you reset the unit after purchase to factory defaults ("exec factoryreset") before configuring it? Beware that this command will delete all of the config.
 
If you do see wirespeed between LAN ports I would investigate the WAN protocol used. How do you connect to your IPS(s)? DHCP, PPPoE, static IP? There is a known gotcha with PPPoE processing in FortiOS. Cheap but dedicated WAN routers use a special chip to handle the protocol, FortiOS doesn't. It might well suck up the CPU performance if the WAN line speed is in excess of 100 Mbps for desktop models, higher speeds for multi-core FGTs like yours.
 
The 600C itself is very capable, with decent memory size, content ASIC (CP) and network ASIC (NP) for offloading chores off the CPU. CPU will mostly handle session setup, negotiations (IPsec, SSLVPN, PPPoE, DHCP), logging and GUI. Plus some more but will usually stay out of the way of running session traffic. That's why you would expect wirespeed performance on GbE ports. IMHO the specs on the datasheet come quite close to realworld figures.
 
Lastly, if you're running FOS v5.4 do update to the latest build (v5.4.6). Each patch version will fix some bugs and possibly improve throughput (while adding features which introduce more bugs...). For a used unit without contract this will be, hm, difficult but you may well ask FTNT for a contract. FortiCare will do (firmware updates, warranty extension) but of course FortiGuard would be more beneficial for you (AV, IPS, botnet IP blacklist, webfilter,...). If such contracts are no longer available from the regular price list you could ask FTNT for a 'coterm quote'.
 
Happy testing!

Ede

" Kernel panic: Aiee, killing interrupt handler!"
#17
Brent
New Member
  • Total Posts : 14
  • Scores: 0
  • Reward points: 0
  • Joined: 2017/11/26 18:13:46
  • Status: offline
Re: Internet Connection Speed 2017/12/07 15:18:41 (permalink)
0
ede_pfau
I would have checked first if this phenomenon is connected to the WAN port and/or protocols.
Please run a performance test between 2 hosts on your LAN, both with GbE ports, AV disabled. I usually use iperf for this as the same exe-file contains the server and the client.
You should see wirespeed here.
 
If you don't there is a problem within FortiOS such as the traffic offloading to the network ASIC (NP) being disabled. Have you reset the unit after purchase to factory defaults ("exec factoryreset") before configuring it? Beware that this command will delete all of the config.
 
If you do see wirespeed between LAN ports I would investigate the WAN protocol used. How do you connect to your IPS(s)? DHCP, PPPoE, static IP? There is a known gotcha with PPPoE processing in FortiOS. Cheap but dedicated WAN routers use a special chip to handle the protocol, FortiOS doesn't. It might well suck up the CPU performance if the WAN line speed is in excess of 100 Mbps for desktop models, higher speeds for multi-core FGTs like yours.
 
The 600C itself is very capable, with decent memory size, content ASIC (CP) and network ASIC (NP) for offloading chores off the CPU. CPU will mostly handle session setup, negotiations (IPsec, SSLVPN, PPPoE, DHCP), logging and GUI. Plus some more but will usually stay out of the way of running session traffic. That's why you would expect wirespeed performance on GbE ports. IMHO the specs on the datasheet come quite close to realworld figures.
 
Lastly, if you're running FOS v5.4 do update to the latest build (v5.4.6). Each patch version will fix some bugs and possibly improve throughput (while adding features which introduce more bugs...). For a used unit without contract this will be, hm, difficult but you may well ask FTNT for a contract. FortiCare will do (firmware updates, warranty extension) but of course FortiGuard would be more beneficial for you (AV, IPS, botnet IP blacklist, webfilter,...). If such contracts are no longer available from the regular price list you could ask FTNT for a 'coterm quote'.
 
Happy testing!




So I connected a laptop to my desktop computer via a $30 gigabit switch and IPerf gave me 950Mbs, but then when I connected the laptop to the otherside of the fortigate (same network so no firewall rules etc), it dropped to 850Mbs (haven't ruled out the cable).  If I connect it to my other network it drops even further to 750Mbs.  Though on saying that, I have also had speedtest.net return 950Mbs after making these changes, and it is commonly in the high 800s.  So I'm reasonably happy now.
 
Though I am curious as to why the local network drops when traversing the fortigate
 
In answer to your other questions, yes, I did a factory reset.  I'm running FortiOS 5.4.0, I would love to update to 5.4.6, but sadly the FortiCare contract is over $1000 which I'm just not willing to pay (nor can I afford to) :(
#18
Jump to:
© 2018 APG vNext Commercial Version 5.5