AnsweredHot!Several problems high memory and cpu usage blocking WAN connection after upgrade to 6.2

Page: 12 > Showing page 1 of 2
Author
gcraenen
New Member
  • Total Posts : 7
  • Scores: 0
  • Reward points: 0
  • Joined: 2018/09/08 01:48:51
  • Status: offline
2019/04/26 01:08:41 (permalink) 6.2
0

Several problems high memory and cpu usage blocking WAN connection after upgrade to 6.2

Hi,
 
After upgrading from 6.0.4 to 6.2 I have problems with WAN connectivity falling out. I'm getting this message in de Fortios GUI:
 
Conserve mode activated due to high memory usage
 
I have tried to downgrade to 6.0.4 but can't with the error message that it failed beacause it cannot download the file from fortiguard. Help.
#1
SMabille
Silver Member
  • Total Posts : 71
  • Scores: 18
  • Reward points: 0
  • Joined: 2013/03/31 15:39:51
  • Status: offline
Re: Several problems high memory and cpu usage blocking WAN connection after upgrade to 6. 2019/04/26 01:12:51 (permalink) ☼ Best Answerby gcraenen 2019/04/26 02:23:45
0
Hi,
 
Download the image from the Fortinet support website and upload/apply it from your browser.
 
Best regards,
Stephane
#2
gcraenen
New Member
  • Total Posts : 7
  • Scores: 0
  • Reward points: 0
  • Joined: 2018/09/08 01:48:51
  • Status: offline
Re: Several problems high memory and cpu usage blocking WAN connection after upgrade to 6. 2019/04/26 02:25:51 (permalink)
0
Thanks, that worked and everything is working ok again. Fortios 6.2 absolutely is not ready for our production environment.
#3
nomeursy
New Member
  • Total Posts : 1
  • Scores: 0
  • Reward points: 0
  • Joined: 2019/05/02 00:11:47
  • Status: offline
Re: Several problems high memory and cpu usage blocking WAN connection after upgrade to 6. 2019/05/02 00:18:12 (permalink)
0
Same problem here. After upgrade a Fortigate 30E, from 6.0.4 to 6.2.0, average MEM usage went from 65% to 75%, causing the Fortigate to go in and out of "Conserve mode". SSL-VPN does not except connections and WAN traffic is blocked several times a day. Downgrading back to 6.0.4 solved the problem.
#4
Andy Bailey
Bronze Member
  • Total Posts : 54
  • Scores: 4
  • Reward points: 0
  • Joined: 2016/06/27 11:21:22
  • Status: online
Re: Several problems high memory and cpu usage blocking WAN connection after upgrade to 6. 2019/05/02 01:56:37 (permalink)
0
Same problem here too.
 
I have a fairly lightly loaded 60E running 6.2 (no more than 800 sessions maximum) and have had at least one "Conserve Mode" event a week since the upgrade.
 
For me it's not too much of a problem- but certainly indicates 6.2 has some issues with memory management.
 
Kind Regards,
 
 
Andy.
 
 
#5
rkhair
Bronze Member
  • Total Posts : 23
  • Scores: 0
  • Reward points: 0
  • Joined: 2004/12/02 07:18:50
  • Status: offline
Re: Several problems high memory and cpu usage blocking WAN connection after upgrade to 6. 2019/05/05 22:26:10 (permalink)
0
We are getting same thing, memory is at about 36% when rebooted, then creeps up slowly every day untill the FW goes into conserve mode, so we have to reboot it and go through the cycle again. Hopefully they fix the memory leak fast.
#6
thuynh_FTNT
Bronze Member
  • Total Posts : 44
  • Scores: -2
  • Reward points: 0
  • Joined: 2014/02/05 09:30:09
  • Status: offline
Re: Several problems high memory and cpu usage blocking WAN connection after upgrade to 6. 2019/05/06 14:39:10 (permalink)
0
Hi all, thank you for reporting the issue. For each of your cases, can you check which process is taking up the memory when the FGT goes into conserve mode? "diagnose sys top"

There is a known memory issue with IPS engine which is fixed in FortiOS 6.2.1
#7
rkhair
Bronze Member
  • Total Posts : 23
  • Scores: 0
  • Reward points: 0
  • Joined: 2004/12/02 07:18:50
  • Status: offline
Re: Several problems high memory and cpu usage blocking WAN connection after upgrade to 6. 2019/05/07 00:21:40 (permalink)
0
Is there any way to get 6.2.1 or at least an ips engine to fix it till 6.2.1 is officially out?
#8
Break16
New Member
  • Total Posts : 2
  • Scores: 0
  • Reward points: 0
  • Joined: 2019/05/14 00:22:09
  • Status: offline
Re: Several problems high memory and cpu usage blocking WAN connection after upgrade to 6. 2019/05/14 00:24:49 (permalink)
0
Hello 
 
same problem here. my company stopped working for nearly 2 hours because we could not find the problem until I arrived in the office to check the firewall ... after getting access to the firewall I saw the error: "Conserve mode activated due to high memory usage"
 
can forginet fortinet/fortigate people help please? why we have to wait so long time to get this fixed?
 
user gcraenen reported this 3 weeks ago!
#9
rkhair
Bronze Member
  • Total Posts : 23
  • Scores: 0
  • Reward points: 0
  • Joined: 2004/12/02 07:18:50
  • Status: offline
Re: Several problems high memory and cpu usage blocking WAN connection after upgrade to 6. 2019/05/14 00:35:55 (permalink)
0
i have an open support ticket with them... its a problem with the ips monitor.. temp fix at the moment is to run this command in a CLI:
 
diagnose test application ipsmonitor 99 
 
doing that restarts the ips monitor and the mem usage drops all the way back down again, they said it will be fixed in 6.2.1.
#10
Break16
New Member
  • Total Posts : 2
  • Scores: 0
  • Reward points: 0
  • Joined: 2019/05/14 00:22:09
  • Status: offline
Re: Several problems high memory and cpu usage blocking WAN connection after upgrade to 6. 2019/05/22 01:21:36 (permalink)
0
still no fix? this happened today again! bug was reported over 4 weeks ago !!! what a bad service.
#11
Andy Bailey
Bronze Member
  • Total Posts : 54
  • Scores: 4
  • Reward points: 0
  • Joined: 2016/06/27 11:21:22
  • Status: online
Re: Several problems high memory and cpu usage blocking WAN connection after upgrade to 6. 2019/05/22 05:37:40 (permalink)
0
Hello,
 
If it's any help I've raised a ticket and had confirmation it is a known issue due to be resolved in 6.2.1 (as others have reported).
 
The release date of 6.2.1 (I'm told) is late May/ early June so perhaps a couple of weeks away.
 
In the meantime as rkhair suggested the work around "diagnose test application ipsmonitor 99" does seem to be work well and support stated it was a valid work around.
 
I have set up an "automation stitch" to run this command hourly (which seems ok for my 60E, you may prefer different time intervals) and that seems to be working well. It does spike CPU usage- so perhaps needs some thought/ review before using in a production network.
 
Essentially the "automation stitch" just executes that command as a script regularly to keep the memory issues under control.
 
That might be a useful way to keep things under control until the 6.2.1 release is available.
 
Hope that's useful.
 
Kind Regards,
 
 
Andy.
 
 
 
#12
thuynh_FTNT
Bronze Member
  • Total Posts : 44
  • Scores: -2
  • Reward points: 0
  • Joined: 2014/02/05 09:30:09
  • Status: offline
Re: Several problems high memory and cpu usage blocking WAN connection after upgrade to 6. 2019/05/22 17:02:03 (permalink)
0
Thanks Andy/rkhair. That is very helpful.

Hi Break16, sorry about the inconvenience. Like Andy said, we already identified the root cause and added a fix as soon as it's reported. However, you will need to upgrade the firmware to FortiOS 6.2.1 (scheduled to release soon) to pick up the fix. For now please use the work around per Andy's suggestion. 
#13
anujdalal
New Member
  • Total Posts : 2
  • Scores: 0
  • Reward points: 0
  • Joined: 2018/08/17 06:11:52
  • Status: offline
Re: Several problems high memory and cpu usage blocking WAN connection after upgrade to 6. 2019/06/12 09:41:17 (permalink)
0
Hi,
 
Having the same issue on Azure FGVM02 (memory leak?). I have 2 running in Active/Active HA. I removed one of them from the Azure Loadbalancer back-end pool ("cluster") at 64% memory usage. Even with close to no traffic going through it, the memory usage stayed at 64% constantly. The usage gradually climbs when the ipsengine is in use. diagnose sys top shows ipsengine using lots of memory, and not releasing it.
 
As for downgrading the firmware, I get the same error message, but downloading the image from https://support.fortinet.com/Download/VMImages.aspx seems to do the trick.
 
Thanks.
#14
toren
New Member
  • Total Posts : 1
  • Scores: 0
  • Reward points: 0
  • Status: offline
Re: Several problems high memory and cpu usage blocking WAN connection after upgrade to 6. 2019/06/25 01:31:07 (permalink)
0
Hi,
 
Have the same problem on 61E. I restart FW every 2-3 days as it goes into "Conserve mode". 
 
Just had a call with support and was told to change web filter enabled security policy mode from flow based to proxy-based. That's a workaround till FortiOS 6.2.1 will be released which according to the support will happen early July.
 
Also diagnose test application ipsmonitor 99 can be used instead of restart to drop memory consumption.
#15
mattf
New Member
  • Total Posts : 1
  • Scores: 0
  • Reward points: 0
  • Joined: 2019/06/25 17:49:22
  • Status: offline
Re: Several problems high memory and cpu usage blocking WAN connection after upgrade to 6. 2019/06/25 17:57:41 (permalink)
0
I'm going to post this here in the hope that it helps others while we not so patiently wait for a fix.
 
If none of the above has helped you, work out where the memory leak is on your own fortigates by using the following command.

get sys perf top

On our firewalls, it's actually the WAD process which has the memory leak. There's 4 consecutive processes amounting to 40% of the total memory usage. This is determined by looking at the right most column in the results.
 
Press ctrl + c to stop the "sys perf" report.
 
Use diagnose test application wad 99 to restart the process which is causing your memory leak.
*wad can be interchanged with any other process name which is responsible for the memory usage.

 
#16
marlonjohn
New Member
  • Total Posts : 1
  • Scores: 0
  • Reward points: 0
  • Joined: 2019/06/25 22:03:16
  • Status: offline
Re: Several problems high memory and cpu usage blocking WAN connection after upgrade to 6. 2019/06/25 22:09:42 (permalink)
0
happening to me also after 6.0 to 6.2 Fortigate 60e
#17
jonoarm
New Member
  • Total Posts : 6
  • Scores: 2
  • Reward points: 0
  • Joined: 2015/01/27 07:39:13
  • Status: offline
Re: Several problems high memory and cpu usage blocking WAN connection after upgrade to 6. 2019/06/26 13:10:11 (permalink)
0
Yep, Same here gone back to 6.05
#18
goodj
New Member
  • Total Posts : 1
  • Scores: 0
  • Reward points: 0
  • Joined: 2019/06/27 15:55:53
  • Status: offline
Re: Several problems high memory and cpu usage blocking WAN connection after upgrade to 6. 2019/06/27 15:59:21 (permalink)
0
This happened to me today.   Updated past week to 6.2 from 5.6 on 1500D HA Cluster
#19
lladereche
New Member
  • Total Posts : 1
  • Scores: 2
  • Reward points: 0
  • Joined: 2019/07/01 11:01:30
  • Status: offline
Re: Several problems high memory and cpu usage blocking WAN connection after upgrade to 6. 2019/07/01 11:04:27 (permalink)
5 (1)
I've created this auto-script to restart ipsmonitor every 6 hours:
 
config system auto-script
edit "memoryscript"
set interval 21600
set repeat 0
set start auto
set script "diagnose test application ipsmonitor 99"
next
end
 
Thanks!
#20
Page: 12 > Showing page 1 of 2
Jump to:
© 2019 APG vNext Commercial Version 5.5