exchange server traffic problem with utm enabled

[fortinetuser2020]

topology :

2 x fortigate 100d with 5.6.2

load balanced vip leading to 2 exchange 2016 server for rpc traffic (port 443)

on the incoming vip policy there are av/ips/waf/proxy/ssl for single server, full inspection

when the utm engines are on, the traffic flows very poorly in speed and continuity. if i turn them off, everything fly perfect.

i've narrowed it down to just the ssl and proxy (one cannot live without the other of course)

if i have just those 2 on, the problem occurs. so that tells me that the problem is not at the av/ips/waf engines

the proxy is in default mode, no special settings, rpc over http disabled (enabling doesn't help), no large file limitations

the ssl profile is a single server protection, with the same certificate installed on the servers themselfs

any ideas what to look for?

[packetpusher]

What volume of ingress & egress traffic traverses the rule during normal operations? How many existing & new created sessions do you see?

[fortinetuser2020]

it's hard to keep track because all of the traffic

why is that important? please educate me :)

[packetpusher]

It's called "The ‘Black Art’ of FortiGate Sizing". Depending on what your UTM setup look like, amount of traffic traversing the Exchange server policy, the overall number of policies, whether you utilize flow or proxy-based method, and etc. you will get a certain performance behavior.

"the proxy is in default mode, no special settings, rpc over http disabled (enabling doesn't help), no large file limitations the ssl profile is a single server protection, with the same certificate installed on the servers themselfs"

From the above, it appears that you covered some of the topics. Try to disable, one UTM profile at a time while recording the output of the following diagnostic commands:

get system performance statusget system performance firewall statistics 

di sys top-summary

di sys top-summary "-s mem"

 As a result, in order to conserve firewall resources, you may need to fine-tune your security profiles, session timeouts, DNS cache, FortiGuard cache TTL, and etc.

[fortinetuser2020]

thank you

as i said, i've narrowed it down to just the proxy and the ssl

but those profiles doesn't contain much options to change

also, the firewall is not very busy in terms of memory, so i don't think it's a bottleneck issue

[packetpusher]

Is this a new install? Was it working before? What did change? When you say "ssl profile is a single server protection" can you share the profile configuration?

[fortinetuser2020]

hi

thank you

i've seperated the VIPS to 3 different ones : one for smtp, one for tls and one for rpc

for each one i've created it's own proxy profile handling only it's port 

it looks better now, i'll keep track of it

thank you all