URL Filter ignoring regexp rule?

[sw2090]

I have the following. A FGT with a policy for internet access that has webfiter and ssl inspection enabled . It runs v5.6.10.

Webfilter is set to allow some Fortiguard (sub)cathegory and not allow the rest. 

Additionally theres web rating overrides and for things man cannot cope with a rating override it also includes a web url filter.

This has several working rules for urls.

Now I applied a new rule for https://www.icloud.com/sharedalbum/de-de/#xxxxxxxxx (where xxxxxxxxxx is the album id) as I don't want to allow the complete icloud. Since the Web Url Filer does not accept url that have a "#" in them (even though that's an allowd character in http at least since 1.1) I had to enter that as regexp. 

So I set up a regexp rule in url filter with this: .*www\.icloud\.com\/sharedalbum\/de-de\/#xxxxxxxxxx.* 

I checked this in a regexp checke online and it said it matches the above url with and without https:// . The action for this rule is set to exempt.

Thus my FGT keeps blocking https://www.icloud.com/sharedalbum/ by cathegory and seems to ignore the regexp in url filter.

Does anyone have some hint for me to find why this happens?

[Dave_Hall]

If you have log security events enabled, check the web filter log for possible reasons why it is still blocked. 

Do keep in mind that web filter rules are processed from top-to-down.  so put your URL ".*www\.icloud\.com\/sharedalbum\/de-de\/#xxxxxxxxxx.*" above any other URL filter blocking www.icloud.com.

Keep in mind that www.icloud.com is hosting on virtual content servers, so it may be possible that those other host/FQDN are being blocked - I do advise monitoring/drilling down to the connection sessions to see what is actual happening.

[sw2090]

there is no other url filter rule blocking icloud.com.

Webfilter log states it is blocking icloud.com/sharedalbum/de-DE/ due to cathegory which brought me to the conclusion that me regexp is ignored even though it should match.

Probably canonical name could be too...

[sw2090]

Well as I found out now with TAC it is not really beeing ignored ;)

There is just a limitation by fortinet: as TAC explained to me FortiOS only interprets the part from bginning tu the last slash as a url and ignores the res. Due to this my regexp never matched.

This means that it is not possible to filter certain specific urls. 

You could e.g. exempt icloud.com/sharedalbum/de-de but not icloud.com/sharedalbum/de-de/#abcdefgh .

Additionally FortiManager will not even allow you to enter a url with a "#" in it into url filter while the FortiGate's gui does allow it.