Helpful ReplyHot!URL filter regexp

Author
figge
New Member
  • Total Posts : 17
  • Scores: 0
  • Reward points: 0
  • Joined: 2017/03/02 04:46:30
  • Status: offline
2019/01/23 06:33:56 (permalink) 5.6
0

URL filter regexp

Have the way the Fortigate interprets regular expressions changed between FortiOS release 5.4 an 5.6 ???
 
\.ru\b  worked in 5.4 to match all domains under the top-domain .ru, it does not work in 5.6.4
 
if I add \.ru\b  in the Static URL filter via GUI the regexp as show in the CLI is \\.ru\\b and it does not match
#1
Dave Hall
Expert Member
  • Total Posts : 1457
  • Scores: 160
  • Reward points: 0
  • Joined: 2012/05/11 07:55:58
  • Location: Canada
  • Status: offline
Re: URL filter regexp 2019/01/23 07:28:28 (permalink)
0
I'm pretty sure this is an intended design - I know it was/is pointed out in the old 4.0.x documentation, that the fgt will insert a \ prefix in url expression before a \ character - you just don't see it in the GUI.  Perhaps your issue is elsewhere - can you provide more info?
 
Edit: see https://help.fortinet.com/fos50hlp/56/Content/FortiOS/fortigate-security-profiles/Other_Profile_Considerations/Using%20wildcards%20and%20Perl%20regular%20expressions.htm?Highlight=expression
post edited by Dave Hall - 2019/01/23 07:44:34

Attached Image(s)


NSE4/FMG-VM64/FortiAnalyzer-VM/5.4/6.0 (FWF40C/FW92D/FGT200D/FGT101E)/ FAP220B/221C
#2
figge
New Member
  • Total Posts : 17
  • Scores: 0
  • Reward points: 0
  • Joined: 2017/03/02 04:46:30
  • Status: offline
Re: URL filter regexp 2019/01/24 01:28:00 (permalink)
0
I have two fortigate, one running 5.4.8 and one running 5.6.4.
The first one (FortiOS 5.4.8) have a VDOM in proxy mode and I use the IPv4 Policy to apply a Webfilter + static URL filter to the traffic. The URL filter have a policy  \.ru\b  Block Enable and it works as intended blocking the top domain ru.
 
The second one (FortiOS 5.6.4) have no VDOMs, runs in proxy mode and I use the Explicit Proxy Policy to apply a Webfilter + static URL. The URL filter have a policy  \.ru\b  Block Enable and it does not work, I have tested many different regexp patterns. The interesting thing though is that using exactly the same regexp as above gives a different result e.g. no match for www.google.ru .and other .ru sites
post edited by figge - 2019/01/24 01:31:57
#3
Dave Hall
Expert Member
  • Total Posts : 1457
  • Scores: 160
  • Reward points: 0
  • Joined: 2012/05/11 07:55:58
  • Location: Canada
  • Status: offline
Re: URL filter regexp 2019/01/24 09:28:03 (permalink)
0
If explicit proxy is setup on the second fgt, are you sure it is working properly?  Have you forced (manual set a proxy server setting) on a client browser?

NSE4/FMG-VM64/FortiAnalyzer-VM/5.4/6.0 (FWF40C/FW92D/FGT200D/FGT101E)/ FAP220B/221C
#4
figge
New Member
  • Total Posts : 17
  • Scores: 0
  • Reward points: 0
  • Joined: 2017/03/02 04:46:30
  • Status: offline
Re: URL filter regexp 2019/01/24 23:06:32 (permalink)
0
Yes, everything works correctly; AV, Webfilter, App-filter, Deep SSL-inspection - no problems.
I use PAC file and have no problem what so ever with that. Forced proxy setting make no difference.
So the logical conclusion is that handling of regular expressions must have changed. I can test the two systems side by side and get different results from the same URL filter regexp.
 
#5
baggins
New Member
  • Total Posts : 11
  • Scores: 0
  • Reward points: 0
  • Joined: 2018/07/12 06:06:20
  • Status: offline
Re: URL filter regexp 2019/01/25 01:07:46 (permalink)
0
Hi,
 
I know it's a long shot but can you try only with .ru
 
so create new url filter and set to regex + block and enter only .ru and try.
 
 
#6
figge
New Member
  • Total Posts : 17
  • Scores: 0
  • Reward points: 0
  • Joined: 2017/03/02 04:46:30
  • Status: offline
Re: URL filter regexp 2019/01/25 01:35:35 (permalink)
0
That regexp surely blocks all .ru sites, but it also blocks
sites like www.rum.se  and I dont want that.
#7
baggins
New Member
  • Total Posts : 11
  • Scores: 0
  • Reward points: 0
  • Joined: 2018/07/12 06:06:20
  • Status: offline
Re: URL filter regexp 2019/01/25 02:17:23 (permalink) ☄ Helpfulby figge 2019/01/28 00:29:14
0
right..
 
Just checked with wildcard option and *.ru/ and it works.
I'm on 6.0.3..
Site that you mentioned works "www.rum.se" and went trough list here on few and all blocked.
try..
#8
figge
New Member
  • Total Posts : 17
  • Scores: 0
  • Reward points: 0
  • Joined: 2017/03/02 04:46:30
  • Status: offline
Re: URL filter regexp 2019/01/28 00:33:10 (permalink)
0
Thanks, the wildcard option solves this particular use case.
I am still curious as to why the very same regexp gives different result in the two Fortigates though.
I have opened a TAC-case with this question. 
#9
baggins
New Member
  • Total Posts : 11
  • Scores: 0
  • Reward points: 0
  • Joined: 2018/07/12 06:06:20
  • Status: offline
Re: URL filter regexp 2019/01/28 00:38:49 (permalink)
0
Hi figge,
 
glad that this helps..
Do you have same fw version on both?
#10
figge
New Member
  • Total Posts : 17
  • Scores: 0
  • Reward points: 0
  • Joined: 2017/03/02 04:46:30
  • Status: offline
Re: URL filter regexp 2019/01/28 00:49:56 (permalink)
0
As I wrote earlier:
I have two fortigate, one running 5.4.8 and one running 5.6.4.
In the first one (FortiOS 5.4.8) regexp works as intended. 
In the second one (FortiOS 5.6.4) regexp does not match
 
The interesting thing is that the same regexp gives a different results in the two units
even though it is applied in identical policys.
#11
baggins
New Member
  • Total Posts : 11
  • Scores: 0
  • Reward points: 0
  • Joined: 2018/07/12 06:06:20
  • Status: offline
Re: URL filter regexp 2019/01/28 01:11:43 (permalink)
0
yap...
 
I guess they "upgraded" the syntax.. :)
Let me know if you get anything from the TAC
#12
figge
New Member
  • Total Posts : 17
  • Scores: 0
  • Reward points: 0
  • Joined: 2017/03/02 04:46:30
  • Status: offline
Re: URL filter regexp 2019/01/29 05:06:14 (permalink)
0
Apparently there is a bug in parsing certain regexps, learned about this in the response to my TAC-case.
#13
Jump to:
© 2019 APG vNext Commercial Version 5.5