Support Forum
The Forums are a place to find answers on a range of Fortinet products from peers and product experts.
cuneyt
New Contributor II

100+ VDOM Design Best Practice

Hi Everyone,

 

We have a Fortigate 3950B and 100+ VDOMs on it. Dont have a fortimanager. I want to ban some malicuos IPs (100+) for all VDOMs and these IPs may change every week. Solutions:

 

1- Go with ACLs on router. (I dont want this solution for some reasons)

2-  Define all IPs on all VDOMs and create a firewall deny policy on all VDOMS. (If you dont die before you finish :))

3- Create a VDOM. Define all IPs and create a firewall deny policy on this VDOM. Route all other VDOMs' trafic to this VDOM.

 

I have not tried 3rd solution yet. Is this a good solution? Why not?

Is there another solution to handle this situation?  

 

Regards

 

 

5 Solutions
emnoc
Esteemed Contributor III

Yes all three proposals will work  and I love your comment about  the  "don't die" if you do a massive  fw.policy  and in 100+ vdoms fw.policies ;)

 

Here's a few better options that you should explore and not ignore

 

1>

If you have a router doing  a dynamic protocol and  the fgt have a means to learn of these routes , do a  route distribute with  the /32 netmask and route to  a blackhole { null  route }. This will allow you  to drop the  /32 hosts in one single swipe. You can  do this static-route on the same  ISP-ACL router that you  mention b4. OSPF/BGP works great for this or do a static route on internet external WAN1 and enable  uRPF

 

{ cisco }  ip route 1.1.1.1 255.255.255.266 null0

 

2>

Now if you  have no access to the router or a different team controls it, you can quickly deploy  the address into the fwpolicys across the 100+ vdom with some  scripting

 

3>

 

or better yet  use the restAPI. You  could  batch job this and upload the configuration into a pre-defined fw.policy.ID ( read more )

 

In one of my recent rule we have 10 vdoms with public access. We maintain a fwpolicy  with  object-group. The policy was at that top of our list and used a deny and bidir, the addrgroup is only modified with the bad actor and we place the single ip.addr into the one  address.group. In this example it's called ITBLACKLISTS, which comprised of  numerous host firewall.address or firewall.addrgrps

 

NOTE: research the max  values in a addrgrp, I believe it 1024 or 1500 entries per group on bigger units , you should be fine in a 3950 unless your blacklist over 1500 addresses. So if you think you  might have   more than 1500   stack address.groups

 

 

e.g

 

config firewall addrgrp

    edit ITBLACKLISTS

      set member BHGRP1 BHGRP2 BHGRP3  OCT182018 SEPT32018

end

 

 

 

config firewall policy

    edit 20

        set uuid c5d8882e-5437-51e6-649d-8f30869eac8e

        set srcintf "PORT1"

        set dstintf "WAN1"

        set srcaddr "ITBLACKLISTS

        set dstaddr "ANY"

        set action deny

        set tag SOC PREMITERSECURITY

        set schedule "always"

        set service "ALL"

        set logtraffic all

        set comments " DROP IT LIKE IT'S HOT "

    next

   edit 21

        set uuid ab0165a0-1073-51e6-ad98-74a3bcce8032

        set dstintf "PORT1"

        set srcintf "WAN1"

        set dstaddr "ITBLACKLISTS

        set srcaddr "ANY"

        set action deny

        set schedule "always"

        set service "ALL"

        set tag  SOC

        set logtraffic all

        set comments " DROP IT LIKE IT'S HOT "

    next

 

 

fwpolicy.id 20/21 was used in ALL of our  vdoms firewall.policies and ITBLACKLISTS is  created in all  vdoms. So we only needed to modified the ITBLACKLISTS address group. We  even  crafted new groups and appended them into the parent group and set the name of the group based on DATE or  TICKET/CASE to make it easier for auditing or setting some value to know when we free the address out of the "jail" so to speak.

 

 

Another option is to null route on the  firewall lew of routing it to another vdom

 

YMMV  pick one and test them all but your option#3 is best or  mine null route on the ISP uplinks. That latter keeps the  back actor from even touch the firewall, doesn't create a log, or consume a session or uses any cpu.

 

 

Ken

 

PCNSE 

NSE 

StrongSwan  

View solution in original post

PCNSE NSE StrongSwan
neonbit
Valued Contributor

If you go with option 3 it will add a little more complexity, and it will also mean that all the WAN sessions will be doubled on the firewall as the WAN VDOM will now have keep sessions for each VDOM sending traffic through it.

 

To be honest I would look at getting a FortiManager if you have the budget for it. You could then create a global header policy and push it to all VDOMs so it will be the first policy on each of them. This way you make a change once on the global level and it will be pushed to all VDOMs.

View solution in original post

emnoc
Esteemed Contributor III

I can't   answer how to  teach you scripting , but  you will have to explore your team  access knowledge . You can do this with pyhton/perl/pwr-shell/etc...

 

We ( in past roles )  had a tool teams that would draft up our scripts ,  and they would do this by setting  expect or  restAPI calls to add or remove address objects in the  fortigate. So we would seed  a line  by line file of our bad-actors in a text file 

 

e.g ( entries pulled by our SIEM or some other device logs  like ; FEYE ,F-Secure,etc.... or you can use a BCL subscription services )

 

1.1.1.1

2.1.122.2

9.1.1.3

1.2.3.4

 

This text file would be  ran upon events or attacks and if the address didn't exist, the restAI will add the  object as an firewall.address and then append this to the firewall.addrgrp  for the ITBLACKLIST

 

I would suggest you test this in a lab or lab-vdom b4 running this in a  full production. Once you work out the  bugs you can  easily  update 1k plus  entries  in  a matter of a few seconds and across multiple vdoms. Granted I nevere worked in any appliance with more than 50+ vdoms ;)

 

e.g ( a restAPI post to add address )

 

curl -b cookie.txt -H "Content-Type: application/json" -H "X-CSRFTOKEN blablahblah "   -d '{ "name"; "$NAME",  "subnet" : "$NAME/32" }' -X POST https://x.x.x.x/api/v2/cmdb/firewall/address

 

 

We would  do  "for loop" for the name which == the ipv4.address, this was also  added as  subnet with  the mask 32.

 

Always make a config backup b4 and after and diff to ensure the changes where as planned. Also last advise , make sure you place safe guards

 

e.g

 

[ul]
  •   do not add a mask smaller than /32
  •   do not add your own subnet or address in your own subnet [/ul]

    I like to mention this since I worked a case where a forcepointNGFW user  blacklisted his own subnet  causing a priority 1 outage and they didn't follow the above 2 rules :)

     

     

    Ken Felix

     

     

  • PCNSE 

    NSE 

    StrongSwan  

    View solution in original post

    PCNSE NSE StrongSwan
    Alexis_G
    Contributor II

    According to feedback, (you dont like to block on router), I would definetely choose third. I m doing this all the time and works fine (normally i do this during new projects (not during production)). BUT !!!

    you need to prepare the vdom links, the firewall rules and last to have ready the new routes for the intervdom routing. Because otherwise you can have a downtime in order to migrate.

    Its good also if you could prepare a script to configure all routing at once.

    --------------------------------------------

    If all else fails, use the force !

    View solution in original post

    -------------------------------------------- If all else fails, use the force !
    Alexis_G

    YES !

    The solution of having one Aggregation VDOM terminating all wan traffic and then dispatch this traffic through vdomlinks to coresponded VDOM , has the effect of this traffic is passing 2 times.

     

    You need to examine the statistics , cpu, mem, performance inorder to procced to this.

    Additionally I would have great concerns for your box which is obsolete (end of sales, end of support, end of FortiOS updates).

    So maybe it would be a good time to procced to hardaware change and compute new sizing needs, so that you migrate to an new hardware with agregation vdom in your topology.

    --------------------------------------------

    If all else fails, use the force !

    View solution in original post

    -------------------------------------------- If all else fails, use the force !
    10 REPLIES 10
    emnoc
    Esteemed Contributor III

    Yes all three proposals will work  and I love your comment about  the  "don't die" if you do a massive  fw.policy  and in 100+ vdoms fw.policies ;)

     

    Here's a few better options that you should explore and not ignore

     

    1>

    If you have a router doing  a dynamic protocol and  the fgt have a means to learn of these routes , do a  route distribute with  the /32 netmask and route to  a blackhole { null  route }. This will allow you  to drop the  /32 hosts in one single swipe. You can  do this static-route on the same  ISP-ACL router that you  mention b4. OSPF/BGP works great for this or do a static route on internet external WAN1 and enable  uRPF

     

    { cisco }  ip route 1.1.1.1 255.255.255.266 null0

     

    2>

    Now if you  have no access to the router or a different team controls it, you can quickly deploy  the address into the fwpolicys across the 100+ vdom with some  scripting

     

    3>

     

    or better yet  use the restAPI. You  could  batch job this and upload the configuration into a pre-defined fw.policy.ID ( read more )

     

    In one of my recent rule we have 10 vdoms with public access. We maintain a fwpolicy  with  object-group. The policy was at that top of our list and used a deny and bidir, the addrgroup is only modified with the bad actor and we place the single ip.addr into the one  address.group. In this example it's called ITBLACKLISTS, which comprised of  numerous host firewall.address or firewall.addrgrps

     

    NOTE: research the max  values in a addrgrp, I believe it 1024 or 1500 entries per group on bigger units , you should be fine in a 3950 unless your blacklist over 1500 addresses. So if you think you  might have   more than 1500   stack address.groups

     

     

    e.g

     

    config firewall addrgrp

        edit ITBLACKLISTS

          set member BHGRP1 BHGRP2 BHGRP3  OCT182018 SEPT32018

    end

     

     

     

    config firewall policy

        edit 20

            set uuid c5d8882e-5437-51e6-649d-8f30869eac8e

            set srcintf "PORT1"

            set dstintf "WAN1"

            set srcaddr "ITBLACKLISTS

            set dstaddr "ANY"

            set action deny

            set tag SOC PREMITERSECURITY

            set schedule "always"

            set service "ALL"

            set logtraffic all

            set comments " DROP IT LIKE IT'S HOT "

        next

       edit 21

            set uuid ab0165a0-1073-51e6-ad98-74a3bcce8032

            set dstintf "PORT1"

            set srcintf "WAN1"

            set dstaddr "ITBLACKLISTS

            set srcaddr "ANY"

            set action deny

            set schedule "always"

            set service "ALL"

            set tag  SOC

            set logtraffic all

            set comments " DROP IT LIKE IT'S HOT "

        next

     

     

    fwpolicy.id 20/21 was used in ALL of our  vdoms firewall.policies and ITBLACKLISTS is  created in all  vdoms. So we only needed to modified the ITBLACKLISTS address group. We  even  crafted new groups and appended them into the parent group and set the name of the group based on DATE or  TICKET/CASE to make it easier for auditing or setting some value to know when we free the address out of the "jail" so to speak.

     

     

    Another option is to null route on the  firewall lew of routing it to another vdom

     

    YMMV  pick one and test them all but your option#3 is best or  mine null route on the ISP uplinks. That latter keeps the  back actor from even touch the firewall, doesn't create a log, or consume a session or uses any cpu.

     

     

    Ken

     

    PCNSE 

    NSE 

    StrongSwan  

    PCNSE NSE StrongSwan
    cuneyt
    New Contributor II

    Thanks a lot @emnoc for the very very comprehensive reply.

     

    Your 3rd solution is good but before, i must create firewall policy on 100+ VDOMS. So it is hard to setup. By the way routing all VDOMs to a one VDOM is also difficult :)

     

    2nd solution would be good but i have problems scripting on fortigate. Do you know how can i execute script on all VDOMS recursively (i mean without naming them statically)?

     

    Your route soluiton is also quite good but i don't want to manage routers. It is like ACLs solution.

    neonbit
    Valued Contributor

    If you go with option 3 it will add a little more complexity, and it will also mean that all the WAN sessions will be doubled on the firewall as the WAN VDOM will now have keep sessions for each VDOM sending traffic through it.

     

    To be honest I would look at getting a FortiManager if you have the budget for it. You could then create a global header policy and push it to all VDOMs so it will be the first policy on each of them. This way you make a change once on the global level and it will be pushed to all VDOMs.

    cuneyt
    New Contributor II

    Thanks for the reply.

     

    "..WAN sessions will be doubled.." is a good point to consider. In 100+ VDOMs this will make sense.

    emnoc
    Esteemed Contributor III

    I can't   answer how to  teach you scripting , but  you will have to explore your team  access knowledge . You can do this with pyhton/perl/pwr-shell/etc...

     

    We ( in past roles )  had a tool teams that would draft up our scripts ,  and they would do this by setting  expect or  restAPI calls to add or remove address objects in the  fortigate. So we would seed  a line  by line file of our bad-actors in a text file 

     

    e.g ( entries pulled by our SIEM or some other device logs  like ; FEYE ,F-Secure,etc.... or you can use a BCL subscription services )

     

    1.1.1.1

    2.1.122.2

    9.1.1.3

    1.2.3.4

     

    This text file would be  ran upon events or attacks and if the address didn't exist, the restAI will add the  object as an firewall.address and then append this to the firewall.addrgrp  for the ITBLACKLIST

     

    I would suggest you test this in a lab or lab-vdom b4 running this in a  full production. Once you work out the  bugs you can  easily  update 1k plus  entries  in  a matter of a few seconds and across multiple vdoms. Granted I nevere worked in any appliance with more than 50+ vdoms ;)

     

    e.g ( a restAPI post to add address )

     

    curl -b cookie.txt -H "Content-Type: application/json" -H "X-CSRFTOKEN blablahblah "   -d '{ "name"; "$NAME",  "subnet" : "$NAME/32" }' -X POST https://x.x.x.x/api/v2/cmdb/firewall/address

     

     

    We would  do  "for loop" for the name which == the ipv4.address, this was also  added as  subnet with  the mask 32.

     

    Always make a config backup b4 and after and diff to ensure the changes where as planned. Also last advise , make sure you place safe guards

     

    e.g

     

    [ul]
  •   do not add a mask smaller than /32
  •   do not add your own subnet or address in your own subnet [/ul]

    I like to mention this since I worked a case where a forcepointNGFW user  blacklisted his own subnet  causing a priority 1 outage and they didn't follow the above 2 rules :)

     

     

    Ken Felix

     

     

  • PCNSE 

    NSE 

    StrongSwan  

    PCNSE NSE StrongSwan
    tanr
    Valued Contributor II

    Ken,

     

    I'm curious.  In these blacklist building scripts, did you

    - Just rebuild the list of IPs from the SIEM reports?

    - Have the list keep getting added to till it was gigantic?

    - Put in a method for blacklists to age out? 

     

    I ask because it seems like there could be some problems with any of these options, though aging out seems best.

     

    If the list gets rebuilt, since you were blackholing and therefore not logging the current crop of bad IPs they wouldn't show up on a rebuilt list (depending on your report time frame) and thus would get allowed again.

     

    Just adding to the list makes it too big, plus it will continue to block IPs that might have been dynamic and are now valid.

     

    Aging out the blacklist (or re-generating it from a SIEM report with a large enough time window) seems like the best solution, though I would think you'd want to also use a static list of definite bad actors that doesn't age out.

     

    Any thoughts on this?   

    As I move from painfully building a list by hand to scripting it myself seeing what others are doing is very helpful.

     

    I really appreciate your pointer to make sure your own subnet can't get automatically added to the list!  That seems glaringly obvious until you do it to yourself...  ;)

    emnoc
    Esteemed Contributor III

    - Just rebuild the list of IPs from the SIEM reports?

     

    Yes, via SIEM , IDS,   and other means for extraction or  defined  Botnetcontrollists ( aka BCL )

     

     

    - Have the list keep getting added to till it was gigantic?

     

    yes, see my warning on max  values in  firewal;

     

    - Put in a method for blacklists to age out?
     

    Yes,  if you a  ADDR.GROUP and set a DATENAME   or case, you  do a schedule sweep at some  set time.

     

    e.g  maybe every  5 business days, you  delete the older defined  addr.group

     

    PCNSE 

    NSE 

    StrongSwan  

    PCNSE NSE StrongSwan
    Alexis_G
    Contributor II

    According to feedback, (you dont like to block on router), I would definetely choose third. I m doing this all the time and works fine (normally i do this during new projects (not during production)). BUT !!!

    you need to prepare the vdom links, the firewall rules and last to have ready the new routes for the intervdom routing. Because otherwise you can have a downtime in order to migrate.

    Its good also if you could prepare a script to configure all routing at once.

    --------------------------------------------

    If all else fails, use the force !

    -------------------------------------------- If all else fails, use the force !
    cuneyt
    New Contributor II

    Hi @jklapas,

     

    Thanks for the reply. This is what I want to learn but previously @neonbit said that in this configuration "..all the WAN sessions will be doubled on the firewall..". Is this correct? 

     

    I serached for fortinet documentation again and find some useful information. In this link:

    http://help.fortinet.com/fos50hlp/50/index.html#page/FortiOS%25205.0%2520Help/inter-VDOM.180.15.html

     

    It is about Meshed VDOM configuration with inter-VDOM links. This is what I want actually. I don't need full meshed configuration (in 100+ VDOMs it is too hard to manage) but partial mesh is the exact solution for me. But again @neonbit warning confusing me.

     

    Are there any disadvantages of this topology?

     

    What are your opinions @emnoc, @tanr, @neonbit?

    Labels
    Top Kudoed Authors