Hot!AWS Fortigate instance incommunicado...

Author
kadey
New Member
  • Total Posts : 12
  • Scores: 0
  • Reward points: 0
  • Joined: 2018/08/24 10:06:23
  • Status: offline
2019/12/23 12:58:51 (permalink)
0

AWS Fortigate instance incommunicado...

I have a Fortigate instance in AWS that I cannot get to with ssh or https. It was accessible since creation (months), then all of a sudden it was not. I can ping it, and (by using nmap) can see that ports 22, 443, 500 and 4500 are accessible and listening. Security is wide open to this instance. Any ideas what could be wrong?
 
The console image shows the following:
 
System is starting...
Serial number is FGTAWS000133210D


FGTAWS000133210D login:
AWS instance-id: i-0133210d2e0c26da

#1

14 Replies Related Threads

    emnoc
    Expert Member
    • Total Posts : 5461
    • Scores: 355
    • Reward points: 0
    • Joined: 2008/03/20 13:30:33
    • Location: AUSTIN TX AREA
    • Status: online
    Re: AWS Fortigate instance incommunicado... 2019/12/23 14:06:34 (permalink)
    0
    Have you check
     
    trusthost ?
    security-group?
     
    Ken Felix
     

    PCNSE 
    NSE 
    StrongSwan  
    #2
    kadey
    New Member
    • Total Posts : 12
    • Scores: 0
    • Reward points: 0
    • Joined: 2018/08/24 10:06:23
    • Status: offline
    Re: AWS Fortigate instance incommunicado... 2019/12/23 20:03:12 (permalink)
    0
    Wouldn't the fact that I can scan the relevant ports mean there's nothing blocking the traffic?
     
    I looked at the flow logs for the WAN interface, and the traffic is being accepted ok.
     
    post edited by kadey - 2019/12/24 07:45:57
    #3
    kadey
    New Member
    • Total Posts : 12
    • Scores: 0
    • Reward points: 0
    • Joined: 2018/08/24 10:06:23
    • Status: offline
    Re: AWS Fortigate instance incommunicado... 2019/12/26 07:25:14 (permalink)
    0
    I did more comprehensive scanning, and it's only finding tcp port 541 open.
     
    Port 541 is used by FortiManager. Could my FortiGate have been somehow put into remote management mode?
     
    #4
    emnoc
    Expert Member
    • Total Posts : 5461
    • Scores: 355
    • Reward points: 0
    • Joined: 2008/03/20 13:30:33
    • Location: AUSTIN TX AREA
    • Status: online
    Re: AWS Fortigate instance incommunicado... 2019/12/26 07:45:49 (permalink)
    0
    What changes did you make in AWS and fortiOS? If it was working and now not working, undo or revert any changes. I would also not over look any local-in policies.
     
    Ken Felix

    PCNSE 
    NSE 
    StrongSwan  
    #5
    kadey
    New Member
    • Total Posts : 12
    • Scores: 0
    • Reward points: 0
    • Joined: 2018/08/24 10:06:23
    • Status: offline
    Re: AWS Fortigate instance incommunicado... 2019/12/26 08:21:39 (permalink)
    0
    I made no changes prior to this issue cropping up.
     
    Can you elaborate on "local-in policies"? Are these in AWS or on the FortiGate?
    #6
    emnoc
    Expert Member
    • Total Posts : 5461
    • Scores: 355
    • Reward points: 0
    • Joined: 2008/03/20 13:30:33
    • Location: AUSTIN TX AREA
    • Status: online
    Re: AWS Fortigate instance incommunicado... 2019/12/26 09:29:09 (permalink)
    0
    You have 3 or 4 things to check
     
    1: route 
     
    2: trusthost for any allowaccess
     
    https://kb.fortinet.com/kb/documentLink.do?externalID=10868
     
    3: local-in policy
     
    https://help.fortinet.com/fos50hlp/52data/Content/FortiOS/fortigate-firewall-52/Security%20Policies/Local-In%20Policies.htm
     
    4: proper SG in AWS 
     
    https://docs.fortinet.com/vm/aws/fortigate/6.2/aws-cookbook/6.2.0/228062/opening-ports-in-the-security-group
     
    If your not getting a login prompt, I would research all of the above. if you did not change the fortios-cfg than the logical step is to look at the SG in AWS. Also I had a client that change the elastic ip and so he was trying to access the Fortigateinstance with the wrong EIP address.
     
    Ken Felix
     
    Ken Felix

    PCNSE 
    NSE 
    StrongSwan  
    #7
    kadey
    New Member
    • Total Posts : 12
    • Scores: 0
    • Reward points: 0
    • Joined: 2018/08/24 10:06:23
    • Status: offline
    Re: AWS Fortigate instance incommunicado... 2019/12/26 11:07:25 (permalink)
    0
    Thanks for those links.
     
    Routing is not an issue, since, like I previously said, I can ping the instance and perform a port scan on it.
     
    I did not enable "Restrict login to trusted hosts" nor configure local-in policies to restrict access.
     
    The AWS Security Group for the instance is wide open, and I have confirmed that the traffic is getting to the instance by looking at the VPC flow logs of the WAN interface.
     
    I believe the problem is that it's in some weird state, because the port scan shows nothing is listening on the usual access ports 22 and 443. The only port open is 541, the FortiManager access port.
    #8
    James_G
    Gold Member
    • Total Posts : 152
    • Scores: 5
    • Reward points: 0
    • Joined: 2016/02/28 02:55:47
    • Status: offline
    Re: AWS Fortigate instance incommunicado... 2019/12/26 11:16:32 (permalink)
    0
    Have you tried a reboot?
    #9
    kadey
    New Member
    • Total Posts : 12
    • Scores: 0
    • Reward points: 0
    • Joined: 2018/08/24 10:06:23
    • Status: offline
    Re: AWS Fortigate instance incommunicado... 2019/12/26 11:26:28 (permalink)
    0
    Yes, I have stopped and restarted the instance multiple times.
     
    #10
    emnoc
    Expert Member
    • Total Posts : 5461
    • Scores: 355
    • Reward points: 0
    • Joined: 2008/03/20 13:30:33
    • Location: AUSTIN TX AREA
    • Status: online
    Re: AWS Fortigate instance incommunicado... 2019/12/26 23:36:20 (permalink)
    0
    SSHD should be running. You can double check by finding the pid
     
    e.g
     
    FIREWALLFGT # fnsysctl ls /var/run/sshd.pid
    /var/run/sshd.pid  
     
    FIREWALLFGT # fnsysctl ls -ltr /var/run/sshd.pid
    ls: invalid option -- 't'
    usage: ls [-aAl] [file ...]
     
    FIREWALLFGT # fnsysctl ls -l /var/run/sshd.pid
    -rw-r--r--    1 0        0       Thu Dec 26 22:57:33 2019                3 /var/run/sshd.pid
     
    FIREWALLFGT # fnsysctl cat  /var/run/sshd.pid
    85
     
    FIREWALLFGT # diag sys process  dump 85
    Status:
    Name: sshd
    State: S (sleeping)
    Tgid: 85
    Pid: 85
     
    You can also use diag sys top -p 
     
    If you have a pid than diag sniffer the interface and monitor
    e.g
     
      diag sniffer packet wan1 "dst port 22"
     
    Ken Felix
     

    PCNSE 
    NSE 
    StrongSwan  
    #11
    emnoc
    Expert Member
    • Total Posts : 5461
    • Scores: 355
    • Reward points: 0
    • Joined: 2008/03/20 13:30:33
    • Location: AUSTIN TX AREA
    • Status: online
    Re: AWS Fortigate instance incommunicado... 2019/12/26 23:40:31 (permalink)
    0
    Also run debug on sshd 
     
    e.g
     
    diag debug en
    diag debug reset 
    diag debug application  sshd -1
     
    That might shed some light also and ensure that the sshd tcp.port is known if it's not #22
     
    Ken Felix

    PCNSE 
    NSE 
    StrongSwan  
    #12
    kadey
    New Member
    • Total Posts : 12
    • Scores: 0
    • Reward points: 0
    • Joined: 2018/08/24 10:06:23
    • Status: offline
    Re: AWS Fortigate instance incommunicado... 2019/12/27 07:35:54 (permalink)
    0
    How am I supposed to do this if I can't access the instance?
    #13
    emnoc
    Expert Member
    • Total Posts : 5461
    • Scores: 355
    • Reward points: 0
    • Joined: 2008/03/20 13:30:33
    • Location: AUSTIN TX AREA
    • Status: online
    Re: AWS Fortigate instance incommunicado... 2019/12/27 07:41:57 (permalink)
    0
    You don't have an inside host that can access or is it all external-facing hosts that has problems ( i.e mgt access from the internet )  ? I still believe you either have 1> SG ( Sec-group 2> or wrong instance public-ip 3> maybe you release the ip and gather a new one 
     
    If nothing changes on fortios , then your problem is elsewhere. How did you determine ssh/https is NOT running as you mentioned previously? if you have no access and the image is "truly" up, your problem might be somewhere else and has nothing todo with FortiOS or the Virt-image
     
    "been there done that before"
     
    if your problem is only mgt traffic and EC2 instances are working through the unit, then that can be a clue as to what is happen or happening.
     
    Ken Felix

    PCNSE 
    NSE 
    StrongSwan  
    #14
    kadey
    New Member
    • Total Posts : 12
    • Scores: 0
    • Reward points: 0
    • Joined: 2018/08/24 10:06:23
    • Status: offline
    Re: AWS Fortigate instance incommunicado... 2019/12/27 08:14:53 (permalink)
    0
    I performed a port scan on the instance, and the only tcp port that came back as open was 541. UDP 500/4500 (the IKE/ISAKMP IPsec ports) are open.
     
     
    #15
    Jump to:
    © 2020 APG vNext Commercial Version 5.5