Hot!Fortigate blocked TCP packets with PSH flag.

Author
Polikarpov
New Member
  • Total Posts : 3
  • Scores: 0
  • Reward points: 0
  • Joined: 2020/02/12 19:12:39
  • Status: offline
2020/02/12 19:59:22 (permalink) 6.2
0

Fortigate blocked TCP packets with PSH flag.

Hello community!
Does someone have any idea what is the issue?
PSH flag in TCP packets is rarely used in common life, but our NMEA-to-IP converter is using this.
Fortigate did not allow it to pass and did not logged it as a blocked.
Session was successfully established - SYN, SYN-ACk and ACK passing through firewall,
but PSH-ACK did not want to pass.
I have played with auto-asic-offload config, np-acceleration and inspection proxy/flow mode it did not help.
 
Wireshark capture:

 
Config:
Firewall (158) # show
config firewall policy
    edit 158
        set name "IT Manager to Nmea converter"
        set uuid fa3c5914-cca8-51e9-819d-57551413207d
        set srcintf "vsw.port16"
        set dstintf "GPS"
        set srcaddr "all"
        set dstaddr "all"
        set action accept
        set schedule "always"
        set service "ALL"
        set profile-protocol-options "NMEA"
        set ssl-ssh-profile "NMEA"
        set logtraffic all
        set fsso disable
    next
end
 
 
Aleksandr
 

Attached Image(s)

#1

6 Replies Related Threads

    Yurisk
    Bronze Member
    • Total Posts : 22
    • Scores: 2
    • Reward points: 0
    • Joined: 2011/12/04 03:30:01
    • Status: offline
    Re: Fortigate blocked TCP packets with PSH flag. 2020/02/13 00:47:31 (permalink)
    0
    I wouldn't be sure the reason is PSH flag, as it is a valid flag in TCP sessions, and I see it regularly passing FGs without a problem. I'd suggest : 
    1. Making sure it arrives at the FG and is indeed being dropped by it:
    FG# dia sniffer packet any 'Ip of oustide host receiving the connection' 4
    And see if the same packet enters and leaves the FG or just enters it.
     
    2. If the above shows packet arrives but doesnt leave the FG, run debug flow to see why:
    diag debug reset
    diag debug flow filter addr <ip of outside host receiving the connection>
    diag debug flow show function-name enable
    diag debug flow trace start 100
     
    Then initiate the connection and post the result.
    #2
    Polikarpov
    New Member
    • Total Posts : 3
    • Scores: 0
    • Reward points: 0
    • Joined: 2020/02/12 19:12:39
    • Status: offline
    Re: Fortigate blocked TCP packets with PSH flag. 2020/02/13 04:17:22 (permalink)
    0
    Thank you Yuri for prompt and detailed help!
    Sniffer shows the packet is receiving by FG, but it does not go out.

    The result of debug below:
    id=20085 trace_id=1043 func=print_pkt_detail line=5501 msg="vd-root:0 received a packet(proto=6, 192.168.202.3:2000->10.49.41.150:56976) from GPS. flag [.], seq 611704, ack 1946171203, win 2920"
    id=20085 trace_id=1043 func=resolve_ip_tuple_fast line=5581 msg="Find an existing session, id-098026bf, reply direction"
    id=20085 trace_id=1043 func=npu_handle_session44 line=1139 msg="Trying to offloading session from GPS to vsw.port16, skb.npu_flag=00000000 ses.state=00140386 ses.npu_state=0x00000001"
    id=20085 trace_id=1043 func=fw_forward_dirty_handler line=449 msg="state=00140386, state2=00000001, npu_state=00000001"
    id=20085 trace_id=1043 func=av_receive line=305 msg="send to application layer"
    id=20085 trace_id=1044 func=print_pkt_detail line=5501 msg="vd-root:0 received a packet(proto=6, 10.49.41.150:56976->192.168.202.3:2000) from local. flag [.], seq 1946171203, ack 611741, win 65535"
    id=20085 trace_id=1044 func=resolve_ip_tuple_fast line=5581 msg="Find an existing session, id-098026bf, original direction"
    id=20085 trace_id=1045 func=print_pkt_detail line=5501 msg="vd-root:0 received a packet(proto=6, 192.168.202.3:2000->10.49.41.150:56976) from GPS. flag [.], seq 611741, ack 1946171203, win 2920"
    id=20085 trace_id=1045 func=resolve_ip_tuple_fast line=5581 msg="Find an existing session, id-098026bf, reply direction"
    id=20085 trace_id=1045 func=npu_handle_session44 line=1139 msg="Trying to offloading session from GPS to vsw.port16, skb.npu_flag=00000000 ses.state=00140386 ses.npu_state=0x00000001"
    id=20085 trace_id=1045 func=fw_forward_dirty_handler line=449 msg="state=00140386, state2=00000001, npu_state=00000001"
    id=20085 trace_id=1045 func=av_receive line=305 msg="send to application layer"


    I have temporary put old Cisco 1841 between these networks and problem disappeared.
    Is there any chance to get this packet passing through Fortigate?
     
    Aleksandr

    Attached Image(s)

    #3
    Yurisk
    Bronze Member
    • Total Posts : 22
    • Scores: 2
    • Reward points: 0
    • Joined: 2011/12/04 03:30:01
    • Status: offline
    Re: Fortigate blocked TCP packets with PSH flag. 2020/02/14 01:31:09 (permalink)
    0
    Strange indeed, in the FG sniffer if you look 3 lines above where 192.168.202.3 returns ACk it also gets blocked (and no PSH flag), seems like FG drops any returning traffic from 192.168.202 except SYN+ACK. On the other hand the traffic is strange as well - first 4 packets establish successfully TCP session, then instead of using this session, the host 10.49 again sends SYN out of the blue ... 
    if it is some SCADA/IOT irregular traffic FG may have problems statefully understanding it. 
    Is it in proxy mode (I see it is, have you tried FLow mode instead?)? Do you have some fancy VIP/NAT ? What does profile do (why custom, not "default") 
    set profile-protocol-options "NMEA" ?


     
    post edited by Yurisk - 2020/02/14 23:41:19
    #4
    darwin_FTNT
    Bronze Member
    • Total Posts : 43
    • Scores: 2
    • Reward points: 0
    • Joined: 2018/04/24 18:12:28
    • Status: offline
    Re: Fortigate blocked TCP packets with PSH flag. 2020/02/14 18:11:18 (permalink)
    0
    id=20085 trace_id=1045 func=av_receive line=305 msg="send to application layer"

    It indicates proxy-based utm feature is used.  If it is flow-based utm, it is "send to ips".

    To debug proxy-based utm: "diag debug application wad -1"
    To debug flow-based utm: "diag debug application ipsengine -1" OR "diag ips debug enable all"
     
    Note that the above commands will print out lot's of debug messages and bring a network traffic to halt.
    You can specify filter like ip address to reduce debug slow down.

    For proxy-based utm, it means 2 connections is done from client to wad daemon, then wad daemon to server (proxy mode).  So it could be the tcp flag is processed by wad daemon but the new connection to server-side didn't use it.

    Disable all utm features first so either "send to application" or "send to ips" in "diag debug flow..." doesn't show up.


    This means the packet is handled by kernel directly or offloaded to npu / hw accel is used/triggered for better performance. Hope it helps.
    #5
    emnoc
    Expert Member
    • Total Posts : 5508
    • Scores: 355
    • Reward points: 0
    • Joined: 2008/03/20 13:30:33
    • Location: AUSTIN TX AREA
    • Status: offline
    Re: Fortigate blocked TCP packets with PSH flag. 2020/02/14 22:25:39 (permalink)
    0
    I would also check and make note of the . set anti-replay option in gloal cfg 
     
     
    e.g
     
       show full sys global | grep -f anti
     
    and adjust and restest.
     
    Ken Felix

    PCNSE 
    NSE 
    StrongSwan  
    #6
    Polikarpov
    New Member
    • Total Posts : 3
    • Scores: 0
    • Reward points: 0
    • Joined: 2020/02/12 19:12:39
    • Status: offline
    Re: Fortigate blocked TCP packets with PSH flag. 2020/02/15 19:36:21 (permalink)
    0
    I have tried on different Fortigate model, disabled anti-reply changed from proxy to flow and from flow to proxy.
    The same result. All ips features are disabled. No additional info was seen then ips debugging was activated.
     
     
     
    id=20085 trace_id=276 func=print_pkt_detail line=4479 msg="vd-root received a packet(proto=6, 192.168.202.3:2000->192.168.100.6:63071) from local. flag [.], seq 1177162469, ack 838393172, win 5840"
    id=20085 trace_id=276 func=resolve_ip_tuple_fast line=4542 msg="Find an existing session, id-02a8a75b, reply direction"
    id=20085 trace_id=277 func=print_pkt_detail line=4479 msg="vd-root received a packet(proto=6, 192.168.202.3:2000->192.168.100.6:63071) from local. flag [F.], seq 1177162469, ack 838393172, win 5840"
    id=20085 trace_id=277 func=resolve_ip_tuple_fast line=4542 msg="Find an existing session, id-02a8a75b, reply direction"
    id=20085 trace_id=278 func=print_pkt_detail line=4479 msg="vd-root received a packet(proto=6, 192.168.100.6:63071->192.168.202.3:2000) from local. flag [F.], seq 1366320358, ack 262568, win 5840"
    id=20085 trace_id=278 func=resolve_ip_tuple_fast line=4542 msg="Find an existing session, id-02a8a75b, original direction"
    id=20085 trace_id=279 func=print_pkt_detail line=4479 msg="vd-root received a packet(proto=6, 192.168.100.6:63071->192.168.202.3:2000) from internal3. flag [.], seq 838393172, ack 1177162470, win 8212"
    id=20085 trace_id=279 func=resolve_ip_tuple_fast line=4542 msg="Find an existing session, id-02a8a75b, original direction"
    id=20085 trace_id=279 func=av_receive line=255 msg="send to application layer"


    #7
    Jump to:
    © 2020 APG vNext Commercial Version 5.5