Helpful ReplyHot!DNS server on loopback not working after upgrade to 5.6.3

Author
MrSinners
Bronze Member
  • Total Posts : 51
  • Scores: 10
  • Reward points: 0
  • Joined: 2014/03/05 09:22:42
  • Status: offline
2018/03/16 06:40:09 (permalink)
0

DNS server on loopback not working after upgrade to 5.6.3

Hello,
 
After upgrading from 5.4.5 to 5.6.3, the DNS service on a loopback interface is not working anymore. The traffic is arriving on the VDOM and the debug flow shows the traffic is accepted and sent to the loopback. However the DNS service itself does not reply. A wireshark shows exactly the same behavior, the packet is incoming on the VDOM, but there's no reply back. DNSproxy debugging shows:
 
udp_receive_request()-2065: vd=8, intf=39, len=32, alen=16, 10.1.0.47:42857=>172.16.10.13
handle_dns_request()-1395: id:0xabb5 pktlen=32, qr=0 req_type=3
get_intf_policy()-876: ifindex=39


A config compare between 5.4.5 and 5.6.3 shows no modified configuration whatsoever related to DNS or the loopback interface. Running the DNS service on a physical interface does seems to be working.
 
Has anyone by chance run into this before? It seems like a bug or behavior change.
#1
emnoc
Expert Member
  • Total Posts : 5397
  • Scores: 355
  • Reward points: 0
  • Joined: 2008/03/20 13:30:33
  • Location: AUSTIN TX AREA
  • Status: offline
Re: DNS server on loopback not working after upgrade to 5.6.3 2018/03/16 07:43:14 (permalink)
0
Yes we had the same issue. Did you reapply the cfg ?  And  issues the  fwpolicy is allowing the traffic ( I know you said yes above )
 
Ken

PCNSE 
NSE 
StrongSwan  
#2
MrSinners
Bronze Member
  • Total Posts : 51
  • Scores: 10
  • Reward points: 0
  • Joined: 2014/03/05 09:22:42
  • Status: offline
Re: DNS server on loopback not working after upgrade to 5.6.3 2018/03/16 14:06:22 (permalink)
0
Hi Ken,
 
Thanks for your response!
 
The config has not been reapplied, at least not that im aware off. The FG has been recently added to FMG and had its first install. Should the config be reapplied? How did you manage to solve this? The policy is allowing traffic, but does not show up in the debug flow trace (Allowed by policy-X) as its a local interface.
#3
emnoc
Expert Member
  • Total Posts : 5397
  • Scores: 355
  • Reward points: 0
  • Joined: 2008/03/20 13:30:33
  • Location: AUSTIN TX AREA
  • Status: offline
Re: DNS server on loopback not working after upgrade to 5.6.3 2018/03/16 21:09:23 (permalink) ☄ Helpfulby MrSinners 2018/03/17 02:40:20
0
I don't know what else to suggest outside re-apply the cfg and make sure  the status is enable. We have ipv6/ipv4 bound on the  loopback and it works 100%
 
e.g 
 
AUN1LSE01:~ root# host -6  -T -t  a www.1plus1eq2.com  '2001:db8:101::1'
Using domain server:
Name: 2001:db8:101::1
Address: 2001:db8:101::1#53
Aliases: 
 
www.1plus1eq2.com has address 1.1.1.1
 
 
AUN1LSE01:~ root# host -t a www.1plus1eq2.com 192.168.1.1
Using domain server:
Name: 192.168.1.1
Address: 192.168.1.1#53
Aliases: 
 
www.1plus1eq2.com has address 1.1.1.1
 
config system interface
    edit "loop0"
        set vdom "mgmt"
        set ip 128.0.0.1 255.0.0.0
        set allowaccess ping https ssh
        set type loopback
        set snmp-index 9
        config ipv6
            set ip6-address 2001:db8:101::1/128
            set ip6-allowaccess ping ssh
        end
    next
end
 
 
config system dns-database
    edit "1plus1eq2.com"
        set status enable
        set domain "1plus1eq2.com"
        set type master
        set view public
        set ttl 300
        set authoritative enable
        unset forwarder
        set source-ip 0.0.0.0
        config dns-entry
            edit 1
                set status enable
                set type A
                set ttl 30
                set hostname "www.1plus1eq2.com"
                set ip 1.1.1.1
            next
            edit 2
                set status enable
                set type MX
                set ttl 7
                set hostname "mail.1plus1eq2.com"
                set preference 1000
 
This an authoritative  DNS for a single zone btw
 v5.6.3 FGT90D
 
Ken
 

PCNSE 
NSE 
StrongSwan  
#4
emnoc
Expert Member
  • Total Posts : 5397
  • Scores: 355
  • Reward points: 0
  • Joined: 2008/03/20 13:30:33
  • Location: AUSTIN TX AREA
  • Status: offline
Re: DNS server on loopback not working after upgrade to 5.6.3 2018/03/16 21:28:15 (permalink) ☄ Helpfulby MrSinners 2018/03/17 02:40:29
5 (1)
Hey MrSinne guess what you found a bug in v5.6.3 and DNS-server running a dual-stack  on loopback causes th e ipv4 answers to be dropped
 
 
e.g ( 19.168.1.1 == phy 128.0.0.1 == loopback )
 
AUN1LSE01:~ root# host -t mx  1plus1eq2.com 192.168.1.1 
Using domain server:
Name: 192.168.1.1
Address: 192.168.1.1#53
Aliases: 
 
1plus1eq2.com mail is handled by 1000 mail.1plus1eq2.com.
 
AUN1LSE01:~ root# host -t a  1plus1eq2.com 192.168.1.1
Using domain server:
Name: 192.168.1.1
Address: 192.168.1.1#53
Aliases: 
 
Host 1plus1eq2.com not found: 3(NXDOMAIN)
AUN1LSE01:~ root# host -t a  www.1plus1eq2.com 192.168.1.1
Using domain server:
Name: 192.168.1.1
Address: 192.168.1.1#53
Aliases: 
 
www.1plus1eq2.com has address 1.1.1.1
AUN1LSE01:~ root# host -t a  www.1plus1eq2.com '2001:db8:101::1'
Using domain server:
Name: 2001:db8:101::1
Address: 2001:db8:101::1#53
Aliases: 
 
www.1plus1eq2.com has address 1.1.1.1
AUN1LSE01:~ root# host -6  -t a  www.1plus1eq2.com '2001:db8:101::1'
Using domain server:
Name: 2001:db8:101::1
Address: 2001:db8:101::1#53
Aliases: 
 
www.1plus1eq2.com has address 1.1.1.1
AUN1LSE01:~ root# 
AUN1LSE01:~ root# 
AUN1LSE01:~ root# host -6  -t a  www.1plus1eq2.com '2001:db8:101::1'
Using domain server:
Name: 2001:db8:101::1
Address: 2001:db8:101::1#53
Aliases: 
 
www.1plus1eq2.com has address 1.1.1.1
 
AUN1LSE01:~ root# host -t a  www.1plus1eq2.com 192.168.1.1
Using domain server:
Name: 192.168.1.1
Address: 192.168.1.1#53
Aliases: 
 
www.1plus1eq2.com has address 1.1.1.1
 
 
( now loopback ipv4  udp with dans-helpers "default" )
 
 
AUN1LSE01:~ root# host -t mx  1plus1eq2.com 128.0.0.1
;; connection timed out; no servers could be reached
 
AUN1LSE01:~ root# host -t mx  1plus1eq2.com 128.0.0.1
;; connection timed out; no servers could be reached
AUN1LSE01:~ root# 
 
 
AUN1LSE01:~ root# host -t a www.1plus1eq2.com 128.0.0.1
;; connection timed out; no servers could be reached
AUN1LSE01:~ root# 
 
if you run to a  physical address it works 100% of  the time { ipv4/6 } . My lab is primary ipv6  so it was working always against the ipv6 address and I never assumed ipv4 was broken ;(
 
 
Strange but again this is FTNT  and v5.6.3 should be avoided ;)
 
Ken
 
 

PCNSE 
NSE 
StrongSwan  
#5
MrSinners
Bronze Member
  • Total Posts : 51
  • Scores: 10
  • Reward points: 0
  • Joined: 2014/03/05 09:22:42
  • Status: offline
Re: DNS server on loopback not working after upgrade to 5.6.3 2018/03/17 02:39:53 (permalink) ☄ Helpfulby Abed ALR 2019/01/09 09:00:46
5 (1)
Hey Ken,
 
Its interesting that you could reproduce the issue with a dual stack configuration. The odd thing is that in this environment there is no dual stack config, ipv6 is not used. I tried to reproduce the issue on my own FG with 5.6.3 but in this case DNS is responding, so its not clear how the issue is triggered, but it has to be a bug. Even if it would work again with reapplying the config, it doesn't feel like a safe solution for its purpose as it may just as well break again. I had to improvise with a workaround by shutting the loopback, then moving the DNS zone to a windows DNS server and using a VIP to make it reachable on the previous IP of the loopback int. Ill create a ticket and see if its a known issue. Thanks for your help!
#6
emnoc
Expert Member
  • Total Posts : 5397
  • Scores: 355
  • Reward points: 0
  • Joined: 2008/03/20 13:30:33
  • Location: AUSTIN TX AREA
  • Status: offline
Re: DNS server on loopback not working after upgrade to 5.6.3 2018/03/17 06:50:33 (permalink)
0
Did you try  with  TCP and UDP and with the dns-udp helper removed? I bet this is a bug, I never  tested  ipv4 until you posted this thread but this problem was found in two other  FGTs running v5.6.3
 
 
 
Ken

PCNSE 
NSE 
StrongSwan  
#7
MrSinners
Bronze Member
  • Total Posts : 51
  • Scores: 10
  • Reward points: 0
  • Joined: 2014/03/05 09:22:42
  • Status: offline
Re: DNS server on loopback not working after upgrade to 5.6.3 2018/05/23 07:48:38 (permalink) ☄ Helpfulby Abed ALR 2019/01/09 09:01:09
5 (1)
Hi Emnoc,
 
For this issue I made a Fortinet ticket. As it turns out, there was a change of behavior in 5.6. There was a security flaw related to DNS in 5.4 which has been fixed. The consequence of this fix is that the DNS service has to be enabled also on the source interface receiving the DNS traffic. This is also why reproducing the issue at home was not successful for me, as DNS was enabled on my LAN also so I couldn't break it. After removing DNS on my local LAN, the DNS service on the Loopback interface indeed stopped responding. It still doesn't make that much sense to enable DNS as a service on the receiving interface, but it is the solution. Thanks for your help :)
#8
emnoc
Expert Member
  • Total Posts : 5397
  • Scores: 355
  • Reward points: 0
  • Joined: 2008/03/20 13:30:33
  • Location: AUSTIN TX AREA
  • Status: offline
Re: DNS server on loopback not working after upgrade to 5.6.3 2018/05/23 12:14:35 (permalink)
0
No thank you , I'm too surprise of this behavior. I should test it in v6.0. Fortinet should update the known issues so others who  read release note can be warn.

PCNSE 
NSE 
StrongSwan  
#9
ekenny
New Member
  • Total Posts : 3
  • Scores: 0
  • Reward points: 0
  • Status: offline
Re: DNS server on loopback not working after upgrade to 5.6.3 2018/11/28 04:13:43 (permalink)
0
Confirmed this is still the behavior in 6.0.x.  Fix was enabling the DNS server on the source interface as MrSinners mentioned.
#10
tinyadmin
Bronze Member
  • Total Posts : 23
  • Scores: 0
  • Reward points: 0
  • Joined: 2014/02/07 11:44:29
  • Location: Germany, Lake Konstanz
  • Status: offline
Re: DNS server on loopback not working after upgrade to 5.6.3 2019/08/11 00:46:28 (permalink)
0
Dear all,
I see same problem on FortiOS 6.2.1 on 60D and same workaround: enable DNS-Server on source interface, too.
Thanks for the hint.
 
Does anyone opened a bug for this issue?
Regards
Tinyadmin
post edited by tinyadmin - 2019/08/11 00:52:24
#11
Jump to:
© 2019 APG vNext Commercial Version 5.5