Hot!Some .gov sites blocked, others are not?

Page: 12 > Showing page 1 of 2
Author
adder666
New Member
  • Total Posts : 3
  • Scores: 0
  • Reward points: 0
  • Joined: 2021/09/30 14:00:18
  • Status: offline
2021/09/30 14:24:59 (permalink)
0

Some .gov sites blocked, others are not?

I'm a novice with this, but I have web filters enabled and still having trouble reaching a few government sites, for example www.pittsburghca.gov. But others, like https://www.dmv.ca.gov work fine. I've explicitly allowed the blocked sites and still same error:
 
[font="'helvetica neue', arial, helvetica, sans-serif; font-size: 18px"]This Connection is Invalid. SSL certificate expired.
A secure connection to www.pittsburgca.gov cannot be established.
When you try to connect securely, sites will present trusted identification to prove that you are going to the right place. However, this site's identity can't be verified.
Sitewww.pittsburgca.govCertificate CNsan-g2.granicusgovaccess.netCertificate AuthorityR3Certificate Validity
Not Before: Sep 20 16:46:45 2021 GMT
Not After: Dec 19 16:46:44 2021 GMT
 
What am I missing? And remember, I'm a real novice with the Fortinet equipment (new job, this is what they had in place) so explain to me like I'm a 5 year old. It's a Fortigate 100E with firmware v7.0.1 build0157 (GA). Thanks!
#1
LucD
New Member
  • Total Posts : 1
  • Scores: 0
  • Reward points: 0
  • Joined: 2021/09/30 23:29:42
  • Status: offline
Re: Some .gov sites blocked, others are not? 2021/09/30 23:35:59 (permalink)
0
use a lets'encrypt issued certificate. There is a problem since yesterday for letsencrypt issued certificates.. Using flow-based mode could help... We are using an explicit proxy so I think we have to use proxy-based mode.
 
#2
ThibM
New Member
  • Total Posts : 1
  • Scores: 0
  • Reward points: 0
  • Joined: 2021/09/30 23:39:03
  • Status: offline
Re: Some .gov sites blocked, others are not? 2021/09/30 23:45:33 (permalink)
0
Hello,
 
Looks like you're impacted by Let's Encrypt's Root Certificate expiration issue.
Here's an article of a similar case, last year:
https://kb.fortinet.com/kb/documentLink.do?externalID=FD49028
 
Somehow the FortiGates still choose the wrong certification path.
When bypassing certificate inspection, you can clearly see that modern browses choose the right path.
Here's a link to an article of Let's Encrypt, written back in May:
 
I hope Fortinet gets this right for future cases...
 
Good luck!
#3
ffischer
New Member
  • Total Posts : 20
  • Scores: 2
  • Reward points: 0
  • Joined: 2016/12/20 00:56:05
  • Status: offline
Re: Some .gov sites blocked, others are not? 2021/10/01 01:57:19 (permalink)
0
Have same problem here on 6.4.7
i.e with "https://mathefachschaft.de"
 
Here is what happens:
They use Let's Encrypt Certs for their server as well.

https://www.ssllabs.com/ssltest/analyze.html?d=mathefachschaft.de shows,
they send the complete chain with an "ISRG Root X1"
Fingerprint SHA256: 6d99fb265eb1c5b3744765fcbc648f3cd8e1bffafdc4c2f99b9d47cf7ff1c24f
which is signed by the DST Root CA X3 expired yesterday.
 
The factory default trusted store on FortiOS contains the old, expired
 
fortigw (ca) # get DST_Root_CA_X3
name                : DST_Root_CA_X3
ca                  :
        Subject:     O = Digital Signature Trust Co., CN = DST Root CA X3
        Issuer:      O = Digital Signature Trust Co., CN = DST Root CA X3
        Valid from:  2000-09-30 21:12:19  GMT
        Valid to:    2021-09-30 14:01:15  GMT
        Fingerprint: 41:03:52:DC:0F:F7:50:1B:16:F0:02:8E:BA:6F:45:C5
        Root CA:     Yes
        Version:     3
        Serial Num:
                44:af:b0:80:d6:a3:27:ba:89:30:39:86:2e:f8:40:6b
        Extensions:
                Name:     X509v3 Basic Constraints
                Critical: yes
                Content:
                CA:TRUE

                Name:     X509v3 Key Usage
                Critical: yes
                Content:
                Certificate Sign, CRL Sign

                Name:     X509v3 Subject Key Identifier
                Critical: no
                Content:
                C4:A7:B1:A4:7B:2C:71:FA:DB:E1:4B:90:75:FF:C4:15:60:85:89:10


range               : global
source              : bundle
ssl-inspection-trusted: disable
scep-url            :
source-ip           : 0.0.0.0

as well as and the new
fortigw (ca) # get ISRG_Root_X1
name                : ISRG_Root_X1
ca                  :
        Subject:     C = US, O = Internet Security Research Group, CN = ISRG Root X1
        Issuer:      C = US, O = Internet Security Research Group, CN = ISRG Root X1
        Valid from:  2015-06-04 11:04:38  GMT
        Valid to:    2035-06-04 11:04:38  GMT
        Fingerprint: 0C:D2:F9:E0:DA:17:73:E9:ED:86:4D:A5:E3:70:E7:4E
        Root CA:     Yes
        Version:     3
        Serial Num:
                82:10:cf:b0:d2:40:e3:59:44:63:e0:bb:63:82:8b:00
        Extensions:
                Name:     X509v3 Key Usage
                Critical: yes
                Content:
                Certificate Sign, CRL Sign

                Name:     X509v3 Basic Constraints
                Critical: yes
                Content:
                CA:TRUE

                Name:     X509v3 Subject Key Identifier
                Critical: no
                Content:
                79:B4:59:E6:7B:B6:E5:E4:01:73:80:08:88:C8:1A:58:F6:E9:9B:6E


range               : global
source              : bundle
ssl-inspection-trusted: enable
scep-url            :
source-ip           : 0.0.0.0
 
Now,
ssllabs shows that there are 2 possible verification paths:
One with the new self-signed X1 as the root CA,
another with the expired DST_Root_CA_X3 as the rootCA; here X1 is only an intermediate CA.
 
The web server sends the X1 cert pointing to expired X3  in the chain:
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            40:01:77:21:37:d4:e9:42:b8:ee:76:aa:3c:64:0a:b7
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: O = Digital Signature Trust Co., CN = DST Root CA X3
        Validity
            Not Before: Jan 20 19:14:03 2021 GMT
            Not After : Sep 30 18:14:03 2024 GMT
        Subject: C = US, O = Internet Security Research Group, CN = ISRG Root X1
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (4096 bit)
                Modulus:
                    ....
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Basic Constraints: critical
                CA:TRUE
            X509v3 Key Usage: critical
                Certificate Sign, CRL Sign
            Authority Information Access:
                CA Issuers - URI:http://apps.identrust.com/roots/dstrootcax3.p7c

            X509v3 Authority Key Identifier:
                keyid:C4:A7:B1:A4:7B:2C:71:FA:DB:E1:4B:90:75:FF:C4:15:60:85:89:10

            X509v3 Certificate Policies:
                Policy: 2.23.140.1.2.1
                Policy: 1.3.6.1.4.1.44947.1.1.1
                  CPS: http://cps.root-x1.letsencrypt.org

            X509v3 CRL Distribution Points:

                Full Name:
                  URI:http://crl.identrust.com/DSTROOTCAX3CRL.crl

            X509v3 Subject Key Identifier:
                79:B4:59:E6:7B:B6:E5:E4:01:73:80:08:88:C8:1A:58:F6:E9:9B:6E
    Signature Algorithm: sha256WithRSAEncryption
         .....
SHA256 Fingerprint=6D:99:FB:26:5E:B1:C5:B3:74:47:65:FC:BC:64:8F:3C:D8:E1:BF:FA:FD:C4:C2:F9:9B:9D:47:CF:7F:F1:C2:4F
 

So the Fortigate simply follows the path suggested by the webservers' chain and fails verification.

I would not say this is a bug incorrect but would consider it as misconfiguration of the webservers cert chain.

If the server admin would simply remove the x1 cert from the chain, the FG would use the built in, new X1 CA and could verify successfully.
 
The only was FN could resolve the issues is by
not only follow the path suggested by the servers' chain,
but check any cert against the factory and user trusted certs as well.
 
Well.. this explanains the errors, but it does not help to get the websites unblocked
until the server admins correct their configs.

As a bad workaround I have set "allow" expired certificate in the ssl inspection for the moment :-(
   
#4
simonorch
Gold Member
  • Total Posts : 346
  • Scores: 14
  • Reward points: 0
  • Joined: 2009/06/05 00:05:08
  • Location: Norway
  • Status: offline
Re: Some .gov sites blocked, others are not? 2021/10/01 02:35:44 (permalink)
0
Great thread guys, many thanks indeed. Currently i also only see 'allow expired' as a temporary workaround.
 
the issue has been so far seen by us on 6.2, 6.4 and 7.0

NSE8
Fortinet Expert partner - Norway
#5
simonorch
Gold Member
  • Total Posts : 346
  • Scores: 14
  • Reward points: 0
  • Joined: 2009/06/05 00:05:08
  • Location: Norway
  • Status: offline
Re: Some .gov sites blocked, others are not? 2021/10/01 03:12:03 (permalink)
0
Word has it Fortiguard are working on an update

NSE8
Fortinet Expert partner - Norway
#6
boneyard
Gold Member
  • Total Posts : 429
  • Scores: 20
  • Reward points: 0
  • Joined: 2014/07/30 11:15:18
  • Status: offline
Re: Some .gov sites blocked, others are not? 2021/10/01 04:42:28 (permalink)
0
simonorch
Word has it Fortiguard are working on an update

heard the same, but quite sceptical they can solve this on the FortiGuard side. it feels like it needs an actual code change.
#7
Alucard
New Member
  • Total Posts : 1
  • Scores: 0
  • Reward points: 0
  • Joined: 2021/10/01 05:18:26
  • Status: offline
Re: Some .gov sites blocked, others are not? 2021/10/01 05:21:32 (permalink)
0
boneyard
simonorch
Word has it Fortiguard are working on an update

heard the same, but quite sceptical they can solve this on the FortiGuard side. it feels like it needs an actual code change.


I think removing the X3 certifcate from the trust store on the Fortigate should also solve this.
#8
Jirka
Gold Member
  • Total Posts : 197
  • Scores: 9
  • Reward points: 0
  • Joined: 2014/07/09 11:34:53
  • Location: Czech Republic
  • Status: offline
Re: Some .gov sites blocked, others are not? 2021/10/01 09:15:50 (permalink)
0
This workaround works for us (200E, 7.0.1, Policy mode Proxy & Deep inspection)

 
Jirka

Attached Image(s)

#9
localhost
Gold Member
  • Total Posts : 138
  • Scores: 28
  • Reward points: 0
  • Joined: 2015/05/21 02:47:51
  • Location: Switzerland
  • Status: offline
Re: Some .gov sites blocked, others are not? 2021/10/01 09:22:58 (permalink)
0
boneyard
heard the same, but quite sceptical they can solve this on the FortiGuard side. it feels like it needs an actual code change.



My thoughts exactly. I guess they will just make an exception for the expired DST_Root_CA_X3 certificate and ignore its expiry date.
 
#10
adder666
New Member
  • Total Posts : 3
  • Scores: 0
  • Reward points: 0
  • Joined: 2021/09/30 14:00:18
  • Status: offline
Re: Some .gov sites blocked, others are not? 2021/10/01 10:20:18 (permalink)
0
I saw the Let's Encrypt thing after I posted the question. I spent a lot of time trying to get it to work. Unfortunately, the original 4 templates for SSL/SSN (certificate-inspection, custom-deep-inspection, deep-inspection, no-inspection) are the only ones there. I wasn't able to clone the certificate-inspection template, which is the one being used and I can't change it.
 
I cloned the custom-deep-inspection and tried modifying it to match the certificate-inspection, but must have missed. It screwed up everyone. I was able to go to the .gov sites that were blocked, but it messed up other sites. Might be easier to tell the staff to wait it out.
 
That or try again with the cloning deep and matching as best I can to the one in use. thanks for all your input everyone!
#11
adder666
New Member
  • Total Posts : 3
  • Scores: 0
  • Reward points: 0
  • Joined: 2021/09/30 14:00:18
  • Status: offline
Re: Some .gov sites blocked, others are not? 2021/10/01 10:53:18 (permalink)
0
One more update. So I copied it all and enabled it, broke things again. Like Google was blocked, though others were allowed now. Like the previous City of Pittsburgh, CA site. That was available, but Google wasn't. Not sure what the hell is happening on this device, but it's obvious I need to spend way more time in there learning.
#12
boneyard
Gold Member
  • Total Posts : 429
  • Scores: 20
  • Reward points: 0
  • Joined: 2014/07/30 11:15:18
  • Status: offline
Re: Some .gov sites blocked, others are not? 2021/10/01 23:01:54 (permalink)
0
found this one today, didnt pop earlier in my search results, but is from Thursday (September 30th) already
 
https://www.fortinet.com/blog/psirt-blogs/fortinet-and-expiring-lets-encrypt-certificates
 
this also seems to indicate that only changing the bundle wont be enough. the bundle which seems now updated to (around 20:00 October 1st for me) but you also need that checked blocked in some way. which hopefully doesnt break other things.
 
communication wise Fortinet isnt doing very well here in my opinion. not reaching out on this when it occurred (or even weeks earlier so we could prepare), but letting people contact them. too many places with parts of information that doesnt match. announcing bundles which aren't out yet.
 

Solution
Fortinet is working on a longer-term solution to improve certificate validation and add additional intelligence to rebuild missing certificate chains in these cases going forward, and will include this in a future release.

 
this has happened twice now, certainly hope not a third time.
#13
boneyard
Gold Member
  • Total Posts : 429
  • Scores: 20
  • Reward points: 0
  • Joined: 2014/07/30 11:15:18
  • Status: offline
Re: Some .gov sites blocked, others are not? 2021/10/01 23:04:14 (permalink)
0
adder666
One more update. So I copied it all and enabled it, broke things again. Like Google was blocked, though others were allowed now. Like the previous City of Pittsburgh, CA site. That was available, but Google wasn't. Not sure what the hell is happening on this device, but it's obvious I need to spend way more time in there learning.

certificate inspection and deep inspecting are quite different things. where deep inspection simply will require quite some exceptions as not all sites work with the man in the middle process. i advise you to try again for one IP (policy source) and work on an acceptable setup before going broader. if that fails start a new question / thread to get help on that.
#14
Toshi Esumi
Expert Member
  • Total Posts : 2741
  • Scores: 273
  • Reward points: 0
  • Joined: 2014/11/06 09:56:42
  • Status: offline
Re: Some .gov sites blocked, others are not? 2021/10/02 16:00:05 (permalink)
0
Through the ticket we opened for our customer who encountered this situation, TAC L2 said that they removed the expired DST Root CA X3 from the certificate bundle and pushed yesterday as version 1.28. I now can see it like below:

FG1xxxxxxx-fg2 (global) # diag autoupdate versions | grep 'Bundle' -A 6
Certificate Bundle
---------
Version: 1.00028
Contract Expiry Date: n/a
Last Updated using push update on Fri Oct  1 13:23:41 2021
Last Update Attempt: Sat Oct  2 17:23:41 2021
Result: No Updates

However, he also said since it's still receiving expired cert from the server as one of multiple chain paths, it would block it since AIA fetching pulls the same expired cert. Therefore DNS blocking is necessary for "apps.indentrust.com". Once apps.identrust.com removes and stops sending the expired cert, it would be solved and you can remove the DNS blocking. All are described below:
https://kb.fortinet.com/kb/documentLink.do?externalID=FD53305
 
For the fix they're intending, when I asked about these situations with cross-signing, he mentioned "... making FGT behavior something similar to web browsers. ... trust the chain that the website is providing and ignore the untrusted certificate chain."
post edited by Toshi Esumi - 2021/10/02 16:02:26
#15
ffischer
New Member
  • Total Posts : 20
  • Scores: 2
  • Reward points: 0
  • Joined: 2016/12/20 00:56:05
  • Status: offline
Re: Some .gov sites blocked, others are not? 2021/10/04 00:36:40 (permalink)
0
Update:
Fortinet removed the expired  DST_Root_CA_X3 from the factory certificate bundle and published an update.
As expected, this does not help if the webserver is misconfigured and
keeps sending a certificate chain still containing the old X1 Cert signed by the expired DST_Root_CA_X3.
 
I'd say, 'fixing' this on the FGT side by ignoring the chain would require a code change...
what means we can expect a behavior similar to browsers only with the next release/build.
 
#16
krzysztof
New Member
  • Total Posts : 5
  • Scores: 0
  • Reward points: 0
  • Joined: 2021/02/15 01:16:57
  • Status: offline
Re: Some .gov sites blocked, others are not? 2021/10/04 03:38:24 (permalink)
0
Hello,
 
I have implemented the workaround 1 from here:
Fortinet and Expiring Let’s Encrypt Certificates
 
and it looks like it did the trick.
 
I pushed following config to all the Fortigates (using the scripting on FortiManager):
config system dns-database
    edit "1"
        set domain "identrust.com"
        config dns-entry
            edit 1
                set hostname "apps"
                set ip 127.0.0.1
            next
        end
    next
end
 
Then I also cleared the DNS cache:
diag test application dnsproxy 1
 
Some of the users had to restart their machines and then it started to work OK.
 
It is a bit dirty solution but definitely better than trusting all invalid certs or entirely disabling deep inspection . 
 
 
#17
boneyard
Gold Member
  • Total Posts : 429
  • Scores: 20
  • Reward points: 0
  • Joined: 2014/07/30 11:15:18
  • Status: offline
Re: Some .gov sites blocked, others are not? 2021/10/04 10:29:02 (permalink)
0
it is also posted in the other thread here but to have it complete, after that trick with the DNS blackhole you need to clear the certificate cache most likely.
 
flow-mode:

# diagnose ips share clear cert_verify_cache

proxy-mode (process restart will have impact):

# diagnose test application wad 99
 
a true fix will be an actual code change indeed, but with this you can move on without having to lower security.
#18
frank0957
New Member
  • Total Posts : 3
  • Scores: 0
  • Reward points: 0
  • Joined: 2021/10/05 03:59:42
  • Status: offline
Re: Some .gov sites blocked, others are not? 2021/10/05 04:08:57 (permalink)
0
I'm facing the same problem on OS FortiOS v7.0.1 build0157 (GA)
 
When I try to change the Flow mode and create a new policy to allow invalid SSL certificates, it still doesn’t work.
 
#19
tuanccs
New Member
  • Total Posts : 5
  • Scores: 0
  • Reward points: 0
  • Joined: 2021/10/06 10:32:25
  • Status: offline
Re: Some .gov sites blocked, others are not? 2021/10/06 10:38:20 (permalink)
0
Still same problem after tried workaround 1 on fortigate 6.2.9.
Does anyone else get it working on fortigate 6.2?
#20
Page: 12 > Showing page 1 of 2
Jump to:
© 2021 APG vNext Commercial Version 5.5