Well, this is interesting. #LetsEncrypt #Firefox #Mozilla

Marcos Dione

@vwbusguy @sesivany Well, they used to have an intermediate certificate C signed by root A and root B¹. Then A expired (the CentOS 6 thing I mention), then C did or was simply replaced by intermediate D, signed only by B.

If I get it right from the error message, you OS trusts B _but ffox doesn't_?

¹ reading that I start to suspect it doesn't make any sense.

Scott Williams 🐧

@deathkitten It's regular Firefox from the Fedora update repositories. Tested on three different Fedora 39 variants on two different networks.

The fact that I don't see it on the same Firefox version with Budgie Atomic but I do with Kinoite is very strange, since they're both Atomic from the same rpm source.

Scott Williams 🐧

I'm able to reproduce this with Firefox 123.0.1 and Firefox 124.0 on two different systems on two different networks, but wasn't able to reproduce it on a third with Firefox 121 or 124.

I got the message with Fedora Workstation 39 and Kinoite 39 but not Budgie Atomic 39. 🫠

I tested with certs generated from certbot, cert-manager, and caddy, both freshly renewed and months old and these all behaved the same.

Raptor :gamedev:

@vwbusguy it's gotta be something with a setting, could it be from the enterprise CA management settings? i'm not getting it on my installs of the same versions from a vanilla source build.

Scott Williams 🐧

@raptor85 Interesting. It also makes no sense to me that I can reproduce it with Kinoite but not Budgie Atomic. Those are both ostree and use the same rpm source for Firefox.

Scott Williams 🐧

For good measure, I don't get this with chromium or curl on the same machines. Only Firefox, which is consistent with the messaging. This suggests it's something to do with Firefox and not Fedora, but the fact that I don't see this on Budgie Atomic has me second guessing everything.

Raptor :gamedev:

@vwbusguy these pictured are auto-enabled in a source build, are they off by default on one of the distros, they might push different settings for the browsers even at the same version.

edit: nope, not this, i thought it could be these settings as they control using your system CAs but i turned them off and all my Lets Encrypt certs don't show the error

Scott Williams 🐧

@mdione @sesivany Correct. The OS trusts it but Firefox does not, but Firefox will still load it anyway because the OS trusts it.

Scott Williams 🐧

@raptor85 Interesting. That is set to true on all three.

Scott Williams 🐧

OK, I think I've figured it out and it might be a me problem. Because cert-manager and Red Hat-ish systems can't agree on whether or not the spec requires an explicit chain, at some point I added the LE intermediary cert into my CA anchors and *that's* what Firefox is now grumping about. I fixed TLS for CLI tools but now Firefox is complaining about it. Never saw that before today and I check TLS certs with it fairly regularly.

Marcos Dione

@vwbusguy where can I read about explicit chains? And are CA anchors what I call the trust store (the store of trusted certificates, usually only roots)?

Scott Williams 🐧

@mdione Here's the very short of it - https://cert-manager.io/docs/faq/#why-isnt-my-root-certificate-in-my-issued-secrets-tlscrt

It's a very longwinded debate however. Red Hat interprets the spec as explicit. Cert Manager, browser vendors, and others don't agree and everyone else in between has to find workarounds for stuff being broken that otherwise shouldn't be because of that disagreement.

Scott Williams 🐧

@mdione As of now, cert-manager refuses to give an option to just do the full chain because they view it as unnecessary and wasteful, but they get requests over it regularly. Likewise, Red Hat has refused to budge in their belief that they are simply following the spec.