Approx 30 mins ago, I suddenly lost access to lemmy.ca due to DNS resolution failures. I’ve managed to restore access by tossing the known good IP into my computers hosts file, in order to make this post.

It’s worth noting I’m running my own DNS resolver (via PfSense router/firewall software) instead of the typical DNS forwarder offered by my ISP. As a result, my DNS ecosystem is likely a bit more fussy about the “correctness” of the DNS configuration.

My DNS server logs some entries complaining about DS and DNSKEY

debug: Failed to match any usable DS to a DNSKEY.
info: Could not establish a chain of trust to keys for lemmy.ca. DNSKEY IN

An attempt to verify the domain name using delv results in the following:

delv lemmy.ca 
;; broken trust chain resolving 'lemmy.ca/A/IN': 127.0.0.53#53
;; resolution failed: broken trust chain

Running an external DNSSEC check tool, currenly reports DNSSEC related issues. The tool I ran was:

https://dnssec-analyzer.verisignlabs.com/lemmy.ca

This gave me two issues:

None of the 2 DNSKEY records could be validated by any of the 2 DS records

The DNSKEY RRset was not signed by any trusted keys

Did something related to DNS get deployed in the last little while?

In any case, I’ve got a workaround in place and am not stuck, however a site admin may want to be aware of this information.

Thanks!!

  • TruckBCMAEnglish
    8·
    1 year ago

    We’re aware and it should be resolved at this time.

    Did something related to DNS get deployed in the last little while?

    Yes, putting the instance behind cloudflare to protect from DDoS.

    Please let me know if you’re still having issues without your workaround.

    • FarthomOPEnglish
      5·
      1 year ago

      Thanks for the prompt reply!!

      Things didn’t immediately work, but if you had genuinely changed something I know DNS can take some time for things to trickle down as caches/TTLs expire etc etc etc.

      I’m happy to report things appear to be working normally this morning. I’m glad this was just a transition hiccup and not an outright misconfiguration!

      Thanks so much for all your hard work keeping this thing running for us!

  • ThiefEnglish
    2·
    1 year ago

    Mine only started working now like 8h later. Just assumed lemmy.ca was down.

  • nimnimEnglish
    2·
    1 year ago

    I still can’t connect to lemmy.ca using my home router unless I use a VPN which is annoying. I thought the server was down the whole day.

    Edit: It had something to do with the AdGuard DNS settings. After changing and then reverting them back, everything now works perfectly fine.

    • ShadowMAEnglish
      3·
      1 year ago

      I think some DNS servers are holding onto our cached dnssec records for a little longer than they’re supposed to. Every DNS health check I can find is reporting everything is healthy.

      Please let us know if the issue persists!