domain name system

a-root-server-at-the-internet’s-core-lost-touch-with-its-peers-we-still-don’t-know-why.

A root-server at the Internet’s core lost touch with its peers. We still don’t know why.

Biz & IT, DNS, DNSSEC, domain name system, root servers, Security / /
A root-server at the Internet’s core lost touch with its peers. We still don’t know why.

For more than four days, a server at the very core of the Internet’s domain name system was out of sync with its 12 root server peers due to an unexplained glitch that could have caused stability and security problems worldwide. This server, maintained by Internet carrier Cogent Communications, is one of the 13 root servers that provision the Internet’s root zone, which sits at the top of the hierarchical distributed database known as the domain name system, or DNS.

Here’s a simplified recap of the way the domain name system works and how root servers fit in:

When someone enters wikipedia.org in their browser, the servers handling the request first must translate the human-friendly domain name into an IP address. This is where the domain name system comes in. The first step in the DNS process is the browser queries the local stub resolver in the local operating system. The stub resolver forwards the query to a recursive resolver, which may be provided by the user’s ISP or a service such as 1.1.1.1 or 8.8.8.8 from Cloudflare and Google, respectively.

If it needs to, the recursive resolver contacts the c-root server or one of its 12 peers to determine the authoritative name server for the .org top level domain. The .org name server then refers the request to the Wikipedia name server, which then returns the IP address. In the following diagram, the recursive server is labeled “iterator.”

Given the crucial role a root server provides in ensuring one device can find any other device on the Internet, there are 13 of them geographically dispersed all over the world. Each root sever is, in fact, a cluster of servers that are also geographically dispersed, providing even more redundancy. Normally, the 13 root servers—each operated by a different entity—march in lockstep. When a change is made to the contents they host, it generally occurs on all of them within a few seconds or minutes at most.

Strange events at the C-root name server

This tight synchronization is crucial for ensuring stability. If one root server directs traffic lookups to one intermediate server and another root server sends lookups to a different intermediate server, the Internet as we know it could collapse. More important still, root servers store the cryptographic keys necessary to authenticate some of intermediate servers under a mechanism known as DNSSEC. If keys aren’t identical across all 13 root servers, there’s an increased risk of attacks such as DNS cache poisoning.

For reasons that remain unclear outside of Cogent—which declined to comment for this post—all 12 instances of the c-root it’s responsible for maintaining suddenly stopped updating on Saturday. Stéphane Bortzmeyer, a French engineer who was among the first to flag the problem in a Tuesday post, noted then that the c-root was three days behind the rest of the root servers.

A mismatch in what's known as the zone serials shows root-c is three days behind.

Enlarge / A mismatch in what’s known as the zone serials shows root-c is three days behind.

The lag was further noted on Mastodon.

By mid-day Wednesday, the lag was shortened to about one day.

By late Wednesday, the c-root was finally up to date.

A root-server at the Internet’s core lost touch with its peers. We still don’t know why. Read More »

microsoft-plans-to-lock-down-windows-dns-like-never-before-here’s-how.

Microsoft plans to lock down Windows DNS like never before. Here’s how.

Biz & IT, domain name system, microsoft, Security, Windows, ztdns / /
Microsoft plans to lock down Windows DNS like never before. Here’s how.

Getty Images

Translating human-readable domain names into numerical IP addresses has long been fraught with gaping security risks. After all, lookups are rarely end-to-end encrypted. The servers providing domain name lookups provide translations for virtually any IP address—even when they’re known to be malicious. And many end-user devices can easily be configured to stop using authorized lookup servers and instead use malicious ones.

Microsoft on Friday provided a peek at a comprehensive framework that aims to sort out the Domain Name System (DNS) mess so that it’s better locked down inside Windows networks. It’s called ZTDNS (zero trust DNS). Its two main features are (1) encrypted and cryptographically authenticated connections between end-user clients and DNS servers and (2) the ability for administrators to tightly restrict the domains these servers will resolve.

Clearing the minefield

One of the reasons DNS has been such a security minefield is that these two features can be mutually exclusive. Adding cryptographic authentication and encryption to DNS often obscures the visibility admins need to prevent user devices from connecting to malicious domains or detect anomalous behavior inside a network. As a result, DNS traffic is either sent in clear text or it’s encrypted in a way that allows admins to decrypt it in transit through what is essentially an adversary-in-the-middle attack.

Admins are left to choose between equally unappealing options: (1) route DNS traffic in clear text with no means for the server and client device to authenticate each other so malicious domains can be blocked and network monitoring is possible, or (2) encrypt and authenticate DNS traffic and do away with the domain control and network visibility.

ZTDNS aims to solve this decades-old problem by integrating the Windows DNS engine with the Windows Filtering Platform—the core component of the Windows Firewall—directly into client devices.

Jake Williams, VP of research and development at consultancy Hunter Strategies, said the union of these previously disparate engines would allow updates to be made to the Windows firewall on a per-domain name basis. The result, he said, is a mechanism that allows organizations to, in essence, tell clients “only use our DNS server, that uses TLS, and will only resolve certain domains.” Microsoft calls this DNS server or servers the “protective DNS server.”

By default, the firewall will deny resolutions to all domains except those enumerated in allow lists. A separate allow list will contain IP address subnets that clients need to run authorized software. Key to making this work at scale inside an organization with rapidly changing needs. Networking security expert Royce Williams (no relation to Jake Williams) called this a “sort of a bidirectional API for the firewall layer, so you can both trigger firewall actions (by input *tothe firewall), and trigger external actions based on firewall state (output *fromthe firewall). So instead of having to reinvent the firewall wheel if you are an AV vendor or whatever, you just hook into WFP.”

Microsoft plans to lock down Windows DNS like never before. Here’s how. Read More »

doing-dns-and-dhcp-for-your-lan-the-old-way—the-way-that-works

Doing DNS and DHCP for your LAN the old way—the way that works

BIND, bind9, Biz & IT, dhcpd, DNS, domain name system, Feature, Features, homelab, it projects, Projects, weekend projects / /
All shall tremble before your fully functional forward and reverse lookups!

Enlarge / All shall tremble before your fully functional forward and reverse lookups!

Aurich Lawson | Getty Images

Here’s a short summary of the next 7,000-ish words for folks who hate the thing recipe sites do where the authors babble about their personal lives for pages and pages before getting to the cooking: This article is about how to install bind and dhcpd and tie them together into a functional dynamic DNS setup for your LAN so that DHCP clients self-register with DNS, and you always have working forward and reverse DNS lookups. This article is intended to be part one of a two-part series, and in part two, we’ll combine our bind DNS instance with an ACME-enabled LAN certificate authority and set up LetsEncrypt-style auto-renewing certificates for LAN services.

If that sounds like a fun couple of weekend projects, you’re in the right place! If you want to fast-forward to where we start installing stuff, skip down a couple of subheds to the tutorial-y bits. Now, excuse me while I babble about my personal life.

My name is Lee, and I have a problem

(Hi, Lee.)

I am a tinkering homelab sysadmin forever chasing the enterprise dragon. My understanding of what “normal” means, in terms of the things I should be able to do in any minimally functioning networking environment, was formed in the days just before and just after 9/11, when I was a fledgling admin fresh out of college, working at an enormous company that made planes starting with the number “7.” I tutored at the knees of a whole bunch of different mentor sysadmins, who ranged on the graybeard scale from “fairly normal, just writes his own custom GURPS campaigns” to “lives in a Unabomber cabin in the woods and will only communicate via GPG.” If there was one consistent refrain throughout my formative years marinating in that enterprise IT soup, it was that forward and reverse DNS should always work. Why? Because just like a clean bathroom is generally a sign of a nice restaurant, having good, functional DNS (forward and reverse) is a sign that your IT team knows what it’s doing.

Just look at what the masses have to contend with outside of the datacenter, where madness reigns. Look at the state of the average user’s LAN—is there even a search domain configured? Do reverse queries on dynamic hosts work? Do forward queries on dynamic hosts even work? How can anyone live like this?!

I decided long ago that I didn’t have to, so I’ve maintained a linked bind and dhcpd setup on my LAN for more than ten years. Also, I have control issues, and I like my home LAN to function like the well-run enterprise LANs I used to spend my days administering. It’s kind of like how car people think: If you’re not driving a stick shift, you’re not really driving. I have the same kind of dumb hang-up, but for network services.

Honestly, though, running your LAN with bind and dhcpd isn’t even that much work—those two applications underpin a huge part of the modern Internet. The packaged versions that come with most modern Linux distros are ready to go out of the box. They certainly beat the pants off of the minimal DNS/DHCP services offered by most SOHO NAT routers. Once you have bind and dhcpd configured, they’re bulletproof. The only time I interact with my setup is if I need to add a new static DHCP mapping for a host I want to always grab the same IP address.

So, hey, if the idea of having perfect forward and reverse DNS lookups on your LAN sounds exciting—and, come on, who doesn’t want that?!—then pull up your terminal and strap in because we’re going make it happen.

(Note that I’m relying a bit on Past Lee and this old blog entry for some of the explanations in this piece, so if any of the three people who read my blog notice any similarities in some of the text, it’s because Past Lee wrote it first and I am absolutely stealing from him.)

But wait, there’s more!

This piece is intended to be part one of two. If the idea of having one’s own bind and dhcpd servers sounds a little silly (and it’s not—it’s awesome), it’s actually a prerequisite for an additional future project with serious practical implications: our own fully functioning local ACME-enabled certificate authority capable of answering DNS-01 challenges so we can issue our own certificates to LAN services and not have to deal with TLS warnings like plebes.

(“But Lee,” you say, “why not just use actual-for-real LetsEncrypt with a real domain on my LAN?” Because that’s considerably more complicated to implement if one does it the right way, and it means potentially dealing with split-horizon DNS and hairpinning if you also need to use that domain for any Internet-accessible stuff. Split-horizon DNS is handy and useful if you have requirements that demand it, but if you’re a home user, you probably don’t. We’ll keep this as simple as possible and use LAN-specific DNS zones rather than real public domain names.)

We’ll tackle all the certificate stuff in part two—because we have a ways to go before we can get there.

Doing DNS and DHCP for your LAN the old way—the way that works Read More »